Creating a simple, quick Borg backup

Here is how to create a simple borg backup. Something more along the lines of a zip file than a careful backup system. I’m using this method for long term storage. The primary benefits are de-duplication and compression.

Again, to clarify – these are not backups meant to be updated each day/week/month. These are simply long term storage that I normally would have made by rsyncing or making a tgz.

To create, run the following:

borg init -e none test1
borg create test1::first /path/to/files
borg info test1

My test case gave 50% savings:

                       Original size      Compressed size    Deduplicated size
All archives:               50.42 GB             35.15 GB             25.58 GB

                       Unique chunks         Total chunks
Chunk index:                  215930               520240

Don’t forget that you can mount borg archives to browse files.

borg mount test1::first /mnt/borg
borg unmount /mnt/borg

Using a private key file with sftp over lftp

I ran into a few issues trying to use a non-default private key file with lftp. Here is how I got it to work:

inside ~/.lftprc put

set sftp:connect-program "ssh -p 23 -a -x -i /home/my_user/.ssh/key_name_here"

The important items are what is passed to the -i option – this should be the private key you want to use. In my example I’m also using a non-standard port (-p 23 for port 23)

To connect, run:

lftp -u your_user_name,blank sftp://your-server.org

For this to work you need to:

  • specify sftp:// as the server type
  • set the password to a junk value, I use “blank”

Slow login and su on Centos 7 (not an ssh or dns problem)

I had a particular Centos 8 box that had very slow logins but over ssh but also using su. The issue was not with dns and ssh. I finally found the problem – or at least the fix (related to systemd, sigh):

systemctl restart systemd-logind

The server was running inside of openvz so perhaps there were resource issues at some point. I did not have time to determine why it broke but the above did fix it.

Reset Centos 8 root password

reboot the server

at grub menu, press up/down to pause auto booting

select top menu item, press e (to edit)

in editing menu, go to end of line starting with “linux” and add “rd.break”

press control-x to boot using this edited grub command line

when the server starts you should not have to login

at the prompt, type :

mount -o remount,rw /sysroot/
chroot /sysroot
passwd root
shutdown -r now

Arch Linux virtualbox install on OSX

If your Virtualbox screen is too small, go to preferences and adjust screen scaling to 300% (or your choice). You’ll need to restart any running virtual machine to see the effect of this.

fdisk /dev/sda

# inside fdisk
n
p
enter
enter
w

# you should now out of fdisk and back in the shell
mkfs.ext4 /dev/sda1

wifi-menu
ping kernel.org

pacman -Syy

pacman -S reflector
reflector -c "US" -f 12 -l 10 -n 12 --save /etc/pacman.d/mirrorlist

mount /dev/sda1 /mnt
pacstrap /mnt base linux linux-firmware vim nano
genfstab -U /mnt >> /mnt/etc/fstab
arch-chroot /mnt
timedatectl list-timezones
timedatectl set-timezone <your-time-zone-here>


# i skipped this
locale-gen
echo LANG=en_GB.UTF-8 > /etc/locale.conf
export LANG=en_GB.UTF-8

echo "arch.metajack.org" > /etc/hostname

largely from : https://itsfoss.com/install-arch-linux/

When mysql won’t start due to innodb errors

Over the years innodb has caused me more trouble than anything else in the LAMP stack. MyISAM tables seem to work and repair without issue. Innodb seem to be endless trouble. Alas, clients need them so I have to make them work.

Today a mysql server crashed and wouldn’t restart. A few of the errors:

File operation call: 'read' returned OS error 71.
InnoDB: Operating system error number 2 in a file operation

among others. I won’t get into the causes – I’m sure 99% is due to poor shutdowns and other issues that are not really the fault of mysql (other than the lack of graceful failure whichh myisam seems to be so good at).

Here is a quick fix when under pressure – this won’t fix your problems if you have a single innodb table you need but if you have a mixed set of databases and tables this will get at least something working – which in my case was better than nothing.

Note that everything I’m saying to do is very bad form and practice. But if you have critical databases that need to be up it may help you out. As always, back everything up first just in case. You’ll have a back-up of a broken database but perhaps it will be a bit less broke than the version you have if the following goes wrong.

First find and move all databases with innodb tables to a temp directory :

mkdir /var/lib/tmp_disable
find /var/lib/mysql -name "*.ibd" -print
# you'll see various files inding in .ibd - look at the directory they are in (the database name) and move that dir out of the mysql data directory
mv <database-dir> /var/lib/tmp_disable

Finally move the global .ibd and logs to the tmp directory:

mv /var/lib/mysql/ibdata1 /var/lib/mysql/ib_logfile* /var/lib/tmp_disable

mysql won’t like this and it’s not really a good idea. However your mysql server should at least start now. A new (empty) ibdata1 will be created. This will break a lot of things if your data is stored in this global datafile. However if you have each innodb table writing to its own ibd (by setting innodb_file_per_table in /etc/my.cnf) you might be ok.

Next, move each database back into /var/lib/mysql on-by-one, restarting mysql each time to see if it will still start. What to do with the database that is causing the problem? No easy answer to that unfortunately. I usually build a temporary mysql server in a VPS to work on it. At least your overall server will run while you are able to troubleshoot the problem database table(s).

Fixing promox lockups

My proxmox 6 test install locked up every day or two. Here is the fix. This in particular seems to apply to common hardware on Hetzner servers.

In /etc/default/grub, update this line :

GRUB_CMDLINE_LINUX_DEFAULT="consoleblank=0 intel_idle.max_cstate=1"

Reboot and you should find your server stable.

Read more on the proxmox forum at https://forum.proxmox.com/threads/random-crashes-reboots-mit-proxmox-ve-6-1-auf-ex62-nvme-hetzner.63597/page-2

Does the Yamaha DX100 send program change commands?

The manual for the DX100 says it sends program changes yet mine refused to do so. Here is how to make the DX100 send program changes.

To send program changes you must have “ch info” set to on (as the manual says). What the manual does not mention is that “sys info” must be set to off. Importantly, after you set “sys info” to off you must also first press the Internal Bank and select any program in that bank to start program change transmission. If you press other banks program changes are not sent until you first press bank 1 (the internal bank). After this program change is always transmitted until you turn off “ch info” or turn on “sys info”

Not sure if this is a bug or not but it will help to think of it as one – turning off “sys info” does not turn on program changing until you first select program 1 (internal bank, patch one).

By the way, when “sys info” is on, the DX100 transmits the sysex for the patch on each patch change.