As a result, a both fresh and advanced user finds himself in a situation where the system – ceases to work at all, or requires frequent restarts, or worse causes quick degradation of the main storage device – which is often (micro) SD card.
Is there a simple and effective solution to increase Domoticz’s availability on Raspberry Pi / Orange Pi?
Sure there is! But first – before you begin – make a copy of your (micro)SD working used in domoticz instance on your Raspberry Pi/Orange Pi.
CAUTION: We will perform operations that may lead to the loss of all data – so make sure you have a copy!
What will you need?
- harddrive either classic HDD or better SSD, small is OK from 32GB up to 128GB
- heatsinks for Raspberry Pi/Orange Pi
- USB 2.0 or 3.0 to SATA adapter – so we can connect the harddrive via USB port
Let’s do it!
The plan is very simple – connect the hard disk, move data and check if we have heat sinks. SSD type disk is recommended due to long life, reliability and low power consumption. Even the older designs are fine or MLC/TLC flash type based- will serve us for a very long time. We will transfer the following resources to the disk:
- /root – this is one of the domoticz possible location
- /home – old style installation folder for domoticz
- /var – fast changing files, databases, possibly influxdb
While switching to a solid disk (SSD) – we cannot neglect the microSD card – avoid any no-name cards, and remember to verify the named ones by registering on the manufacturer’s website. The “endurance” cards can be also used, but usually their price is close to the price of the SSD.
The risk of the damage to the micro SD will be significantly reduced after the switch, especially the unfamous problem of the physical damage to the card’s cells when power cycling the computer
Don’t forget to put the heat sinks on the CPU, memory and controller – RPi 3 gets quite hot!
Hardware – the SSD
The SSD needs to have:
- SATA interface
- 5V only voltage
- power (mA), the best if it reaches a 50% of the max. 800mA of the Raspberry Pi3’s USB port, enable max_current=1 for older models or Zero
You will need to purchase a USB-SATA adapter/bridge for 2.5″ SATA disks, to be able to connect a SSD with a SATA interface, with the integrated, USB-sourced power. Adapters with one USB 3.0 connector (blue), compatible with USB 2.0 and based on the JMicron chipset are recommended. I’m using today (3 years) an old one – without trim/ATA support.
Please test the connection of the disk to an already working Raspberry Pi, and during the restart – please note the behavior of the Pi’s red LED – it cannot go off, neither a lightning bolt icon might appear in the case you’re uning monitor via HDMI / CVSB.
Issue the “dmesg” command after connecting new SSD, output should be similar:
[ 5.649443] usb 1-1.3: new high-speed USB device number 6 using dwc_otg
[ 5.782360] usb 1-1.3: New USB device found, idVendor=05e3, idProduct=0718
[ 5.786326] usb 1-1.3: New USB device strings: Mfr=0, Product=1, SerialNumber=2
[ 5.790444] usb 1-1.3: Product: USB Storage
[ 5.794449] usb 1-1.3: SerialNumber: 000000000033
[ 5.806179] usb-storage 1-1.3:1.0: USB Mass Storage device detected
[ 5.814256] scsi host0: usb-storage 1-1.3:1.0
[ 5.922077] gpiomem-bcm2835 3f200000.gpiomem: Initialised: Registers at 0x3f200000
[ 6.362614] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null)
[ 6.902236] scsi 0:0:0:0: Direct-Access INTEL SS DSA2CT040G3 0016 PQ: 0 ANSI: 4
[ 6.917156] sd 0:0:0:0: [sda] 78165356 512-byte logical blocks: (40.0 GB/37.3 GiB)
[ 6.933290] sd 0:0:0:0: [sda] Write Protect is off
[ 6.933314] sd 0:0:0:0: [sda] Mode Sense: 03 00 00 00
[ 6.934475] sd 0:0:0:0: [sda] No Caching mode page found
[ 6.934491] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 6.941523] sda: sda1
[ 6.944631] sd 0:0:0:0: Attached scsi generic sg0 type 0
[ 6.949765] sd 0:0:0:0: [sda] Attached SCSI disk
This gives us the following information: this is the only external drive, system knows it as /dev/sda, first partition is /dev/sda1. Next up is the task of formatting that partition (deletes all data!) with the ext4 file system (side note: B-tree File System or Butter FS is also good choice, but we talk stable here).
First – let’s install new software:
apt install rsync monit
Then issue the mkfs command on your Pi (make sure it’s the right Pi, the right partition – as the process deletes all data).
sudo mkfs.ext4 /dev/sda1
Now, lets mount the new drive to check if it works:
mkdir -p /mnt/ssd
mount /dev/sda1 /mnt/ssd
The dmesg should show us:
[ 7.744129] EXT4-fs (sda1): mounting with "discard" option, but the device does not support discard
[ 7.754021] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: discard,commit=600,errors=remount-ro
Now update the /etc/fstab with the following from the top:
dev/sda1 /mnt/ssd ext4 discard,noatime,commit=600,errors=remount-ro 0 1
/mnt/ssd/var/lib /var/lib none bind 0 1
/mnt/ssd/home /home none bind 0 1
/mnt/ssd/opt /opt none bind 0 1
It’s time to turn off all services – except sshd – so we can tranfer all the files to SSD. It sould be done on other computer with system not working but the risk is rather low.
sudo su -
systemctl stop domoticz.sh
systemctl stop influx.service
systemctl stop grafana.service
systemctl stop cron.service
systemctl stop rsyslog.service
Copy the /home /root i /var to SSD.
sudo rsync -avz /home /root /var /mnt/ssd/
ls -alR /mnt/ssd
This should give us a long list of files.
Now restart your Pi and after booting up – check:
sudo mount | grep sda1
/dev/sda1 on /mnt/ssd type ext4 (rw,noatime,discard,errors=remount-ro,commit=600,data=ordered)
/dev/sda1 on /root type ext4 (rw,noatime,discard,errors=remount-ro,commit=600,data=ordered)
/dev/sda1 on /home type ext4 (rw,noatime,discard,errors=remount-ro,commit=600,data=ordered)
/dev/sda1 on /var/ type ext4 (rw,noatime,discard,errors=remount-ro,commit=600,data=ordered)
Stable software – monit
Domoticz is quite a complex software, often incorporates code of different quality and development dynamics. As a result, adding even an innocent sensor or function – may cause destabilization – even if using official stable version. A good remedy for the “Domoticz offline” syndrome is monit – a piece of software that can be used to monitor domoticz’s operation (of course not only domoticz), restarting it after crash while having a nice web interface.
Since we installed monit in previous step – let’s edit the /etc/monit/monitrc – find this section and modify it – the PASSWORD and “httpd port” value
## Monit has an embedded HTTP interface which can be used to view status of
## services monitored and manage services from a web interface. The HTTP
## interface is also required if you want to issue Monit commands from the
## command line, such as 'monit status' or 'monit restart service' The reason
## for this is that the Monit client uses the HTTP interface to send these
## commands to a running Monit daemon. See the Monit Wiki if you want to
## enable SSL for the web server.
set httpd port 8888 and
use address localhost # only accept connection from localhost
allow localhost # allow localhost to connect to the server and
allow admin:PASSWORD # require user 'admin' with password 'monit'
use address 0.0.0.0 # only accept connection from localhost (comment to connect from other hosts)
allow @monit # allow users of group 'monit' to connect (rw)
allow @users readonly # allow users of group 'users' to connect readonly
Then add the domoticz section (assuming it’s there using port 8080, and has no username/password for localhost):
check process domoticz with pidfile /var/run/domoticz.pid
start program = "/etc/init.d/domoticz.sh start"
stop program = "/etc/init.d/domoticz.sh stop"
and content = '"status" : "OK"'
for 2 cycles
if 5 restarts within 5 cycles then exec "/sbin/reboot"
sudo systemctl enable monit.service
sudo systemctl restart monit.service
Check the monit’s web interface – you should get similar results:
Some final thoughts on the first part: The best way is to have trim support – but most of the adapter/bridges doesn’t pass the ATA/trim command, which means that it’s good to do manual fstrim – at the time of regular micro SD card backup – by means of connecting the SSD to other computer.
Don’t forget this is not 100% proof solution, it’s a step to extend life and make the system more stable. The next part will add the ongoing and needed backup to external filesystem (NFS), and proper installation of external power (UPS).