There are many reasons for running a Linux service. Linux provides multiple mechanisms to so. Before systemd, it was difficult to start a service and keep it running. Like any program, it's possible for the process to die, systemd will restart it.

The example below shows how to use systemd to run multiple simulated sensors. simsensor.js uses NodeJS to read a file one line at a time. The JSON obtained is published to an MQTT topic. I won't describe this program, I'll just describe how to run multiple simsensor.js processes as a service.

Using systemd(8), a service is managed with these characteristics:

  • The program is started at boot after all of the required services have been started
  • The program is killed by the systemctl stop service command
  • The program is started by the systemctl start service command
  • The program status is provided by the systemctl status service command
  • The program is restarted if it fails

systemd(8) requires a configuration file for each service.  This configuration file, simsensors.service, is used for the simsensors example service:

[Unit]
Description=Sim Sensors Service
Wants=network-online.target

[Service]
ExecStart=/bin/bash -c /home/bitnami/simsensors/simsensors.sh
ExecStop=/bin/bash -c /home/bitnami/simsensors/simsensors.stop.sh
Restart=always

[Install]
WantedBy=multi-user.target

Let's create the simsensors.sh and simsensors.stop.sh scripts and make them executable

simsensors.sh

#!/bin/bash
#
# simsensors.sh
#
shopt -s expand_aliases

# for debugging
# exec > /dev/pts/0
# exec 2> /dev/pts/0

exit_handler() { echo EXIT: killing ${PIDS}; kill ${PIDS}; }
int_handler() { kill ${PIDS}; echo INT: killing ${PIDS}; }
kill_handler() { kill ${PIDS}; echo KILL: killing ${PIDS}; }

trap exit_handler EXIT
trap int_handler SIGHUP
trap kill_handler SIGKILL

PIDS=""

if [ -d /home/bitnami/simsensors ]
then
    cd /home/bitnami/simsensors
    alias node=/opt/bitnami/nodejs/bin/node
fi

. ./simsensor.env.sh
for i in 1221_RTP 1221_garage_f 1221_outside 1221_sunroom_f 1221_sunroom_hum
do
    node simsensor.js $i.json &
    PIDS="${PIDS} $!"
    sleep 1
done
echo $PIDS running
wait

simsensors.stop.sh

#!/bin/bash
/usr/bin/pkill simsensors.sh

Hopefully you are using Debian or Ubuntu, in which case systemd  is already running. This file can reside in a few locations, man systemd will list the valid locations for your Linux system. In this example we'll use this executable script, simsensors.install.sh to install the simsensors.service file:

simsensors.install.sh

#!/bin/bash
cp -f simsensors.service /lib/systemd/system
cd /etc/systemd/system
ln -sf /lib/systemd/system/simsensors.service .

Execute the script:

% sudo ./simsensors.install.sh

At this point, the file is in the right place, but systemd does not know about it.

We'll enable the service, which creates the appropriate symbolic links. We'll execute disable first to clean up any incorrect links from previous attempts. Then enable the service

% sudo systemctl disable simsensors
% sudo systemctl enable simsensors
% sudo systemctl start simsensors

The service is now running. Let's check it's status:

$ sudo systemctl status simsensors
● simsensors.service - Sim Sensors Service
   Loaded: loaded (/lib/systemd/system/simsensors.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-04-06 14:56:12 UTC; 14s ago
  Process: 14788 ExecStop=/bin/bash -c /home/bitnami/simsensors/simsensors.stop.sh (code=exited, status=0/SUCCESS)
 Main PID: 14793 (simsensors.sh)
    Tasks: 56
   Memory: 114.9M
      CPU: 1.571s
   CGroup: /system.slice/simsensors.service
           ├─14793 /bin/bash /home/bitnami/simsensors/simsensors.sh
           ├─14798 /opt/bitnami/nodejs/bin/.node.bin simsensor.js 1221_RTP.json
           ├─14814 /opt/bitnami/nodejs/bin/.node.bin simsensor.js 1221_garage_f.json
           ├─14830 /opt/bitnami/nodejs/bin/.node.bin simsensor.js 1221_outside.json
           ├─14846 /opt/bitnami/nodejs/bin/.node.bin simsensor.js 1221_sunroom_f.json
           └─14865 /opt/bitnami/nodejs/bin/.node.bin simsensor.js 1221_sunroom_hum.json

Apr 06 14:56:12 ip-172-26-2-217 systemd[1]: Stopped Sim Sensors Service.
Apr 06 14:56:12 ip-172-26-2-217 systemd[1]: Started Sim Sensors Service.
Apr 06 14:56:17 ip-172-26-2-217 bash[14793]: 14798 14814 14830 14846 14865 running

The output shows the parent process and subprocesses, this is correct. The service will now run forever.

Magic in simsensors.sh

This script must manage the processes running in the background. This requires knowing their PIDs and trapping any signals to make sure all the processes are killed, otherwise they'll become zombies.

These lines run the programs in the background and keeps track of their PIDs:

node simsensor.js $i.json &
PIDS="${PIDS} $!"

Traps are used to run code when signals are received, including if the script exits normally (which shouldn't occur unless it has a bug).

It is possible to send kill to a process with causes the signal to propogate down the process tree. This would eliminate the need to keep track of PIDs and to avoid calling kill in the trap handlers. However, killing an individual child does not propogate the signal to the parent. Therefore, it's better to just kill in the trap handler.

Trap handling can be used in any script. Trapping EXIT can be useful for removing temporary files for instance.