Light At The End Of The Tunnel

Capturing the data

My scenario is that I wish capture some multicast data and analyse it later.  I’d like this process to be automated, start at a specific time, stop at a specific time, and then process the captured data file.

I’ll be using CentOS 7 which is based on systemd.

Fortunately, the ExaNIC cards come with a great utility called exanic-capture,  this tool is one of the features of the ExaNIC card outlined in this post on the Exablaze site.  The  exanic-capture utility can produce a pcap file that has nanosecond timestamp resolution (6.2 nanoseconds).

We will also sync the ExaNIC‘s  internal timer to the host’s clock.  The host in this case is running NTP to discipline the system clock.

systemd has the capability to create services and associated timers, in this case I wanted to capture mulitcast data between the hours of 0830 and 1700.  I first created a mulitcast listener service that registers the host to listen for the data, and then a bit later I start a service to capture the raw data.  Note I did not filter the data as I also wanted the data to be passed up to the kernel. If you add a filter to the exanic-capture utility it keeps the data to itself.

Multicast listener

For the services on CentOS 7  I will define them under

/etc/systemd/system

mdlisten.service

[Unit]
Description=Multicast Data listener
After=NetworkManager.service
Requires=NetworkManager.service

[Service]
TimeoutStartSec=0
ExecStart=/usr/bin/socat UDP4-RECVFROM:30011,ip-add-membership=239.192.1.11:10.3.78.13,fork -
Restart=on-abort

[Install]
WantedBy=multi-user.target

mdlisten.timer

[Unit]
Description=Start registering for multicast data

[Timer]
OnCalendar=Mon-Fri *-*-* 08:00:00
Unit=mdlisten.service

[Install]
WantedBy=basic.target

Multicast data capture service

I called this service datacapture.service

[Unit]
Description=Multicast Data collector
After=mdlisten.service
Requires=mdlisten.service

[Service]
TimeoutStartSec=0
ExecStart=/bin/sh -c "export D=\"`date +\"%Y%%m%%d\"`\"; /usr/bin/exanic-capture -i exanic0:0  -w /data/logging/$(hostname -s)/\"$D\".pcap "
ExecStop=/bin/sh -c "export D=\"`date +\"%Y%%m%%d\"`\"; cd /data/logging/$(hostname -s)/;/usr/local/bin/process-data.sh \"$D\".pcap \"$D\""
#Restart=on-abort

[Install]
WantedBy=multi-user.target

Multicast data capture timer

I have this setup to start 5 minutes after the listener

[Unit]
Description=Start listening to market data

[Timer]
OnCalendar=Mon-Fri *-*-* 08:05:00
Unit=datacapture.service

[Install]
WantedBy=basic.target

Clock sync service

This service is going to run all the time, and requires that NTP service is running

[Unit]
Description=ExaNIC Clock Sync to host
After=ntpd.service
Requires=ntpd.service

[Service]
TimeoutStartSec=0
ExecStart=/usr/bin/exanic-clock-sync exanic0:host
Restart=on-abort

[Install]
WantedBy=multi-user.target

Next Steps

systemctl daemon-reload
systemctl status mdlisten
systemctl status mdlisten.timer

You will see that both services are disabled by default, we will need to enable each of new services

systemctl enable mdlisten
systemctl enable mdlisten.timer
systemctl enable market
systemctl enable datacapture.timer
systemctl enable datacapture

Then ensure that the status is correct for each service and that the timers fire their respective services at the correct times.