How to set up the RADAR Platform on local server and private network - RADAR-base/RADAR-Docker GitHub Wiki

These instructions describe how the RADAR Platform can be used for streaming data collection on a local network with the Empatica E4 devices (which do need an outbound connection to data.empatica.com). The context here was the RADAR Epilepsy Patient in-hospital data collection study where data from the E4 devices are being benchmarked for seizure detection against the gold standard of Video-EEG. Synchronization between the Android Device and Video-EEGs uses a local timeserver for synchronisation.

The goal of this wifi is to set up a private network for a pilot study and provide access to a restricted number of web sites on the public Internet.

The hardware components are described as follows:

  • the Operating System is Ubuntu Server 16.04. Can check it running on the command-line:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:    16.04
Codename:   xenial
  • the machine has 3 network interfaces named enp2s0, enp5s0 and wlp5s1, which are respectively two ethernet NICs and a wifi NIC. To check your interfaces name run on the command-line:
$ netstat -i
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
enp2s0     1500 0      4622      0      0 0          2947      0      0      0 BMRU
enp5s0     1500 0      4622      0      0 0          2947      0      0      0 BMRU
wlp5s1     1500 0     85841      0      0 0         53685      0      0      0 BMRU

Note: The entire wiki can be generalised just substituting NIC names.

Note: The Operating System must be up to date. Before starting the set up, connect the machine to internet and run apt-get update && apt-get upgrade.

The environment we are taking into account is composed by:

  • our Ubuntu server with three NICs:
    • enp2s0 192.168.1.200
    • enp5s0 192.168.1.100
    • wlp5s1 10.131.135.186 connected to the public-wifi
  • four Windows machines connected via ethernet with IPs in range 192.168.1.2/31
  • four Android devices connected via wifi (i.e. private-wifi) with IPs in range 192.168.1.12/30
  • two networks:
    • private-network accessible via either ethernet or wifi (i.e. private-wifi)
    • public-wifi that provides access to Internet

We want to allow Windows machines and Android devices to reach our Ubuntu server using the private-network and Android devices to contact another server available on the public Internet via public-wifi.

The idea is to expose our platform and the ntp server on the network private-network (i.e. 192.168.1.0/24), and set as gateway the public-wifi for accessing Internet. To do that we need:

  • the RADAR-Platform listening on 192.168.1.200
  • a HTTP Proxy listening on 192.168.1.100 to provide access to internet
  • a NTP server listening on 192.168.1.100 to provide synchronisation
  • a Firewall to control the inbound and outbound traffic

As Proxy we have selected an open source project called Tinyproxy; as Firewall another open source project named UFW and as Network Time Server the standard linux package.

First, specify static IPs in /etc/network/interfaces:

auto enp2s0
    iface enp2s0 inet static
    metric 300
    address 192.168.1.200
    netmask 255.255.255.0
    network 192.168.1.0

auto enp5s0
    iface enp5s0 inet static
    metric 200
    address 192.168.1.100
    netmask 255.255.255.0
    network 192.168.1.0

Restart the network service

$ sudo service networking restart

To manage the connection with the public-wifi is suggested to use the NetworkManager instead of the interfaces. This is needed due to possible authentication issues related to the certificates. First of all, install the NetworkManager 

$ sudo apt-get install network-manager
$ sudo systemctl enable NetworkManager
$ sudo reboot

After rebooting, find the public-wifi's bssids running nmcli -f in-use,ssid,bssid dev wifi. From the returned list, copy the needed bssids

*  SSID                  BSSID
   2.4-RADAR             50:C7:BF:1D:99:4A
   SGDP-DOBSON           20:C9:D0:1E:15:53
   eduroam               DC:A5:F4:9C:3E:10
   The Cloud             DC:A5:F4:9C:3E:15
   KINGSWAP              DC:A5:F4:9C:3E:12
   PLOCAL                DC:A5:F4:9C:3E:13
   KINGSWAP              DC:A5:F4:65:10:52
   PLOCAL                DC:A5:F4:65:10:53
   eduroam               DC:A5:F4:65:10:50
   The Cloud             DC:A5:F4:65:10:55

For instance, for eduroam the bssids are DC:A5:F4:9C:3E:10 and DC:A5:F4:65:10:50.

Run ifconfig wlp5s1 to find the MAC-Address of the wifi NIC, it is listed as HWaddr

wlp5s1    Link encap:Ethernet  HWaddr 84:16:f9:f3:bc:99
          inet6 addr: fe80::fc94:f910:d176:1478/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3798 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3676 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:518300 (518.3 KB)  TX bytes:407318 (407.3 KB)

In this case 84:16:f9:f3:bc:99.

Go to /etc/NetworkManager/system-connections and create a file with the same name of the public-wifi. For eduroam, run sudo vim eduroam and paste the following configuration file inserting the correct parameters

[connection]
id=<WIFI-SSID>
uuid=
type=wifi
permissions=
secondaries=
timestamp=

[wifi]
mac-address=<HW-ADDRESS>
mac-address-blacklist=
mac-address-randomization=0
mode=infrastructure
seen-bssids=<AP-HW-ADDRESS>
ssid=eduroam

[wifi-security]
group=
key-mgmt=wpa-eap
pairwise=
proto=

[802-1x]
altsubject-matches=
eap=peap;
identity=<USERNAME>
password=<PASSWORD>
phase2-altsubject-matches=
phase2-auth=mschapv2

[ipv4]
dns-search=
method=auto

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
ip6-privacy=0
method=auto

Note: in case of multiple bssids, insert them as a semicolon separated list DC:A5:F4:9C:3E:10;DC:A5:F4:65:10:50;.

Now, reboot and check the public-wifi has been automatically connected running nmcli d

DEVICE           TYPE      STATE      CONNECTION
wlp5s1           wifi      connected  eduroam
enp2s0           ethernet  unmanaged  --
enp5s0           ethernet  unmanaged  --
lo               loopback  unmanaged  --

Run route -n and check the routing table is now listing all NICs with the correct rooting policy: the only available gateway must be the public-wifi

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.41.128.1     0.0.0.0         UG    600    0        0 wlp5s1
1.1.1.1         10.41.128.1     255.255.255.255 UGH   600    0        0 wlp5s1
10.41.128.0     0.0.0.0         255.255.128.0   U     600    0        0 wlp5s1
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 enp2s0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 enp5s0

To establish a synchronisation on the private-network, install the Network Time Server and enable it on boot

sudo apt-get install ntp
sudo systemctl enable ntp

The Network-Manager has ntpdate as dependency. Due to this bug, ntp cannot start on boot if ntpdate is installed. To remove ntpdate run sudo apt-get purge ntpdate.

Reboot the system and then run ntpq -p. If the Network Time Server is working as expected, the last command will return the list of known ntp servers.

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 0.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 1.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 2.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 3.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 ntp.ubuntu.com  .POOL.          16 p    -   64    0    0.000    0.000   0.000
 LOCAL(0)        .LOCL.          10 l    -   64    0    0.000    0.000   0.000
+139.59.199.215  158.43.128.33    2 u  168  512  377    3.610    1.284   6.029
+85.199.214.102  .GPS.            1 u   36   64  377    4.661   -0.575   2.365
+ntp0.pipex.net  .PPS.            1 u  144  512  377    3.421   -0.516   2.187
+host-87-242-168 .UPPS.           1 u  170  512  275   12.786   -0.516   4.679
*85.199.214.100  .GPS.            1 u   33   64  377    4.698   -0.501   0.623
+y.ns.gin.ntt.ne 249.224.99.213   2 u  193  512  377    3.234   -0.170   7.358
+greenore.zeip.e 140.203.204.77   2 u  159  512  377    3.289    0.240   0.930
+sundown.speedwe 87.242.168.84    2 u  156  512  377    3.615   -0.454   1.488
+ldn-web.globaln 85.199.214.99    2 u  155  512  377    3.337   -0.260   4.971

Note: The * is the one currently synchronised.

To install Tinyproxy run on the command-line:

$ sudo apt-get install tinyproxy

Once the installation is complete, open the Tinyproxy configuration file with your editor of choice

$ sudo vim /etc/tinyproxy.conf

By default Tinyproxy will only accept connections from 127.0.0.1. To allow the Android device to access it, we need to modify the authorisation controls section adding the Android IP address to the allow list:

Allow 127.0.0.1
Allow 192.168.1.0/24

By default, Tinyproxy listens on port 8888, but it can be changed to whatever you like.

Note: If you want to run your Tinyproxy server on a port lower than 1024 you will need to start Tinyproxy using root.

Save the configuration and restart the service.

$ sudo /etc/init.d/tinyproxy restart

To start Tinyproxy on boot

$ sudo systemctl enable tinyproxy

Now, we can connect the Android devices on the private-wifi setting as proxy

host: 192.168.1.100
port: 8888

and specifying to bypass the proxy for 192.168.1.200.

Note: Bypassing Tinyproxy to access the RADAR Platform allows us to achieve better performance, data sources can reach the server with less network hopes.

We need to limit the number of address that can be reached via Tinyproxy. The next step is configuring ufw since it is pre-installed in Ubuntu. We want to allow:

  • all traffic in both direction on the private-network
  • allow access to a specific public IP on HTTPS
  • allow access to a specific range of public IPs on HTTPS
  • allow traffic for the Network Time Protocol
  • allow access to public DNSs
  • allow traffic for git

By default, ufw denies all traffic in both directions.

  1. enable ufw

    sudo ufw enable

  2. allow all traffic on the private-network

    sudo ufw allow from 192.168.1.0/24
sudo ufw allow to 192.168.1.0/24

  3. set default behaviour

    sudo ufw default deny incoming
sudo ufw default deny outgoing

  4. allow all traffic on port 8888, 8081 and 8082 within the private-network

    sudo ufw allow out on enp5s0 to any port 8888
sudo ufw allow in on enp5s0 to any port 8888
sudo ufw allow out on enp2s0 to any port 8082
sudo ufw allow in on enp2s0 to any port 8082
sudo ufw allow out on enp2s0 to any port 8081
sudo ufw allow in on enp2s0 to any port 8081
sudo ufw allow out on enp2s0 to any port 8888
sudo ufw allow in on enp2s0 to any port 8888

  5. provide HTTPS access to a specific public IP address on NIC wlp5s1

    sudo ufw allow out on wlp5s1 to 184.169.135.131 port 443

    Note: If you want to allow access to a public IP but you know the URL you can use the command dig to find the public IP. For example

    $ dig data.empatica.com
    ; <<>> DiG 9.10.3-P4-Ubuntu <<>> data.empatica.com
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17860
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 4096
    ;; QUESTION SECTION:
    ;data.empatica.com.     IN  A  
    ;; ANSWER SECTION:
    data.empatica.com.  300 IN  A   184.169.135.131
    ;; Query time: 7 msec
    ;; SERVER: 127.0.1.1#53(127.0.1.1)
    ;; WHEN: Thu Feb 23 19:11:45 GMT 2017
    ;; MSG SIZE  rcvd: 62

    Note: This configuration is valid as long as the public server that you want to access does not change IP address. In case it changes, run again dig and update the firewall rule.

    Repeat this step also for www.empatica.com

  6. allow the Network Time Protocol

    sudo ufw allow in on wlp5s1 to any port 123
sudo ufw allow out on wlp5s1 to any port 123

  7. allow DHCP

    sudo ufw allow out on wlp5s1 to any port 67 proto udp
sudo ufw allow out on wlp5s1 to any port 68 proto udp

  8. provide access to the public DNS servers

    sudo ufw allow out on wlp5s1 to any port 53

  9. allow SMTP traffic for RADAR-CNS monitors

    sudo ufw allow in on wlp5s1 to any port 25
sudo ufw allow out on wlp5s1 to any port 25

  10. provide access to git and Github.com on HTTPS via NIC wlp5s1 

    sudo ufw allow git
sudo ufw allow out on wlp5s1 to 192.30.252.0/22 port 443
sudo ufw allow in on wlp5s1 from 192.30.252.0/22 port 443

    Note: The Github's IP address range has been extracted form here. These IPs may change (or be added to) in the future, but in any case following the previous link you can find the up-to-date information.

By default, ufw is not automatically enabled. To start it upon boot, modify the configuration file /etc/ufw/ufw.conf

ENABLED=yes

Reboot the server. To check that everything is working as expected, run on the command-line:

curl -H 'Cache-Control: no-cache' -x 192.168.1.100:8888 https://data.empatica.com  \ 
     --connect-timeout 8 --max-time 20
curl -H 'Cache-Control: no-cache' -x 192.168.1.100:8888 https://184.169.135.131  \ 
     --connect-timeout 8 --max-time 20
curl -H 'Cache-Control: no-cache' -x 192.168.1.100:8888 https://www.google.com \
     --connect-timeout 8 --max-time 20

While the first two commands should be allow, the third will return an error saying that the address cannot be reached. Install the RADAR-CNS Platform following this link and check that the REST-PROXY can be contacted:

curl -H 'Cache-Control: no-cache' -x 192.168.1.100:8888 http://192.168.1.100:8082/topics \
     --connect-timeout 8 --max-time 20

This should show you allow available Kafka topics.

Useful commands

  • to restart Tinyproxy run sudo /etc/init.d/tinyproxy restart
  • to check the Tinyproxy status run sudo service tinyproxy status
  • to check the ufw log run grep UFW /var/log/syslog
  • to check the Tinyproxy log run sudo tail -f /var/log/tinyproxy/tinyproxy.log
  • to check the sudo ufw status
  • to remove a firewall rule, run sudo ufw status numbered and then sudo ufw delete #. Anytime, you remove a rule the list numbers may change. To remove multiple rules, run first the status command and the remove.

Android Network

Tinyproxy's log reports connections to connectivitycheck.gstatic.com/generate_204. This URL is used by Android, since Android Marshmallow 6.0.0_r1, to check whether the device is connected to the public internet. Since ufw is denying all traffic by default, this Google service cannot be accessed and the device will notify that it is connected to a network without internet access.

Improvements

If the router has the functionality to set a whitelist of MAC addresses allowed to join the network, we strongly recommend to enable it jointly with the usual password credentials. Add all UDOOS and server MAC addresses, and then add the MAC addresses of all staff members NICs that are allow to access the environment.

⚠️ **GitHub.com Fallback** ⚠️