mwptools in the cloud - stronnag/mwptools GitHub Wiki
mwptools in the cloud
Unmaintained Article Please check the manual for more up-to-date information.
The problem
You have (heaven forbid) a local, physical non-Linux computer and access to a Linux instance in the cloud (for example an AWS EC2 node). How can you upload mwp designed missions into the FC when the FC is connected to your local machine?
Two solutions
In summary, you need to be able to make the FC's USB device available over the network to the cloud instance of mwptools. Two solution are offered; in both cases there is an assumption that the cloud instance can connect to a TCP/IP port on the local machine. How to acheive that is beyond the scope of this artice.
usbip
usbip is a cross-platform technology to facilitate the use of USB devices over an IP network. Implementations exist for (at least) Linux, MacOS and MS Windows. If necessary see usbip on sourceforge. On Linux at least, this should not be necessary as your distro should provide a package for usbip.
Set up the physical side
On the (local) machine with the physical hardware, it is necessary to run the usbipd daemon. On most Linux distros:
sudo systemctl start usbipd
sudo systemctl enable usbipd # optionally, make this survive reboots
Then connect the FC and find its busid
$ usbip list -l
- busid 3-10.1 (0a5c:2101)
Broadcom Corp. : BCM2045 Bluetooth (0a5c:2101)
- busid 3-6.1 (0483:5740)
STMicroelectronics : STM32F407 (0483:5740)
In this case, I want to make busid 3-6.1 (a MatekF405) available to the network.
$ sudo usbip bind --busid 3-6.1
usbip: info: bind device on busid 3-6.1: complete
Configure the remote (cloud) machine
We now need to tell the cloud device about the exported USB serial port. In the following examples, the physical machine with the attached FC is called eeyore and has IPv4 address 172.31.0.244.
So to see the exported USB device:
$ sudo usbip list -r 172.31.0.244
Exportable USB devices
======================
- 172.31.0.244
3-6.1: STMicroelectronics : STM32F407 (0483:5740)
: /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1
: Communications / Abstract (modem) / None (02/02/00)
I know that this is Linux device /dev/ttyACM0, at the moment on the remote (assumed cloud) machine, does this exist?
$ ls -l /dev/ttyACM0
ls: cannot access '/dev/ttyACM0': No such file or directory
No it does not, so attach the remote device:
$ sudo usbip attach --remote 172.31.0.244 --busid 3-6.1
Now:
$ ls -l /dev/ttyACM0
crw-rw---- 1 root uucp 166, 0 Mar 29 18:51 /dev/ttyACM0
This is Arch Linux, so the ownership (root,uucp) and permissions are correct.
$ sudo usbip port
Imported USB devices
====================
Port 00: <Port in Use> at Full Speed(12Mbps)
STMicroelectronics : STM32F407 (0483:5740)
5-1 -> usbip://172.31.0.244:3240/3-6.1
-> remote bus/dev 003/007
And it's the correct USB device.
So now we can upload a mission:
$ mwp-uploader -d /dev/ttyACM0 -m ~/Projects/quads/missions/nm_multi2.mission -s
2018-03-28T21:48:20+0100 Opening /dev/ttyACM0
2018-03-28T21:48:22+0100 WP: 48
2018-03-28T21:48:22+0100 Mission validated
2018-03-28T21:48:22+0100 WP_GETINFO: 48/60/1
uploaded 48/60 WP, valid
2018-03-28T21:48:22+0100 Saving mission
2018-03-28T21:48:23+0100 Confirmed mission save
We could otherwise have used the mwp UI. As this appears to the kernel to be a real USB device, USB detection will work too:
$ mwp-uploader -m ~/Projects/quads/missions/nm_multi2.mission -s
Very neat.
Ubuntu 16.04 woes
There had to be a but, and as usual it's broken Ubuntu and friends. In Ubuntu 16.04, the obvious usbip package is broken
$ sudo usbip -D -l 172.31.0.244
usbip dbg: usbip_network.c: 221 (tcp_connect ) trying 172.31.0.244 port 3240
usbip dbg: usbip_network.c: 241 (tcp_connect ) connected to 172.31.0.244:3240
- 172.31.0.244
usbip err: usbip_network.c: 119 (usbip_recv_op_common) recv op_common, -1
usbip err: vhci_attach.c: 202 (query_exported_devices) recv op_common
usbip err: vhci_attach.c: 417 (show_exported_devices) query
It is necessary to install a kernel specific set of linux-tools-generic. As I'm using the 'hwe' kernel:
$ sudo apt purge usbip
$ sudo apt-get install linux-tools-generic-hwe-16.04
$ sudo modprobe usbip-core
$ sudo modprobe vhci-hcd
I've also removed the Matek FC on the "physical device" side and replaced it with a Dodo, so the device will be /dev/ttyUSB0 vice /dev/ttyACM0.
$ sudo usbip list -r 172.31.0.244
Exportable USB devices
======================
- 172.31.0.244
3-6.1: Cygnal Integrated Products, Inc. : CP210x UART Bridge / myAVR mySmartUSB light (10c4:ea60)
: /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1
: (Defined at Interface level) (00/00/00)
: 0 - Vendor Specific Class / unknown subclass / unknown protocol (ff/00/00)
$ sudo usbip attach --remote 172.31.0.244 --busid 3-6.1
$ mwp-uploader -d /dev/ttyUSB0 -m ./area.mission -s
2018-03-29T17:17:35+0100 Opening /dev/ttyUSB0
2018-03-29T17:17:35+0100 WP: 17
2018-03-29T17:17:35+0100 Mission validated
2018-03-29T17:17:35+0100 WP_GETINFO: 17/60/1
uploaded 17/60 WP, valid
2018-03-29T17:17:35+0100 Saving mission
2018-03-29T17:17:35+0100 Confirmed mission save
So now it works on Ubuntu 16.04 as well (which is useful, as this is popular AWS host OS).
socat and other "serial to net" solutions
This is fine, but it's somewhat messy as we have to do a bit of sudo messing around, so here's a second solution using socat (available for Linux and MacOS, possibly MS Windows). There are other "serial to network" solutions across all platforms (e.g. ser2net on Linux) that will also work.
On the machine with the USB device physically attached:
socat /dev/ttyUSB0,nonblock,raw,echo=0,b115200 TCP-LISTEN:4321
where /dev/ttyUSB0 is the device node, for a VCP device on Linux it will be someting like /dev/ttyACM0 and on MacOS, perhaps something like /dev/tty.usb1241.
And on the (cloud) box with mwptools:
$ mwp-uploader -d tcp://eeyore:4321 -m ./area.mission -s
2018-03-29T17:30:13+0100 Opening tcp://eeyore:4321
2018-03-29T17:30:13+0100 WP: 17
2018-03-29T17:30:13+0100 Mission validated
2018-03-29T17:30:13+0100 WP_GETINFO: 17/60/1
uploaded 17/60 WP, valid
2018-03-29T17:30:13+0100 Saving mission
2018-03-29T17:30:13+0100 Confirmed mission save
where eeyore is the host name of the box with the usb device, and I'm using port 4321, which must be accessible from the cloud machine. Success (again).
Conclusion
Two working methods for cross-platform remote USB access. Neat, very neat.