MicroDrop v2.0 software architecture - sci-bots/microdrop GitHub Wiki
Motivation
As described here, all MicroDrop v1.0 code:
- is written in Python (version 2.7)
- executes within a single process executing GTK 2.0 main run loop
Within MicroDrop v1.0, all code is organized in plugins based on the PyUtilib Component Architecture plugin framework from the PyUtilib project, which itself was based on the framework used in the Trac project.
Since all code is contained within plugins (either core plugins or user plugins) the plugin framework is the heart of the MicroDrop application.
Since the PyUtilib Component Architecture plugin framework was first introduced into MicroDrop in 2011, it has encouraged us to develop code that is modular and extensible.
However, a lot has changed since then:
- MicroDrop is now used by labs around the world
- MicroDrop interfaces with various hardware (through plugins), including:
- Digital microfluidics control systems (DropBot and OpenDrop)
- Syringe pumps
- Auto-sampler (using Analyst control plugin to interface with Analyst software from SCIEX)
- Users are interested in developing custom functionality.
- Embedded systems such as the Raspberry Pi and C.H.I.P. present incredible potential for new system paradigms.
- Javascript has matured into a popular server/application language (thanks in large part to Node.js)
With MicroDrop v2.0, our goal is to pivot development to take these points (and more) into consideration.
At a high-level, the goals of MicroDrop v2.0 are to:
- Migrate to plugin framework that is:
- Language agnostic
- Network compatible (e.g., supports communication over TCP)
- Remove all UI code from core MicroDrop application by migrating all UI (i.e., GTK 2.0) code to non-core plugins.
Since the most significant changes in MicroDrop v2.0 involve changes to the plugin framework, below we describe the properties of the plugin framework of MicroDrop v1.0 followed by the proposed MicroDrop v2.0 plugin framework.
MicroDrop v1.0 (PyUtilib) plugins
As described above, MicroDrop v1.0's plugin framework was based on the Plugin Component Architecture from the PyUtilib project.
Advantages:
- Leverage existing framework for registering, enabling, and disabling plugins
Disadvantages:
- Python-specific; requires plugins to be written in Python
- Plugins are automatically registered through Python import "magic"
- No straight-forward mechanism to update a plugin while MicroDrop is running
- All plugins run within single process; any single plugin can cause MicroDrop to crash
MicroDrop v2.0 (ZeroMQ-based) plugins
To address the short-comings of the existing MicroDrop v1.0 plugins framework, we are developing a new ZeroMQ-based plugin framework for MicroDrop v2.0. This framework has to the following goals:
- Support plugins written in any language
- Process isolation through a multi-process architecture
(similar to the approach used in the Chrome browser):
- Run each plugin in a separate process
- Run main MicroDrop event loop in independent process
- Language agnostic and network compatible plugin communication
Advantages:
- The execution environment of each process can be managed completely independently as long as all process environments support connection to a common network
- No single plugin process can crash the main MicroDrop process
- Plugins can be written in any language
Disadvantages:
- Existing MicroDrop v1.0 plugins must be ported to new MicroDrop v2.0 plugin format
- If communicating over a public network, security must be addressed
Current status
See the MicroDrop development roadmap.