Background and history - micwik/ArnLib GitHub Wiki

My name is Michael Wiklund. I've been doing electronics and programming since 1980. Both on a hobby basis and as a profession. I have a M.Sc.C.E degree from Chalmers University of Technology, Sweden.

Almost 15 years ago I started working on a home automation project for my new house. In focus were controlling of room thermostats, sun shaders, irrigation and some more. Sensors included weather station, sun radiation and a lot of temperature values from the thermostats. All this were spread out to different units and a diverse of buses and interfaces.

My first plan was to gather all values, preferably in a tree structure, like a file system. Traditional databases are focused on storing, not fast changing values. I couldn't find anything suitable and good enough to meet my requirements. It also had to be available for Linux as the system should be robust and running 24/365.

I decided to skip an existing standard system and make my own tree-structured storage system, as this was quite a simple task. This Registry was available via TCP/IP and some simple commands. It was relatively easy to add new kinds of units. Logic controllers had just one API to get and set all values, independently of the target unit type. However, this collection of programs worked by polled loops and all communication details cluttered the overview of the programs main logic.

The next step came with Qt. This is really an awesome library. Its object system and signal slots made a good base for the new Registry. All details were now hidden in a library. My program collection is using a class called ArnItem to represent each data object. The objects are still in a tree structure, but all communication is automatic. When the ArnItem gets a value assigned, the value will be replicated to all connected programs that have a corresponding ArnItem, i.e. those having the same path in the tree.
The Qt signal slot fits nicely, as the ArnItem will signal any change of its value. Further, any connected slots can act directly on the changed object instead of using a polling loop that iterates all objects.

As my goal was to have programs with a clear main logic without any clutter from inter program communication and likewise, all such issues had to be handled by ArnLib automatically. When TCP/IP connections are broken there should be attempts to restore connection and resync relevant information. This is handled by ArnLib client routines in the background. The result was quite satisfactory. The illusion for the programs is constant connection with blocked communication during a broken connection.

One of the most important pieces for the Arn concept was the making of a browser. Qt offered a tree view widget with a custom modell. This made up the main part of the GUI. The browser connects as any other programs connected to the Arn Registry. It visualized all the changing objects in the Arn Registry. For debugging it was also nice to be able to alter the value of any object.

Many more improvements were made for ArnLib during the years. To mention some major:

  • Support for JavaScript Jobs in ArnLib made it possible to move most of the house controlling logics to scripts. Actually, I store these javaScripts as objects in the Arn Registry. Easy to change and the scripts get automatically restarted.

  • Streams were added to promote logging and message sendings between programs. Streams are using normal Arn objects set in pipe mode.

  • RPC (remote procedure call) made it possible to implement different service API's. The programs could now call each other using "remote signal slot". I.e. just connect Qt signals and slots across programs. I use it both for end-to-end calls and broadcast calling slots in many programs at the same time.

  • A web based Arn browser acting like a proxy between the Arn Registry and a normal web browser. Thereby I could use any common computer to log in and monitor my Arn Registry. Even change a JavaScript at a distance felt so awesome.