Architecture - NetLogo/Tortoise GitHub Wiki

The Tortoise project contains a compiler and a runtime engine for compiling and running NetLogo models in JavaScript environments. It's tied closely to the NetLogo desktop project for its parser and the Galapagos project for its model view, user interface, and web site.

Compiler

The NetLogo project contains a parser for parsing NetLogo code into a syntax tree. This parser code is almost entirely written in Scala. It publishes the project code to JavaScript using Scala.js to create the parserJS artifact. This setup allows NetLogo desktop and NetLogo Web to share common parsing code. The FrontEnd is the entry point to the NetLogo parser.

The Tortoise project takes a dependency on the parserJS artifact, which is used by the Compiler to transform NetLogo code into a syntax tree, and then into JavaScript. The actual entry point for use by the Galapagos project is the BrowserCompiler which exposes JavaScript methods.

Most of the Tortoise compiler code is written in Scala, with some small bits in JavaScript when necessary. But it is important to remember that the compiler is translating NetLogo code into JavaScript and will itself be compiled to JavaScript with Scala.js.

Engine

The Tortoise engine duplicates much of the NetLogo runtime environment. The entry point for the engine is the MiniWorkspace class. The compiler takes care of making sure everything the MiniWorkspace needs is setup and then passed in at runtime when the JavaScript is executed.

  • This is the code that actually runs when a statement or command is executed in order to update the model state.
  • This code is almost entirely CoffeeScript, with some regular JavaScript thrown in (as that is the final output language).
  • The runtime engine is stand-alone. It can run models "headlessly" anywhere you run JavaScript; we use it that way to run models for testing, for instance. To run the model, we need to execute widgets (especially forever buttons), and then check the Updater to see what happened in the model since the last command was run.

User Interface

Galapagos uses the Tortoise BrowserCompiler in the tortoise.coffee code. Then it uses the Tortoise engine via the created MiniWorkspace and widgets to run the code, taking the engine updates and transforming those into the view.

  • The SessionLite class is the Galapagos component that handles coordinating everything for the running model.
  • The eventLoop() method is where the widgets (forever buttons) are run and updates from the engine are collected and applied in the WidgetController and its ViewController.
  • The WidgetController is also stores the widget state and UI state as the model is live in Ractive.js.
  • Ractive.js is the UI framework we use in Galapagos. It's very simplistic, but it has just enough to do what we need - events, observables, computed values, and templating.

HTML Pages and Play Framework

Galapagos also contains a Play Framework project that is the basis for the web site https://netlogoweb.org, including the HTML views, CSS files, JavaScript files, and the NetLogo models library. The Play Framework site is scraped and published as static files; the final https://netlogoweb.org site all runs client-side with the web server just providing those files.

UI Config Shims

Because the engine is headless, we tried to avoid directly relying on any UI-specific features or APIs. For example, we can't assume alert() is available. For situations where we need to call into UI or front-end specific code, the config shims are available. The shims can then be filled in by Galapagos at runtime.

Compilation Results

The output of the Tortoise compiler is JavaScript code. We aim to generate JavaScript code that will run in modern browsers. For testing purposes during development, we're using GraalVM. Here is an example of the generated JavaScript code for the Ants library model. We generate the JavaScript code to check for correctness and no unintended changes as we do work on the compiler.

Scala.js Notes

The Tortoise compiler is cross-compiled into Scala.js for in-browser use. We build both a JVM (compilerJVM) and a Scala.js (compilerJS) artifact because we want to be able to use JVM in the Galapagos server and for testing, as well as being able to have in-browser compilation in static Galapagos. A few quick notes about cross-building in Scala.js:

Since Scala.js lacks reflection any code that needs to vary between JVM and Scala.js must have two class implementations - one for the JVM and one for Scala.js. Typically, these are kept in separate directories, with mirrored structure, but when there is only a single file (as in the current Tortoise implementation) sbt can be convinced to compile replacing only that file. Typically, the reason that you'll want a separate Scala.js implementation will be either (a) the library you want to use hasn't yet been ported to Scala.js, (b) the Scala code uses an aspect of the JVM (like reflection or local file input/output) that hasn't yet been ported to Scala.js. Creating a separate implementation for other reasons should be looked at very carefully.

Since Scala.js doesn't support reflection, any time something reflection-like is needed you may be required to write a macro. These exist in both NetLogo-Headless and Tortoise. In both projects, they typically generate a lot of very trivial code that would be tedious and error-prone to maintain by hand. They probably could use code generation instead of macros, but Scala macros are relatively straightforward to use. They can search for implicits and look up classes in the code under compilation which allows them to act as reflection tools without the need for JVM runtime reflection.

JavaScript Artifacts

Scala.js artifacts are compiled into .sjsir classfiles, which can then be jar'ed and used by other scala.js projects. The Tortoise project includes a separate build netLogoWeb project that packages the files tortoise-compiler.js and tortoise-engine.js into a jar for independent deployment. The netLogoWeb project is also the home of the various tests we run against the compiler and engine both, see Tortoise Tests for more info on those.

On the Galapagos side, tortoise-engine.js is served as a renamed netlogo-engine.js while tortoise-compiler.js is served as the same (defined in conf/routes). Both are served at the root of https://netlogoweb.org as they are nonstandard JavaScript dependencies that need to be extracted from their distrubtion .jar file before use.

Engine Notes

State

The main model state is maintained by the Tortoise engine in an instance of the World class, stored as the world global (also available via the workspace global as workspace.world).

The state of the widgets is all kept by the Galapagos code (or whatever is using the engine), and the engine is simply asked to execute certain code without necessarily knowing that it's from a forever button and polled for updates.

The state of the widgets is generally not important to the engine code. Monitors and labels are UI-only, plots and the view can be exported by the engine but that code comes from the shims, and buttons simply cause code to run. That leaves the global variable widgets, and when those change the appropriate values are set in the observer's variable list by Galapagos, so the engine has the appropriate updated values.

Galapagos does maintain its own representation of the model state, that is the AgentModel class. It applies the updates it gets from polling to an instance of that class then uses it to do things like draw the turtles in the world. This actually happens in the view-controller.coffee in the update() method. As of this writing in May 2023 I (Jeremy B) don't know exactly why this separate structure is used for the view updates, instead of directly accessing the world to query the necessary information; it could be to ease the process of drawing the view or it was originally done to ease cross-testing with NetLogo desktop (via the generated updates) and wound up getting used for the view, too.

Runtime Error and Type Checking

Runtime error and type checking is incomplete.

The runtime error and type checking middle layer entry point is the Checker class. The checks for primitives are divided by type: agentsets, lists, math ops, and more.

The compiler will automatically generate argument type checks for primitives as needed (see makeCheckedArgOps()), and skip type checks when the types are known.

Example: n-of (count turtles - 10) turtles will not generate any runtime type checks as the compiler can see count turtles is a number so (count turtles - 10) is a number and turtles is the agentset arg. There will be a runtime check for n-of that the integer argument is valid for the number of turtles in the agentset as that cannot be determined from the type information alone.

We also use the checker middle layer to do simple tests for which implementation to use when prims support different arg types. For example, remove can take a list or string arg, so we switch to the proper implementation based on the found arg type at runtime.

The RuntimeException class is the "expected engine runtime error" class, and it's been dropped into all appropriate places in the engine.

Many exceptions.runtime() errors still exist sprinkled in the happy path engine code. Ideally those would be moved into the middle checking layer as well, but that's work in progress.

Internationalizations (I18N)

Internationalization and translation of NetLogo Web is very incomplete.

The runtime error checks use a simple I18N bundler to get their error messages. At the moment only bundles for English and Simplified Chinese are available. Galapagos uses the switch() method to swap locales in the engine as set by the browser or user preferences.

The bundles use a simple key/value system to store messages. The keys use underscores where arguments will be provided at runtime. The values are functions that take in those arguments and produce the appropriate error in the designated language.

Module System and Bundling

We're using Browserify. Browserify uses the same module system as Node.js, which allows us to leverage Node/NPM libraries. It also bundles the engine code and the libraries together into one JavaScript file for easy distribution.

Because the require() used for Tortoise and Browserify is not the "real" require() when things are running in a web browser, we need to take an extra step when translating the engine code for release. We use a custom Grunt task to rename all instances of require() in the engine code to tortoise_require() which will get the engine components finding each other at runtime inside the browser.

We want to switch the Tortoise project over to ES6 modules soon-ish, which should simplify a lot of this setup, but that work isn't underway yet.