Architecture - NetLogo/NetLogo GitHub Wiki

NetLogo has two levels of functionality: headless and application.

Package structure

NetLogo's package structure is shown in dist/depend.graffle, an OmniGraffle document. (OmniGraffle, a diagramming application for Mac OS X, has a free viewing mode. Editing costs money.) We call this diagram "the graffle" for short. It has not been updated since March 2013 and does not reflect changes since then. There is a pdf version of this file in dist/depend.pdf

The package structure also exists in textual form in project/Depend.scala, so that sbt depend can detect and report illegal dependencies using Classycle. This is up-to-date.

The packages are arranged in ascending layers of increasing functionality. Lower packages may not have compile-time dependencies on higher packages.

Dependencies are transitive, so if A may depend on B and B may depend on C, then it's implicitly allowed for A to depend on C, too.

The Engine Architecture and Package Guide says what each package does.

Dependency injection

In the graffle, you'll see that many packages are dependency "roots", that is, they have no packages depending on them at compile-time. That's because some dependencies that exist at runtime happen via dependency injection, where the underlying technique used is Java reflection.

We use PicoContainer for dependency injection in high level packages like app and headless.

For jar size reasons we don't want the applet to depend on PicoContainer though, so in lower packages we use a simple class called Femto that we wrote ourselves for doing elementary DI.

Main classes

NetLogo can run as an application (org.nlogo.app.App), or as an embedded component (org.nlogo.lite.InterfaceComponent) or headlessly (org.nlogo.headless.Main, org.nlogo.headless.Workspace, and the undocumented org.nlogo.headless.Shell).

Events

NetLogo has its own event system, which is totally separate from the AWT event system. It serves a different purpose than AWT events: NetLogo events are used so that different GUI components can communicate with each other, and with the execution engine, without needing to know about each other's method names, implementation details, or even existence.

Most events are in org.nlogo.window.Events; a few are in org.nlogo.app.Events. All are subclasses of org.nlogo.window.Event. They each have a Handler trait which a class must implement in order to be allowed to handle that event. Both events source files are autogenerated from project/events.txt. The structure of events is simple and uniform.

Events should always be raised only on the AWT event thread. To assist with this, a raiseLater() method exists which is based on invokeLater() in java.awt.EventQueue (and javax.swing.SwingUtilities).

Once raised, a NetLogo event is handled immediately, synchronously, and in the same thread (the event thread) from which it was raised -- they do not queue up. Thus, when code raises a NetLogo event, control flow does not return to that code until all event handlers for that event have run. If you want an event to be handled later instead, use raiseLater().

If there are multiple handlers for a single event, the handlers run in no particular order. (Come to think of it, I don't even know if it's the same from run to run... Probably it is...)

Event handlers cannot find out who raised an event. This helps enforce that events have clean semantics.

As a matter of style, events should only contain immutable data.

There are two general kinds of events. Some events are requests that something should happen. Other events are notifications that something has just happened. We attempt to distinguish these types in their names, according to whether the name is in the past tense or not. So for example CompileAllEvent is a request to recompile the whole model, but CompiledEvent is a notification that the compilation of something has just completed.

Events normally find their handlers through the AWT containment hierarchy. In order to raise an event, ordinarily you must be a component in the same Window, or in a Window with an ancestor in the same Window, as the object you expect to handle it. It is possible to evade this system though and cause events to be passed in a custom way, even among objects that are not AWT components, using the EventLinkComponent and EventLinkContainer interfaces. Using EventLinkComponent lets an object raise events by specifying a "link parent" through which the events are transmitted. Conversely, EventLinkContainer lets events "pass through" an object that implements to it to its "link children".

Q: Why is the AWT containment hierarchy used?

A: (Seth answers) This design dates back to Tufts days (predating me). At one time, it was common for the propagation of an event to be restricted to just a subtree of the component hierarchy, and there were additional mechanisms, since removed, to support this. So for example originally, events were used internally for communication between the various subparts of a PlotWidget. Since there may be multiple PlotWidgets, the Tufts developers didn't want crosstalk between their events, so the various plotting events would propagate up to the enclosing PlotWidget, but then stop there. Nowadays, I think all or nearly all of the events are global. But there might still be a few places where it's relied on that there are multiple "roots" and events don't cross over. I couldn't say for sure. An example of a place where I'm afraid that might be true is multiple applets or InterfaceComponents running in the same JVM, another is the System Dynamics Modeler window in the app, another is the HubNet Client Editor in the app. That's not necessarily an exhaustive list, but those are the two that come to mind. But aside from the global vs. local issue, there's another reason to use the Swing component tree. How else would the event system know what receivers exist? That's the big thing that would break if you just put in a global event bus. You'd have to require event receivers to register. Currently you only need to explicitly register if you aren't already in the component hierarchy. I'm not saying that would be a bad change, but it would require work to change over.

Threads

All NetLogo code runs in the job thread (org.nlogo.job.JobThread). Never, never, never run NetLogo code in any other thread. That includes even just evaluating reporters, as well as running commands. Also don't even access agent data structures (World, Patch, Turtle, AgentSet, etc.) from any other thread, since those classes are not threadsafe. Use JobManager to schedule NetLogo code for execution that will inspect or alter the data structures as needed. Evaluator provides some convenient ways to schedule jobs without having to interact directly with JobManager.

Code on the AWT event thread must never stop and wait for something to happen on the job thread. The job thread often waits on the event thread, for example when it pauses while the event thread updates the display, so if the event thread were to ever wait for the job thread, then deadlock could easily occur. (Exception: when you use "Halt", or open a new model, the event thread must wait for the job thread to shut down all current jobs. So deadlock can't occur, this is accomplished by calling interrupt() on the job thread.)

org.nlogo.window.ThreadUtils supplies two convenience methods called waitFor and waitForResult for use in the perform() methods of GUI commands that let you schedule work on the event thread and make the job thread wait until that work is finished.

Most other code runs in the event thread. (There are various places in NetLogo that locally create other, temporary threads but these usages are rare and localized.)

Collections

Don't use java.util.Hashtable, java.util.Vector, or java.util.Enumeration; they are obsolete. Use the Scala collections API instead, or (in legacy code) the Java Collections API. (bin/findbadcollections.scala checks this)

System Dynamics Modeler

The System Dynamics Modeler lives in two packages:

org.nlogo.sdm holds the classes that represent the system dynamics diagram (Binding, Variable, etc.) and the code that translates the diagram into NetLogo code.

You'll notice that the class names of diagram elements don't match the names presented to the user. We changed the public names after building the tool. We can't change (at least, it's not trivial to change) the internal names without breaking compatibility with old models (the serialization uses reflection on the class names), so the two are different. To translate:

  • internal name => public name
  • stock => stock
  • converter => variable
  • flow => rate
  • link => binding

org.nlogo.sdm.gui holds the GUI for the SDM. The diagram editor is based on JHotDraw, an amazing framework for structured drawing editors. StockFigure holds the JHotDraw component that represents the Stock figure element, etc.

Note that we don't allow classes in org.nlogo.sdm to depend on JHotDraw; only org.nlogo.sdm.gui is allowed to depend on JHotDraw. We make this restriction for several reasons. One is just to keep things modular so they are easier to understand and maintain as NetLogo evolves. Another, more immediately practical reason, is that we want aggregate models to be able to run headless or as saved applets without requiring the JHotDraw JAR.

One difficulty with this arises because we want the classes that represent the parts of the diagram to implement the org.jhotdraw.util.Storable interface, so that JHotDraw will take care of serializing and deserializing whole diagrams for us (The serialized form is stored in the .nlogo file.) The actual classes themselves, such as Converter and Stock, are in org.nlogo.sdm though, so they can't implement the interface themselves, since the interface is part of JHotDraw. So in the org.nlogo.sdm.gui package, we define classes called WrappedConverter, WrappedStock, etc., that are containers for the original classes and implement the Storable interface. These wrapper classes are then used for serialization and deserialization.

One final note: JHotDraw includes serialization and deserialization code, but when running headless or in a saved applet, we can't use the deserialization part because we can't depend on JHotDraw in those contexts. So we had to write our own alternate deserialization code that read the JHotDraw format but ignores the visual information about the diagram, since it's only important when displaying or editing the diagram, which you can't do headless or in the applet. The custom deserialization code is in org.nlogo.sdm.HeadlessAggregateManager.

Renderer

The renderer for the 2D view is the org.nlogo.render package.

The main class is render.AbstractRenderer. It has two subclasses, render.Renderer and hubnet.client.ClientRenderer. The former is used in NetLogo itself, the latter in HubNet clients.

Only one render.Renderer object exists even if there are multiple views. (Agent monitors contain additional views.)

The render package is not allowed to depend on the agent package. Code in render deals only with "renderable" objects (api.Renderable).

AbstractRenderer has several helpers: TopologyRenderer and individual "drawer" classes for the different things that can appear in a view: patches, turtles, links, trails (the drawing layer), and spotlight (for watch/follow).

Rendering topologies

TopologyRenderer primarily deals with wrapping the graphical elements in the world as necessary and converting NetLogo coordinates to screen coordinates. There are subclasses for each topology. Note: some of the view size information is cached in the topology renderer be careful to update it every time you start rendering a new view or when these values have changed. Nothing above the TopologyRenderer (AbstractRenderer, the drawers) should ever deal with screen coordinates and nothing below (drawables, shape package, etc) should ever deal with NetLogo coordinates. The same goes for sizes: sizes above should be in patches, sizes below should be in pixels. TopologyRenderer wraps two types of objects: Drawables are objects that have a draw method and are rendered like an object as opposed to a background (This includes turtles, links, spotlight, patch highlights); The second type are backgrounds, essentially, the patches and the drawing.

Coordinate conversions

A big part of rendering is translating from NetLogo coordinates to screen coordinates.

The two coordinate systems differ in various respects. In NetLogo coordinates up is positive and down is negative; in screen coordinates it's the reverse. Sizes and distances in NetLogo coordinates are measured in patches, in screen coordinates they are measured in pixels.

Coordinate conversion is done by the individual topology renderers (box, torus, etc).

High-level rendering code uses only NetLogo coordinates. Low-level code uses only screen coordinates. Keeping the coordinate system distincts in both the code and in your mind will prevent you from going mad. As an aid to this, variables that are NetLogo coordinates should be named xcor and ycor. Variables that are screen coordinates should be named x and y.

Application Initialization

NetLogo's main lives in App.scala. This file is pretty large and complex, so I'll offer a 10,000 ft view of what has to happen to start up the app and then drill into the minutiae of how this happens.

From a high level, initializing the NetLogo application is complex because there are a lot of interdependencies. These could be considered design flaws, but it isn't obvious how to do things better in many cases. The most important objects and their roles are listed here:

  • App: Global state container. Holds on to virtually everything in the application. Note that the App.app method is a static reference to the global state. This has been used by extensions and other parts of the application historically, but new uses should be avoided where possible as global state creates real problems down the line.
  • AppFrame: The top-level frame that contains NetLogo. This frame doesn't actually have that much custom code, but does serve as the root for the event system (which is a pretty big deal in the world of the NetLogo UI).
  • Tabs: This is the container for the three NetLogo tabs (interface, info, code). This class is responsible for creating each of the three tabs and is used by various classes that want access to them.
  • GUIWorkspace: This class is defined in the window package, but is held as a field of App. A large function of App is to connect other classes with the workspace.
  • World / World3D: This class is initialized by App instead of by the workspace and contains the vast majority of the state of the model.
  • FileManager: This class handles the substance of operations listed in the file menu like saving and loading.
  • ModelSaver: Manages the runtime state of the model. Offers a consistent view of the model suitable for saving by aggregating the disparate components (code, interface, shapes) into a NetLogo Model object.
  • turtleShapesManager / linkShapesManager: Hold the state of the model shapes. Provide actions for changing those shapes.
  • MenuBar: The main NetLogo menu bar. Initialized as a collection of empty menus and has actions added dynamically (see below).

MenuBar

The NetLogo MenuBar holds the NetLogo menu items. These are added as javax.swing.Actions using the offerAction method. The items are placed and positioned in menus based on the values they have set for various fields in Action's internal key-value store (accessed using putValue / getValue). These keys are defined in org.nlogo.swing.UserAction:

  • ActionCategoryKey: Defines which menu an action's item will appear in. FileCategory indicates that the item belongs in the "File" menu, EditCategory indicates that it belongs in the "Edit" menu, etc.
  • ActionGroupKey: Defines which actions this will be grouped with. The menus will place menu separators between different groups. Additionally, each menu can define an order in which groups appear.
  • ActionSubcategoryKey: Defines the submenu in which an item will appear.
  • ActionRankKey: Controls the action's placement within its group. This is a java.lang.Double. The lower the rank, the earlier in the group an action will be placed.

Note that only ActionCategoryKey is required to be set for an Action to appear in a menu.

Testing

We use ScalaTest for all tests.

Some tests use ScalaTest's ScalaCheck wrappers. ScalaCheck is like QuickCheck in Haskell; it automatically generates large numbers of random and/or borderline test cases.

The tests for a class named Foo should be in a class (in the same package) called FooTests. Tests with no corresponding class in the same package are named e.g. TestFoo.

Some of our test classes narrowly test the functionality of a particular class. Such tests are what are most traditionally considered "unit tests". But we also have tests that test broader swaths of NetLogo functionality. These are in the org.nlogo.headless package. For example, TestImportExport tests import-world and export-world. TestCommands and TestReporters test the NetLogo language. The tests themselves are in test/commands/*.txt and test/reporters/*.txt.

When to add tests

Whenever you add a new (non-GUI) primitive to the language, you should add tests for it to the language tests.

Whenever you add any non-GUI code at all to NetLogo, you should strongly consider adding tests for that code. In fact, you may want to intertwine the development of the code with the development of tests for that code. This needn't be just an unpleasant duty -- it can actually make development both faster and more pleasant if you write tests as you go. This is known as "Test-Driven Development" (TDD for short).

If you are writing GUI code, it is best (if possible) to separate the GUI parts of the code from the underlying logic (We can't currently test GUI, but even if we had, it is still, of course, a very good practice to separate GUI from underlying logic). Then you can still write tests for the underlying logic. For example, in BehaviorSpace the underlying logic is in the org.nlogo.lab package, and has tests. The GUI that goes on top of that logic is separate, and lives in the org.nlogo.lab.gui package. This approach to structuring code has many other benefits besides just facilitating testing.

Running tests

within SBT,

  • fast:test (just the really fast tests)
  • medium:test (some slower tests)
  • slow:test (only the slowest tests)
  • test (all the tests in all three categories)

Running just certain tests

tc, tr, te, and tm are shortcuts for TestCommands, TestReporters, TestExtensions, and TestModels. They all take an optional argument so you can do e.g. tc Lists to just run test/commands/Lists.txt, or tc Lists::Test5 to run just the one test with that name. See Language tests for more details.

For other suites besides those special ones you can use sbt's standard test-only command to run just one test suite, or test suites matching a pattern (use *).

If you want to run only certain tests within a suite, and the suite isn't a language test suite, you can either tag the tests you want to run, or filter them by name.

Tagging a test involves changing its source code, so e.g. test("my test") { ... } becomes test("my test", org.scalatest.Tag("foo")) { ... }. Then run it through sbt with e.g. test-only *MySuite -- -n foo.

If you want to filter tests by name instead,

  • In the headless branch of NetLogo and in the Tortoise repo, ScalaTest now supports this to run only tests with "foo" in the name:
  • test-only *MySuite -- -z foo
  • In the 5.x branch of NetLogo, we're stuck on an older ScalaTest version so life is harder. You either fall back on the tagging method, or use this ugly workaround: you can bypass the sbt/ScalaTest integration and just run the ScalaTest runner directly, like this: test:run-main org.scalatest.tools.Runner -o -s my.package.MySuite -z foo

Compiler and engine

see Compiler architecture and Engine Architecture and Package Guide

Primitives

You might want to look at How to Add a Primitive