I can add coverage of particular topics on request. — Seth

In this page you can find information on Packages, a Walkthrough, and a Package Guide.

See also Compiler architecture, since the design of the compiler and the design of the engine are of course intertwined.

Packages

The execution engine is in the org.nlogo.agent, org.nlogo.nvm, and org.nlogo.job packages. job depends on nvm; nvm depends on agent. And they all use the api package, of course. They have no other dependencies.

agent package

This is where classes like World, Observer, Turtle, Patch, and Link are. Agents, agent variables, agent movement and measurement, agentsets, birth, death, all of that is implemented here.

This package could be used as a library to support building of NetLogo-style agent-based models in other languages besides NetLogo. For example, you could write a model in Java or Scala that uses only the agent package.

The nvm and job packages, on the other hand, are very much about NetLogo the language.

nvm package

Most of the language execution engine is here. Important classes include:

• Instruction, an abstract class with two concrete subclasses Command and Reporter. These classes are in turn extended by the individual primitives in the prim package. The key methods are Command.perform and Reporter.report.
• Procedure represents a user-defined procedure. The body of the procedure is an Array[Command].
• Activation is an activation record (aka stack frame); it represents an active procedure call at runtime. The NetLogo call stack is a chain of Activations. The arguments supplied to the procedure at runtime are an Array[AnyRef] stored in Activation.args.
• Job represents the execution of code by an agentset. Every use of ask creates a job. It has two subclasses:
• ConcurrentJob: a job whose execution can be paused and resumed, so other jobs can have a chance to run. Used for top-level jobs (e.g. for button and command center jobs) and by ask-concurrent.
• ExclusiveJob: a job that hogs the engine until it finishes. Simpler and faster than a concurrent job.
• Context. When an agentset runs a job, each agent runs the job in its own Context. The main fields in Context are Context.agent (what agent is running code), Context.activation (what procedure is it in), and Context.ip (= instruction pointer; what command in the Procedure's command array is the agent running).
• Binding. Holds the values variables bound with let. (But note that not all uses of let will be represented using Binding; in the compiler, LocalsVisitor converts some let variables to invisible extra inputs to the procedure, for efficiency.)
• Task. Represents a command task or reporter task.

job package

A fairly small package containing only two classes, JobManager and JobThread. JobManager implements nvm.JobManagerInterface. The other parts of NetLogo only know about that interface; they don't know what implements it.

JobManager has methods for adding and removing jobs and for halting the engine. JobThread actually runs the jobs (by making calls into the nvm package). There is exactly one job thread per workspace. (A workspace represents an open model.)

Jobs can be added from any thread, but they always run on the job thread, which is always a separate thread that only runs NetLogo code.

Walkthrough

You fire up NetLogo, type ask patches [ set pcolor red ] into the Command Center, and press return. Voila, red patches. What happened to make that happen?

Touching only lightly on GUI aspects, mainly focusing on the engine:

Procedure wrapper

The NetLogo compiler can't compile standalone snippets of code, only entire procedure definitions. app.CommandLine (the part of app.CommandCenter that accepts input) wraps the code you typed in a procedure definition, as follows:

to __commandline []
  __observercode
  ask patches [ set pcolor red ]
  __done
end

Some notes on the wrapping:

• It doesn't matter what the procedure is called, because we're never going to invoke it by name.
• __observercode is an internal observer-only primitive that does nothing except, by its observer-only-ness, force the rest of the procedure body to be interpreted as observer code
• Since this a top-level procedure with no enclosing procedure for it to return to, we need __done at the end so execution stops before running off the end.

Compilation

CommandLine then raises a window.CompileMoreSourceEvent to hand the procedure definition off to the compiler. A middleman class, window.CompilerManager, will receive the event and actually invoke the compiler for us.

The compiler has two entry points, one (compileProgram) for compiling the Code tab, and the other (compileMoreCode) for compiling additional, auxiliary procedure definitions. compileMoreCode returns an nvm.CompilerResults object, which CompilerManager then relays back to CommandLine via a window.CompiledEvent.

CompilerResults is a wrapper containing an nvm.Procedure object containing compiled code. This Procedure object is what we'll need to hand the engine in order to run the code.

Job creation

CommandLine doesn't know how to interact directly with the engine, but instead raises a window.AddJobEvent (containing the Procedure) which is handled by window.GUIWorkspace. GUIWorkspace calls job.JobManager.addJob on the Procedure.

At this point we're still on the AWT Event Thread. But we can't actually run NetLogo code on that thread, because then the GUI would lock up while it run. So the code will actually run on a background thread, an instance of job.JobThread. JobManager wraps the Procedure in a job.Job and adds it to JobThread.primaryJobs, which is a synchronized java.util.ArrayList, synchronized so it's OK that we're writing to it from one thread and reading it from another.

(Why "primary" jobs? For historical reasons, monitor jobs are "secondary" jobs and all other jobs are "primary". Don't worry about it right now.)

Job execution

JobThread was already created and started when NetLogo first started. If there are no jobs for it run, it sleeps, waiting for someone to call notify() on JobThread.newJobsCondition. JobManager does this. The AWT event thread is now done and returns to the AWT's top level event loop, ready to process any further user actions in the GUI.

Meanwhile JobThread wakes up and enters JobThread.runPrimaryJobs. Before calling Job.step() on the Job it fetches from primaryJobs, it locks the agent.World object. (By convention, nobody is supposed to read or write from any of the data structures in the agent package without first locking World.)

nvm.Job.step() is abstract because Job has two concrete subclasses, ConcurrentJob and ExclusiveJob. ConcurrentJob exists to support interleaving of execution inside ask-concurrent, as well as the similar interleaving that can occur in a turtle, patch, or link forever button. The jobs created by regular ask are "exclusive" jobs, meaning each agent gets exclusive control of execution in turn, no interleaving.

For historical reasons, all top level jobs are ConcurrentJobs, even if they aren't doing agent interleaving. So the flow of execution passes to ConcurrentJob.step().

A Job is run by all of the agents in an agentset. That agentset might only contain a single agent, but still, we must put that agent in an agentset in order to run the job. For each agent in the job, we make an nvm.Context object to represent the current state of execution in that job for that agent. So the first thing ConcurrentJob.step() does is see if we have already made Context objects, and if we haven't, call ConcurrentJob.initialize() to make them. In this case, we're making a single Context object referring to the observer. We also make an nvm.Activation object to represent the invocation of the Procedure we are running.

Now we're ready to actually execute some NetLogo code.

The Procedure we are running contains an Array[Command] containing compiled code. Context.ip, where "ip" stands for "instruction pointer", is an integer index into that array. We're at the beginning of the procedure, so ip is currently 0.

ConcurrentJob.step() loops through all of its Contexts (here, we have only one) and calls stepConcurrent on each context. Context.stepConcurrent retrieves the Command in the Procedure that the instruction pointer points to and calls its perform() method.

Command has many subclasses, and in each subclass, perform() does something different. Here we are running prim._ask.perform, since that's the first thing in the compiled code.

Rather than discuss what happens in detail inside _ask.perform() now, we'll return to it later. For now, suppose that the ask finishes, all the patches run the code in the body of the ask, and all the patches turn red. perform() methods are expected to move the instruction pointer to the next Command themselves, so when Control returns to Context.step(), ip is 1 activation.procedure.code[ip] is an instance of __done.

So we call __done.perform(), which sets the context.finished flag to true. Context.stepConcurrent checks this flag as it loops, and seeing it is now true, exits. ConcurrentJob.step knows whether any unfinished contexts remain, and if none remain, it exits too and tells JobThread the job is done. JobThread removes the Job from JobThread.primaryJobs. Since primaryJobs is now empty, JobThread goes back to sleep.

Job completion

But before going to sleep, it does one last thing: notify GUIWorkspace that the job is done, by calling its ownerFinished method. GUIWorkspace responds by updating the view and emitting a window.JobRemovedEvent on the event thread so that if the job's creator cares that execution finished, it can do something. But actually app.CommandLine doesn't care when a command the user typed finishes executing, so it ignores the JobRemovedEvent.

What about ask?

So that's everything... except the big part we skipped over, which is, what actually happens in the body of ask?

Just briefly (I can expand this on request), _ask.perform itself creates a new Job object. But there's no need to add it to JobThread.primaryJobs, because we are going to the run the job immediately using ExclusiveJob.run(). The body of run() is very similar to the body of ConcurrentJob.run() except that instead of making a fresh Context for each agent, we just use the same Context over and over again, modifying context.agent each time. (This reuse is a performance optimization.) ConcurrentJob.run() calls Context.runExclusive(), which is almost identical to Context.stepConcurrent.

(FURTHER DETAIL ON REQUEST)

Exploring further

To see what the compiler does, it's helpful to examine its output. Bytecode generation tends to obscure the structure, so unless you're specifically interested in the bytecode, you'll want to disable bytecode generation. (NetLogo runs just fine with bytecode generation disabled; code runs slower, that's all.) Use the nogen command in sbt prior to run to disable the generator.

With the generator disabled, if I type this in the command center:

print __dump1 ask patches [ set pcolor red ]

The result I get is:

Command Center:[]{O---}:
[0]_print
      _dump1
[1]_ask:+4
      _patches
[2]_setpatchvariable:PCOLOR
      _constdouble:15.0
[3]_done
[4]_done
[5]_return

The numbers here are the indices of the Commmand objects in Procedure.code — the successive values that Context.ip will take on. Arguments to the Commands are indicated by nested indentation. ask:+4 means that when the ask completes, ip will have 4 added to it to go to the next command. The first _done is inside the body of the ask. The second _done terminates the procedure as a whole. The _return at the end is never executed, since this is a top level job.

Package Guide

This is most illuminating if used as a key to the graffle, rather than read in order.

agent

Agents (observer, turtles, and patches) and agentsets, as well as the World object which is a class that collects a lot of agent-related functionality in one place.

api

The extensions API that lets users write their own commands and reporters in Java. also includes lots of "API"s that are only used internally — typically interfaces that are declared here but implemented in other packages.

app

The parts of the NetLogo GUI that are used only in the full application, not the applet -- menus, tabs, toolbars & their buttons, and so on.

awt

GUI code that isn't NetLogo-specific, depending only on AWT. There's also a swing directory for similar code that uses Swing.

compile

Compiles NetLogo source to NetLogo byte code. Also has the routines that parse Logo constant syntax and return Logo objects (e.g. to support read and read-from-string).

core

Contains API and data classes used by both NetLogo and NetLogo Web.

editor

The syntax-highlighting code editor. Doesn't know NetLogo language details; those are added from org.nlogo.window and/or org.nlogo.ide.

fileformat

Classes for file loading and autoconversion.

generator

Bytecode generator — generates JVM byte code from parsed and assembled NetLogo code.

gl.render and gl.view

Code for the 3D view. gl.render depends only on JOGL; gl.view uses Swing too.

headless

Top-level classes of headless (command line, no GUI) support.

hubnet

HubNet. The GUI portions are divided into hubnet.client and hubnet.server, both of which depend on the low-level connection, mirroring, and protocol packages.

ide

Classes which understand the structure of NetLogo code and can be used to display information about it or transform it.

job

Classes for scheduling and executing "jobs" (of running NetLogo code).

lab and lab.gui

BehaviorSpace

lex

The part of the compiler that only lexes (that is, tokenizes) NetLogo code. This package is used for syntax highlighting as well as during compilation.

lite

Classes used only in applets or the embedded interface tab.

log

Support for logging of end-user actions in the application (or embedded component) for research purposes.

mc

Stands for "Modeling Commons". This code primarily relates to/performs the "Upload to Modeling Commons..." menu option. Largely consists of a slew of dialog boxes and HTTP abstractions.

nvm

Stands for "NetLogo Virtual Machine". This is the model execution engine. Important classes include Job, Context, and Activation. Context represents the current state of execution of a particular agent on a particular job. See also org.nlogo.job.

parse

Contains the NetLogo parser which turns a stream of tokens into an AST.

plot

Plotting subsystem. Includes drawing code (which works under the headless AWT).

prim

Primitives which don't fall into any other categories, and which other classes have compile-time dependencies on. if a prim is here instead of prim.etc, it's because the compiler knows about it specifically and handles it specially somehow.

prim.dead

Primitives not in the language anymore, but still exist in stub form in order to support opening and converting models from old releases.

prim.etc

"Normal" primitives that nothing has any compile-time dependencies on.

prim.gui

Primitives that depend on GUI classes.

prim.hubnet, prim.plot, prim.threed

More primitives.

properties

Code for editing items in the Interface tab (buttons and sliders and so forth).

render

Code for rendering turtles & patches in 2D.

sdm, sdm.gui

NetLogo's system dynamics (aka "aggregate" modeling tool). The top level classes are HeadlessAggregateManager and GUIAggregateManager.

shape, shape.editor

Vector turtle shapes and the editor for creating them.

swing

GUI code that isn't NetLogo-specific, but uses Swing. There's also an awt directory for similar code that doesn't use Swing.

util

Grab bag of misc non-GUI utility code -- nothing in here is supposed to be NetLogo-specific (nothing about models, agents, Logo, etc.). Don't put code here unless it's used from multiple packages.

widget

This package represents an attempt to move the implementations of individual GUI widgets in the Interface tab to a separate package, outside of window, so that code that depends on window isn't depending on every individual widget implementation, either. At present, only a few widgets have so far been moved here.

window

The parts of NetLogo's GUI that are used in both the applet and the full-blown application -- basically, the contents of the Interface tab.

workspace

Holds the AbstractWorkspace class and related classes -- AbstractWorkspace implements the Workspace interface from the nvm package -- Workspace is supposed to represent a model that is open -- currently we can only have one open at a time so there's only one Workspace object, but eventually we plan to support multiple open models. GUIWorkspace (in the window package) and HeadlessWorkspace (in the headless package) both subclass AbstractWorkspace. Workspace has methods for retrieving many of the objects associated with a model, so it's kind of a central point for getting references to objects.