Node library dependencies - nodebox/nodebox GitHub Wiki

Every NodeBox document potentially contains useful nodes that can be re-used in other documents. A document should be able to re-use those nodes by referring to them.

In addition, Each NodeBox nodes has code behind them. We want the code to be "just code", that is, a Python module with basic functions that take in and return values. Each node refers to a function using a function identifier.

Imagine a NodeBox document that contains car parts: it has nodes that represent a wheel, engine, steering wheel and headlight. When starting the "sports car" project, we'd like to refer (or depend) on the parts document and use its nodes to assemble a complete car.

This set of dependencies (documents we refer to, or use) is unique per document. That is, every NodeBox document can refer to a different set of documents. In addition, all documents have access to a core set of nodes that are considered essential. These start with "core", e.g. "corevector", "coreimage".

• A node library is the data structure for a NodeBox document. It contains all the nodes (as a tree structure) and connections between nodes. Also, it contains dependencies to other node libraries.

• A node library dependency is a reference to another node library. It is stored in the referring (dependent) node library.

• A node repository is a list of node libraries. Each node library has a node repository that contains the node library dependencies for the library.

• A function identifier is a link to a Python/Clojure/Java function. Example: math/add

• A function library wraps a .py or .clj file. It describes the functions available in the module.

• A function library dependency is a reference to a function library. It is stored in the referring (dependent) node library.

• A function repository is a list of function libraries. Each node library has a (possibly empty) function repository that contains the function dependencies for the library.

We make a distinction between these types of node libraries:

• System libraries: Core node libraries (e.g. corevector, coreimage) included with the application

  • They are available from every NodeBox document.
  • The dependency link is implicit and not written to the ndbx file.

• Global libraries: Libraries that are installed under the library path.

  • These could be installed by a (to-be-implemented) NodeBox package manager.
  • They are available from every NodeBox document.
  • Once we use nodes from them, they are explicitly included using a short name.
  • Available under /Users//Library/NodeBox/Nodes

• Custom libraries: Libraries that are stored somewhere on the file system.

  • We link to them explicitly.
  • The href can refer to a absolute or relative path.

As an example, let's build a sports car. One document contains all the parts and another document contains the behavior. The car document combines this into a fully operational vehicle.

Parts -- Contains visual parts of the car, but no behavior.
  +-- parts.ndbx

Drive -- Contains the behavior of the car. Links to the parts project.
  +-- drive.ndbx
  +-- drive.py -- custom code for the "steer" node.

Car -- Contains a fully assembled car. Refers to the parts and drive projects.
  +-- car.ndbx

To link to a node library, use the link tag with rel="nodes" and a href pointing to the file. Here's an example from car.ndbx:

<ndbx>
  <link rel="nodes" href="path/to/parts.ndbx"/>
  <link rel="nodes" href="path/to/drive.ndbx"/>
  <node name="wheel1" prototype="parts/wheel"></node>
  <node name="steer1" prototype="drive/steer"></node>
</ndbx>

To link to a function library, use the link tag with rel="functions" and a href pointing to the identifier.

The identifier is in the format language:filename.ext. The last part does not have to be filename.ext: the library loading function can choose how to implement this.

• Python: Use the file name of the Python module. The name is the namespace. The filename needs to consist of only lowercase letters and underscores. python:math.py
• Clojure: Use the file name of the clojure file. The clojure file needs to declare a namespace using (ns math): clojure:math.clj
• Java: Use the class name. We assume it contains a public static final field LIBRARY: java:nodebox.functions.MathFunctions

Here's an example from drive.ndbx:

<ndbx>
  <link rel="functions" href="python:drive.py"/>
  <node name="steer" function="drive/steer">
    <port name="speed">12</port>
  </node>
</ndbx>

Let's demonstrate how NodeBox loads our example:

• Load car.ndbx.
• Detect that it has a dependency to parts.ndbx.
  • Load parts.ndbx.
  • Add exported nodes to the node repository.
• Detect that it has a dependency to drive.ndbx.
  • Load drive.ndbx
  • Detect that it has a dependency on parts.ndbx.
  • Parts.ndbx is already loaded, so continue.
  • Add exported nodes to the node repository.
• Load the nodes of car.ndbx.
• Finish loading car.ndbx.

Depending on the type of code, NodeBox takes different paths.

NodeBox doesn't provide a mechanism for specifying dependencies between function libraries. Instead, we rely on the native package manager (easy_install, pip, or some clever sys.path hacks) to do the job for us.

Suppose we want to build our Car project and we're starting from scratch. We should have a user interface element where we can add / edit our dependencies. We suggest making a menu under Node > Dependencies.

• A window appears with a list of our current dependencies (None) and an Add-button.
• Press Add.
• A list of available dependencies appears.
• The libraries on the /Application Support/NodeBox path
• The libraries which were loaded through having loaded dependencies before. (ie if we added drive as a dependency before, parts wouldn't have been added but through drive it would already have been known by the system)
• A file field with button to open directories / files to add new libraries.

When referring to another node library, we need to make sure the reference is unambiguous. If we just refer by name, two node libraries with the same name could potentially conflict.

Using an unique identifier (UUID) could resolve this problem.

public class NodeLibrary {
    private final String name;
    private final Node root;
    private final FunctionRepository functionRepository;
    private final NodeRepository nodeRepository;
    ...
}

It's not wise to only be able to link to specific (absolute) locations on a users computer. What if the user wants to share a library that depends on one or more other libraries which he can't be certain of whether the receivers will also own?

As a solution we could offer the possibility to save (or export, or pack) a project complete with dependencies, so everything is ready to use for another user. A project structure could like this:

Car
  +-- car.ndbx
  +-- parts.ndbx
  +-- drive.ndbx
  +-- drive.py

All the paths to the dependencies could be rewritten:

<link rel="nodes" href="parts.ndbx"/>
<link rel="nodes" href="drive.ndbx"/>

In this case the user would have to load car.ndbx to get the project working. Another option is to have the entire structure wrapped in a zip or jar file. In the case of a jar some meta-information will be needed to indicate which file should be loaded (car.ndbx).

It's important that the user can choose if he wants to save a project as described above (including all dependencies) or if he just wants to save locally (using absolute paths). Usually the latter will be done, the first option is somewhat overkill for everyday use.

If we export a project with all its dependencies we might want to choose whether we also want to include libraries that are on the NodeBox user library path /Application Support/ . This might be handy because we don't know if a receiving user will already have access to those libraries. However when we would load such a complete project (with ALL depending libraries included) ourselves, how will we make sure we're using the correct nodes? Because here there will be some duplication. We're loading libraries that are copies of those stored on the /Application Support/ location. We could:

• Do nothing and have a duplicate set of nodes available to us.
• Automatically override the standard set with the "new" set.
• Detect that we have two similarly named sets of libraries and inform the user. The user could then choose to:
  • Do nothing and have a duplicate set of the same nodes.
  • Dse the standard set of nodes and hide the nodes from the new set.
  • Use the new set of nodes and hide the nodes from the standard set.
  • Other ?

A library can also be a single file. It could be made up of a few nodes that are connected and that have some of their parameters modified. The nodes themselves are of types that are general (the core nodes and the nodes in the standard search path) so a separate library containing code is not needed in this case.