Want to add your own domain of mathematics to Penrose, and define custom visualization styles for it? You're in the right place! ✨

STEPS:

1. Build and run the system by following the instructions on the page Building and running.

2. Next, fork this repository, and set penrose/penrose as your upstream.

3. Make a new folder in penrose/packages/examples/src/. Let's say your domain is called DOMAIN. Then you'll have a working folder penrose/packages/examples/src/DOMAIN/.

4. Make Domain, Substance, and Style programs in that folder: touch DOMAIN.dsl; touch myfirstprogram.sub; touch onestyle.sty.

5. Set up syntax highlighting in your editor: we recommend (and officially support) VSCode with Typescript and our extension installed. We also offer unofficial syntax highlighting for other editors here.

6. Keep working on your domain in your DOMAIN folder. If you need to add objectives/constraints/functions, you can add them in penrose/penrose-web/src/contrib. (See "Contributing to the Penrose platform" below.)

7. Pull from upstream's main branch periodically, as we may push changes. Change logs for new releases are documented here. (See this page for more info on releases.)

8. As you're working, feel free to ask us any questions on Slack (if we've already added you) or via email at team@penrose.ink.

9. When you're ready for review, follow this workflow to standardize your code, then open a pull request, and request review from us. (We're still putting together guidelines for contributor commit style; coming soon.)

If you have a problem, please file a GitHub issue or email us at team@penrose.ink.

If you would like a more guided walkthrough of how to make diagrams in Penrose, check out our tutorial.

If you're already very comfortable with writing code (especially graphics code), read on for our more condensed documentation below.

Now that you've gotten a skeleton project set up, you're ready to write Domain, Substance, and Style programs in them. We call a group of these programs that can be used together a "triple."

For a quick overview of these languages, check out our the SIGGRAPH video from 6:30-11:30 and SIGGRAPH paper, Sections 3 and 5.

For other working example triples, see Example diagrams. (These can all be run in our system, though note they may have TODOs, cryptic comments, etc.)

Currently syntactic sugar does not work in Domain and Substance programs. We hope to re-add this feature in the future.

We have added new language features to Style beyond what's documented in the paper. These features are local variables, vectors/matrices, and configurable canvas dimensions. For an example of all these features in action, check out linear-algebra-domain/linear-algebra-paper-simple.sty. For a more complete description of these features, check out this page.

Writing a Style program involves the most moving parts. We have provided some shapes, objectives, constraints, and functions that are built into our system that you can call out-of-the-box in Style.

For shape documentation (the properties and their types), check out this page.

For objective/constraint/computation documentation, check out this page.

Read "Contributing to the Penrose platform" (below) if the shapes, objectives, constraints, and computations that are built in to Penrose don’t meet your needs. (These are written in TypeScript, a version of JavaScript with types.) The built-in libraries are not comprehensive, so it’s likely that you’ll need to extend them yourself.

RECOMMENDED WORKFLOW:

• Think about how to model your domain in Substance; that is, what are the abstract logical relationships involved?
• Start with some example diagrams and break them down into Substance programs.
• Write a small core Domain, paired with several small Substance programs that use that small Domain.
• Think about how objects might are represented visually, and start a small Style program for it.
• Extend that Style program to handle visual relationships between objects.
• Run your triple of programs.
• Alternate between looking at the resulting diagrams (resampling them a few times to get a sense of what's possible), debugging your programs (more on this below), extending the Style/Substance/Domain programs to handle more cases, and (if needed) extending the provided library of objectives/constraints/functions to handle your Style of choice.

EXPORTING DIAGRAMS:

• For web: you can download the diagram as SVG using the button at the top. This code can be opened in vector graphics applications for further applications, or embedded straight into a webpage. (Note: if you're embeddding multiple Penrose diagrams, make sure the CSS styles are compatible. Talk to us if you hit problems.)
• For print: our PDF export is currently broken, but here is a temporary workaround. You can get around it by printing the webpage in Chrome, hitting "more settings", and choosing "open PDF in Preview." That will get you a PDF that you can then edit (to remove extraneous Penrose elements) and embed as usual.

Read "Contributing to the Penrose platform" (below) if the shapes, objectives, constraints, and computations that are built in to Penrose don’t meet your needs. (These are written in TypeScript, a version of JavaScript with types.) The built-in libraries are not comprehensive, so it’s likely that you’ll need to extend them yourself.

HANDLING CHANGES:

• If you change a Penrose program, you need to restart the backend (unless you have the CLI installed, per the install instructions).
• If you change a TypeScript file and save it, if you have the node server running, you do not need to restart the frontend; it watches for changes automatically.
• If the frontend crashes, you'll need to restart it
• If you pull new code from GitHub, you may have to rebuild and rerun the Haskell backend and/or the TypeScript frontend (including an npm install if new libraries were added).

The main thing to know is that Penrose is not super debug-friendly right now, but we would like to improve it! In particular, the error messages are not very reliable. Something may fail and give a misleading error message, or a different part of the system (that you weren't expecting) might fail, instead of the part that you were working with.

If you hit a weird bug and get stuck for more than 10 minutes, your best bet is to ping us on Slack (if we are already collaborating), or file a GitHub issue. Don't worry, we aren't bothered; we would be very happy to help!

BUG REPORT INFO TO BE INCLUDED:

• The Penrose programs you are running (including Substance, Style, and Domain)
• A link to the base repository or branch you're on, and if possible, latest commit (this is so we can go in and debug things ourselves)
• The error(s) or unexpected results you're getting
  • Please check the terminal output for the backend, as well as the console output for the frontend
• The results that you're expecting
• If you are able to isolate the bug to specific lines of code (e.g. it happens with this line of code in, but goes away when the line of code is removed), or find minimal reproducible programs (i.e. removing as much of each program as possible that is unrelated to the bug), let us know! This is extremely helpful.

So you hit a bug. In our experience, most bugs are in Style programs, and sometimes in the optimization. Some things to check in your Style program:

• Are you assigning a value to a property of a shape that doesn't exist?
• Are you using an int when you should be using a float? (Check the type of the property, the input function, etc.)
• Are you passing in arguments of the right type and number to the objective/constraint/function you're calling in Style?
• Are all the variables that you're using in your Style block present in the environment defined by its selector?
• Is your objective function or constraint actually satisfiable? (Sometimes the optimizer will reach NaNs when given an unsatisfiable problem.)
• Did you try sampling different variations on your diagram?

(We hope to check these things automatically in the future. 🤖)

Some things to check in general:

• Did you rebuild the backend and frontend after pulling?
• Did you rerun the backend after changing a Penrose program? (Not needed if you're running the Penrose command-line assistant.)

For more advanced users, you can use the following techniques to debug:

• The frontend "frames," "shapes," and "mod" tabs allow you to inspect the state of the diagram, e.g. specific property values of specific shapes.
  • A diagram is also just an SVG in Chrome, so you can inspect the elements in the DOM, modify it, change the styles, save the SVG and open it in another application, etc.
• Visualizing quantities in Style (e.g. with "debug" shapes and lines)
• console.log statements in your objectives/constraints/functions (NOTE: This will only work to a limited extent, as log statements are compiled out in the optimization, so don't be surprised when you don't see console logs)
  • To print values in an objective/constraint/function, you should use the special debug function as detailed here
  • You can also visualize derivatives in Style using the special derivative or derivativePreconditioned function applied to a varying float

So you want to add a new objective/constraint/computation or new shape, or look at an existing implementation, or extend the renderer (or system in general). Many of the key source files are located in packages/core:

• types: Typescript definitions of types used internally by the Penrose system. (Note that these are of course different from types defined in a Domain program.) Changing these files will require the frontend to be rebuilt.
• contrib: Contains objective/constraint definitions (in Constraints.ts) and function definitions (in Functions.ts).
• engine: Contains the optimizer, evaluator, autodiff, and other utils. You probably don't need to change this unless you're debugging optimization.
• utils: Contains various utils. If you're trying to add a utility function, you probably want OtherUtils.ts.

Other important directories:

• packages/core/src/renderer/: Contains shape rendering definitions in Typescript. Our renderer uses the shape properties, plus this file, to render a shape.
• packages/browser-ui/src/inspector: Contains code for the frontend inspector. If you want to extend the system with new visual debugging support, do it here.

Our system uses a technique called "automatic differentiation" (autodiff for short) on all of your code, so it can give you gradients. That means, in short, that you'll have to use special number types and operations in all of your code so it can be differentiated. Please read the first few sections of the Autodiff guide (before the "Definitions" section) so you are familiar with how to do this.

On function types:

Inputs: The input parameters to these functions are any kind of argument value (modeled in the type ArgVal<VarAD> in types/types.d.ts) without the tag wrapper. What that means is that the input to a function can be of type GPI<T> or Value<T>.

GPI<T> is a shape. It is defined as a pair of a string (which is the type of the shape, like "Circle") and a map from shape property to shape value. So you can write something like [t1, s1]: [string, any] in a function type signature, which deconstructs a shape input into t1 (the name of the shape) and s1, the shape information. One common operation is to access the parameter of the shape, which is done via shapeName.propertyName.contents, which will return a VarAD (differentiable floating point number) containing a single number. For example, if you had a circle c as input, and you want its radius, doing c.r.contents will give you something like 5.0 (wrapped in a VarAD).

A Value<T> is any value that's not a shape. It can be one of the following:

• float
• int
• bool
• string
• path data (this is a slightly complicated type that models an SVG path)
• color
• palette (list of colors)
• file string
• style string
• vector of floats (2D only)
• matrix of floats (2D only)
• tuple (2D only)
• list of vectors

See Value<T> in type.d.ts for their definitions. (Any branches that are not listed on this wiki page are deprecated, such as HMatrix). To tag a value as a return value you'll have to wrap it, e.g. { tag: "FloatV", contents: constOf(0.0) }.

Output: The output type of a function differs based on the kind of function, whether it's an objective or constraint (must return a VarAD representing a penalty energy), or computation (must return a Value<VarAD>), as described below.

Body: In the body of a function, you must use the provided helper functions for arithmetic on VarADs or vector operations on lists of VarADs, which are described in the Autodiff guide (before the "Definitions" section). You cannot use built-in JavaScript math operations on VarADs, though you can use arbitrary JavaScript on lists. Note that you must write these functions in straight-line functional style (i.e. no imperative style, no mutating state, no for-loops or if statements). These restrictions are so the Penrose system can automatically differentiate your code.

Note that constraints and objectives are currently defined based on the type of their input shapes, e.g. repel might have one case for shapes with centers, then another case for a line and a rectangle, and so on. There may be missing cases, in which case feel free to add an implementation!

• Add another one to objDict in Constraints.ts. (Check out the existing functions there for examples.)
• The inputs must be of type ArgVal<VarAD> (without the tag wrapper).
• It should output the "badness" of the inputs (as a number or Tensor), and have local minima where you want the solution to be.

• Add another one to constrDict in Constraints.ts. (Check out the existing functions there for examples.)
• The inputs must be of type ArgVal<VarAD> (without the tag wrapper).
• Let's say I want the constraint f(x) <= c to be true.
• I translate it to the zero-based inequality f(x) - c <= 0.
• I translate the inequality constraint into an energy (penalty) E(x) = f(x) - c — it is greater than 0 iff the constraint is violated, and the more the constraint is violated, the higher the energy is (e.g. if f(x) is way bigger than c then the energy is a lot bigger than 0). That is the form of all the constraints in the system (and the form you'll be writing them in). It should return a number or Tensor.
  • You may need to multiply it by some weight d depending on how it compares to the overall magnitude of other constraints/energies in the system. E(x) = d * (f(x) - c)
• The Penrose runtime automatically substitutes each individual constraint into the penalty function p(x) = max(x, 0)^2. This has the effect of ignoring any energy value for a constraint that is satisfied (e.g. if f(x) <= c then E(x) < 0 then you can ignore it), and of making the penalty quadratic for any constraint that's violated.
  • The choice of exponent is arbitrary though squaring seems to work well in practice, as detailed in the notes on exterior point method. Just be careful with how the squaring changes the asymptotic behavior of how you defined the energy originally, e.g. squaring a linear energy works well, but squaring a quadratic may not.
• The Penrose runtime increases a separate weight on all the constraints (multiplicatively by a factor of 10) for each unconstrained optimization (this is also part of the exterior point method, and you don't have to worry about it, though you should know it's happening).

• Add it to compDict in Functions.ts. (See compDict for examples.)
• The only types a computation can take and return are defined in types/types.d.ts, the type ArgVal<VarAD> as input and Value<VarAD> as output.
• You'll have to tag the result of computation with the right type.

Note that the optimizer takes many optimization steps per display step (set in numSteps in Optimizer.ts). So what you see in the frontend does not represent the results of every optimization step.

• A "display step" is what happens when you click "step" in the frontend and you see the diagram update. It lets you see more of the process, at the cost of speed.
• An "optimization step" is one iterative update to the state of the diagram via a descent method.
• If you're bent on debugging optimization, read these tips on debugging optimization (warning: some may be outdated)
• For further reading, e.g. if you want to add a new differentiable operation, you can check out the latter half of the Autodiff guide.

• Add a new file in packages/core/src/renderer for your shape. Call it <shapename>.ts.
  • This file should export a function that returns the rendered shape element.
• Add and export a variable with ShapeDef type to packages/core/src/renderer/ShapeDef.ts that corresponds to your new shape.
• Add a mapping to packages/core/src/renderer/shapeMap.ts.
• Carefully select the properties of your shape. If you want the optimizer to be able to optimize a property of your shape then this property has to exist. For example, add a single positional property (e.g. center) if you want the property to be optimized.

Here is an end-to-end example of adding a new shape called PathString that takes a path string and passes it through to the frontend to render.

If you add new functions in our system libraries, please also write a brief description of its inputs, outputs, and purpose. Please write comments in typedoc syntax right above your function's definition, as described on this page.

We're still working on our test infrastructure; some old code can be found in penrose-web/__tests__. Basically, though, we do regression tests by running new code against our list of working example diagrams and making sure that they all still work.