Oil Ninja Build - oilshell/oil GitHub Wiki

Back to Contributing / Oil Dev Cheat Sheet

How it Works

  1. ./NINJA-config.sh does two things:
    1. invokes build/dynamic_deps.py to figure out dependencies of Python code generators
    2. then calls build/NINJA_main.py to write build.ninja, which is the default file that ninja looks at.
  • build.ninja has many targets due to the build variants we use: dbg, opt, asan, ubsan, coverage, etc. All of these tools find bugs, help measure performance, or let us use other C++ introspection tools! They help us with the difficult task of writing and testing the garbage-collected C++ runtime.

  1. NINJA_main.py imports the files {build,cpp,mycpp}/NINJA_subgraph.py

  2. Each of those files writes a subset of the NINJA rules (a "subgraph").

    • This is a little but like a BUILD file in Google's build system Bazel, but not quite. We should probably move more toward that structure, where the build config lives next to the source files. That is, EVERY dir will have NINJA_subgraph.py, and the build and tests for osh could be in osh/NINJA_subgraph.py, not in build/NINJA_subgraph.py.

On Ninja

There are essentially only two statements in Ninja: rule and build.

  • build defines a graph edge from multiple inputs to (potentially) multiple outputs
  • rule tells what action to perform
    • which often delegates to build/NINJA-steps.sh
    • or sometimes _bin/shwrap/${foo}_main, which allows us to get incremental builds right with textual code generation
      • for example, if you change frontend/consts.py, then that should invalidate frontend/consts_gen.py, which will invalidate _gen/frontend/consts.{cc,h}.
      • Note: this is the thing where it's unclear how other build systems like CMake handle it.
      • It's analogous to gcc -MD with the Makefile fragments and .d files.

Caveats

The incremental build is correct in almost all cases. EXCEPTIONS:

  • If you change the lexer (e.g. frontend/lexer_def.py), ninja _bin/cxx-dbg/osh_eval won't reflect it. Because that is shared with the Python build build/py.sh all.
  • If you change */NINJA-steps.sh, it won't be reflected. (TODO: add these as implicit deps?)
  • If you set CXXFLAGS='-D ALLOCLOG' ninja, then objects in _build/obj/cxx-dbg will be compiled differently. And then future incremental builds may not be correct.

Adding Files

2022-09-06

  • there is a GC_RUNTIME var in mycpp/NINJA_subgraph.py
    • do ./NINJA-config.sh; ninja
  • there is a list of C++ unit tests in mycpp/NINJA_subgraph.py

Include style

We are using the clang-format / IWYU / "Google" style.

Details: https://google.github.io/styleguide/cppguide.html

Summary:

  • "traditional" .cc and .h structure (not unity builds)
  • Every header has #include guards
  • #includes at the top of every file
  • run test/lint.sh format-cpp
    • avoids the problem where you have to "jiggle" include order to make the build pass
    • there is ONE correct build order for dependencies