Recent Dev Friction - oilshell/oil GitHub Wiki

Back to Where Contributors Have Problems

Summarizing some of #oil-dev > Dev Friction / Smells

2023-04

Some global constants are in weird places

To do the C++ translation, we moved some constants from their natural place to say frontend/consts.py or frontend/flag_def.py. I would like to move them back at some point.

(The translation involves a lot of code generation: these files have counterparts frontend/consts_gen.py and frontend/flag_gen.py.)

RuntimeError exception swallowed at the top level

Came up while implementing compgen -A builtin with Chris.

I removed the except:, so you can more easily see a stack trace from the Python interpreter.

Inverted Python import from asdl/ -> core/

This caused a Python traceback due to a circular import during the "big parser refactoring".

Solution: duplicate a small log() function in asdl/.

The reason it's an inverted dependency is that ASDL is a tool that operates on the OSH interpreter, which includes core/. So it shouldn't import that code too.

Some advice: Run the type checker and unit tests very often during refactoring. I generally do it multiple times a minute. This way I avoid building on top of a bad state.

Spec Test Flakiness on Local Machine

The spec tests start a ton of processes in parallel. The work pretty reliably in the CI, but on my local machine, I got

__ glob
__ here-doc
__ if_
__ interactive

[1]+  Stopped                 test/spec.sh osh-all
andy@lenny:~/git/oilshell/oil$

There are random stoppages.

Hilariously, this appears to be longstanding bug / race condition in shell job control in some shells, which we probably also have: https://github.com/oilshell/oil/issues/330 -- bug going back to 2019!

The issue is probably that when a child process is not in the foreground, and it tries to write to the terminal, the kernel will send it SIGTTIN or SIGTTOU.

This is an extremely difficult to debug, and we need help! There should be a better way of deterministically reproducing it.

In practice, it's not a big blocker, because I just run single spec tests on my machine serially, and then rely on the CI to run the whole suite.

build/py.sh minimal left stale .so files from Python extensions

We have this split between build/py.sh all (creates .so files) and build/py.sh minimal which I should probably get rid of. The idea was to let people get started quickly while building less, but it can cause more confusion than it's worth. See Contributing.

mycpp got confused about a local variable loc vs. module name loc

It generated the wrong code, and this resulted in a C++ compile error.

The workaround was to rename the variable to blame_loc.

This should ideally be fixed, but realistically it's hard given the architecture of mycpp, and how we build on top of MyPy.

Though mycpp is getting better, and it would be nice to have someone to fix it up a bit. It's a good intro to compilers.

Most shell scripts ("task files") in the repo don't have help (can be hard to use)

There are A LOT of shell scripts that highly automate our dev process. See the end of Where Contributors Have Problems.

But they don't have help, and they use a somewhat unfriendly "$@" pattern.

Chris fixed one of them, but it would be nice to fix all of them.

Ironically, this is a deficiency of shell that we should fix in OSH and YSH!!! We can use help!

So for now, you have to read the comments at the top of the file. I always have an editor and a terminal open side by side for this reason.

Or use the hints in the web UI for http://travis-ci.oilshell.org/github-jobs/.