spikes - LucPrestin/Hidden-Modularity GitHub Wiki

As described in issue #36, the current workflow is not yet optimal due to system boundaries that we encounter. Specifically, we have three issues that should be addressed:

1. the feedback loop is longer than usual in a live-programming system, too long.
2. because we load d3-force over a CDN we are dependent on an internet connection
3. when creating and exporting the graph, the way back from the graph to the base data is lost

So the question is, can we port the visualization to Squeak while remodeling the graph construction so that we still have a path back to the base data?

Currently we use d3-force for the layouting and visualization of the graph. The source code is open source and licensed under ISC License. So we could reimplement the code in Smalltalk.

The simulation would use the reimplemented d3-force source code. The visualization would be based on Morphic.

For the way back to the data, we could model the idExtractionBlock in a way that when parsing the trace, we create an object for each extracted id that holds a reference to all objects that got that id. The graph node would then be based on that intermediary object and thus allow us to access the base data of each node.

Creating and including the intermediary object presents no difficulty.

Drawing the graph using Morphic is also straight-forward.

The simulation, on the other hand, is more complicated. The simulation code is relatively clear, but uses a quad-tree for its layout, which does not exist in Squeak yet. Accordingly, this data structure would have to be implemented first for the d3-force implementation to be a manual translation from JavaScript to Smalltalk.

At the time we evaluated this spike, we were primarily concerned with problems 1 and 2. The visualization is a tool that gave us interesting results even as it is. The use case to go back to the base data came later.

Since it was more of a convenience and availability issue for us, but it worked well enough for now, the conclusion was that implementing a quad tree would take too much time away. Implementing an own layout system and adapting the d3-code to it would also be time consuming. We could also put that time into improving the HMTraceParser or finding new and evaluating existing scenarios.

The issue was put on hold for the time being.

As described in issue #37, an image boot would be an interesting scenario to trace. However, this could prove to be more difficult than expected, as we have only ever traced small to medium sized scenarios. The start of a new image, though, is something very large, with many objects and classes involved.

So the question is, is it performance-wise feasible to trace an image boot at all?

To investigate this, we need data from experts on two fields:

1. Ask @LeonMatthes about how we could start an image from within an image as that is his current projekt
2. Compare that information to how long it takes to trace the medium-sized scenarios
3. If question 2 suggests that it is feasible, do it.

According to @LeonMatthes, an image boot from within an image takes approximately 30 seconds on his machine. Since his machine has more processing power than the one we are using, an image boot will also take at least about 30 seconds, if not longer.

Now we compare this with opening a SystemBrowser. On our machine, there is no time delay from human perception when opening normally via a workspace. However, if we trace this scenario with the TraceDebugger, it takes 15 minutes. Even if we assume as a basis of calculation that it normally takes one second to open a SystemBrowser, we would therefore expect $15min \times {30sec \over 1sec} = 450min$, assuming there is enough memory for the TraceDebugger to work as usual.

An image boot is too big of a scenario to be traced with the resources we currently have. Our time is probably better spent tracing more small to medium sized scenarios.