Performance & Scale

Patterns and Anti-Patterns

Chatty Clients

Any API call that has to go over the network is one that we should be suspicious of from a performance perspective. In particular, at very large scales we need to be concerned with any server API calls that take place within a loop.

Anti-Pattern: Linearly growing number of network calls

var items = service.fetchSomeItems(predicate);
for (var i=0; i < items.length; i++)
    service.doSomething(items[i]);

The number times that this code needs to communicate over the network grows linearly with the number of items in question. Since a network call is orders of magnitude more expensive than executing JavaScript locally, this code could perform unacceptably slow for large numbers of items.

To solve these issues we can make a single service call to a new API which does the iteration on the server side.

Pattern: Coarse Grained APIs

service.doSomethingForEach(predicate);

In a "Coarse Grained API" call we tell the server what we want done and describe which items it should be done to. The server can handle iterating over the items and applying the action to each. The number of network calls needed here is constant. No matter how large the set of items described by predicate is, only one API call is needed.

Collections of Items

A lot of Java code involves the use of Collections (Lists, Sets, Maps, etc.). When we are concerned about scale, we need to be suspicious of patterns that leverage collections.

Consider this common (anti-)pattern for dealing with the result set of a query.

Anti-Pattern: Convert query page to list of anything

List<IItemHandle> results = new ArrayList<>(page.getResultSize());
while (true) {
    List<IItemHandle> handles = page.getItemHandles();
    results.addAll(handles);
    if (!page.hasNext()) {
        break;
    }
    page = (IItemQueryPage) qSvc.fetchPage(page.getToken(), page.getNextStartPosition(), Integer.MAX_VALUE);
}

In the above example, the paged query result set is flattened to a simple List of IItemHandles. This makes the result set much easier to process but exposes the application to potential performance and scale issues. The query is paged for a reason, when we flatten the result set we run the risk that not all pages can fit into memory. Even if they do, they may be consuming a huge portion of our heap and driving up the frequency of garbage collections to compensate.

New Ideas

New QM Repository API

Brainstorming about an API that could wrap the complexity of writing scalable code.

Types

Repository

Repositories are abstractions over the Foundation DB access layer. They hide the Jazz Foundation APIs from the rest of the QM application. Repositories can be wide-open, or may be constrained to a a specific project area, configuration, component, or artifact type.

Statement

A "statement" is a non-executable definition of the author's intent. Think of this like writing a SQL statement. These are just words that represent what the author wants to do. Those words then need to be parsed and turned into an executable object inside the DB application.

Executable

This is the result of parsing a statement. It is an object that can be executed to carry out the behavior that was defined in the statement.

Examples

Archive test cases from previously run query

TestCaseRepository repository = new JfsTestCaseRepository(/*required services, project area, configuration, etc*/);
Statement statement = Statement.startReadWriteTransaction()
    .then(Statements.archive(token, exclusions))
    .build();
repository.parse(statement).execute();

Client-side Cache

What if there was a cache on the client. When a service call is made we could cache the results, repeated service calls would return the same results. Invalidating the cache would be the trick here. Some service calls might invalidate results (e.g. refresh button). We could perhaps use a timer to check if query results are still cached on the server and invalidate our cache if they are not.

One thing I like about this is that it frees the client code to act as though the QM database was local. We don't need the whole team to be experts on how to write performant and scalable code.

New Mechanism for Generating Large DBs Faster

• Datagen isn't fast enough to create 100s of millions of resources (would take 100s of weeks)
  • Probably need to cut out network access