Design Of Objectify 6

Objectify is coming up on eight years old, with v5 released over four years ago. In all this time, the underlying Google-provided API to the datastore hasn't changed much. This stability has been great for App Engine users, but the API is based on a proprietary binary protocol that reflects an era when App Engine was Google's entire cloud offering. GAE may be the gem at the center, but Google's Cloud Platform is now something much, MUCH bigger than just the handful of services historically provided within App Engine.

As part of creating a broader ecosystem, Google has been breaking out pieces of App Engine into standalone services with standards-based APIs that can be accessed from anywhere on the internet. It's pretty clear that these new APIs are the future, even for App Engine users. Objectify needs to be rebuilt on top of the new Cloud Datastore SDK.

Goals

• Objectify 6 will be usable from everywhere the new Cloud Datastore SDK is supported. It will not reference the old GAE SDK.
• Objectify 6 should preserve the public API as much as possible. Upgrading v5 -> v6 should not require changes to the load/save client code (although it may produce deprecation warnings).
• Objectify 6 should offer an improved API, incorporating lessons learned from the last several years.
• Objectify 6 will integrate with Google's new memcache API.

Nongoals

• Objectify 6 will not be an exact drop-in replacement. Different client code will be needed to configure the datastore connection; you may have to change one or two parts of your code, but not "all of your code".
• Objectify 6 can not coexist in the same codebase with any previous version of Objectify. To preserve API compatibility, the java package names must remain the same. Maven artifact group/name will not change.
• Objectify 6 will not necessarily perform exactly the same as prior versions.

Changes

A not-necessarily-exhaustive list of the specific changes that need to be implemented, and the challenges they present:

New Test Harness

The old GAE SDK provides a "local unit test framework" that simulates the datastore in-memory and can be reset with a function call. Objectify's test harness relies on this framework. The new Cloud Datastore SDK provides a separate process that can be run locally to simulate the datastore. Objectify's test harness will need to be adapted to start/stop this process and reset the content between tests.

A number of Objectify's tests directly interact with Low-Level API objects; these tests will need to be updated.

While doing this work, might as well convert to Junit5/Truth.

Always Deferred

After many years using Objectify, I've come to the conclusion that saves and deletes should always be defer()ed. That is, these two calls should be equivalent:

ofy().save(thing);
ofy().defer().save(thing);

In the unusual case that someone needs a write to occur immediately, they can call ofy().save(thing).now() or subsequently call ofy().flush(). But in almost all business processing it's best to group writes into a single batch call at the end of a transaction/unit-of-work.

Minor API Simplification

Existing:

ofy().save().entity(thing);
ofy().delete().entity(thing);

Better:

ofy().save(thing);
ofy().delete(thing);

Of course, preserve and deprecate the old versions.

Memcache

The underlying memcache API is totally changing; we need to reflect those changes.

Documentation

The docs are gonna have to change. Yup.