Kryo v6 Ideas - EsotericSoftware/kryo GitHub Wiki

Compatibility

Kryo 6 should be serialization compatible with Kryo 5.

Kryo 6 reads data written by Kryo 5 with the Kryo 5 format, kryo.setFormat(Format.KRYO5) before registering classes, which makes Kryo's serializers and reference resolvers write and read the Kryo 5 format where the default format changed (https://github.com/EsotericSoftware/kryo/pull/1477, https://github.com/EsotericSoftware/kryo/pull/1478; replaces Kryo5Compatibility). See MIGRATION.md.

  • The Kryo 5 format is tested against the released Kryo 5.7.1, a test dependency, in both directions: Kryo 6 reads what Kryo 5 writes, Kryo 5 reads what Kryo 6 writes with the Kryo 5 format, and both write the same bytes (https://github.com/EsotericSoftware/kryo/pull/1479)

✅ JDK 17 Baseline

We should move to JDK17 to be able to integrate records into FieldSerializer.

See https://github.com/EsotericSoftware/kryo/commit/20cc80579062e3f6dfe1e9e16eb7851a325e50b5

Done on the kryo-6 branch, see https://github.com/EsotericSoftware/kryo/pull/1313, https://github.com/EsotericSoftware/kryo/pull/1314 and https://github.com/EsotericSoftware/kryo/pull/1315

✅ FieldSerializer for records

We should move record serialization from RecordSerializer into FieldSerializer.

See https://github.com/EsotericSoftware/kryo/pull/931

✅ Enable additional JDK serializers

We should enable all additional safe serializers.

See https://github.com/EsotericSoftware/kryo/pull/1318

Disable advanced generics optimization

We should switch to a less aggressive version of generics optimization by default.

Superseded by "Fix generics" below: the bugs that motivated a less aggressive generics strategy are caused by how type variables are resolved, not by the optimization itself. https://github.com/EsotericSoftware/kryo/pull/879 was closed.

✅ Fix generics

DefaultGenerics assigned the type arguments of the declared type (eg the type of a field) by position to the type parameters of the class of the value. This is only correct if both have the same type parameters in the same order, and otherwise fails with a ClassCastException. The guard in pushTypeVariables only hid the cases where the number of type arguments differed. Map the type arguments through the super classes and interfaces of the class instead, and remove the guard.

See https://github.com/EsotericSoftware/kryo/pull/1329

✅ Disable unsafe by default

Unsafe should be disabled by default and users should opt-in to use it.

Done: FieldSerializer uses Unsafe only where Java doesn't warn about it (before Java 24 or with --sun-misc-unsafe-memory-access=allow), reflection on Android, and VarHandles otherwise. FieldSerializerConfig.setFieldAccess or the system property kryo.fieldAccess selects it explicitly. See https://github.com/EsotericSoftware/kryo/pull/1339 and https://github.com/EsotericSoftware/kryo/pull/1397

VarHandle field access is safe and warning free, but 10% to 21% slower than Kryo 5 with Unsafe on an object graph of 78 classes (and 60% faster for the chunked encoding, which has a new format). Unsafe field access is 20% to 27% faster than Kryo 5, code generation 2.1 to 3.6 times. The hidden classes that made VarHandles constants (https://github.com/EsotericSoftware/kryo/pull/1393) were removed again in favor of code generation, which generates one class per serialized class instead of one per field (https://github.com/EsotericSoftware/kryo/pull/1444). Final fields are read with VarHandles and set with reflection (https://github.com/EsotericSoftware/kryo/pull/1447). Chart: https://github.com/EsotericSoftware/kryo/blob/kryo-6/benchmarks/charts/kryo5VsKryo6.svg

✅ Migrate away from terminally deprecated Unsafe methods

See https://github.com/EsotericSoftware/kryo/issues/1246

Done for the default configuration: no Unsafe warnings on any Java version. The serializers for unmodifiable and synchronized collections use method handles if java.util is open to Kryo (https://github.com/EsotericSoftware/kryo/pull/1341). Final fields are set with reflection, which Java 26+ warns about; Kryo explains how to allow it (https://github.com/EsotericSoftware/kryo/pull/1340). ClosureSerializer no longer reads JDK internals, so it needs no --add-opens (https://github.com/EsotericSoftware/kryo/pull/1422). Unsafe buffers and the UNSAFE field access remain as opt-ins.

✅ Support GraalVM native image

Kryo should work in GraalVM native images without metadata for Kryo itself. Users only register their own serialized classes, eg with the tracing agent.

See https://github.com/EsotericSoftware/kryo/issues/936

✅ Support Android

Kryo 6 supports Android from API level 26 (Android 8.0), see the README section "Android". APIs that are only available on newer Android versions are guarded (https://github.com/EsotericSoftware/kryo/pull/1338), and fields are accessed with reflection on Android (https://github.com/EsotericSoftware/kryo/pull/1339). Verified once on an Android 16 emulator, there is no Android CI yet.

✅ Provide built-in serializers for unmodifiable and synchronized collections

See https://github.com/EsotericSoftware/kryo/pull/1154

✅ Chunked encoding that doesn't lose data in skipped fields

With chunked encoding, skipping a field lost the class names, field names and references first written in it, so the rest of the data could not be read. Fields are now written with their length, and the class names and field names first written in an object before it. The new format is also smaller and faster. The lengths are variable length longs, so a future version can write fields larger than 2 GiB, which Kryo 6 buffers in memory (https://github.com/EsotericSoftware/kryo/pull/1438, https://github.com/EsotericSoftware/kryo/pull/1445).

See https://github.com/EsotericSoftware/kryo/issues/1247 and https://github.com/EsotericSoftware/kryo/pull/1382

✅ Performance

✅ No required dependencies

✅ Final fields if Java denies setting them with reflection

Java will deny setting final fields with reflection by default (JEP 500). Set the final fields of serializable classes with the method handles that Java provides for deserialization.

Done: if setting final fields with reflection is denied, FieldSerializer sets the final fields of serializable classes with these method handles on Java 24+ (https://github.com/EsotericSoftware/kryo/pull/1383). The setter is resolved when a final field is first set, so a process that only writes sees no warning (https://github.com/EsotericSoftware/kryo/pull/1441), and final fields are read with VarHandles (https://github.com/EsotericSoftware/kryo/pull/1447).

Misc

Open items found in the reviews of kryo-6 are in Kryo v6 Follow-ups.