scala.collection.immutable.ListMap fails to serialize since 2.9.3

[dejanlokar1]

Serializing a class with a property of type scala.collection.immutable.ListMap fails with a JsonMappingException. Serializing a ListMap as a root value works as expected. Other scala.collection.immutable.Map implementations also work. The last working version is 2.9.2. Issue is reproduced here.

This may be related to #2111 and #2080. Currently, we are setting serialization typing to static as a workaround.

[info]   com.fasterxml.jackson.databind.JsonMappingException: Failed to specialize base type scala.collection.immutable.Map<java.lang.String,java.lang.String> as scala.collection.immutable.ListMap$Node, problem: Type parameter #1/2 differs; can not specialize java.lang.String with 
java.lang.Object (through reference chain: SerializationTest$SampleCaseClass["map"])     
[info]   at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:394)
[info]   at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:353)    
[info]   at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316)
[info]   at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:727)
[info]   at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)                                                                                                                                                                                          
[info]   at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
[info]   at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
[info]   at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3905)            
[info]   at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3201)           
[info]   at SerializationTest.serialize(SerializationTest.scala:67)                                               
[info]   ...                                                                                    
[info]   Cause: java.lang.IllegalArgumentException: Failed to specialize base type scala.collection.immutable.Map<java.lang.String,java.lang.String> as scala.collection.immutable.ListMap$Node, problem: Type parameter #1/2 differs; can not specialize java.lang.String with java.lang.
Object                                                                                                                     
[info]   at com.fasterxml.jackson.databind.type.TypeFactory._bindingsForSubtype(TypeFactory.java:432)
[info]   at com.fasterxml.jackson.databind.type.TypeFactory.constructSpecializedType(TypeFactory.java:401)
[info]   at com.fasterxml.jackson.databind.cfg.MapperConfig.constructSpecializedType(MapperConfig.java:297)
[info]   at com.fasterxml.jackson.databind.DatabindContext.constructSpecializedType(DatabindContext.java:161)
[info]   at com.fasterxml.jackson.databind.ser.BeanPropertyWriter._findAndAddDynamic(BeanPropertyWriter.java:893)                                                                                                                                                                         
[info]   at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:705)
[info]   at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
[info]   at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)          
[info]   at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
[info]   at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)

[cowtowncoder]

Hmmh. This exception actually sounds familiar -- there was tightening in resolution code in TypeFactory, but with some relaxation for specific case where there are common cases where seeming conflict should be ignored for convenence, so:

// line 471:
    private String _resolveTypePlaceholders(JavaType sourceType, JavaType actualType)
        throws IllegalArgumentException
    {
         ....
                // 14-May-2018, tatu: As per [databind#2034] it seems we better relax assignment
                //   rules further -- at least likely "raw" (untyped, non-generic) base should probably
                //   allow specialization.
                if (exp.hasRawClass(Object.class)) {
                    continue;
                }
                // 19-Apr-2018, tatu: Hack for [databind#1964] -- allow type demotion
                //    for `java.util.Map` key type if (and only if) target type is
                //    `java.lang.Object`
                if (i == 0) {
                    if (sourceType.hasRawClass(Map.class)
                            && act.hasRawClass(Object.class)) {
                        continue;
                    }
                }

but, alas, Scala's Map is not same as java.util.Map.

However, the first check (added in 2.9.6) would seem like that should avoid the problem.

[cowtowncoder]

One significant challenge is that of reproduction: test in jackson-databind can not rely on Scala core (too much baggage to add, even for tests). I think I could probably hack another exclusion here using "simple" class name... or maybe better yet, since Scala "map" is MapLikeType, I think I could just check that instead.

But if a test could be added to Scala module (for 2.9 branch ideally), I could add a tentative fix in databind, and we could verify fix for 2.9.10?

[cowtowncoder]

@dejanlokar1 I changed code in jackson-databind to allow "downgrade" of key type for all Map-like types, and I think that has a good chance to prevent problem you see. But I don't have a good way to verify this, so if you could locally build jackson-databind and test against Scala module, that'd be good. Change in databind is fine in itself so I can leave it in regardless.

[dejanlokar1]

@cowtowncoder that is great. I will try it out after vacation at the start of September.

[cowtowncoder]

@dejanlokar1 sounds good Have a nice vacation!

[dejanlokar1]

@cowtowncoder I tested the change and it works as expected.

[cowtowncoder]

Thank you again for verification.

[cowtowncoder]

Quick note: will be in 2.10.0, but no current plans to backport into 2.9 due to minor (but non-null) risk of regression.