Input mismatch with case-insensitive properties ('MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES')

[jetersen]

Besides the #252 I tried to adapt my XML to nonself-closing, some preprocessing is always good 🃏
Issue is somewhat similar to #255

My XML provider, accurev is not very friendly.
I would very much like to abstract as much XML as possible.
Sadly when case insensitivity is turned on the XML mapper it seems to add a rogue element.
In this case, it treats a "/" as an Element that then needs to chuck through to Depot class but chokes because of a null pointer.

I can easily treat the symptom by adding a null pointer check for JsonToken however I would like to know the root cause.

First element is detected like this:

Second element, notice the nextToken is null

Third rogue context who is apparently child to Element 😕

Caused by: java.lang.NullPointerException
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:168)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:161)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:285)

fun main(args : Array<String>) {
    val bob = "hello world"
    val xml = """
<?xml version="1.0" encoding="utf-8"?>
<AcResponse
    Command="show depots"
    TaskId="1260">
  <Element
      Number="1"
      Name="accurev"
      Slice="1"
      exclusiveLocking="false"
      case="insensitive"
      locWidth="128"></Element>
  <Element
      Number="2"
      Name="second accurev"
      Slice="2"
      exclusiveLocking="false"
      case="insensitive"
      locWidth="128"></Element>
</AcResponse>""".trimIndent()
    val mapper : ObjectMapper = XmlMapper()
    mapper.registerModule(KotlinModule())
    mapper.configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true)
    val response : Depots = mapper.readValue(xml)

    println(response)
}

data class Depots(
    val command: String,
    val taskId: String,
    @JacksonXmlElementWrapper(useWrapping = false)
    var element: ArrayList<Depot>
)

@JsonIgnoreProperties(ignoreUnknown = true)
data class Depot(
    @JacksonXmlProperty(isAttribute = true)
    val number : String = "",
    @JacksonXmlProperty(isAttribute = true)
    val name : String = ""
)

Full stack trace

Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: org.jenkinsci.plugins.accurevclient.model.Depots["element"]->java.util.ArrayList[2])
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:391)
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:363)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:306)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:519)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:527)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:416)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1265)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:325)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.dataformat.xml.deser.WrapperHandlingDeserializer.deserialize(WrapperHandlingDeserializer.java:113)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3011)
	at org.jenkinsci.plugins.accurevclient.PlayGroundKt.main(PlayGround.kt:40)
	... 5 more
Caused by: java.lang.NullPointerException
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:168)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:161)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:285)
	... 17 more

[jetersen]

Well seems to be the same case as #255 after a closer look. I suddenly couldn't reproduce by turning case sensitivity on and off.

[cowtowncoder]

Ok, part about case-insensitivity is important to rule out (if possible). I saw a mention of this possibly having to with namespace difference (same local name, diff ns uri); if so, that is (alas) a known problem at this point.

But I will see whether XML module has case-insensitive tests at any rate: handling is not trivial so it better be tested.

[jetersen]

I ended up using JAXB but would have preferred to use Jackson due to the Kotlin module which was SUPER awesome. Hopefully, we can fix the bug and then I can switch over 👍

[cowtowncoder]

@Casz Understandable: purely considering XML part, JAXB is awesome lib/framework, since it started with and focuses solely on XML. I hope this problem (and related) can be solved as well as there's on the other hand lots of potential being able to cross formats, languages and datatypes.

[cowtowncoder]

Ok, I was able to reproduce the issue (and added a failing unit test). Based on that I managed to improve things slightly so that instead of NPE you now get "unexpected end of input" exception.

Actual underlying reasons is still unknown and unresolved. I did observe that replace start and end tags of Element with empty <Element ... /> tag did, for some reason, actually remove the exception (although did not make things work fully as expected).
This is weird, but maybe would help figure out the underlying problem.

[cowtowncoder]

Had another look, but still not sure what causes the problem. But since property name matching was rewritten for 3.0, including handling of case-sensitivity, seems likely this might only get fixed for 3.0.
Behavior at this point is slightly better for 3.0, but not yet correct.

[cowtowncoder]

Having another look at this, finally. I think I know where to look now: chances are this has to do with unwrapped Lists; probably start/end tag matching is not doing case-insensitivity as it should.

But the tricky part is, maybe surprisingly, that low-level XML parser (streaming API impl) has no concept of case-insensitivity, yet, all handled (currently) at databind layer.
But will see if knowledge is necessary or if it'd be possible to just rely on matching of start/end elements instead.