ObjectMapper.valueToTree does not work with @JsonRawValue

[pimlottc]

The handy valueToTree shortcut method on ObjectMapper does not appear to support pojos with the @JsonRawValue annotation:

java.lang.IllegalArgumentException: Called operation not supported for TokenBuffer

Looking at the TokenBuffer implementation of JsonGenerator used internally for this operation, I find that all the writeRaw methods are unimplemented.

This annotation works as expected in a full round-trip conversion (pojo->JSON->JsonNode). The valueToTree method is described as "Functionally same" so I would expect it to support this annotation (and all others) as well.

[pimlottc]

Test case:

@Test
public void testValueToTreeConversion() {
  ObjectMapper objectMapper = new ObjectMapper();
  Object pojo = new Object() {
    @JsonRawValue
    public String getJson() {
      return "{ }";
    }
  };

  JsonNode root = objectMapper.valueToTree(pojo);

  assertTrue(root.get("json").isObject());
}

[cowtowncoder]

Alas, I don't know if this can be supported: raw values are not portable or inter-operable, and buffering them as such does not necessarily make sense. They are to be used only for injecting exact content in output, using specific output format. This means that Tree Models can't really work too well with them.

[cowtowncoder]

Or, more specifically: while Tree Model could (if we wanted to) support "raw" content as special nodes, it's TokenBuffer that is problematic. It does support passing of byte[], but that content gets Base64 encoded.

[pimlottc]

Couldn't the raw value be parsed?

At the very least, the javadoc should be updated to note this caveat, as this method is not truly

Functionally same as if serializing value into JSON and parsing JSON as tree, but more efficient.

[cowtowncoder]

Raw value is just a snippet of output document, in some format (not necessarily JSON), and there is no functionality for partially parsing things. So while it might work, it is not really guaranteed. The idea of trying to parse it is interesting however. Dunno. I would be open to a patch; working for some cases would be improvement over "never works".

And yes, I think adding a javadoc note makes sense: I will add that in, regardless.

[poornerd]

+1

[pulse00]

so what's the preferred method to create unquoted values then? for example, https://github.com/l0rdn1kk0n/wicket-jquery-selectors needs to be able to generate raw values which represent javascript objects, e.g:

{
   "queryParser" : SomeGlobalObject.someproperty,
   "query": "foo"
}

With previous (< 2.4) version of jackson-databind, it was possible by using a custom JsonSerializer which simply called writeObject(String) on the JsonGenerator. But with >= 2.4, the resulting json looks like this (note the quotes):

{
   "queryParser" : "SomeGlobalObject.someproperty",
   "query": "foo"
}

As it's not possible to call any of the rawValue methods on the TokenBuffer, the above json object cannot be created with >= 2.4.

[cowtowncoder]

There is no supported mechanism for this. But if there really is strong desire to support "raw values" through Tree node representation, a new node type can be added. I am hesitant, however, since I do not want to be promoting use of unquoted values: I think that is wrong usage and should be eliminated, not facilitated.

Adding new JsonNode types is quite possible, of course, as well as defining custom (de)serializers for JsonNode. So it is technically possible to add custom handlers that would add support for raw values, regardless of whether standard databind does that or not.

[pulse00]

Are there any examples on how one would add a custom JsonNode type which can then be used by a custom JsonSerializer? All i could find is the Custom Serializer wiki - but i guess there's still a step necessary of adding a new JsonNode type for raw values?

[cowtowncoder]

It's no different than registering serializers/deserializers for any other type. For deserialization it'd be tricky to do in general (would need to override standard JsonNode deserializer, figure out logic etc), but for serialization you simply register it for the new type you are defining, and register using Serializers implementation provided by module.

I am not quite sure as to your reference on custom JsonSerializer, but if you mean something a custom serializer delegates to, the main options are to:

1. Fetch additional serializer(s) in createContextual() (and implement ContextualSerializer), by a lookup from SerializerProvider given at that point (and, if you want, info via BeanProperty concerning actual property serializer is used with)
2. Use dynamic per-call lookup directly from serialize method(s)

Of these, 1 is more efficient, but 2 is simpler to use.

Then again many custom serializers just override handling of dependant pieces without delegation; this may work if you know your exact settings and do not need to delegate to types provided by external datatype modules (like Joda or Guava)

[gsmet]

Hi,

As I'd like to use wicket-jquery-selectors with Jackson 2.5, I'm as well trying to ouput raw values using Jackson 2.5 and I haven't made it work so far.

Jackson is used by a lot of Java libraries to generate Json used as a configuration for Javascript libraries. At least, it's something very common in the Wicket world. And I'm pretty sure, it's something common in other contexts.

Having the ability to output raw values is really something needed in this case, either to reference a Javascript function or a Javascript variable.

It would be nice to have a supported way to do so, even if it requires some specific configuration.

Is the solution to add a new type really something doable. AFAICS we need to override:

public abstract JsonNodeType getNodeType();

and JsonNodeType is an enum?

Would you reconsider implementing the writeRaw* method in TokenBuffer? It looks like the easiest way to perform this and people would still be forced to write custom code to use it so it won't encourage them to do so it not really required.

If not, I would appreciate some more guidance on how to be able to do so. And it would be nice to have a nice future-proof way to do it documented in this issue.

Thanks for your feedback!

[martin-g]

+1 for supporting at least serialization of raw values

[cowtowncoder]

@gsmet On addition in TokenBuffer... I'll have to think about this. Since a common use case is to write things in buffer for later reading, and since there is no good way to expose raw values for reading yet, this is also problematic. One could either expose it as VALUE_EMBEDDED_OBJECT, or, perhaps, expose a new token type altogether. But adding a new token type could cause many more new issues, so it probably is better to figure out how to distinguish such raw values.

Could you file a separate issue for adding raw values in TokenBuffer?
I can then consider this issue as the "other part" of allowing them to be stored and accessed as JsonNode (or perhaps creating second issue later on).

If and when raw values may be buffered it'd be easier to solve the other part.

[gsmet]

Done: #737 .

[cowtowncoder]

Due to implementation of #737 and #743 it is now possible to use ObjectMapper.valueToTree() with @JsonRawValue annotated properties. Result will NOT re-parse raw value, but will rather contain an embedded RawValue node (of type POJONode, i.e. JsonToken.VALUE_EMBEDDED_OBJECT).
This node may, further, be fully serialized as JSON with mapper.writeValue() and related methods.

So, I will consider this implemented to the degree it can be, without forcing conversion to go through actual serialization, which would defeat the purpose of the method as efficient buffered transformation.