Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache Serialization serializes empty contents #90

Closed
wlfbck opened this issue Jun 26, 2021 · 6 comments
Closed

Cache Serialization serializes empty contents #90

wlfbck opened this issue Jun 26, 2021 · 6 comments
Milestone

Comments

@wlfbck
Copy link

wlfbck commented Jun 26, 2021

I'm trying to serialize a Guava Cache<String,String>, so nothing fancy. But this seems to either fail (using default guava version 21.0) with an InvalidDefinitionException or return {} (using guava 30.1.1-jre).

Here's my minimal example, i hope i'm just stupid and missing something here:

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.guava.GuavaModule;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;

public class Testy {
	public static void main(String[] args) throws Exception {
		Testy test = new Testy();
	}
	
	private Cache<String, String> cache;
	
	public Testy() throws Exception {
		cache = CacheBuilder.newBuilder().build();
		
		for(int i = 0; i < 10; i++) {
			cache.put("abc"+i, "def");
		}
		
		ObjectMapper mapper = new ObjectMapper().registerModule(new GuavaModule());
		
		System.out.println(mapper.writeValueAsString(cache));
		System.out.println(cache.size());
	}
}

My pom is minimal:

<project xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>Test</groupId>
	<artifactId>Test</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<dependencies>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-core</artifactId>
			<version>2.12.3</version>
		</dependency>
		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava</artifactId>
			<version>21.0</version>
		</dependency>
		<dependency>
		    <groupId>com.fasterxml.jackson.datatype</groupId>
		    <artifactId>jackson-datatype-guava</artifactId>
		    <version>2.12.3</version>
		</dependency>
	</dependencies>
</project>
@cowtowncoder
Copy link
Member

I don't think Guava datatype module has explicit support for Cache, which would suggests it was dealt as just any old POJO.

But assuming handling was added, typically cache types would be considered transient in the sense that only an empty instance would be serialized/read. Would you expect fully contents to be serialized/deserialized back?
This behavior could be made configurable, I think, for GuavaModule, if that is desired.
It should be possible to make it work similar to regular Maps, but that is likely significantly more work than just doing minimum of creating an empty instance.

@wlfbck
Copy link
Author

wlfbck commented Jun 26, 2021

I see, saw what i'm seeing with Guava 30.1.1 is expected, right?

Would you expect fully contents to be serialized/deserialized back?

Yeap :) It would be very desirable to have a possibility to serialize these caches easily.

Maybe i can sketch my use case a little: We have system connected to industrial machinery. Connected means we are also on the power supply of said machinery. These kinds of machines are simply switched off, so basically there is no real shutdown happening, we are just "gone" at one point. The intention right now is to send several small packets over the network to some server, but in case the server is not reachable for a moment, this stuff needs to be cached. Here enters guava cache. Easy to use, fast, easy to configure eviction policy, fits perfectly.

Only part missing is that i need to make it persistent so i can restore that cache when we get turned on again. The persistence doesn't have to 100% perfect, that's why i was aiming to just serialize it into JSON and save it with the other stuff we already save for this type of getting shutdown.

@cowtowncoder
Copy link
Member

Ok, so yes; 21.0 fails to serialize since there are no properties found. 30.x is interesting since API itself is not different, but I am guessing LocalCache$LocalManualCache possibly implements some tag interface that prevents Jackson from throwing exception (not sure what that'd be).

As to supporting Cache instances, things gets trickier.

First of all: supporting serialization would be relatively easy:

  1. Serializing all Cache instances as empty JSON Object would be very easy
  2. ... or, as an alternative, could make it "ignorable type", something that would actually be skipped (if there is no value to serialize any value)
  3. Serializing contents would be more work but doable; would essentially need to replicate what MapSerializer does

However trying to support deserialization would be much more work:

  1. Supporting various subtypes would require code for creating instances in cases where caller specifies type; as well as for polymorphic handling, if any -- this regardless of whether there are contents
  2. Deserializing typed contents (not to mention non-String key types) would also be more work: MapDeserializer does that, so it'd be doable, just quite a bit of work.

I probably won't have time to work on this, except if it made sense to support "serialize as empty Object" which seems like a small step that might be useful -- and if so, perhaps counterpart on deserialization side.
I could help with a PR if anyone wants a challenge, however.
Serializers/deserializers for existing Guava Maps (Immutable-) could be a useful starting point, supporting most things general JDK Map[De]Serializer does.

@cowtowncoder
Copy link
Member

@wlfbck Instead of directly serializing/deserializing Cache instances, what would probably work well and be easy enough to support would be to use a wrapper: basically declaring sort of external type to be Map<K,V> and handling conversion yourself.

So something like:

public class CacheSerialization {
   private Cache<String, Value> cache;

   @JsonCreator(mode = JsonCreator.Mode.DELEGATING)
   public CacheSerialization(Map<String, Value> map) {
       // build cache instance
   }

   @JsonValue
   protected Map<String, Value> external() {
      // construct and return `Map` with cache contents
   }
}

(or, if you prefer, just a logical property with Map-valued getter and setter)

The idea here being that the hard part of serializing/deserializing entries is fully implemented by JDK Map / Jackson Map[De]Serializer -- and adding converters between that and Cache is simple enough at Java level.

@cowtowncoder
Copy link
Member

Added CacheSerializer which for now will explicitly write empty Object (as opposed failing or doing that depending on version).

@cowtowncoder cowtowncoder added the good first issue Issue that seems easy to resolve and is likely a good candidate for contributors new to project label Mar 6, 2023
@cowtowncoder
Copy link
Member

Might be simple enough to implement, marking as "good first issue"

@cowtowncoder cowtowncoder changed the title Cache Serialization throws InvalidDefinitionException / returns empty Cache Serialization serializes empty contents May 1, 2023
@cowtowncoder cowtowncoder added 2.16 and removed good first issue Issue that seems easy to resolve and is likely a good candidate for contributors new to project labels May 1, 2023
cowtowncoder added a commit that referenced this issue May 1, 2023
@cowtowncoder cowtowncoder added this to the 2.16.0 milestone May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants