Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SmileGenerator.Feature.LENIENT_UTF_ENCODING for lenient handling of broken Unicode surrogate pairs on writing #276

Closed
kireet opened this issue May 7, 2021 · 1 comment
Labels
Milestone

Comments

@kireet
Copy link

kireet commented May 7, 2021

When encoding some invalid user generated data, we encounter JsonGenerationExceptions with the message Unmatched first part of surrogate pair. This didn't occur when using text serialization.

It seems there's a CBOR option to avoid this exception, CBORGenerator.Feature.LENIENT_UTF_ENCODING, but not one for Smile. It would be great to add this feature.

To reproduce, just try to serialize an invalid string, such as "\uD83D".

@cowtowncoder
Copy link
Member

cowtowncoder commented May 7, 2021

Sounds like a good idea, hoping to implement when I have time!

Note: was done for CBOR via #222.

@cowtowncoder cowtowncoder added this to the 2.13.0 milestone Jun 26, 2021
@cowtowncoder cowtowncoder changed the title Smile: Unmatched first part of surrogate pair error during serialization Add SmileGenerator.Feature.LENIENT_UTF_ENCODING for lenient handling of broken Unicode surrogate pairs on writing Jun 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants