Migrating from JSON and Newtonsoft - SmithSoft-Studios/JAZMIN GitHub Wiki

Migrating from JSON and Newtonsoft

JAZMIN is designed to slot in where you already use JSON for lists of records. Most changes are one line.

The suggested approach

  1. Keep JSON at your API edges: browsers and third parties expect it.
  2. Use JAZMIN for files, caches, exports and data you query.
  3. Change one data flow at a time. Where the two formats meet, convert with one call (toJSON / fromJSON, JazminConvert.ToJson / FromJson).

JavaScript: JSON to JAZMIN

Before After Notes
JSON.stringify(rows) JAZMIN.stringify(rows) Returns a Buffer, not a string
JSON.parse(text) JAZMIN.parse(bytes) Returns the array of rows
fs.writeFileSync(p, JSON.stringify(rows)) write(p, rows, options) Add columns, indexes, key, sortedBy as needed
JSON.parse(fs.readFileSync(p)).filter(...) [...open(p).find(filter)] Reads only what can match
Loading a big JSON file to find one record open(p).find({ id }) One chunk, not the whole file
JSON.stringify for an API response toJSON(reader, { filter, select }) JSON straight from the file
A JSON file of any size importJSONFile(jsonPath, jzmPath, options) Streams: never loads the whole file

.NET: Newtonsoft.Json to JAZMIN

Newtonsoft.Json JAZMIN Notes
JsonConvert.SerializeObject(list) JazminConvert.SerializeObject(list) Returns byte[] (binary), not string
JsonConvert.DeserializeObject<List<T>>(json) JazminConvert.DeserializeObject<List<T>>(bytes)
JsonConvert.DeserializeObject<T>(json) JazminConvert.DeserializeObject<T>(bytes) The first row
JsonSerializer.Serialize(writer, value) new JazminSerializer(settings).Serialize(stream, value)
JsonSerializer.Deserialize<T>(reader) serializer.Deserialize<T>(stream) / DeserializeEnumerable<T>(stream) Streaming
JsonSerializerSettings JazminSerializerSettings Plus key or password, codec, indexes, metadata
Formatting.Indented Formatting.Indented Applies to JSON output
NullValueHandling.Ignore NullValueHandling.Ignore Applies to JSON output
[JsonProperty("name")] [JazminProperty("name")] [JsonPropertyName] also works
[JsonIgnore] [JazminIgnore] System.Text.Json's [JsonIgnore] also works
JsonConverter JazminConverter<T> in settings.Converters, or [JazminConverter] See .NET Serializer and LINQ
CamelCasePropertyNamesContractResolver NamingStrategy = JazminNamingStrategy.CamelCase Also SnakeCase, KebabCase
DefaultValueHandling DefaultValueHandling Same values
TypeNameHandling [JsonDerivedType] on the base type Opt-in and closed: data can't name a type to load
PreserveReferencesHandling.Objects PreserveReferencesHandling.Objects $id / $ref, as Newtonsoft writes them
Reading a JSON file, then LINQ JazminConvert.FromJsonFile(json, jzm) once, then reader.AsQueryable<T>() Queries read only what they need
JSON for an API JazminConvert.ToJson(bytes, settings, filter) System.Text.Json inside
JObject / JToken JazminRow, JsonNode No JToken-style editing of the DOM

Before and after, in C#

using Jazmin;
using Jazmin.Query;
using Jazmin.Serialization;

var customers = Enumerable.Range(1, 1000).Select(i => new Customer
{
    Id = i, Name = $"Customer {i}", Country = i % 3 == 0 ? "ZA" : "NA", Balance = i * 10.25m,
}).ToList();

// Before: string json = JsonConvert.SerializeObject(customers);
byte[] bytes = JazminConvert.SerializeObject(customers);
// Before: var back = JsonConvert.DeserializeObject<List<Customer>>(json);
var back = JazminConvert.DeserializeObject<List<Customer>>(bytes)!;
Console.WriteLine($"{back.Count} customers round-tripped, last balance {back[^1].Balance}");

// Where JSON is still needed (an API response): straight from the bytes, filtered
string json = JazminConvert.ToJson(bytes, new JazminSerializerSettings { NullValueHandling = NullValueHandling.Ignore },
    JazminFilter.Eq("Id", 3) | JazminFilter.Eq("Id", 6));
Console.WriteLine(json);

// And back: JSON from a request into a JAZMIN file
byte[] fromRequest = JazminConvert.FromJson(json);
Console.WriteLine($"{JazminConvert.DeserializeObject<List<Customer>>(fromRequest)!.Count} customers from JSON");

public class Customer
{
    public int Id { get; set; }
    public string Name { get; set; } = "";
    public string? Country { get; set; }
    public decimal Balance { get; set; }
}
1000 customers round-tripped, last balance 10250.00
[{"Id":3,"Name":"Customer 3","Country":"ZA","Balance":30.75},{"Id":6,"Name":"Customer 6","Country":"ZA","Balance":61.50}]
2 customers from JSON

Converting the JSON files you already have

importJSONFile (JavaScript) and JazminConvert.FromJsonFile (.NET) stream a JSON array or JSON Lines file of any size: they read it twice (once to work out the columns, once to write the rows) and never hold it in memory. Pass columns to skip the first pass.

import fs from 'node:fs';
import { importJSONFile, open, toJSON } from '@smithsoft-studios/jazmin';

// An existing JSON file: 50,000 statement lines
const lines = Array.from({ length: 50_000 }, (_, i) => ({
  statement: `ST-${String(Math.floor(i / 100)).padStart(4, '0')}`,
  line: (i % 100) + 1,
  description: i % 7 === 0 ? 'Interest' : 'Card purchase',
  amount: ((i * 7919) % 100_000) / 100,
}));
fs.writeFileSync('statements.json', JSON.stringify(lines));

// Once: convert, sorted by statement so each statement's lines are found by chunk statistics alone
importJSONFile('statements.json', 'statements.jzm', { sortedBy: ['statement', 'line'] });
const kb = (p) => `${Math.round(fs.statSync(p).size / 1024).toLocaleString('en-US')} KB`;
console.log('statements.json', kb('statements.json'), '-> statements.jzm', kb('statements.jzm'));

// Then: one statement at a time, without loading the rest
const reader = open('statements.jzm');
console.log(reader.columns.map((c) => `${c.name}: ${c.type}`).join(', '));
console.log(toJSON(reader, { filter: { statement: 'ST-0321', line: { lte: 2 } } }));

// Nothing lost: the whole file converts back to the same JSON
console.log('round trip identical:', toJSON(reader) === JSON.stringify(lines));
statements.json 3,856 KB -> statements.jzm 148 KB
statement: string, line: int, description: string, amount: float
[{"statement":"ST-0321","line":1,"description":"Card purchase","amount":999},{"statement":"ST-0321","line":2,"description":"Card purchase","amount":78.19}]
round trip identical: true
// .NET: the same conversion
JazminConvert.FromJsonFile("statements.json", "statements.jzm", new JazminSerializerSettings
{
    Indexes = new() { ["statement"] = [JazminIndexKind.Sorted] },
});

Differences to watch for

  • Bytes, not text. SerializeObject and JAZMIN.stringify return bytes. Store them as files or blobs; base64 them only if a text channel is unavoidable.
  • Decimals stay exact. JSON numbers become float columns unless you declare decimal (or use decimal in C#). In JavaScript, decimal values come back as strings ('1520.75') so nothing is rounded.
  • Dates are real dates. A datetime column gives Date / DateTime (UTC). When importing JSON in JavaScript, ISO-8601 strings stay strings unless you pass detectDates: true or declare the column.
  • Every row has every column. Missing properties become nulls; a property that appears in only some objects makes a nullable column.
  • Nested objects and arrays go into json columns and come back as they were. To filter on a nested value, give it a column of its own; to produce nested output from flat rows, use Export Shapes.
  • Case of names. JavaScript matches column names exactly. The .NET serializer matches them case-insensitively, as Newtonsoft does.
  • Small payloads. A file has about 300 bytes of fixed overhead, so a 200-byte API body is better as JSON.
⚠️ **GitHub.com Fallback** ⚠️