Filter Language - SmithSoft-Studios/JAZMIN GitHub Wiki

Filter Language

One filter language works everywhere: JavaScript find(), .NET Find(json), the browser reader, the viewer, the command-line tool and GraphQL where arguments. .NET also has a filter builder and LINQ, which turn into the same filters.

The shape

{
  "country": "ZA",                                   // shorthand for { "eq": "ZA" }
  "age": { "gte": 18, "lt": 65 },                    // several operators on one column: AND
  "or":  [ { "name": { "icontains": "smith" } }, { "vip": true } ],
  "not": { "status": { "in": ["closed", "void"] } },
  "email": null                                      // shorthand for { "isNull": true }
}

Keys in one object are combined with AND. and, or and not nest to any depth.

Operators

Operator Matches Example Index that helps
eq Equal { country: 'ZA' } or { country: { eq: 'ZA' } } sorted
ne Not equal (and not null) { status: { ne: 'void' } }
gt, gte, lt, lte Ranges { amount: { gte: 100, lt: 500 } } sorted
in Any of a list { region: { in: ['ZA', 'NA'] } } sorted
contains Text contains (case-sensitive) { memo: { contains: 'refund' } } trigram
icontains Text contains (ignoring case) { name: { icontains: 'smith' } } trigram
startsWith Text starts with { account: { startsWith: 'ACC-1' } } sorted
isNull Is (or isn't) null { email: { isNull: true } }, or { email: null } sorted
and All of { and: [ {...}, {...} ] }
or Any of { or: [ {...}, {...} ] }
not Not { not: { region: 'ZA' } }

Without an index, every operator still works: chunks whose min/max statistics rule them out are skipped, and the rest are checked row by row. Indexes and statistics affect only speed, never results.

Long in lists. From 1.3.0, rows are checked against a hash set of the listed values, and chunk statistics against the values in order: 20,000 values against 100,000 rows take 18 ms in Node and 36 ms in .NET. Before 1.3.0, every row was compared with every listed value: the same query took 24 s and 7.3 s. On 1.2, keep in lists to a few hundred values. In .NET, LINQ's ids.Contains(x.Id) becomes an in condition.

Selective filters return whole rows cheaply (1.3.0). A filtered query decodes the filter's columns first. It decodes the columns it returns only for chunks with matching rows, and makes their text, decimal, json and binary values only for those rows. On 2.5M order lines, returning every column of one product's 2,474 lines takes 344 ms in Node (1.1 s before) and 231 ms in .NET (369 ms before).

Lists and objects (1.4). Nested columns take three more operators, each with a filter of its own: any and all on a list's items, and match on an object's fields. The conditions of one any apply to the same item, and a null list or object matches nothing:

{
  "lines": { "any": { "sku": "A", "qty": { "gt": 5 } } },  // a line with sku A and more than 5 (the same line)
  "steps": { "all": { "done": true } },                     // every step done (an empty list: yes)
  "head":  { "match": { "city": "Durban" } },               // the object's fields
  "tags":  { "any": "vip" }                                 // items that are not objects: the item itself
}

Every operator, run

import { open, write } from '@smithsoft-studios/jazmin';

write('people.jzm', [
  { id: 1, name: 'Ann Smith', country: 'ZA', age: 34, email: '[email protected]', balance: '120.50' },
  { id: 2, name: 'Bob Smithers', country: 'NA', age: 17, email: null, balance: '0.00' },
  { id: 3, name: 'Carla Jones', country: 'ZA', age: 66, email: '[email protected]', balance: '12.5' },
  { id: 4, name: 'Dumi Nkosi', country: 'BW', age: 45, email: null, balance: '980.00' },
  { id: 5, name: 'Eve SMITH', country: null, age: 29, email: '[email protected]', balance: '12.50' },
], {
  columns: [
    { name: 'id', type: 'int', nullable: false },
    { name: 'name', type: 'string', index: 'trigram' },
    { name: 'country', type: 'string', index: 'sorted' },
    { name: 'age', type: 'int' },
    { name: 'email', type: 'string' },
    { name: 'balance', type: 'decimal' },
  ],
});

const reader = open('people.jzm');
const ids = (filter) => [...reader.find(filter, { select: ['id'] })].map((r) => r.id).join(', ') || '(none)';
const examples = {
  "{ country: 'ZA' }": { country: 'ZA' },
  "{ country: { ne: 'ZA' } }": { country: { ne: 'ZA' } },
  '{ age: { gte: 18, lt: 65 } }': { age: { gte: 18, lt: 65 } },
  "{ country: { in: ['NA', 'BW'] } }": { country: { in: ['NA', 'BW'] } },
  "{ name: { contains: 'Smith' } }": { name: { contains: 'Smith' } },
  "{ name: { icontains: 'smith' } }": { name: { icontains: 'smith' } },
  "{ name: { startsWith: 'D' } }": { name: { startsWith: 'D' } },
  '{ email: null }': { email: null },
  '{ email: { isNull: false } }': { email: { isNull: false } },
  "{ balance: '12.5' }": { balance: '12.5' },
  "{ balance: { gt: '100' } }": { balance: { gt: '100' } },
  "{ id: { eq: '3' } }": { id: { eq: '3' } },
  "{ or: [{ age: { lt: 18 } }, { age: { gt: 65 } }] }": { or: [{ age: { lt: 18 } }, { age: { gt: 65 } }] },
  "{ not: { country: 'ZA' } }": { not: { country: 'ZA' } },
  "{ country: 'ZA', not: { name: { icontains: 'jones' } } }": { country: 'ZA', not: { name: { icontains: 'jones' } } },
};
for (const [text, filter] of Object.entries(examples)) console.log(text.padEnd(56), '->', ids(filter));
{ country: 'ZA' }                                        -> 1, 3
{ country: { ne: 'ZA' } }                                -> 2, 4
{ age: { gte: 18, lt: 65 } }                             -> 1, 4, 5
{ country: { in: ['NA', 'BW'] } }                        -> 2, 4
{ name: { contains: 'Smith' } }                          -> 1, 2
{ name: { icontains: 'smith' } }                         -> 1, 2, 5
{ name: { startsWith: 'D' } }                            -> 4
{ email: null }                                          -> 2, 4
{ email: { isNull: false } }                             -> 1, 3, 5
{ balance: '12.5' }                                      -> 3, 5
{ balance: { gt: '100' } }                               -> 1, 4
{ id: { eq: '3' } }                                      -> 3
{ or: [{ age: { lt: 18 } }, { age: { gt: 65 } }] }       -> 2, 3
{ not: { country: 'ZA' } }                               -> 2, 4, 5
{ country: 'ZA', not: { name: { icontains: 'jones' } } } -> 1

Rules worth knowing

  • Nulls behave as in SQL. A comparison with a null cell is false, so { country: { ne: 'ZA' } } does not match row 5, whose country is null. not is the exact opposite of what it wraps, so { not: { country: 'ZA' } } does include row 5. To include nulls in a ne, write { or: [{ country: { ne: 'ZA' } }, { country: null }] }.
  • Decimals compare by value: '12.5' matches both 12.5 and 12.50.
  • Strings are converted to the column's type, so { id: { eq: '3' } } works on an int column, and a date column accepts '2026-03-01' or '2026-03-01T12:00:00Z'. Handy when filters come from query strings.
  • Text operators compare characters exactly (no accent folding); icontains ignores upper and lower case only.
  • Unknown columns are errors: Invalid filter: unknown column 'x'. With an access key, columns the key can't see are unknown too.
  • json columns can be tested only with { isNull: true } or { isNull: false }, written out (the bare null shorthand works for them from 1.4). To filter on a value inside the JSON, give it a column of its own, or store the list or object as a nested column (1.4).

The same filter in .NET

Three ways, all producing the same filter:

using Jazmin;
using Jazmin.Query;

var columns = new[]
{
    new JazminColumn("id", JazminType.Int) { Nullable = false },
    new JazminColumn("country", JazminType.String),
    new JazminColumn("age", JazminType.Int),
};
using (var writer = JazminWriter.Create("people.jzm", columns))
{
    writer.WriteValues(1L, "ZA", 34L);
    writer.WriteValues(2L, "NA", 17L);
    writer.WriteValues(3L, "ZA", 66L);
    writer.WriteValues(4L, null, 45L);
}
using var reader = JazminReader.Open("people.jzm");
string Ids(IEnumerable<JazminRow> rows) => string.Join(", ", rows.Select(r => r["id"]));

// 1. JSON, exactly as in JavaScript
Console.WriteLine(Ids(reader.Find("""{ "country": "ZA", "age": { "gte": 18, "lt": 65 } }""")));

// 2. The builder: & (and), | (or), ! (not)
var filter = JazminFilter.Eq("country", "ZA") & JazminFilter.Gte("age", 18L) & JazminFilter.Lt("age", 65L);
Console.WriteLine(Ids(reader.Find(filter)));

// 3. Parsed once, reused: for filters that arrive as JSON (a GraphQL where argument, a query string)
var parsed = JazminFilter.Parse("""{ "or": [ { "age": { "lt": 18 } }, { "country": null } ] }""");
Console.WriteLine(Ids(reader.Find(parsed)));
Console.WriteLine(Ids(reader.Find(!JazminFilter.Eq("country", "ZA"))));
1
1
2, 4
2, 4
Builder JSON
JazminFilter.Eq("c", v), Ne, Gt, Gte, Lt, Lte { "c": v }, { "c": { "ne": v } }, ...
JazminFilter.In("c", a, b) { "c": { "in": [a, b] } }
JazminFilter.Contains("c", "x"), IContains, StartsWith { "c": { "contains": "x" } }, ...
JazminFilter.IsNull("c"), IsNull("c", false) { "c": null }, { "c": { "isNull": false } }
a & b, a | b, !a (or And, Or, Not) { "and": [...] }, { "or": [...] }, { "not": ... }

LINQ predicates (reader.Query<T>(c => c.Country == "ZA" && c.Age >= 18)) are translated into these filters as far as possible, and checked exactly on each candidate: see .NET Serializer and LINQ.

GraphQL

JAZMIN's filters are GraphQL where inputs, so a resolver passes them straight through. No GraphQL dependency is needed.

input StringFilter { eq: String ne: String in: [String!] contains: String icontains: String startsWith: String isNull: Boolean }
input IntFilter    { eq: Int ne: Int gt: Int gte: Int lt: Int lte: Int in: [Int!] isNull: Boolean }
input CustomerWhere { id: IntFilter name: StringFilter country: StringFilter and: [CustomerWhere!] or: [CustomerWhere!] not: CustomerWhere }

type Customer { id: Int! name: String country: String balance: Float }
type Query {
  customers(where: CustomerWhere, limit: Int = 50, offset: Int = 0): [Customer!]!
  customerCount(where: CustomerWhere): Int!
}
// Resolvers, for any GraphQL server (Apollo, Yoga, graphql-js...)
function columnsOf(info, reader) { // decode only the fields the query asks for
  const selections = info.fieldNodes[0].selectionSet?.selections ?? [];
  if (selections.some((s) => s.kind !== 'Field')) return undefined; // fragments: every column
  const asked = new Set(selections.map((s) => s.name.value));
  return reader.columns.map((c) => c.name).filter((name) => asked.has(name));
}

const resolvers = {
  Query: {
    customers: (_, { where, limit, offset }, context, info) =>
      [...reader.find(where ?? null, { select: columnsOf(info, reader), limit, offset })],
    customerCount: (_, { where }) => reader.count(where ?? undefined),
  },
};
  • Only the fields asked for are decoded: { id name } reads two columns however wide the file is.
  • Shared files: open the file with the caller's access key, and each query sees only that person's rows and columns.
  • .NET (HotChocolate, GraphQL.NET): serialize the where argument to JSON and call reader.Find(json, new JazminQueryOptions { Select = fields, Limit = limit }).
⚠️ **GitHub.com Fallback** ⚠️