Home - grownith/jsontex GitHub Wiki

Welcome to JSONTEx wiki

JSONTEx stand for JSON Transform EXpression

Which is an opensource query and transformation language for JSON data

Inspired by jsonpath jmespath jsonata and javascript itself, this language mainly adopt many familiar feature from those languages, but there would be some difference in details

Aims to be C# and JavaScript first for supported library, especially compatible with unity3d in common platform

For this project, we use antlr4 parser generator and g4 grammar specification

Core feature

  1. JSON parser
    1. By concept, jsontex is superset of json. So jsontex library can be used to parse pure json string on it's own
  2. Arithmetic and Boolean algebra
  3. JSON query. JSONTEx can query and filter large and complicate chunk of json and json-ish data structure and output
    1. Also JSONTEx can be used to project or aggregate data from collection

In C# it can return json primitive or dictionary or list of JSON-ish values by default. And will be extensible to support custom json-ish library, such as NewtonSoft.Json, as extension

The most simplest form of JSONTEx API is

var result = new JSONTEx("{string expression}").Evaluate(jsonInput);

Core Concept

jsontex philosophy is that; you want output in json format, so you should be able to write what you want as json expression

jsontex is superset of json

All of json syntax is baseline of jsontex and is also jsontex expression itself

This below is valid jsontex expresion

{ "fieldName" : 0.14285714 }

Yes, just a json can be expression

Below is also valid jsontex expresion

// valid jsontex, include this comment, but will not be output on the result
/* multiline and inline comment is also available with this comment syntax */
{ "fieldName" : 1 / 7 }

And will be evaluated into

{ "fieldName" : 0.14285714285714285 }

jsontex can output every json type, not only object as example obove

jsontex evaluated json output
null null
false false
1 / 7 0.14285714285714285
"1 / 7" "1 / 7"
[] [] // empty array
{} {} // empty object
["1/2",1/3,"1/5",1/7] ["1/2",0.3333333333333333,"1/5",0.14285714285714285]
1 / 5 > 1 / 7 true
"" + 1 / 5 "0.2"

jsontex use 64bit IEEE-754 as number. Which is compatible number type of both C# (double) and javascript (Number) which is primary target of jsontex library

jsontex is functional

jsontex accept one json input into variable $ and can pipe expressions with |>

|> is placeholder operator as of now. Maybe changed to | or more suitable syntax in actual version, still in consideration

By functional we means, jsontex cannot mutate input value, and all operation will consider input of operation as immutable. This is limitation that we would design any operation base on concept of functional. Any operation in jsontex expression that not a direct children will be ensured independent from each other

The most important but almost always invisible operator of jsontex in pipe operator |>

Whenever you have json input and try to evaluate it with jsontex expression from API, it would actually be the same as writing it as

{ input } |> { expression }

new JSONTEx(expression).Evaluate(input);
// is expectably the same as
new JSONTEx(input + "|>" + expression).Evaluate(null);

Another most important token is always obvious. The $ that stand for input variable. And whenever you try to evaluate plain jsontex, it would be evaluated as if you input null. And $ will be null

new JSONTEx("[$,null,true,false,5]").Evaluate();
// output : [null,null,true,false,5]
input\expr $ $ + 1 $ * 2 $ / 10 $ + " ea"
null null null null null " ea"
0.5 0.5 1.5 1 0.05 0.5 ea
"text" "text"` "text1" null null text ea
input\expr [$] { "output" : $ } [$,true,false,null]
null [null] { "output" : null } [null,true,false,null]
0.5 [0.5] { "output" : 0.5 } [0.5,true,false,null]
"text" ["text"] { "output" : "text" } ["text",true,false,null]

While pipe is just another binary operator. So we could chain multiple jsontex expression with piping operator freely

5 / 10 |> $ * 2 |> $ + 1
// output : 2

$ is scoped only on the block of pipe. It take evaluated value on the left to be input on the right of the |> operator The above example is the same as

5 / 10 |> $ * 2
// output : 1
1 |> $ + 1
// output : 2

jsontex reuse familiar operator

As the example so far. jsontex main purpose is to evaluate mathematical operation with familiar boolean and algebraic operator. We try to support almost all common operator in programming language

still in consideration of bitwise operator

algebraic operator example result
+ 1+1 2
- 3-2
-4
5 |> -$
1
-4
-5
* 5*6 30
/ 8/32 0.25
% 13%5
122.5%1
3
0.5
^ 2^3
2^0.5
8
1.4142135623730951

0 / 0 will be evaluated to null

Still in the process of consider support infinity for 1 / 0 but could be use double.MaxValue instead

boolean operator example result
! !false
!true
true
false
& true & false false
| true | false true
?: false ? 1 : 0
true ? 1 : 0
0
1

comparison operator for equality (== | !=) will compare for deep equality

relational operator (<= | < | >= | >) will be based on json collation specification for couchdb with some modification (still in consideration)

Operator precedence is almost identical to C family. Most binary operator in the same precedence will be evaluate left to right

operator
0. pipe |>
1. parentheses ( ) jsontex also has parentheses precede over most query operation
2. unary - + !
3. binary pow ^
4. Multiplicative * / %
5. Additive + -
6. Relational < <= > >=
7. Equality == !=
8. Logical AND &
10. Logical OR |
11. Propagation AND &&
12. Propagation OR ||
13. Propagation NULL ??
14. Logical Ternary ? :

For compatibility with javascript. jsontex double boolean operator will be used for value propagation.

  • chain of && will return first falsy value, or the last value
  • chain of || will return first truthy value, or the last value
  • ?? was borrowed from C#. will return first value that is not null

However jsontex falsy value is not exactly the same as javascript

jsontex javascript
null false false
false false false
0 true false
"" // empty string false false
[] // empty array false true
{} // empty object false true

So 0 && {} will return {} while 0 || {} will return 0

while single & and | is exactly boolean operator. Which means any combination of binary input between difference type will return null

Some operator is polymorphic, which means it could be used for multiple purpose base on type of input on the left value and right value. But not always, any unsupported combination result in null. Any operation on null is almost always return null

+ : concatenation description sample
string + primitive
primitive + string
any primitive
that try to operate with string
will be converted to string
then concatenated
"Hello " + 1
//"Hello 1"
collection + collection object will be converted to array by value
and concatenate as array
[1,2,"3"] + [4,5,"6"]
// [1,2,"3",4,5,"6"]
[1,2,"3"] + {"f":4,"b":"5"}
// [1,2,"3",4,"5"]
- : subtraction description sample
collection - array collection will remove member by a set of key
object use string key and array use integer key
[1,2,"3",4,5,"6"] - ["0",1,5]
// [1,"3",4,5]
{"f":4,"b":"5"} - ["f",1]
// {"b":"5"}
* : permutation description sample
collection * collection array of permutation of each member by value
left side will be on array index 0,right side is index 1
[1,"2"] * {"f":3,"b":"4"}
//[ [1,3],[1,"4"],["2",3],["2","4"] ]
/ : convolution (zip) description sample
collection / collection array of convolution of each member by value
left side will be on array index 0
right side is index 1
filling null on unmatch
[1,"2",3] / {"f":4,"b":"5"}
//[ [1,4],["2","5"],[3,null] ]
& : intersect description sample
array & collection intersect set by value [1,"2",3] & {"s":0,"f":1,"b":"2"}
//[ 1,"2" ]
object & object intersect object by key. value will become array
left side will be on array index 0
right side is index 1
remove all unmatch
{"s":0,"f":1} & {"f":0,"b":"1"}
//{ "f": [1,0] }
| : union description sample
array | collection union set by value [1,"2",3] | {"s":0,"f":1,"b":"2"}
//[ 1,"2" ]
object | object union object by key. value will become array
left side will be on array index 0
right side is index 1
filling null for unmatch
{"s":0,"f":1} | {"f":0,"b":"1"}
//{"s":[0,null],"f":[1,0],"b":[null,"1"]}

jsontex query json

Another main purpose of jsontex is to query complex json object. Queried json mostly result as array, sometime object, and possibly be single object

The simplest form is similar to jsonpath with some tweak

expression input : { "foo" : { "bar" : 5 } }
$.foo.bar 5
$.*.bar [5]
$.*.bar[0] null // while it seem unexpected,
bar is still evaluated as 5 not an array
so bar[0] is invalid
($.*.bar)[0] 5 // grouping evaluation with parentheses is possible

jsontex also support map/filter. .*. is mapping and [?_expression_] is filtering

While jsontex iterate on array. jsontex will also assign special variable @ with a value of iteration

variable description
@ JSON value that was the current evaluated value when iterate collection
@index numeric value for the index when iterate collection
@key string value for the key when iterate object, null when iterate array
@count numeric value for the length of current iterated collection
@last JSON value that was the last returned value when iterate collection, null for the first item
["a","b","c","d"] |> $.[@,@index,@key,@count]
// [ ["a",0,null,4],["b",1,null,4],["c",2,null,4],["d",3,null,4] ]

The @last variable mostly be used for scan or reduce operation

[1,3,5,7,9] |> $.[(@last ?? 0) + @] // folding addition
// [1,4,9,16,25]
[1,3,5,7,9] |> $.[(@last ?? 0) + @][-1] // summation
// 25
⚠️ **GitHub.com Fallback** ⚠️