Home - grownith/jsontex GitHub Wiki
Welcome to JSONTEx wiki
JSONTEx
stand for JSON Transform EXpression
Which is an opensource query and transformation language for JSON data
Inspired by jsonpath
jmespath
jsonata
and javascript
itself, this language mainly adopt many familiar feature from those languages, but there would be some difference in details
Aims to be C# and JavaScript first for supported library, especially compatible with unity3d in common platform
For this project, we use antlr4 parser generator and g4 grammar specification
- JSON parser
- By concept, jsontex is superset of json. So jsontex library can be used to parse pure json string on it's own
- Arithmetic and Boolean algebra
- JSON query. JSONTEx can query and filter large and complicate chunk of json and json-ish data structure and output
- Also JSONTEx can be used to project or aggregate data from collection
In C# it can return json primitive or dictionary or list of JSON-ish values by default. And will be extensible to support custom json-ish library, such as NewtonSoft.Json
, as extension
The most simplest form of JSONTEx API is
var result = new JSONTEx("{string expression}").Evaluate(jsonInput);
jsontex philosophy is that; you want output in json format, so you should be able to write what you want as json expression
- jsontex is superset of json
- jsontex is functional
- jsontex reuse familiar operator
- jsontex query json
All of json syntax is baseline of jsontex and is also jsontex expression itself
This below is valid jsontex expresion
{ "fieldName" : 0.14285714 }
Yes, just a json can be expression
Below is also valid jsontex expresion
// valid jsontex, include this comment, but will not be output on the result
/* multiline and inline comment is also available with this comment syntax */
{ "fieldName" : 1 / 7 }
And will be evaluated into
{ "fieldName" : 0.14285714285714285 }
jsontex can output every json type, not only object as example obove
jsontex | evaluated json output |
---|---|
null |
null |
false |
false |
1 / 7 |
0.14285714285714285 |
"1 / 7" |
"1 / 7" |
[] |
[] // empty array |
{} |
{} // empty object |
["1/2",1/3,"1/5",1/7] |
["1/2",0.3333333333333333,"1/5",0.14285714285714285] |
1 / 5 > 1 / 7 |
true |
"" + 1 / 5 |
"0.2" |
jsontex use 64bit IEEE-754 as number. Which is compatible number type of both C# (double
) and javascript (Number
) which is primary target of jsontex library
jsontex accept one json input into variable $
and can pipe expressions with |>
|>
is placeholder operator as of now. Maybe changed to|
or more suitable syntax in actual version, still in consideration
By functional
we means, jsontex cannot mutate input value, and all operation will consider input of operation as immutable. This is limitation that we would design any operation base on concept of functional. Any operation in jsontex expression that not a direct children will be ensured independent from each other
The most important but almost always invisible operator of jsontex in pipe operator |>
Whenever you have json input and try to evaluate it with jsontex expression from API, it would actually be the same as writing it as
{ input } |> { expression }
new JSONTEx(expression).Evaluate(input);
// is expectably the same as
new JSONTEx(input + "|>" + expression).Evaluate(null);
Another most important token is always obvious. The $
that stand for input variable. And whenever you try to evaluate plain jsontex, it would be evaluated as if you input null
. And $
will be null
new JSONTEx("[$,null,true,false,5]").Evaluate();
// output : [null,null,true,false,5]
input\expr | $ |
$ + 1 |
$ * 2 |
$ / 10 |
$ + " ea" |
---|---|---|---|---|---|
null |
null |
null |
null |
null |
" ea" |
0.5 |
0.5 |
1.5 |
1 |
0.05 |
0.5 ea |
"text" |
"text"` | "text1" |
null |
null |
text ea |
input\expr | [$] |
{ "output" : $ } |
[$,true,false,null] |
---|---|---|---|
null |
[null] |
{ "output" : null } |
[null,true,false,null] |
0.5 |
[0.5] |
{ "output" : 0.5 } |
[0.5,true,false,null] |
"text" |
["text"] |
{ "output" : "text" } |
["text",true,false,null] |
While pipe is just another binary operator. So we could chain multiple jsontex expression with piping operator freely
5 / 10 |> $ * 2 |> $ + 1
// output : 2
$
is scoped only on the block of pipe. It take evaluated value on the left to be input on the right of the |>
operator
The above example is the same as
5 / 10 |> $ * 2
// output : 1
1 |> $ + 1
// output : 2
As the example so far. jsontex main purpose is to evaluate mathematical operation with familiar boolean and algebraic operator. We try to support almost all common operator in programming language
still in consideration of bitwise operator
algebraic operator | example | result |
---|---|---|
+ |
1+1 |
2 |
- |
3-2 -4 5 |> -$
|
1 -4 -5
|
* |
5*6 |
30 |
/ |
8/32 |
0.25 |
% |
13%5 122.5%1
|
3 0.5
|
^ |
2^3 2^0.5
|
8 1.4142135623730951
|
0 / 0
will be evaluated to null
Still in the process of consider support infinity
for 1 / 0
but could be use double.MaxValue
instead
boolean operator | example | result |
---|---|---|
! |
!false !true
|
true false
|
& |
true & false |
false |
| |
true | false |
true |
?: |
false ? 1 : 0 true ? 1 : 0
|
0 1
|
comparison operator for equality (==
| !=
) will compare for deep equality
relational operator (<=
| <
| >=
| >
) will be based on json collation specification for couchdb with some modification (still in consideration)
Operator precedence is almost identical to C family. Most binary operator in the same precedence will be evaluate left to right
operator | |||
---|---|---|---|
0. | pipe | |> |
|
1. | parentheses | ( ) |
jsontex also has parentheses precede over most query operation |
2. | unary |
- + !
|
|
3. | binary pow | ^ |
|
4. | Multiplicative |
* / %
|
|
5. | Additive |
+ -
|
|
6. | Relational |
< <= > >=
|
|
7. | Equality |
== !=
|
|
8. | Logical AND | & |
|
10. | Logical OR | | |
|
11. | Propagation AND | && |
|
12. | Propagation OR | || |
|
13. | Propagation NULL | ?? |
|
14. | Logical Ternary | ? : |
For compatibility with javascript. jsontex double boolean operator will be used for value propagation.
- chain of
&&
will return first falsy value, or the last value - chain of
||
will return first truthy value, or the last value -
??
was borrowed from C#. will return first value that is notnull
However jsontex falsy value is not exactly the same as javascript
jsontex | javascript | |
---|---|---|
null |
false |
false |
false |
false |
false |
0 |
true |
false |
"" // empty string |
false |
false |
[] // empty array |
false |
true |
{} // empty object |
false |
true |
So 0 && {}
will return {}
while 0 || {}
will return 0
while single &
and |
is exactly boolean operator. Which means any combination of binary input between difference type will return null
Some operator is polymorphic, which means it could be used for multiple purpose base on type of input on the left value and right value. But not always, any unsupported combination result in null
. Any operation on null
is almost always return null
+ : concatenation |
description | sample |
---|---|---|
string + primitive primitive + string
|
any primitive that try to operate with string will be converted to string then concatenated |
"Hello " + 1 // "Hello 1"
|
collection + collection
|
object will be converted to array by value and concatenate as array |
[1,2,"3"] + [4,5,"6"] // [1,2,"3",4,5,"6"] [1,2,"3"] + {"f":4,"b":"5"} // [1,2,"3",4,"5"]
|
- : subtraction |
description | sample |
---|---|---|
collection - array
|
collection will remove member by a set of key object use string key and array use integer key |
[1,2,"3",4,5,"6"] - ["0",1,5] // [1,"3",4,5] {"f":4,"b":"5"} - ["f",1] // {"b":"5"}
|
* : permutation |
description | sample |
---|---|---|
collection * collection
|
array of permutation of each member by value left side will be on array index 0,right side is index 1 |
[1,"2"] * {"f":3,"b":"4"} // [ [1,3],[1,"4"],["2",3],["2","4"] ]
|
/ : convolution (zip) |
description | sample |
---|---|---|
collection / collection
|
array of convolution of each member by value left side will be on array index 0 right side is index 1 filling null on unmatch |
[1,"2",3] / {"f":4,"b":"5"} // [ [1,4],["2","5"],[3,null] ]
|
& : intersect |
description | sample |
---|---|---|
array & collection
|
intersect set by value |
[1,"2",3] & {"s":0,"f":1,"b":"2"} // [ 1,"2" ]
|
object & object
|
intersect object by key. value will become array left side will be on array index 0 right side is index 1 remove all unmatch |
{"s":0,"f":1} & {"f":0,"b":"1"} // { "f": [1,0] }
|
| : union |
description | sample |
---|---|---|
array | collection
|
union set by value |
[1,"2",3] | {"s":0,"f":1,"b":"2"} // [ 1,"2" ]
|
object | object
|
union object by key. value will become array left side will be on array index 0 right side is index 1 filling null for unmatch |
{"s":0,"f":1} | {"f":0,"b":"1"} // {"s":[0,null],"f":[1,0],"b":[null,"1"]}
|
Another main purpose of jsontex is to query complex json object. Queried json mostly result as array, sometime object, and possibly be single object
The simplest form is similar to jsonpath
with some tweak
expression | input : { "foo" : { "bar" : 5 } }
|
---|---|
$.foo.bar |
5 |
$.*.bar |
[5] |
$.*.bar[0] |
null // while it seem unexpected,bar is still evaluated as 5 not an arrayso bar[0] is invalid |
($.*.bar)[0] |
5 // grouping evaluation with parentheses is possible |
jsontex also support map/filter. .*.
is mapping and [?_expression_]
is filtering
While jsontex iterate on array. jsontex will also assign special variable @
with a value of iteration
variable | description |
---|---|
@ | JSON value that was the current evaluated value when iterate collection |
@index | numeric value for the index when iterate collection |
@key | string value for the key when iterate object, null when iterate array |
@count | numeric value for the length of current iterated collection |
@last | JSON value that was the last returned value when iterate collection, null for the first item |
["a","b","c","d"] |> $.[@,@index,@key,@count]
// [ ["a",0,null,4],["b",1,null,4],["c",2,null,4],["d",3,null,4] ]
The @last
variable mostly be used for scan or reduce operation
[1,3,5,7,9] |> $.[(@last ?? 0) + @] // folding addition
// [1,4,9,16,25]
[1,3,5,7,9] |> $.[(@last ?? 0) + @][-1] // summation
// 25