PythonAST - nus-cs4215/x-slang-t1-xz-jj GitHub Wiki

AST Generation

To restrict the syntaxes/ keywords in each of the SICPy chapters, we first generate the the abstract syntax tree of Python and then perform the restrictions based on the output of the AST see Syntax Analysis/ Restriction.

To generate the AST, we rely on Python's built in ast built-in module. To ease the parsing of the AST, we converted it into JSON format. The generation of the AST and the conversion to JSON was done natively in Python via Pyodide.js x-slang.

Example:

import ast
import json
result_ast = json.dumps(ast2json(ast.parse("` + code.replace(/\r\n/g,'\n').replace(/\n/g, '\\n').replace(/\"/g, `\'`) + `")))
result_ast

Note that additional pre-processing on the user input is required to remove unwanted escape sequences such as the annoying CRLF of Windows operating systems.

The following shows an example of the AST for the program x = 1+1:

{
    "_type": "Module",
    "body": [
        {
            "_type": "Assign",
            "col_offset": 0,
            "end_col_offset": 9,
            "end_lineno": 1,
            "lineno": 1,
            "targets": [
                {
                    "_type": "Name",
                    "col_offset": 0,
                    "ctx": {
                        "_type": "Store"
                    },
                    "end_col_offset": 1,
                    "end_lineno": 1,
                    "id": "x",
                    "lineno": 1
                }
            ],
            "type_comment": null,
            "value": {
                "_type": "BinOp",
                "col_offset": 4,
                "end_col_offset": 9,
                "end_lineno": 1,
                "left": {
                    "_type": "Constant",
                    "col_offset": 4,
                    "end_col_offset": 5,
                    "end_lineno": 1,
                    "kind": null,
                    "lineno": 1,
                    "n": 1,
                    "s": 1,
                    "value": 1
                },
                "lineno": 1,
                "op": {
                    "_type": "Add"
                },
                "right": {
                    "_type": "Constant",
                    "col_offset": 8,
                    "end_col_offset": 9,
                    "end_lineno": 1,
                    "kind": null,
                    "lineno": 1,
                    "n": 1,
                    "s": 1,
                    "value": 1
                }
            }
        }
    ],
    "type_ignores": []
}

Refer to runInContext() in x-slang/src/index.ts for the implementation.

More details about Python AST can be found here.