Unexpected parsing error in json module - ebranca/owasp-pysec GitHub Wiki

Classification

• Affected Components : json

• Operating System : Linux

• Python Versions : 2.6.x, 2.7.x, 3.1.x, 3.2.x

• Reproducible : Yes

Source code

import sys
import json

try:
    b = json.loads('42')
    print ("JSON LOADS STRING %r") % (b,)
except Exception:
    print "JSON UNABLE TO LOAD STRING"

try:
    c = json.dumps(int(3.141592))
    print ("JSON LOADS INTEGER %r") % (c,)
except Exception:
    print "JSON UNABLE TO LOAD INTEGER"

try:
    d = json.dumps(float(3.141592))
    print ("JSON LOADS FLOAT %r") % (d,)
except Exception:
    print "JSON UNABLE TO LOAD FLOAT"

try:
    e = json.dumps(complex(3.141592))
    print ("JSON LOADS COMPLEX! %r") % (e,)
except Exception:
    print "JSON UNABLE TO LOAD COMPLEX"

try:
    f = json.dumps([3], [5], [7](/ebranca/owasp-pysec/wiki/3],-[5],-[7))
    print ("JSON LOADS LIST! %r") % (f,)
except Exception:
    print "JSON UNABLE TO LOAD LIST"

try:
    g = json.dumps({ 'abc': 456 })
    print ("JSON LOADS DICTIONARY! %r") % (g,)
except Exception:
    print "JSON UNABLE TO LOAD DICTIONARY"

try:
    h = json.dumps(bytearray("hello"))
    print ("JSON LOADS BYTEARRAY! %r") % (h,)
except Exception:
    print "JSON UNABLE TO LOAD BYTEARRAY"

sys.exit(0)

Steps to Produce/Reproduce

To reproduce the problem copy the source code in a file and execute the script using the following command syntax:

$ python -OOBRtt test.py

Alternatively you can open python in interactive mode:

$ python -OOBRtt <press enter>

Then copy the lines of code into the interpreter.

Description

According to rfc4627 a JSON object is either a list or a dictionary with other elements in it.

However

Execution of the test script produces the following output.

JSON LOADS STRING 42
JSON LOADS INTEGER '3'
JSON LOADS FLOAT '3.141592'
JSON UNABLE TO LOAD COMPLEX
JSON LOADS LIST! '[3], [5], [7](/ebranca/owasp-pysec/wiki/3],-[5],-[7)'
JSON LOADS TUPLE! '[1, 2, 3]'
JSON LOADS DICTIONARY! '{"abc": 456}'
JSON UNABLE TO LOAD BYTEARRAY

The expected behaviour for json module would be to load only lists and dictionaries but Python's implementation took liberty to parse also basic types, like integers, strings, floats, etc.

EXPECTED BEHAVIOUR

JSON LOADS LIST! '[3], [5], [7](/ebranca/owasp-pysec/wiki/3],-[5],-[7)'
JSON LOADS DICTIONARY! '{"abc": 456}'
JSON UNABLE TO LOAD BYTEARRAY
JSON UNABLE TO LOAD COMPLEX

UNEXPECTED BEHAVIOUR

JSON LOADS STRING 42
JSON LOADS INTEGER '3'
JSON LOADS FLOAT '3.141592'
JSON LOADS TUPLE! '[1, 2, 3]'

To note that a new RFC has been proposed that would relax the requirements to make them more similar to what is the de-facto standard json implementation used in common web browsers.

The new superseding JSON RFC can be found at the link:

https://tools.ietf.org/html/rfc7159

The errata to the new RFC at this link:

http://www.rfc-editor.org/errata_search.php?rfc=7159

And the reference standard "ECMA-404" at the link:

http://www.ecma-international.org/publications/standards/Ecma-404.htm

Workaround

We are not aware on any easy solution other than trying to avoid using 'json' in cases like the one examined.

Secure Implementation

WORK IN PROGRESS

References

[Python json module][01] [01]:https://docs.python.org/2/library/json.html

[RFC 4627][02] [02]:http://www.ietf.org/rfc/rfc4627.txt

[RFC 7159][03] [03]:https://tools.ietf.org/html/rfc7159

[RFC 7159 Errata][04] [04]:http://www.rfc-editor.org/errata_search.php?rfc=7159

[Standard ECMA-404][05] [05]:http://www.ecma-international.org/publications/standards/Ecma-404.htm

[Python bug 13212][06] [06]:http://bugs.python.org/issue13212

[Python bug 21514][07] [07]:http://bugs.python.org/issue21514