String with NULs and ctypes - ebranca/owasp-pysec GitHub Wiki

Classification

Affected Components : ctypes
Operating System : Linux
Python Versions : 2.6.x, 2.7.x
Reproducible : Yes

Source code

import ctypes
import sys

class A(ctypes.Structure):
	_fields_ = [('b',ctypes.c_char * 1024)]

print "ctypes version:", ctypes.__version__ 

a = A()
stringnulls = 'x\000y\000'
print "Original String with nulls => %s" % (repr(stringnulls))
print "Interpreted String supposed with nulls:" , stringnulls
a.b = stringnulls
# Should print both X and Y but prints only Xs
print "a.b=", a.b
# Print the type of the object
print "type(a.b)", type(a.b)
buf = buffer(a)
print "Content of the buffer has no 'y' but only 'x' !"
for i in buf[:10]:
	print((i), (ord(i)))

Steps to Produce/Reproduce

To reproduce the problem copy the source code in a file and execute the script using the following command syntax:

$ python -OOBRtt test.py

Alternatively you can open python in interactive mode:

$ python -OOBRtt <press enter>

Then copy the lines of code into the interpreter.

Description

Reading the source code seems that the ctypes modules truncates strings containing NUL characters when assigning to structure fields of type c_char*1024.

What is the difference between NUL and NULL?

'NULL' is the null pointer
'NUL' is the ASCII character represented by '\0'

When a string containing NUL is assigned to a char array is tranformed from 'x\000y\000' to x.

'x\000y\000' ==> ctypes char array ==> 'x'

By checking the buffer output of the script we can assume that the part of string after '\000' has not been copied at all into the array.

In the source code we can also find several important clues:

In Modules/_ctypes/cfield.c the interested part of code uses a size = strlen(data) which is questioned directly in a comment. This call does not guarantee that all bytes are copied while using Py_SIZE(value) would copy all the bytes.
The library is operating in string mode rather than buffer mode, this condition would generate other problems as for example it adds a byte for a terminating NUL. As a result if a 5-byte value is passed a new 6-byte value is generated and this is obviously wrong.
Even by changing s_set to use Py_SIZE would not be possible to see the copied bytes when accessing the attribute because the code in s_get skips out at the first NUL byte. Given that next operation would be to construct using PyBytes_FromStringAndSize and the truncated size the original string would be lost anyway.

If we run the test code using python 2.7.7 we can see the effect:

python -OOBRtt test.py 
ctypes version: 1.1.0
Original String with nulls => 'x\x00y\x00'
Interpreted String supposed with nulls: xy
a.b= x
type(a.b) <type 'str'>
In the content of the buffer there is no 'y' but only x !
('x', 120)
('\x00', 0)
('\x00', 0)
('\x00', 0)
('\x00', 0)
('\x00', 0)
('\x00', 0)
('\x00', 0)
('\x00', 0)
('\x00', 0)

Meybe this behavior was intended and designed to avoiding the display of large quantities of NUL, but this could lead to data corruption or even data loss in some cases.