audit python security - pprados/pysandboxes GitHub Wiki
This page is a red-team report on the Python-level guards โ the dynamic
patching that denies file, socket, import, environment and sensitive-call
access from inside the running interpreter (guard_files, guard_socket,
guard_import, guard_envs, guard_api, and the arming machinery). It
documents, objective by objective, what hostile code can and cannot reach once
a profile is armed, which layer stops each attempt, and โ crucially โ the
residual limits the layer does not claim to cover.
It is the companion of the eval-* assessment, which covers
the dynamic-code sub-language, and of weaknesses, which lists
the limits of the package in prose. This page turns those prose limits into a
structured, test-pinned matrix and names the enforcing function or class for
every row. Read implementation first if you need to know
how the patching works before reasoning about what it stops.
Verification date: 2026-10-06. The focused test modules cited by this
assessment reported 261 passed, 6 failed, 3 skipped and 4 expected failures on
Python 3.13.5. All six failures require creating or binding sockets; the
execution environment returns PermissionError: [Errno 1] Operation not permitted. This run therefore does not revalidate those socket rows. The
individual evidence labels below remain scoped to their named tests and
configurations.
The separate native guard study evaluates C wrappers and CPython audit hooks as ways to reduce introspection of original functions. It records the prototype results and why a general native layer was not pursued.
This assessment concerns the Python interception layer, not the OS boundary.
It documents both tested refusals and routes that remain open to arbitrary
Python code. Treat Blocked โ demonstrated as a result for the named test and
configuration, not as proof that every equivalent attack is blocked. Treat
Reasoned as source analysis that has not been pinned by an isolated payload.
The weaknesses page gives the short operational summary.
The two reports do not share a threat model, and conflating them would be a false guarantee.
-
In the
eval-*report the attacker controls a source string evaluated inside a bounded namespace. There the report can honestly say "every attribute read passes through_check_attr": the attacker never holds a real object reference the guard did not hand out. -
Here the attacker controls arbitrary bytecode running in the host interpreter. Any granted callable, any module object, any frame is a live Python object with full introspection. weaknesses.md states the consequence plainly: "arbitrary Python can always call native code โฆ the OS-sandboxes remain the real barrier."
So the useful columns here are not "blocked vs allowed" in an absolute sense. They are: what the layer costs the attacker, what learning mode makes visible, and which OS layer actually stops what the patching cannot. The Python layer's job is to deny sensitive operations from non-hostile code and to surface them in learning mode โ not to contain hostile bytecode. That is the OS sandbox's job (firejail, bwrap, unshare, qemu, landlock).
Naming the reader this layer is built for decides what is worth building, so it is stated here rather than left to be inferred from the matrix.
The target is code an LLM produced while solving the wrong problem, not an
adversary. Such code reaches for open, requests.get or subprocess in the
open, and stops at the first refusal. The layer exists to make that refusal
happen, to name the rule that caused it, and to record it in learning mode.
Code written to break out is explicitly out of scope, and not for lack of
effort: it needs none of the paths listed in
section 6. One ctypes call, or one
C extension, leaves every Python-level guard untouched and intact. Hardening
this layer against that reader would buy nothing the layer does not already
concede on the line above.
Two consequences, both deliberate:
-
No C extension holding the arming flag, and no PEP 578 audit hook guarding
it, are planned. Either would close one row of section 6 while the rows below
it stay open, and neither survives a reader who can call native code anyway.
The one audit hook the layer does install, in
guard_import, is not that: it keeps the import finder at the head ofsys.meta_path. Frameworks put their own finder there when they are imported (crewai, wrapt post-import hooks, httpx2 aliases, py-key-value), so overtaking the guard is the ordinary case, not an attack; the hook moves it back before each import instead of refusing. - The residual entries in section 6 are documented, not scheduled. What the layer does owe its target is that an accidental disarming be loud rather than silent.
- Two enforcement states: disarmed and armed
- Attack matrix by objective
- Guards that are inert by design
- Residual limits โ denial of service
- Reproducing the assessment
Every row is marked demonstrated (pinned by an executed test) or reasoned (established from the source, not run as an isolated payload).
Nothing is enforced until lifecycle.arm() flips a module global
(_armed, set in lifecycle.arm(), read by lifecycle.is_armed()).
Framework code runs disarmed; user code runs armed. lifecycle is the single
owner of that state: guard_api reads it through the _lc_is_armed alias (see
objective 6), and guard_socket's
_rules_loaded flag tracks a different concept.
The wrappers short-circuit when disarmed โ guard_api's control point returns
the raw function if not _lc_is_armed() (guard_api._wrap_guarded). This is by
design, but it also means the arming flag is a single point whose reset
disarms everything; see
objective 6.
| Attack | Layer | Status |
|---|---|---|
Read/write a path outside expose-ro/expose-rw
|
decision on the canonical path in guard_files._apply_dest_to_src_rules: startswith match against each FSExposeRule โ raise RulePermissionError
|
Blocked โ demonstrated: test_guard_escape_fixes.py::test_symlink_leaving_the_exposed_dir_is_denied asserts the out-of-scope secret resolves to (None, None)
|
| Symlink inside the scope pointing outside it | decision taken on the guard_files._safe_realpath-resolved path, not the raw one |
Blocked โ demonstrated (test_symlink_leaving_the_exposed_dir_is_denied[absolute,relative]) |
| Create a symlink whose target is out of scope |
guard_files._wrap_os_symlink validates the target (os.symlink in the guard's patch table) |
Blocked โ demonstrated (test_creating_a_symlink_out_of_scope_is_refused) |
Name the file relative to an open directory (dir_fd, src_dir_fd, dst_dir_fd), so a check against the current directory would see the wrong path |
guard_files._check_dir_fd resolves the descriptor (_dir_fd_path: /proc/self/fd/<n> on Linux, F_GETPATH on macOS) and applies the rules to the effective path, for os.open, metadata and access checks, symlink, and one- or two-path operations (unlink, rmdir, mkdir, rename, replace, link); unlink, rmdir and mkdir count as writes. The call keeps its original arguments: no remapping under dir_fd
|
Blocked โ demonstrated (test_guard_files_coverage.py::test_a_dir_fd_removal_is_checked_against_the_directory_it_names, ::test_body_two_filenames_denies_rename_through_a_read_only_dir_fd, ::test_wrap_os_open_denies_a_write_through_a_read_only_dir_fd, ::test_wrap_os_open_denies_an_ignored_path_with_dir_fd, ::test_wrap_os_symlink_denies_a_read_only_dir_fd; test_guard_shutil.py::test_shutil_rmtree_cannot_empty_a_read_only_tree) |
Pass a dir_fd the guard cannot resolve |
_dir_fd_path denies the call instead of falling back to the current directory |
Blocked โ demonstrated (test_guard_files_coverage.py::test_dir_fd_path_denies_a_descriptor_it_cannot_resolve) |
| Rename the directory between the descriptor lookup and the operation | the lookup and the syscall are two separate steps | Open โ reasoned (time-of-check/time-of-use race, noted in weaknesses.md); the OS sandbox is the barrier |
Use a descriptor already held (open(fd), os.open, os.stat, os.listdir, os.chdir on an int, or an inherited fd) |
an int names no path, so the wrappers pass it through: the call that opened it was checked |
Open by design โ demonstrated (test_guard_files_coverage.py::test_wrap_filename_passes_through_a_file_descriptor, ::test_wrap_os_open_passes_through_a_file_descriptor); a held descriptor is a capability, as a held dir_fd is |
Reach the original io.open via its closure cell |
wrapper keeps a reference to the original (patch-factory closure) |
Open โ demonstrated (test_escape_pysandbox.py::test_escape_with_closure asserts the original is reachable) |
The canonicalization is itself re-entrant-safe: _safe_realpath calls
os.lstat/os.readlink, which are patched, so a _canonicalizing context var
suppresses the inner re-check โ it is read at the head of
_apply_dest_to_src_rules. The path
returned to the syscall stays unresolved, so link semantics are preserved;
only the decision uses the resolved path.
| Attack | Layer | Status |
|---|---|---|
connect()/bind() to a host outside net=
|
guard_socket._check_address โ _check_address_with_rules on the patched socket.socket
|
Blocked โ demonstrated (test_guard_socket.py::test_connect_enforces_the_rules, ::test_no_rules_denied_connection) |
| AF_UNIX socket path exempted from the rules |
_check_unix_socket applies the file rules to the socket path |
Blocked โ demonstrated (test_guard_escape_fixes.py::test_unix_socket_out_of_scope_is_denied) |
bind(("", port)) wildcard host slips the rule |
guard_socket._resolve_wildcard_host rewrites "" to 0.0.0.0/:: before the check |
Blocked โ reasoned |
import _socket and build the C base class directly |
socket.socket is a Python subclass of the C _socket.socket and does not redefine connect, so patching the subclass alone would leave the base class reachable; the rule replaces the class the C module publishes with a guarded subclass of it (guard_socket.py, _guarded_socket_class) |
Blocked โ demonstrated (test_guard_socket_twin.py::test_connect_through_the_c_class_is_refused refuses the connection under python-sb) |
Resolve a name through _socket instead of socket
|
socket.py does from _socket import *, so socket.gethostbyname is _socket.gethostbyname โ the posix relationship. getaddrinfo differs: socket.py redefines it, leaving the raw C one a separate unnamed function. Both C names carry the socket wrapper |
Blocked โ demonstrated at the table (::test_every_guarded_resolution_call_has_its_c_twin; removing a twin fails it). The end-to-end check compares the two names rather than asserting a refusal: with no resolver the call dies of gaierror before the guard is consulted |
gethostbyaddr, getnameinfo
|
shared with _socket by the same star-import, but no rule guards them under either name
|
Open โ demonstrated (absent from patch_rules); reverse lookup is not covered by the DNS rules |
Reach the original class through the type graph (socket.socket.__mro__[1]) |
none โ attribute patching replaces names, not the inheritance chain | Open โ by construction, same shape as the closure cell in ยง1. Out of scope for the wayward-LLM target |
Lower-level fd handoff (socket.fromfd, socket.socket(fileno=), socketpair, an fd inherited from the parent) |
not in the socket patch table โ no _check_address on an fd the process already holds. On Windows only, socket.socketpair is patched, but to exempt rather than check: see the next row |
Open โ reasoned; contained only by the OS network namespace (unshare/netfilter) |
Take over the daemon's own sockets: socket.socket(fileno=N) on the SSE server's listening socket (then accept the host's next requests) or on the connection of the current call (then write its HTTP response) |
none โ the daemon serves the call in the process that runs the user's code, and its sockets are opened before the guards are armed. Refusing a fileno that matches them would leave os.read/os.write on the raw fd, and the server objects reachable through gc, open |
Open โ demonstrated (a @sandbox call lists the listening socket on 127.0.0.1 and the connection of the call). Same class as the closure cell in ยง1. A forged response goes through the same from_b85_restricted as a returned result, so it carries no more than the code could have returned; it can however answer the host's later calls in place of the daemon |
Connect to a loopback port from inside socket.socketpair on Windows |
the stdlib emulates socketpair over a loopback listener, so guard_socket._wrap_socket_socketpair sets the thread-local _socketpair_scope.active for the call, and _check_address lets any loopback address through while it is set |
Open โ reasoned: a third switch of the objective 6 class, reachable as a module global; loopback only, Windows only |
| Attack | Layer | Status |
|---|---|---|
os.system, os.exec*, os.fork, subprocess.Popen/run/...
|
denied by the sensitive-function registry, independently of import rights: guard_api.SENSITIVE_API, wrapped at guard_api._wrap_guarded, which raises RuleApiPermissionError
|
Blocked โ demonstrated (test_guard_api.py, test_guard_os.py) |
subprocess.Popen / ctypes.CDLL patched as classes |
patched on __init__, not the class object, to keep isinstance/subclassing (guard_api._PATCH_TARGET) |
Blocked โ demonstrated (test_guard_api.py::test_class_entry_denies_and_allows_construction) |
ctypes.CDLL.__new__(ctypes.CDLL) then setting _name, _FuncPtr and _handle = _ctypes.dlopen(...) by hand, skipping the guarded __init__ entirely |
_ctypes.dlopen (posix), _ctypes.dlsym and the Windows _ctypes.LoadLibrary are registered in the native category too, and their ctypes-module aliases (_dlopen/_LoadLibrary) patched alongside the canonical name (guard_api._ALIASES), the same double guard as _posixsubprocess.fork_exec
|
Blocked โ demonstrated (test_guard_api.py::test_ctypes_new_bypass_of_cdll_init_is_blocked, ::test_ctypes_new_bypass_is_refused_end_to_end) |
ctypes.pythonapi native calls |
it is a PyDLL instance built at import time โ no __init__ runs, so it is never wrapped (documented in guard_api, next to the registry) |
Open โ reasoned (matches weaknesses.md; OS layer's job) |
Reach an original sensitive function via __wrapped__
|
_wrap_guarded decorates with guard_wraps.guard_wraps, which drops the back-reference |
Closed โ see objective 6 for the evidence; the wrapper's closure cell still holds the original |
| Attack | Layer | Status |
|---|---|---|
Read an env var outside env=
|
os.environ filtered to the whitelist at parse time; learning wrapper guard_envs.LearnEnviron
|
Blocked โ demonstrated (test_guard_env.py::test_unknown_variable_leaves_the_key_absent, test_learn_environ.py) |
Read the parent's env via /proc/$PPID/environ
|
a file read โ subject to the file rules (objective 1) if /proc is out of scope; not otherwise |
Reasoned; the real barrier is the OS sandbox hiding /proc (noted in weaknesses.md) |
| Read process memory / another module's globals | any function's __globals__ exposes its module namespace |
Open โ demonstrated (test_escape_pysandbox.py::test_escape_with_globals_introspection, xfail-pinned as open) |
| Attack | Layer | Status |
|---|---|---|
import a module outside python-import=
|
guard_import.GuardFinder.find_spec denies via _is_import_allowed โ raise RuleModuleNotFoundError; the finder is inserted at sys.meta_path[0] when the rules are activated |
Blocked โ demonstrated (test_guard_import.py, test_guard_escape_fixes.py::test_find_spec_denies_a_module_outside_the_rules) |
Unhook the finder from sys.meta_path, or insert a finder ahead of it |
sys.meta_path stays a writable list, but guard_import._audit_import, a PEP 578 audit hook installed with the finder, sees the live list on every import event and moves the guard finder back to its head (restoring it if it was dropped) before CPython walks it, so the next import is checked and the other finders, reached through GuardFinder's delegation, keep working; a sys.meta_path that is no longer a list refuses the import. Python offers no way to remove an audit hook |
Blocked โ demonstrated (test_escape_pysandbox.py::test_tampering_with_meta_path_leaves_the_next_import_guarded, test_a_framework_finder_ahead_of_the_guard_keeps_working) |
| Empty rule set treated as "no filter" | empty _rules is deny-all, wildcard ("*",) is allow-all (guard_import._is_import_allowed) |
Blocked โ demonstrated (test_no_import_rule_denies_every_module, test_wildcard_import_rule_allows_every_module) |
Reach a module already in sys.modules (e.g. os, pre-imported by the framework) |
find_spec is not consulted for a cached module โ but its sensitive functions are still denied by objective 3 |
Reasoned: import rights and call rights are deliberately distinct (guard_api.py docstring); a reachable os module is not a reachable os.system
|
| Attack | Layer | Status |
|---|---|---|
Walk object().__subclasses__() to the guard modules and reset their state |
nothing blocks the class-hierarchy walk |
Open โ demonstrated (test_escape_pysandbox.py::test_escape_with_subclasses, xfail-pinned; reaches guard_import and sets _rules=()) |
Reset the _armed flag through that same walk |
_armed is a plain module global of lifecycle, re-read on every is_armed()
|
Open โ reasoned (same class as above; weaknesses.md names it explicitly) |
Rebind the call-site alias instead of the flag: guard_api._lc_is_armed = lambda: False
|
guard_api imports is_armed under the name _lc_is_armed, binding the function by value, so the alias is a second, independent switch read by every wrapper |
Open โ reasoned |
Reach the unguarded original through __wrapped__
|
guard_wraps.guard_wraps() replaces functools.wraps in the five guards: it restores __signature__, then drops the back-reference |
Closed (the guarded callable does not carry __wrapped__; test_guard_escape_fixes.py) |
Reach the original through the wrapper's own closure cell (__closure__[i].cell_contents) or gc.get_referents
|
a Python wrapper necessarily holds a reference to what it wraps; hiding the name does not hide the object | Open โ by construction |
Obfuscated name construction (getattr(os, "sy" + "stem"), chr() chains, hex-decoded names) |
guard_api wraps the function, not the source text, so the check runs at the call however the name was built; a name that reaches no guarded call ('__sub'+'classes__') is the __subclasses__ row above |
Closed for guarded calls (test_escape_pysandbox.py::test_a_name_built_at_run_time_is_refused_at_the_call) |
These are the defining limits of a patch-based layer, not defects to be fixed in it. They are why the OS sandbox exists.
| Attack | Layer | Status |
|---|---|---|
| Overwrite a decision dict in place | rules stored in immutable_dict.ImmutableDict, a tuple subclass with no mutating API; guard_api._allowed is one |
Blocked โ demonstrated (test_immatuable_dict.py) โ for direct mutation |
Rebind the module global that holds the ImmutableDict
|
the binding itself is a plain global, reachable via objective 6 | Open โ reasoned: immutability protects the container, not the name that points at it |
The main process talks to the sandboxed child over SSE, on local HTTP.
| Attack | Layer | Status |
|---|---|---|
| Call the SSE endpoint without the token | Bearer-token check, compared in constant time, request refused on mismatch (remote/sse_server_daemon) |
Blocked โ reasoned (token is a per-run UUID minted in python_sb) |
| Reach the endpoint from another local process | the daemon listens on 127.0.0.1 when the child shares the host network (subprocess, landlock, bwrap with the host network, firejail without --net); on every interface only inside its own network namespace or VM (unshare, bwrap with slirp4netns, firejail --net=<bridge>, qemu), where the host reaches it through the forward. Random free port (remote/client_subprocess_sse_daemon.find_free_port) + the token above |
Reasoned: raises cost; a co-resident process that can read the port and token is not stopped by this layer (weaknesses.md) |
This objective is not in weaknesses.md and is specific to the project's design, so it is stated in full.
In learning mode the verified guards do not enforce: they record the
attempt and then let the real operation proceed. For example, the sensitive-call wrapper
calls the real function anyway (guard_api._wrap_guarded), and the file and
import guards record instead of raising (guard_files._apply_dest_to_src_rules,
guard_import.GuardFinder.find_spec). Every observed file, socket, import, env
and call is serialized verbatim into the next run's profile
(learning.generate_config_from_learning), with no
validation of whether the behavior was legitimate.
Consequence: hostile code run under learning mode both executes unguarded and writes its own permissions into the whitelist the next run trusts.
| Layer | Status |
|---|---|
| The only barrier is human review: the generated file prints "Check and update this file to validate the rules", and every sample profile warns "Never use learning mode with untrusted code" |
Open by design โ demonstrated that rules are written without validation (test_learning.py::TestGenerateConfigFromLearning); the safeguard is operational, not enforced by any layer |
One subject, pickle, is covered only in part, by design. It is not a discovered vulnerability -- do not report it as an escape.
-
Pickle deserialization is covered only in part, by design.
pickle.Unpickleris_pickle.Unpickler, an immutable C type, so neither its__init__nor itsloadcan be patched. Rebinding the module name to a function would break the subclassing a restricted unpickler needs: this is the same shape asctypes.pythonapi.Unpickler(fp).load()therefore stays reachable, pinned astest_guard_api.py::test_the_unpickler_route_is_guarded(xfail).What is covered runs through
guard_api:pickle.loads,pickle.loadand their_pickletwins are registry entries under thedeserializationcategory, denied by default like every other sensitive call and grantable withpython-api=ALLOW:deserialization(warned at load, asdynamic-codeis). A pickle stream is a program: its opcodes name a callable and call it, which reachesos.systemwithout an import and without a source string theeval-*layer could parse. Pinned end to end bytest_guard_api.py::test_pickle_loads_and_its_twin_are_refused_end_to_end.Like the rest of this layer, that denies the call from cooperative code and gives learning-mode visibility. It is not a barrier against hostile bytecode, which reaches
os.systemby a hundred other routes; the OS sandbox is the barrier for that class.test_escape_with_pickle_blockedstaysxfail: removingpicklefromsys.modulesblocks nothing.The only untrusted data this package unpickles is the child's result and exception payloads, rebuilt in the trusted parent by
remote/base_sse_daemon._rebuild_remote_exceptionand the result branch of the same module's event dispatch. A crafted payload would be arbitrary code execution in the host process, outside the sandbox. Both child->parent sites therefore go throughremote/tools.py::from_b85_restricted, in two steps. First, an opcode prescan (allowlist by family plus anti-DoS budgets). Then, afind_classthat triggers no import from the stream and vets each global:- the exception channel is fail-closed (Exception subclasses,
tblib, base data types); - the result channel is a fail-open denylist of dangerous gadgets keyed by
every C-twin (
posix.system,_socket.socket); - a shape check runs on
tblib.Tracebackbeforeas_traceback().
The result guard is toggled by
remote-result-guard(default on). Being fail-open, it raises the cost of a gadget on return values rather than closing the class; the exception guard is fail-closed. Because an exception's state routinely holds a type that guard refuses (httpx.Request, aPath, an application object), the child also sends a descriptor of the exception โ class, message, denials, all primitives โ and the parent falls back to it when the rich form is refused, rather than losing the refusal itself. An exception crossing that way arrives without its attributes. The transport bindspickle.dumps/pickle.loadsat import so the framework's own serialization is not charged to the user's profile, and the registry does not cover this call. See transport-unpickle-guard.md.parent->child(args/kwargs) stays on the rawfrom_b85: there the trust runs the other way. - the exception channel is fail-closed (Exception subclasses,
These match the package's stated position: the Python layer raises cost and adds visibility; the OS sandbox is the barrier.
Exhausting CPU, memory or file descriptors is out of scope for this layer, by
the same reasoning as the eval-* report: a patch on a
Python function cannot bound native resource use. The OS sandbox (cgroups,
ulimit, the VM providers) owns denial of service.
# blocked escapes (pinned as passing) + known-open escapes (pinned as xfail)
uv run pytest tests/unit_tests/test_escape_pysandbox.py \
tests/unit_tests/guard/test_guard_escape_fixes.py -v
# the per-guard enforcement corpus grounding the "blocked" rows
uv run pytest tests/unit_tests/guard/test_guard_api.py \
tests/unit_tests/guard/test_guard_import.py \
tests/unit_tests/guard/test_guard_socket.py \
tests/unit_tests/guard/test_guard_os.py \
tests/unit_tests/test_learning.py -v
# the dir_fd and descriptor rows of objective 1
uv run pytest tests/unit_tests/guard/test_guard_files_coverage.py \
tests/unit_tests/guard/test_guard_shutil.py -v -k "dir_fd or descriptor or rmtree"The three xfail entries of the escape files are the open escapes:
test_escape_with_subclasses (objective 6), test_escape_with_globals_introspection
(objective 4), and test_escape_with_pickle_blocked (the pickle entry of
inert guards). Tampering with sys.meta_path,
a hostile pickle and a name built at run time are not among them: the import audit
hook puts the finder back at the head once it is unhooked or overtaken, as
test_tampering_with_meta_path_leaves_the_next_import_guarded shows. Once
armed, guard_api refuses pickle.loads and the call a built name reaches, as
test_a_hostile_pickle_is_refused_once_armed and
test_a_name_built_at_run_time_is_refused_at_the_call show. The corpus xfail
is test_guard_api.py::test_the_unpickler_route_is_guarded, which records that
pickle.Unpickler is an immutable C type. All are
strict=True, so if any is ever closed, the XPASS fails the suite and this page
must change with it.
Bottom line: the Python layer denies every sensitive operation from
cooperative code and pins each denial to a test. Against hostile bytecode, it
only raises cost and provides learning-mode visibility: the class-hierarchy
walk, the wrappers' closure originals, ctypes.pythonapi,
pickle, and the arming flag remain reachable by design. Containing those is
the OS sandbox's role, stated plainly rather than papered over.
- Treat the Python layer as a guardrail for code that is wrong, not hostile. Against code written to break out, use a kernel provider.
- Never use learning mode with untrusted code: it executes unguarded and writes its own permissions into the whitelist.
- Review every generated profile before trusting it. That review is the only barrier.
- Grant
python-api=ALLOW:deserializationonly when a trusted payload requires it. - Leave
remote-result-guardon, and rely on the OS sandbox for denial of service.
- Security assessment of the
eval-*guard - Transport unpickle guard, weaknesses, implementation
- Kernel providers: landlock, bwrap, firejail, unshare, qemu