• Joined the Discord, completed the quiz, started downloading and collecting the necessary software needed for RE/Exploit Development in both Linux and Windows 11.

GOAL: Borrow design elements from the current state of AI vulnerability research in general and as applied to PLCs. This goal is considered done when an architecture is created from the chosen design elements.

• Gave Carter and the greater team a list of articles that I have been collecting to understand the state of the art in finding vulnerabilities in PLCs using AI/ML.
• Reviewed the article here: https://arxiv.org/pdf/2605.02346 (APIOT)
  • APIOT automates vulnerability discovery, exploitation, mitigation, and verification in OT environments through a purple team framework.
  • It achieved 90% mission success on emulated Cortex-M3 and Cortex-M4F targets with intentionally vulnerable CoAP and Modbus application parsers.
  • A controlled ablation showed that its rule-based overseer improved reliability to 100%.
  • The framework provided protocol primitives for MCU interaction and command execution tools for Linux hosts, but did not isolate the contribution of primitive tooling.
  • The evaluation focused on DoS and actuation.
  • Potential further reading for this upcoming week:
    • https://arxiv.org/abs/2310.11409
    • https://arxiv.org/abs/2404.08144
    • https://www.usenix.org/conference/usenixsecurity24/presentation/deng
  • Gaps observed
    • This focused primarily on runtime/reported status analysis versus static or binary analysis.

GOAL: Borrow design elements from the current state of AI vulnerability research in general and as applied to PLCs. This goal is considered done when an architecture is created from the chosen design elements.

• Reviewed the article here: https://arxiv.org/pdf/2504.10112v2
  • Happe and Cito reviewed 19 articles covering 18 offensive security prototypes and their testing environments.
    • All 18 measured whether a task succeeded, but only six tracked progress through individual steps and only one included a human baseline.
  • The authors recommended documenting model settings and tool configurations, establishing baselines, and repeating tests to make results more reliable. My designed experiment will incorporate this.
  • Progress should be measured by what the agent accomplished.
  • Using vulnerabilities disclosed after a model’s training cutoff does not rule out contamination, since related research or exploit information may have been available earlier. Something I suspect also is that organizations tend to have best practices and individuals have their own preferences when coding this means that if a vulnerable segment of code exists, it is likely to appear elsewhere.
  • This supports building a tools-only baseline for our project before measuring whether AI improves the results.
  • Gaps observed
    • This focused on how offensive security tools are evaluated rather than directly testing whether AI improves vulnerability discovery.
    • The reviewed environments focused primarily on Linux, Windows, web applications, and CTF challenges versus PLC firmware.
    • Some reported counts were inconsistent. Table 3 listed 26 tasks for the NYU CTF dataset, while the text listed 200.

• No Entry Family Emergency

• GOAL: Design an analysis system architecture for this project that maximizes verifiability and handles redundant and timed-out tool calls. This goal is considered done when open questions I-IV, VI, and VII are answered.
• Created a diagram showing the working architecture for the experimental design. I will update it as findings from the literature suggest architectural changes might benefit the efficiency or accuracy of our results.
• Architectural Design Elements
  • Research Inputs: Firmware images and revisions, protocol specifications, traffic captures, target configurations, and lab observations.
  • Workflow: Research inputs -> analysis agents -> candidate findings -> evidence coordinator -> verifier -> reproduction package -> manual human verification -> Evidence.
  • Agents
    • Overseer / Runner: Plans campaigns, schedules work, manages harness construction and test setup, and handles hangs, timeouts, and retries.
    • Component Identification: Identifies embedded components and versions.
    • Delta Review: Examines security-relevant changes between builds.
    • Protocol Analysis: Interprets traffic and maps services to handlers.
    • Reachability Analysis: Traces external inputs to candidate functions.
    • Code Analysis: Identifies defects and trigger conditions.
    • Cryptography and Authentication: Examines trust checks and credential handling.
    • Failure Triage: Investigates failures in the target or harness.
    • Evidence Coordinator: Correlates findings across agents by linking network traffic analysis (NTA), payloads, candidate findings, firmware components, and the code that handles the input, then checks for gaps, combines duplicates, and prioritizes verification.
    • Verifier: Recreates the reported crash or effect and records the setup, test steps, logs, and results in a reproduction package.
  • Evidence
    • Findings are added only after the verifier confirms the result and a human manually verifies the reproduction package.
    • Each record includes the verified finding, linked traffic and payloads, firmware and code locations, hashes, tool outputs, configurations, and verification results.
    • Candidate findings remain separate from Evidence while they are being investigated.
  • Training
    • Analysis knowledge includes protocol specifications, vendor documentation, vulnerability classes, decompiler patterns, and reviewed findings.
    • Each agent has instructions for its role, tools, inputs, and outputs.
    • Training grows as reviewed findings are added, and each run records the version and material it used.
• Design Decisions
  • APIOT, Reviewed in Week 2
    • Influenced the overseer's handling of repeated actions and stalled execution, along with the use of tool results to check reported outcomes.
    • I expanded the overseer's role to include campaign planning and harness construction.
  • Happe and Cito, Reviewed in Week 3
    • Influenced the plan to record model and tool settings, record results at each step, repeat tests, and include a tools-only baseline.
• Open Questions
  1. How should the system be orchestrated?
  2. How should the overseer assign work and track dependencies?
  3. How should agents exchange candidate findings and supporting records?
  4. How should the coordinator establish links between traffic, payloads, firmware, and input-handling code?
  5. Which tasks can run in parallel?
  6. How should failed or incomplete tasks return for more work?
  7. Where should we store candidate findings and failed verification attempts?
  8. Which evaluation metrics should we use?
• Supporting Materials

• GOAL: Choose 5069-L320ER firmware revisions for comparison and find out how to download them. This goal is considered done when the revisions, notices, and access blocker are documented.
• Used Rockwell's Product Compatibility and Download Center (PCDC) to look for firmware for the 5069-L320ER, Series A, a CompactLogix 5380 controller.
• Created a Rockwell account with my Georgia Tech address. The Engineer role was approved, but downloads remain blocked because the account is not linked to an approved location. I selected ten revisions but have not downloaded the files yet.
• Firmware Selection
  • Selected ten revisions: 38.011, 37.013, 37.012, 37.011, 36.014, 36.013, 36.011, 35.017, 35.016, and 35.011.
  • This gives me revisions across four major lines and four adjacent comparisons within a line: 37.011/37.012, 37.012/37.013, 36.013/36.014, and 35.016/35.017.
  • The green checks in the PCDC grid show compatibility with Studio 5000 Logix Designer. They do not show whether my account can download the firmware.
• Important Notice
  • Seven selected revisions showed an Important Notice. I opened all seven and found the same notice each time, pointing to Rockwell's PN1494 VxWorks advisory about the URGENT/11 CVE group.
  • Rockwell's advisory identifies the CompactLogix 5380 family as using the VxWorks real-time operating system (RTOS), so I know the RTOS before opening a firmware image. I plan to look for the VxWorks network stack components in the firmware images and compare them across revisions.
  • PN1494 lists 30.015, 31.013, or 32.013 and later as the upgrade path for CompactLogix 5380 (5069-L3). Every selected revision is above those levels. The notice still appears beside 38.011, so its presence alone does not mean that revision is affected.
  • Rockwell's affected-products table marks eight CVEs for the 5069-L3 family: CVE-2019-12256, -12258, -12260, -12261, -12262, -12263, -12264, and -12265. None of the selected revisions gives me a known affected version for this advisory.
• Acquisition Blocker
  • I asked Hayden on Discord on October 6 for the BPID needed to link my account to the correct location.
  • Some retired revisions show Available downloads instead of Download firmware now. I do not yet know whether that link will provide the image once my account is linked.
• Open Questions
  • Can I download the retired revisions through Available downloads?
  • Why do some newer revisions in the 35, 36, and 37 lines lack the notice while earlier revisions display it?
• Next Step
  • Resolve the account-location issue. For each acquired file, keep the vendor filename and record the catalog number, revision, download date, source link, file size, and SHA-256 hash before analysis.
• Supporting Materials
  • Rockwell PN1494 advisory

• GOAL: Review OSS-CRS for design elements we can use in our PLC research system. This goal is considered done when the design elements, results, and gaps are documented.
• Reviewed the article here: OSS-CRS: Liberating AIxCC Cyber Reasoning Systems for Real-World Open-Source Security.
  • OSS-CRS is a framework for running and combining cyber reasoning system (CRS) components locally, outside the infrastructure used during DARPA's AI Cyber Challenge (AIxCC).
  • The framework integrates components from all seven finalist teams. Table 1 shows that none of the original systems provided interfaces for combining their components with those of other systems.
  • Work moves through three phases: prepare reusable container images -> build the target -> run the selected components.
  • Components exchange seeds, crash-triggering inputs, patches, and candidate findings through shared files. A separate exchange service handles synchronization and deduplication using content hashes.
  • An LLM proxy routes model requests and enforces API budgets. I want to track our AI API usage and cost.
  • Running ATLANTIS through OSS-CRS found seven previously unknown bugs across six of eight tested projects. The authors rated one high severity, two medium, and four low. Crashes in the other two projects were not verified as security bugs.
  • Each campaign had a 24-hour timeout, 16 CPU cores, 64 GB of RAM, and a $50 LLM budget cap. The paper does not report actual LLM spending.
  • Our proposed system fits the CRS approach. We will start with discovery and verification, with code fixes and remediation controls in a later phase. Patching a PLC is not always possible, feasible, or economically sensible.
  • The artifact exchange approach matches the direction of the architecture I drew in Week 5. I want our analysis scripts to produce the supporting evidence and retain them across runs, with hashes and run identifiers so we can count how often each finding appears across runs.
  • I will rename the analysis material labeled Training in Week 5 to Knowledge Base. Findings will be added after the verifier confirms them and a human manually verifies the reproduction package. Each agent will still have instructions for its role.
  • I am considering runs with an empty knowledge base, past CVEs, our discoveries only, and past CVEs plus our discoveries. I have not decided whether to include the discoveries-only condition.
  • For our project, I am planning three levels: preparation covers acquisition and setup, a campaign groups repeated runs, and a run is one execution. I am starting with an arbitrary range of 50 to 100 runs per campaign and will adjust it based on diminishing returns in new verified findings. For diffing, preparation needs to cover the revisions being compared, which may involve pairs or several revisions.
  • Our targets are closed-source PLC firmware. I plan to investigate decompilation and recompilation for analysis and testing.
• Gaps Observed
  • OSS-CRS expects a standardized build script, a containerized environment, a supported language, and at least one fuzz harness. Our PLC firmware images do not come with source code, a build script, a containerized test environment, or a harness, so we cannot use that workflow directly.
  • The authors evaluated ATLANTIS against open-source projects. We still need to test whether the approach works on PLC firmware. The paper also leaves the benefit of combining components unmeasured.
  • Different inputs can trigger the same defect. Recording that defect as several findings would inflate the unique-finding count and split its recurrence across separate records. I am still working out how to handle this. My current approach is to keep the full record and hash a chosen subset of fields, but I have not decided which fields to use or how to check that they identify the same defect.
• Open Questions
  • How should we determine whether different crashes have the same root cause?
  • How can we check whether decompiled and recompiled code behaves the same as the original firmware for the tests we run?
  • Should firmware diffing be a separate experimental condition, and how should we handle comparisons across several revisions?
  • Which verified findings should enter the knowledge base, and when should they become available to later runs?
  • How should test inputs be delivered to the target? We have not chosen between calling a lifted function directly, injecting packets at an emulated network interface, or sending packets to a live controller. Each would require a different setup and implementation.
• Supporting Materials
  • Week 5 architecture diagram.
  • OSS-CRS paper.