Custom Firmware Development Guide for Full Device Emulation

---

A Note from the Author & Guide Status

I'm sharing this transparently, as recent times have been incredibly challenging. Beyond a significant financial setback from a fraudulent chargeback, I've faced multiple other difficult living and health problems that have severely impacted my ability to be online and dedicate time to projects. Frankly, continuing to create comprehensive resources like this guide has been a profound struggle amidst these personal difficulties.

This is anticipated to be the final major iteration of the main guide. For more experienced users already familiar with fundamental hardware concepts (e.g., the function of an FTDI chip), a concise, minified version will also be made available.

If you find this work valuable and are in a position to help, any form of support would be profoundly appreciated. Your generosity enables me to continue contributing to this community despite the ongoing challenges. I sincerely hope this guide has been and continues to be a valuable resource.

---

In Memoriam & Dedication

Ross Freeman (1947–1989)

This guide is humbly dedicated to the memory of Ross Freeman (1947–1989).

A visionary engineer, pioneering Michigander, and co-founder of Xilinx, Ross Freeman is widely recognized as the father of Field-Programmable Gate Array (FPGA) technology, which revolutionized computing.

In 1984, at a time when the semiconductor industry predominantly focused on fixed-function chips, Freeman dared to imagine a different paradigm: hardware that could be reprogrammed after manufacturing. His revolutionary patent (#4,870,302) and tireless advocacy for reconfigurable computing unlocked a technological paradigm that continues to transform our world four decades later.

His groundbreaking innovation enabled the rapid prototyping and deployment of custom silicon solutions without the prohibitive costs of traditional ASIC development, democratizing hardware design and accelerating technological progress across countless domains.

Today, Freeman's vision powers cutting-edge advancements in artificial intelligence, high-performance computing, telecommunications, automotive systems, aerospace applications, and numerous other fields that were merely dreams during his lifetime.

Posthumously inducted into the National Inventors Hall of Fame in 2009, his legacy endures not merely in silicon, but in the spirit of technological audacity that challenges us all to question established limitations and imagine new possibilities.

"The ultimate goal of the FPGA was to make programmable logic devices that could replace standard digital chips." — Ross Freeman

---

Introduction

Welcome to the Custom Firmware Development Guide for Full Device Emulation. This guide will walk you through the process of creating specialized firmware for FPGA-based devices to emulate other PCIe hardware.

1.1 Purpose of the Guide

The overarching goal of this guide is to empower you with the knowledge and practical skills to develop custom Direct Memory Access (DMA) firmware for Field-Programmable Gate Array (FPGA)-based devices. This specialized firmware allows your FPGA to accurately emulate the identity and behavior of other PCIe (Peripheral Component Interconnect Express) hardware devices. Such emulation is a powerful technique with profound implications across several advanced domains including Hardware Security Research, Red Teaming & Penetration Testing, System Debugging & Diagnostics, Hardware Testing & Validation, and Legacy System Support & Interoperability.

By progressing through this guide, you will gain proficiency in:

• Meticulously extracting identifying attributes and configuration details from a physical "donor" PCIe device.
• Modifying and extending existing open-source FPGA firmware frameworks (with a primary focus on the widely-used PCILeech-FPGA project) to adopt the identity of the donor device.
• Configuring and utilizing a professional FPGA development toolchain, centered around Xilinx Vivado, alongside essential code editing tools like Visual Studio Code.
• Developing a solid understanding of the PCIe architecture's layered model, the mechanics of DMA data transfers, and the nuances of crafting firmware that faithfully replicates hardware behavior at a low level.

1.2 Target Audience

This guide is tailored for individuals who already possess a foundational to intermediate knowledge of computer systems, hardware principles, and software development. The content is technically demanding and assumes a capacity for detailed, low-level work. Specifically, it caters to Firmware Developers, Hardware Engineers, Cybersecurity Professionals & Researchers (including Vulnerability Researchers, Exploit Developers, Red Team Members, Digital Forensics & Incident Responders), and FPGA Enthusiasts & Advanced Hobbyists.

Prerequisite Knowledge (Recommended):

• Solid understanding of digital logic and computer architecture.
• Familiarity with at least one Hardware Description Language (HDL), such as Verilog or VHDL.
• Basic proficiency in a Linux environment and command-line interfaces.
• Experience with a C/C++ or a scripting language (like Python).
• A conceptual understanding of operating system principles.
• Patience and a methodical approach to debugging.

1.3 How to Use This Guide

This guide is segmented into three logically progressing parts:

• Part 1: Foundational Concepts: Introduces core terminology, principles, hardware/software setup, and initial procedures.
• Part 2: Intermediate Concepts and Implementation: Covers advanced firmware customizations, PCIe parameters, device-specific capabilities, and TLP emulation.
• Part 3: Advanced Techniques and Optimization: Explores sophisticated debugging, optimization, troubleshooting, and best practices.

Working Through the Guide:

• Sequential Progression: Follow sections in order.
• Hands-On Practice: Actively perform steps and experiments.
• Adapt to Your Environment: Understand concepts to adapt instructions.
• Consult External Resources: Refer to PCIe specs and FPGA documentation.
• Iterative Development: Expect to iterate, debug, and refine.

---

Table of Contents

Part 1: Foundational Concepts

1. Introduction (Covered on this Home Page)
2. Key Definitions
3. Device Compatibility
   • Supported FPGA-Based Hardware
   • PCIe Hardware Considerations
   • System Requirements
4. Requirements
   • Hardware Requirements
   • Software Requirements
   • Environment Setup
5. Gathering Donor Device Information
   • Using Arbor for PCIe Device Scanning
   • Extracting and Recording Device Attributes
6. Initial Firmware Customization
   • Modifying Configuration Space
   • Inserting the Device Serial Number (DSN)
7. Vivado Project Setup and Customization
   • Generating Vivado Project Files
   • Modifying IP Blocks

Part 2: Intermediate Concepts and Implementation

8. Advanced Firmware Customization
   • Configuring PCIe Parameters for Emulation
   • Adjusting BARs and Memory Mapping
   • Emulating Device Power Management and Interrupts
9. Emulating Device-Specific Capabilities
   • Implementing Advanced PCIe Capabilities
   • Emulating Vendor-Specific Features
10. Transaction Layer Packet (TLP) Emulation
    • Understanding and Capturing TLPs
    • Crafting Custom TLPs for Specific Operations

Part 3: Advanced Techniques and Optimization

11. Building, Flashing, and Testing
    • Synthesis and Implementation
    • Flashing the Bitstream
    • Testing and Validation
12. Advanced Debugging Techniques
    • Using Vivado's Integrated Logic Analyzer (ILA)
    • PCIe Traffic Analysis Tools
13. Troubleshooting
    • Device Detection Issues
    • Memory Mapping and BAR Configuration Errors
    • DMA Performance and TLP Errors
14. Emulation Accuracy and Optimizations
    • Techniques for Accurate Timing Emulation
    • Dynamic Response to System Calls
15. Best Practices for Firmware Development
    • Continuous Testing and Documentation
    • Managing Firmware Versioning
    • Security Considerations

Supplementary Pages

16. Additional Resources
17. Contact Information
18. Support and Contributions