NXESP Firmware Format - Threetwosevensixseven/espupdate GitHub Wiki
The Spectrum Next .espupdate dot command has a built-in default firmware (v3.3.0.1), which you can use simply by typing the following at the NextBASIC prompt:
.espupdate
However, you may often wish to update to a different or more recent firmware, or build your own firmware from source.
Sometimes, firmwares are available in multiple files, each of which are loaded to separate addresses, selected and configured in other firmware update tools.
At other times, a factory firmware is uploaded to the entire ESP flash memory, starting at address 0x00000000. For the ESP-01 modules supported by the Next and .espupdate, factory firmware files are always exactly 1MB in size. The file format is described in detail here.
.espupdate, however, compresses the factory firmware until it is around half the size, and appends a custom NXESP header. It does this to ease the burden of zlib compression on the Next, and other retro computers. Other accommodations are also made, for example the header also containts progress percentage messages, to avoid the need for 32 bit division in the client.
In the rest of this document, we will refer to these compressed files with headers as NXESP files. By convention, they always have an .esp file extension, and can be loaded from the NextZXOS browser on the Spectrum Next.
A custom tool, PackageFW.exe, is provided to turn a factory firmware file into an NXESP file. Currently the tool only runs on 32-bit and 64-bit versions of Windows, with the .NET Framework 4.7.2 installed. For linux and MacOS users, an alternative tool will be provided later.
Download the four files PackageFW.exe, zlibnet.dll, zlib32.dll and zlib64.dll into a directory. From the command line, change to this directory, and type:
PackageFW.exe path\to\factory_firmware.bin path\to\firmware\to\be\created.esp
-f=0x0221 -v=<VersionNo> -b=16384
Where the first argument is the 1MB factory firmware file, the second argument is the NXESP file you wish to create, and <VersionNo> is a version string such as 3.3.0.1. Paths can be absolute, or relative to the current directory, and must be quoted if they contain spaces. For example:
PackageFW.exe "c:\My ESP\factory_3.3.0.1.bin" ..\FW\nxesp_3.3.0.1.esp
-f=0x0221 -v=3.3.0.1 -b=16384
The tool will output information similar to this:
Packaging ESP factory firmware into NXESP format...
Running in interactive mode
Flash params: 0x0221
Version: 3.3.0.1
Block size: 0x4000
MD5 Hash: 03192f512d08b14be06b74f98e109ee0
Input firmware: 1048576 bytes
Converting to NXESP format...
Appending header: 554 bytes
Appending firmware: 457535 bytes
Output file: 458089 bytes
Writing output file...
By convention, NXESP files always have an .esp file extension. Copy the NXESP file anywhere on your Next's SD card, and navigate to the directory containing it in the NextZXOS Browser. Select the file, and press the ENTER key. .espupdate will launch with the NXESP ready to update.
Updating your ESP with an NXESP file is easy. Copy the NXESP file anywhere on your Next's SD card, and type the following:
.espupdate "path/to/nxesp_file.esp"
The path and filename only needs to be enclosed in double quotes if it contains any spaces. Paths can be absolute (on any drive) or relative. If the NXESP file is in your SD card's current working directory, you only need to specify the filename, not the path. As an example:
.espupdate "c:/Data/nxesp_3.3.0.1.nxesp"
If successful, the screen should look something like this:
An NXESP file consists of exactly one Header Section, immediately followed by one or more Block Sections, immediately followed by exactly one Data Section.
The Header Section consists of a Fixed Header Section followed by a Variable Header Section.
| Field Name | No of Bytes | Status | Data Type | Value/Notes |
|---|---|---|---|---|
| MagicID | 5 | Mandatory | Unterminated string, Spectrum charset | Always "NXESP" |
| VariableHeaderLen | 2 | Mandatory | Unsigned Int16, little-endian | Length of Variable Header Section, e.g. 547
|
| Field Name | No of Bytes | Status | Data Type | Value/Notes |
|---|---|---|---|---|
| VersionLen | 1 | Mandatory | Unsigned Int8 | Length of Version field (0..10 max), e.g. 7
|
| Version | VersionLen | Optional | Unterminated string, Spectrum charset | e.g. "3.3.0.1" |
| FlashParams | 2 | Mandatory | Unsigned Int16, big-endian | e.g. 0x0221
|
| MD5HashLen | 1 | Mandatory | Unsigned Int8 | Always 16
|
| MD5Hash | MD5HashLen | Mandatory | Byte[16] array | e.g. \x03\x19\x2f\x51 \x2d\x08\xb1\x4b \xe0\x6b\x74\xf9 \x8e\x10\x9e\xe0
|
| DataBlockLen | 2 | Mandatory | Unsigned Int16, little-endian | Maximum of, and typically 16384 (16KB) |
| Compressed DataLen | 4 | Mandatory | Unsigned Int32, little-endian | Length of Data Section, e.g. 457535
|
| BlockSectionLen | 1 | Mandatory | Unsigned Int8 | Fixed length of every Block Section, e.g. 18
|
| BlockSectionCount | 2 | Mandatory | Unsigned Int16, little-endian | The number of Block Sections, e.g. 28
|
| Compressed DataLen StrLen | 1 | Mandatory | Unsigned Int8 | The length of CompressedDataLen when represented as a decimal string, e.g. 6
|
| DataSectionLenStr | Compressed DataLen StrLen | Mandatory | Unterminated string, Spectrum charset |
CompressedDataLen represented as a decimal string, e.g. "457535" |
The 1MB uncompressed firmware is compressed into a size of CompressedDataLen. This is divided by DataBlockLen to give a number of data blocks (BlockSectionCount) necessary to upload the entire compressed firmware. There will be <BlockSectionCount> Block Sections, each in the following format, and each with the fixed length of BlockSectionLen.
| Field Name | No of Bytes | Status | Data Type | Value/Notes |
|---|---|---|---|---|
| BlockSize | 2 | Mandatory | Unsigned Int16, little-endian | The amount of compressed data in this chunk. For every Block Section except the last, this will be DataBlockLen. For the last Block Section, it can be any value between 1 and Block Section. |
| BlockOffsetStr | 8 | Mandatory | Unterminated string, Spectrum charset | The address offset this Block Section will be written too, as a hex string, e.g. "00000000". This value increases by DataBlockLen for every Block Section. Used to display progress messages in .espupdate. |
| PercentageStr | 8 | Mandatory | Null-terminated string, Spectrum charset | Cumulative BlockOffsetStr as a percentage of CompressedDataLen, formatted to 7 chars with spaces and brackets, followed by a zero byte terminator. e.g. " (4%) \x00". Used to display progress messages in .espupdate. |
| Field Name | No of Bytes | Status | Data Type | Value/Notes |
|---|---|---|---|---|
| CompressedData | CompressedDataLen | Mandatory | Byte[] array | The 1MB factory firmware, compressed as a zlib stream. The PackageFW.exe implementation is binary-compatible with Python's zlib.compress(data, 9) algorithm. |
Strings will be displayed in espupdate using the ZX Spectrum character set, which is almost identical to ASCII in the important ranges. We suggest you do not include characters which differ between ASCII and the ZX Spectrum character set. Client programs ported to other retro computers will need to translate strings to the Spectrum character set before displaying them.
FlashParams is a big-endian short integer constructed from the Flash Frequency, Flash Mode and Flash Size. For the ESP-01 these values are usually 26MHz, DIO and 32Mbits respectively, leading to a FlashParams value of 0x0221. More information can be gleaned by reading the esptool.py updater tool.
.espupdate and esptool.py update compressed data to the ESP flash in chunks. Due to the Z80 architecture and perhaps the ESP architecture, it is convenient to do this using a maximum chunk size of 16KB. .espupdate will support smaller sizes, but there is no good reason for doing so.