FEP 001 - FujiNetWIFI/fujinet-firmware GitHub Wiki

Using URLs in Client Applications

Name Value
PEP ID 001
TItle Using URL in Client Applications
Author Andrew Diller
Status Draft
Type Informational
Created 2025-02-27
Version 1.0

Abstract

This document defines a standard approach for using URLs in client applications on supported platforms with the FujiNet. This includes the Atari , Apple2, ADAM, CoCO, and others, leveraging the FujiNet device for protocol adaptation and all network communication. It describes the structure of a URL, encoding and decoding mechanisms, and how a client application can interact with FujiNet to retrieve data from RESTful APIs.

This document proposes a clear set of guidelines and best practices for parsing, encoding, and decoding URLs in the FujiNet firmware. Historically, different code paths and incomplete solutions led to confusion around handling special characters, query parameters, and path segments. This proposal outlines how to separate the concerns of URL parsing from encoding/decoding so that HTTP requests are consistently formed.

Motivation and Background

With FujiNet providing a powerful network interface for retro computing platforms, client applications must efficiently interact with remote resources. The goal of this FEP is to establish a structured methodology for constructing, parsing, encoding, and decoding URLs, ensuring seamless communication between client applications and FujiNet's ESP32-based protocol adapter.

FujiNet operates on resource-constrained hardware (Atari 8-bit computers and similar). It is often used to retrieve files over HTTP or other protocols in a manner similar to local file systems. Historically, “mode 4 vs 8 vs 12” attempts to differentiate between purely file-based operations and general-purpose HTTP requests. The resulting code often merges path and query parameters into a single “path” field, then applies naive encoding with no knowledge of the characters that should remain unencoded (for example, &, ? when used as separators).

Discussions revealed that robust handling requires:

  1. A dedicated parser that splits a valid URL into components (scheme, host, port, path, query, fragment).
  2. A dedicated builder/encoder that can accept raw components and encode them properly where needed.

Relying on a simple urlEncode() function for every piece of the URL often leads to incorrect escaping of separators or failure to parse unencoded brackets and spaces.

Rationale

  1. Separation of Concerns

    • Parsing should not guess which parts need encoding. It is responsible for splitting the URL string into constituent parts, assuming a syntactically valid URL.
    • Encoding/Decoding should be performed either by the user or by a separate builder utility, ensuring only the necessary segments (e.g., query parameter values) are escaped.

  2. Clarity in Firmware

    • Many developers expect a library function that can “fix” or encode arbitrary paths, but this approach can fail when query separators or special characters get incorrectly escaped.
    • A robust approach fosters correct usage: if a file path must be encoded, the user or a dedicated builder should do so before sending it to the parser.

  3. Mode 4 vs 8 vs 12 - Atari Specific

    • mode=4 or 8 has traditionally signified “file-based” requests for direct directory/path listings.
    • mode=12 is used for general-purpose HTTP requests.
    • Any approach to unify them must specify how query strings and special characters are handled, so FujiNet can properly convey the request upstream without guesswork.

Parsing

So we are on the same page lets illustrate URL parsing and agree. Consider the following example URLs:

Example URLs:

  • http://example.com/resource
  • https://user:pass@api.server.com:443/data?format=json#section
  • http://api.example.com/data/info?query=super#section1

We will use this complex example in order to expose each specific component:

https://user:pass@api.example.com:8080/data/info?query=super#section1

URL Structure

A URL (Uniform Resource Locator) consists of multiple components as defined by RFC 3986:

scheme://[user:password@]host[:port]/path[?query][#fragment]

Breaking it down into components:

Component Value
Scheme https
Userinfo user:pass
Host api.example.com
Port 8080
Path /data/info
Query query=super
Fragment section1

After parsing, FujiNet should prepare the URL for encoding by replacing special characters with their respective percent-encoded values.

Encoded URL

https%3A%2F%2Fuser%3Apass%40api.example.com%3A8080%2Fdata%2Finfo%3Fquery%3Dtest%23section1

This encoded URL can then be used in network requests, ensuring proper handling of special characters.

Encoding and Decoding

URL Encoding (Percent-Encoding)

Certain characters must be encoded for proper transmission. FujiNet’s URL encoder will replace special characters with % followed by the hexadecimal ASCII code.

Character Encoded Form
Space %20
# %23
? %3F
& %26
/ %2F

URL Decoding

FujiNet will decode incoming URLs by replacing percent-encoded characters with their original representations before processing the request.

Sample Flow using Atari

The FujiNet firmware running on the ESP32 should handle URL parsing and translation. The Atari 8-bit client application will:

  1. Define a URL string in ATASCII format.
  2. Send the URL to FujiNet over SIO.
  3. FujiNet will:
    • Convert ATASCII to ASCII.
    • Parse the URL and extract components.
    • Encode an HTTP request.
    • Send the request over TCP/IP.
    • Receive and decode the HTTP response.
  4. The parsed response is returned as key-value pairs to the client.

Detailed Specification

4.1. Parser Requirements

  • Parse only syntactically valid, already encoded URLs.
  • Distinguish components: [scheme]://[user]:[password]@[host]:[port]/[path]?[query]#[fragment].
  • If an invalid character set is encountered (e.g., spaces in the raw URL), the parser should reject or handle it as an error (unless a local extension is agreed upon for backward compatibility).

4.2. Builder/Encoder Requirements

  • Provide helper methods, such as:
    • encodePathSegment(segment): Encodes reserved characters within a path segment but preserves /.
    • encodeQueryParamValue(value): Encodes characters that are not safe in a query parameter, preserving &, =, etc.
    • encodeFragment(fragment): Encodes fragment-specific reserved characters.
  • Accept unencoded strings (e.g., user input) and produce a fully valid URL string.

Implementation Notes

Backward Compatibility For existing code that relies on naive approaches, minimal changes can maintain compatibility. In the future, consider adding a “strict mode” that rejects unencoded spaces or special characters in the parser.

Performance Considerations On embedded hardware, every byte of code and data matters. Whichever builder or parser solution is adopted (e.g., a minimal custom parser or a trimmed-down uriparser) should remain memory-efficient.

Open Questions How should “file-based” HTTP operations be integrated without conflating path segments and query parameters? Should mode 4 or 8 remain or be replaced by more explicit HTTP-like modes that parse queries properly?

Conclusion

By separating the concerns of parsing and encoding, FujiNet’s firmware can reliably handle standard HTTP operations, file-based requests, and other protocols without conflating path and query segments. Developers gain clarity in how to submit valid URLs, and the firmware ensures consistency by rejecting or redirecting invalid requests. This approach aligns with modern best practices, reduces code complexity, and can scale with future enhancements.

By offloading URL handling and network requests to FujiNet, we enable efficient client applications with minimal overhead on Atari 8-bit systems. This FEP establishes a structured methodology for URL usage, ensuring compatibility and ease of development for FujiNet-enabled applications.

Links