Web Requests Framework - decentraland/unity-explorer GitHub Wiki
General Structure
We are using a custom generic structure for performing Web Requests
in an asynchronous way. It's located in DCL.WebRequests
assembly.
Currently, the exposed API looks like this:
public interface IWebRequestController
{
UniTask<GenericGetRequest> GetAsync(
CommonArguments commonArguments,
CancellationToken ct,
string reportCategory = ReportCategory.GENERIC_WEB_REQUEST,
WebRequestHeadersInfo? headersInfo = null,
WebRequestSignInfo? signInfo = null);
UniTask<GenericPostRequest> PostAsync(
CommonArguments commonArguments,
GenericPostArguments arguments,
CancellationToken ct,
string reportCategory = ReportCategory.GENERIC_WEB_REQUEST,
WebRequestHeadersInfo? headersInfo = null,
WebRequestSignInfo? signInfo = null);
UniTask<GetTextureWebRequest> GetTextureAsync(
CommonArguments commonArguments,
GetTextureArguments args,
CancellationToken ct,
string reportCategory = ReportCategory.GENERIC_WEB_REQUEST,
WebRequestHeadersInfo? headersInfo = null,
WebRequestSignInfo? signInfo = null);
}
Every type of request creates a strongly typed structure to define and restrain the set of possible operations. E.g. to prevent reading text
from GetTexture
request. This capability is not provided by Unity itself: it will just throw an exception.
The whole API is designed to provide an allocation-free way of setting parameters:
- Everything is designed with
Value Types
unless it's restricted by Unity API itself WebRequestHeadersInfo
uses the pool and is disposed of automatically when aWeb Request
fails or succeeds
Common Arguments
public readonly URLAddress URL;
public readonly int AttemptsCount = 3;
public readonly int Timeout = 0;
Repetitions are handled inside the implementation of IWebRequestController
. If needed (for test purposes) this behavior can be overridden.
Signing
Some requests require a signature to validate the identity of the caller. Those are sensitive APIs that should not be abused by third-party consumers.
The identity itself is provided by the IWeb3Authenticator
implementation. Then with the capabilities of AuthChain and Nethereum Library x-identity-auth-chain-N
headers are set in accordance.
For this to work the following structure should be provided as an argument to the request:
public readonly struct WebRequestSignInfo
{
public readonly URLAddress SignUrl;
public WebRequestSignInfo(URLAddress signUrl)
{
SignUrl = signUrl;
}
}
Common Response Parsing Scenarios
Our scenarios of handling responses are common, therefore, several utilities are presented to unify the approaches and get rid of potential boiler-plate code.
GenericDownloadHandlerUtils
provides capabilities for genericPost
andGet
requestsOverwriteFromJson<T>
- parse JSON into the existing object withUnity
/Newtonsoft
off/on the main threadCreateFromJson<T>
- create an object from JSON withUnity
/Newtonsoft
off/on the main threadGetDataCopy()
- produce a managedbyte[]
- Calling these methods will provide data and automatically release the underlying request
- Texture creation from
GetTextureWebRequest
(can be further expanded to include Sprites creation)
Retry Policy
Idempotency
Idempotency is the property of an operation that can be applied multiple times without changing the result beyond the initial execution.
By default, only GET
requests are idempotent and, therefore, can be repeated without an extra explicit instruction. Those include not only generic GET
requests but also assets such as Textures
and Audio
.
Additionally, in our environment, "Signed" requests are not idempotent as the signature may expire.
Delay between retries
If a request is to be repeated, a backoff delay is applied:
- The initial default delay is 1 second
- With every unsuccessful attempt, it gets multiplied by the backoff multiplier, which is equal to
3
by default
Default Policy
By default, a request can be repeated up to 2
times (1 original execution + 2 retries) under the following circumstances:
- If the network error is qualified for repetition and the request is idempotent
- If the DNS Lookup error occurred and the request is idempotent
Error Codes Qualification
Code | Name | Retry? | What to Do | Notes |
---|---|---|---|---|
400 | Bad Request | β | Fix payload/params; validate before send. | Malformed JSON, missing fields. |
401 | Unauthorized | β | Refresh token once, then retry. | If second fail β force reβlogin. Can't be applied automatically |
403 | Forbidden | β | Block UI path or request proper scope. | Auth OK, but not allowed. |
404 | Not Found | β | Check URL/IDs; refresh local cache. | Common for stale resource IDs. |
405 | Method Not Allowed | β | Use correct HTTP verb. | Server disallows current method. |
406 | Not Acceptable | β | Change Accept header/format. | Unsupported response type. |
408 | Request Timeout | β | Retry with backoff. | Server didnβt receive body in time. |
409 | Conflict | β | Resolve state/version conflict, then manual retry. | E.g., duplicate, version mismatch. |
410 | Gone | β | Remove/redirect feature. | Resource intentionally removed. |
411 | Length Required | β | Ensure Content-Length is set. | Usually auto by UnityWebRequest. |
412 | Precondition Failed | β | Refresh ETag/version, retry once. | Optimistic concurrency failed. |
413 | Payload Too Large | β | Chunk/compress upload. | Split files or use CDN. |
414 | URI Too Long | β | Move params to body (POST). | Overlong query string. |
415 | Unsupported Media Type | β | Fix Content-Type. | e.g., send JSON as application/json. |
421 | Misdirected Request | β | Fix host/routing. | HTTP/2 specific. |
422 | Unprocessable Entity | β | Fix semantic validation errors. | JSON OK syntactically, rules fail. |
425 | Too Early | β | Wait briefly, retry once. | Rare; HTTP/2 early data. |
428 | Precondition Required | β | Add conditional headers. | Similar to 412. |
429 | Too Many Requests | β | "Retry-After" is respected and obligatory to be qualified for retrying | Rate limiting. |
431 | Header Fields Too Large | β | Trim headers/cookies. | Reduce auth blobs. |
451 | Unavailable For Legal Reasons | β | Inform user; disable feature. | Legal/geofence block. |
500 | Internal Server Error | β | Backoff; maybe switch region. | Generic server crash/bug. |
501 | Not Implemented | β | Downgrade client / change call. | Endpoint/method not supported. |
502 | Bad Gateway | β | Backoff; transient proxy/CDN issue. | Upstream sent bad response. |
503 | Service Unavailable | β | "Retry-After" is respected and obligatory to be qualified for retrying | Overload/maintenance. |
504 | Gateway Timeout | β | Retry; | Upstream took too long. |
505 | HTTP Version Not Supported | β | Use supported protocol. | Rare with Unity. |
507 | Insufficient Storage | β | Can't make any assumptions | Server disk full. |
508 | Loop Detected | β | Can't make any assumptions | WebDAV-specific. |
511 | Network Auth Required | β | Prompt captive-portal login. | Intercepting proxy. |
521/522/523/525/526 | CDN Origin Errors | β | Treat like 502/504; backoff. Can recover | Cloudflare/Akamai variants. |
Enforced policy
By calling RetryPolicy.Enforce()
, it's possible to qualify the request for retries even if it's not qualified by default rules.
JS API Policy
The requests originating from SimpleFetch
API are not qualified for repetitions unless "Retry-Delay" is specified in the 429
or 503
response.
ECS originated requests
Streamable Assets are not repeated directly by the WebRequestController
but use their own system with promises which obey budgeting and prioritization.
Analytics
IWebRequestsAnalyticsContainer
records currently ongoing requests. At the moment it's used just for counting and displaying the value in the Debug View via ShowWebRequestsCountersSystem
but in the future can be expanded to hold additional information about requests.