0.5 Numerics - JulTob/Data GitHub Wiki
Numeric Data Types
Range • Granularity • Representation
Pick numeric types that match the phenomenon (units, scale, error) and your tooling (Pandas). The right choice prevents silent rounding bugs, invalid stats, and plotting weirdness.
1) Three axes that define a numeric type
-
Range/domain:
- Admissible values
0..360°- non-negative
- money
- Admissible values
-
Granularity (Δ):
- smallest meaningful step
- cents
- arc-seconds
- Plank Length
- smallest meaningful step
-
Representation:
- floating-point (relative error)
- fixed-point (absolute error).
Decimal + quantize
We can encode these as a binary range, with simple 'translation' rules to the real number.
type My_Fixed_Point_Type is delta N range Start .. End;
type Angle is 1.0 / (60 * 60) range 0.0 .. 360.0;
2) Low Level
The Ada view (semantics first)
Ada encourages you to declare problem-specific numeric types with explicit constraints. Core families:
- Signed integers:
type T is range Start .. End;(bounded, discrete). (Juan Cruz Viotti) - Modular integers:
type T is mod N;(wraparound, bitwise ops). - Floating-point:
type T is digits N [range …];(min significant digits, relative error). - Ordinary fixed-point:
type T is delta Δ range …;(Δ ≈ power-of-two step; max abs error ≤ Δ/2). - Decimal fixed-point:
type T is delta 0.01 digits 10;(Δ must be 10^−k; exact decimal scaling).
Examples
-- Money with 2 decimals (decimal fixed-point)
type Money is delta 0.01 digits 12; -- exact cents, 10-based scale
-- Angle with arc-second resolution (ordinary fixed-point)
type Angle is delta 1.0 / (60 * 60) range 0.0 .. 360.0;
3) Python / Pandas view (practical tooling)
-
Built-ins:
int(arbitrary precision),float(IEEE-754; relative error). -
Exact decimal:
decimal.Decimalwithquantizeto enforce Δ at boundaries of compute (e.g., cents). (Python documentation) -
Pandas dtypes:
Minimal patterns
# A) Decimal with Δ (e.g., cents)
from decimal import Decimal, ROUND_HALF_UP
def to_cent(x): return Decimal(str(x)).quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)
# B) Pandas: nullable ints, Arrow decimal
import pandas as pd, pyarrow as pa
df = pd.DataFrame({"qty": [1, 2, None], "price": ["12.345", "8.1", "0.5"]})
df["qty"] = df["qty"].astype("Int64") # nullable integer
df["price"] = df["price"].map(to_cent).astype(pd.ArrowDtype(pa.decimal128(12, 2))) # precision=12, scale=2
Note: arithmetic on Arrow decimals propagates precision/scale; audit downstream ops and re-
quantizeif needed. (Stack Overflow)
4) Interop map (concept → Ada → Python/Pandas)
| Concept | Ada declaration | Python / Pandas idiom |
|---|---|---|
| Bounded count | type Count is range 0 .. 1_000_000; |
Int64 with validation (e.g., 0 ≤ x ≤ 1e6) |
| Money (2 dp) | delta 0.01 digits N |
Decimal + quantize("0.01"); store as ArrowDtype(pa.decimal128(p,2)) |
| Angle (arc-sec) | delta 1.0/3600 range 0.0 .. 360.0 |
Decimal + quantize("0.0002777778") or float + final rounding; prefer Decimal if exactness matters |
| Bit-masked ints | mod 2**k |
numpy.uint* or int + bit ops; document wrap semantics |
| Scientific real | digits N |
float (IEEE-754), track tolerance; don’t store IDs as float |
5) Guardrails (hygiene)
- Choose Δ at the boundary. Apply
quantizeafter multiplies/divides to avoid drift. (Python documentation) - Use nullable dtypes (
Int64, Arrow decimals) to avoid silent float coercion for missing values. (Pandas) - Make scale explicit in schema (precision/scale for decimals; units for ratio data).
- Don’t average identifiers (never compute means of codes; treat as nominal/ordinal).
6) Micro-snippets for your data dictionary
# declare schema intent next to the data
NUMERICS = {
"qty": {"dtype": "Int64", "range": (0, None)},
"price": {"arrow": ("decimal128", 12, 2)}, # (type, precision, scale)
"angle": {"decimal_step": "0.0002777778"} # arc-second
}
References
- Ada numeric families and declarations (integers, modular, floating, ordinary & decimal fixed-point).
- Python
decimalandquantizeusage. (Python documentation) - Pandas + PyArrow dtypes (Arrow-backed decimals; nullable integers). (Pandas)
If you like, I can add a tiny validator block (pandera) so your Minion can scream when someone tries to sneak a float into price or a negative into qty.