Market Data and Database - cafe-auvers/quant_app GitHub Wiki
Market Data and Database
Database roles
- PC MySQL: canonical market history, indicators, scanner metrics, fundamentals, Market Pulse, and alignment data in the two-machine setup.
- Laptop SQLite mirror: offline, pull-only copy of supported market tables.
- Supabase PostgreSQL: 20 private
quant_coordinationtables for durable control, cards, commands, orders, account reservation locks, daily trading events, leases, alerts, and synchronized settings/state. - Former TiDB source: optional migration archive; no runtime authority or fallback.
- Local JSON: shutdown-safe compatibility/planning state and local order ledger.
Main tables
The Supabase migration copied and verified every source row/ID before runtime writes. Runtime SQL uses verified TLS through the IPv4 Session pooler on port 5432, UTC server time, transactional row locks, and a restricted application role. PC-local MySQL on port 3306 is a separate historical-data service. See Supabase Deployment.
price_history,hourly_price_history,intraday_price_historychart_indicators,scanner_metrics, refresh manifests/failuresstock_profiles,earnings_events,fundamental_sync_statemarket_pulse_*,stock_market_alignment_daily, alignment batches- trade cards, execution commands/orders/ownership, alerts, and runtime state in the operational schema
Data flow
flowchart LR
Sources[Yahoo / verified KIS sources] --> Refresh[Historical refresh process]
Refresh --> MySQL[(Canonical MySQL)]
MySQL --> Metrics[Indicators and scanner metrics]
MySQL --> Desktop[Scanner and charts]
MySQL --> Copy[Checkpointed one-way copy]
Copy --> SQLite[(Laptop SQLite mirror)]
SQLite --> Desktop
The (interval, date) price-history index supports global freshness
watermarks. Database reads return empty/unavailable results conservatively when
the optional cache cannot be reached; execution workflows have stricter
fail-closed database requirements.
Mirror Health deliberately evaluates two scopes. Daily coverage uses the full configured stock universe. Hourly coverage uses only symbols currently relevant to Scanner, Watchlist, and Buylist workflows, matching the selective hourly copy policy. A symbol outside that hourly scope does not make the mirror look unhealthy.