Company Metrics Collector - ram290476/investor-dashboard GitHub Wiki
The company metrics collector (company_ir) gathers company-specific metrics from SEC filings and feeds the per-stock company page at https://investor.vellamsetti.com/#stock/TICKER (code in apps/web/stock-page.js).
Last updated Sat Oct 10, 2026, about 12:12 PM PT. All times are Pacific.
- Built in PR #132 ("collect company metrics from SEC exhibits"), commit
4e73183. - Deployed to production successfully at 10:05 AM PT on Oct 10, 2026.
- Code:
services/data-jobs/src/functions/company_ir/(company_ir.py). The metric catalog is incatalog/TSLA.json, built into the image. - Wired up in Terraform (
infra/terraform/variables.tfdefines the handlers and S3 prefixes).
| Job | What it does | Output (lake bucket) |
|---|---|---|
| collect | Fetches 10-Q, 10-K, XBRL and 8-K exhibits from SEC EDGAR | raw/company_ir/<TICKER>/ |
| extract | Turns collected documents into metric rows | curated/company_metrics/ticker=<TICKER>/ |
| serve | Publishes approved metrics for the company page | serving/stock/<TICKER>.json |
Extract starts automatically after collect, and serve starts after extract. All three Lambdas run the new image and were healthy, with zero errors, as of noon today.
- Collect is scheduled weekdays at 4:15 AM PT.
-
plan_collectioncollects only on Mondays (UTC) or inside an earnings window. The scheduler payload{"source":"schedule"}carries no earnings dates, so on any other day collect skips before it contacts EDGAR, and extract and serve skip with "upstream skipped". - First scheduled run: Mon Oct 12, 4:15 AM PT. The status feed shows Tue Oct 13 because it skips Columbus Day, but the scheduler doesn't. That mismatch should be fixed.
| Time (PT) | Trigger | Result |
|---|---|---|
| about 7:04–7:05 AM | 3 manual test runs (before the deploy) | Succeeded with 0 sources; nothing to process |
| 12:04 PM | On-demand run with the scheduler payload | All three jobs skipped (Saturday, no earnings window). No errors. |
| 12:06–12:07 PM | Forced weekly run, payload {"reason":"weekly"}, approved by Ram |
Collect was partial. Extract and serve succeeded. Details below. |
- Collect stored 35 documents across 12 tickers (10-Q, 10-K and XBRL for each).
- All 45 8-K index fetches returned 404, so no press releases or investor decks came in.
-
TSLA:
raw/has 6 objects.curated/has 4 objects with 356 XBRL rows, all unapproved.serving/stock/TSLA.jsonexists (2.7 KB) withmetrics: []because nothing is approved yet. - 10 of the 47 TSLA metrics have data, all from XBRL: revenue, net income, operating income, R&D, SG&A, EPS, deferred revenue, capex, operating cash flow and cash.
- The operating metrics (deliveries, FSD and others) have no data yet. They depend on the 8-K exhibits.
- All 47 TSLA catalog metrics are
proposed. The company page shows a metric only after it's approved. Useservices/data-jobs/scripts/approve_metric.py. - Until something is approved, the company page shows "No company-specific metrics discovered."
- The 10 metrics with XBRL data above are stored and waiting on approval.
-
8-K index URL bug: the code requests
.../{accession}-index.json, but SEC serves.../index.json. Every 8-K fetch gets a 404, so there are no press releases or decks and no operating metrics. - Cash metric is stale (latest value 2018Q3) because it maps to the wrong XBRL tag.
- Collect logs
outcome: successeven when the run is partial. - The status feed skips Columbus Day, but the scheduler doesn't, so the feed shows the wrong next-run date.
No code changes have been made for these yet.