Company Metrics Collector - ram290476/investor-dashboard GitHub Wiki

Company Metrics Collector

The company metrics collector (company_ir) gathers company-specific metrics from SEC filings and feeds the per-stock company page at https://investor.vellamsetti.com/#stock/TICKER (code in apps/web/stock-page.js).

Last updated Sat Oct 10, 2026, about 12:12 PM PT. All times are Pacific.

Build and deployment

  • Built in PR #132 ("collect company metrics from SEC exhibits"), commit 4e73183.
  • Deployed to production successfully at 10:05 AM PT on Oct 10, 2026.
  • Code: services/data-jobs/src/functions/company_ir/ (company_ir.py). The metric catalog is in catalog/TSLA.json, built into the image.
  • Wired up in Terraform (infra/terraform/variables.tf defines the handlers and S3 prefixes).

The three jobs

Job What it does Output (lake bucket)
collect Fetches 10-Q, 10-K, XBRL and 8-K exhibits from SEC EDGAR raw/company_ir/<TICKER>/
extract Turns collected documents into metric rows curated/company_metrics/ticker=<TICKER>/
serve Publishes approved metrics for the company page serving/stock/<TICKER>.json

Extract starts automatically after collect, and serve starts after extract. All three Lambdas run the new image and were healthy, with zero errors, as of noon today.

Schedule and the collection rule

  • Collect is scheduled weekdays at 4:15 AM PT.
  • plan_collection collects only on Mondays (UTC) or inside an earnings window. The scheduler payload {"source":"schedule"} carries no earnings dates, so on any other day collect skips before it contacts EDGAR, and extract and serve skip with "upstream skipped".
  • First scheduled run: Mon Oct 12, 4:15 AM PT. The status feed shows Tue Oct 13 because it skips Columbus Day, but the scheduler doesn't. That mismatch should be fixed.

Run history (Oct 10)

Time (PT) Trigger Result
about 7:04–7:05 AM 3 manual test runs (before the deploy) Succeeded with 0 sources; nothing to process
12:04 PM On-demand run with the scheduler payload All three jobs skipped (Saturday, no earnings window). No errors.
12:06–12:07 PM Forced weekly run, payload {"reason":"weekly"}, approved by Ram Collect was partial. Extract and serve succeeded. Details below.

Forced weekly run results (12:06 PM PT)

  • Collect stored 35 documents across 12 tickers (10-Q, 10-K and XBRL for each).
  • All 45 8-K index fetches returned 404, so no press releases or investor decks came in.
  • TSLA: raw/ has 6 objects. curated/ has 4 objects with 356 XBRL rows, all unapproved. serving/stock/TSLA.json exists (2.7 KB) with metrics: [] because nothing is approved yet.
  • 10 of the 47 TSLA metrics have data, all from XBRL: revenue, net income, operating income, R&D, SG&A, EPS, deferred revenue, capex, operating cash flow and cash.
  • The operating metrics (deliveries, FSD and others) have no data yet. They depend on the 8-K exhibits.

Approval and the company page

  • All 47 TSLA catalog metrics are proposed. The company page shows a metric only after it's approved. Use services/data-jobs/scripts/approve_metric.py.
  • Until something is approved, the company page shows "No company-specific metrics discovered."
  • The 10 metrics with XBRL data above are stored and waiting on approval.

Known issues

  1. 8-K index URL bug: the code requests .../{accession}-index.json, but SEC serves .../index.json. Every 8-K fetch gets a 404, so there are no press releases or decks and no operating metrics.
  2. Cash metric is stale (latest value 2018Q3) because it maps to the wrong XBRL tag.
  3. Collect logs outcome: success even when the run is partial.
  4. The status feed skips Columbus Day, but the scheduler doesn't, so the feed shows the wrong next-run date.

No code changes have been made for these yet.

⚠️ **GitHub.com Fallback** ⚠️