Fund Data Collection Effort Estimates - ram290476/investor-dashboard GitHub Wiki

Fund Data Collection Effort Estimates

Effort estimates for the fund data collector and the Fund holdings panel tracked in issue #223. The full design is on Design: Fund Data Collector (#223).

Last updated Wed Oct 14, 2026, about 8:10 PM PT. All times are Pacific.

What these numbers are

  • These are engineering effort estimates in person-days, for one person building and testing the feature.
  • They are not AI usage cost and not dollar amounts.
  • No measured self-rate exists for how long an agent would take, in wall-clock time, on a scope this size. These figures should not be read as one.
  • Every figure comes from the effort table in the scope doc (fund-collector-scope.md, section 10), which is also copied in the #223 issue body. Nothing here is a new estimate.

Original estimate (about 8 days), by piece of functionality

# Piece Scope line items Days
1 SEC N-PORT holdings and ARK's 13F from the real SEC endpoints (sample filings: ARKK NPORT-P for 2026-07-31, ARKVX NPORT-P for 2026-07-31, ARK 13F-HR for Q2 2026) Fund detection + mf-map cache (0.5) · N-PORT fetch/parse/curate (1.5) · 13F fetch/parse + CUSIP→ticker (1.0) · Terraform job, IAM, backfill (0.5) 3.5
2 Yahoo fund summaries (expense ratio, net assets, yield, returns, sector weights), best-effort weekly refresh Yahoo fund-profile collector + Fund facts row 1.0
3 ARK daily holdings files, designed but switched off because ARK's Terms of Use bar automated downloads and scraping ARK daily CSV collector + history + precedence merge 1.0, add-on only, not in the 8 days
4 Fund holdings panel: top 10 with Show all / Show top 10, hidden for regular stocks Serving doc + instrument_kind + prefs-api /fund/<T> (1.0) · panel (1.5–2.0) · hide company-only panels, NAV-only Technical Summary (0.5) · source labels and fallback notes (0.25) 3.25–3.75
5 Quarterly timing caveat: SEC fund data is public only on a delay No separate line item. It is covered by the source as-of labels and the empty state in piece 4 0 (included in 4)
Total (pieces 1, 2, 4, 5) 7.75–8.25, quoted as about 8
With the ARK daily add-on (piece 3) 8.75–9.25, quoted as about 9

ARKVX change: live SEC lookup instead of a hard-coded case

  • The original design had a hard-coded ARKVX special case, because ARKVX is missing from SEC company_tickers_mf.json.
  • It is replaced by a live SEC EDGAR lookup. EDGAR full-text search for the ticker resolves to filer ID 1905088 (ARK Venture Fund). The lookup also derives the 13F manager from the fund's N-CEN adviser.
  • The resolved mapping is saved in S3 (curated/fund_universe/fund_map.json), not in code or config. Fund tickers come only from the existing watchlist union.
  • In the scope table, fund detection grows from 0.5 to 1.25 days: +0.75 days.
Days
Original (pieces 1, 2, 4, 5) 7.75–8.25 (about 8)
ARKVX data-driven lookup +0.75
New total 8.5–9.0, quoted as about 8.75
With the ARK daily add-on 9.5–10.0 (about 9.75)

Reconciliation notes

  • "About 8 days" is a rounded range. The line items add up to 7.75–8.25 days. The range comes only from the panel itself (1.5–2.0). So "8.75" is the midpoint of 8.5–9.0, not an exact sum.
  • The ARK daily files (piece 3) are not in the 8 days. They are a separate +1.0 day, to be built only if ARK grants written permission.
  • The timing caveat (piece 5) has no days of its own. Its cost sits inside the panel's labels and empty state.
  • "Monthly" N-PORT holdings: SEC makes NPORT-P public only for each fund's fiscal-quarter-end month, about 50–60 days later. The estimate covers those quarterly public filings.
  • Not included: a fuller 13F manager or overlap view beyond the watchlist highlight, at about +1 day.
  • Added after this estimate, not sized here: the set-once is_fund flag rule (draft PR #224) and the Fund summary panel (draft PR #225).
  • Also added after this estimate, not sized here: the admin-only fund-flag reset (Admin tab in draft PR #225, POST /admin/fund-flag/{ticker}/reset in draft PR #224). See Design: Fund Data Collector (#223).

Related