Recommended Data Stack

This page turns the data source catalog into decisions. The project's working budget is $100–1,000/month — "serious retail." Three tiers follow: the best all-free stack (what you can do for $0), the core stack (the primary recommendation, a concrete pick per category with a total), and institutional context (what unconstrained money buys, and why knowing that matters even when you cannot buy it). Every cost below is a verified catalog figure as of 2026-07-11; anything unofficial is flagged. Provider names link to their profiles in the provider directory.

Free tier ($0/mo)

The best genuinely free stack, category by category:

• Price history — Stooq daily continuous CSVs and Yahoo Finance for exploration, Barchart's free 2-year daily downloads, Kinetick free EOD inside NinjaTrader, and CME Group settlement pages as ground truth. Databento's $125 intro credits buy a real one-time slice of tick data.
• Market depth — no meaningful free depth exists. The free layer is CME's MBO/MBP documentation and fee list (for understanding and budgeting); delayed CME depth via Sierra Chart still requires its ~$36/mo platform package.
• Volatility & options — Cboe free daily CSVs (VIX family to 1990, VX futures settlements to 2004, daily put/call stats), FRED mirrors for API access, and CME Group QuikStrike for options OI/skew eyeballing. Tradier adds free live SPX/VIX chains with greeks if you open an account.
• Macro — the category is nearly all free: FRED/ALFRED (vintages — the anti-look-ahead backbone), BLS, BEA, Census Bureau, US Treasury auction APIs, Federal Reserve Board FOMC materials, and the regional Fed nowcasts (Atlanta Fed GDPNow, Cleveland Fed inflation nowcast, New York Fed SOFR/EFFR + Staff Nowcast, Chicago Fed NFCI).
• Energy fundamentals — EIA API (WPSR, Cushing, SPR, STEO), OPEC MOMR, Baker Hughes rig count, JODI, TSA throughput, and free calendars for the leaked Tuesday API headline.
• Metals fundamentals — CME Group daily warehouse/delivery files (scraping engineering required), LBMA vault and clearing data, SPDR Gold Shares and iShares daily holdings, World Gold Council Goldhub, SGE/SHFE for the China leg, Swiss Federal Customs flows, MetalCharts for monitoring.
• Positioning & flows — CFTC COT via Socrata API (1986+), CME Group daily volume/OI, SqueezeMetrics free DIX/GEX CSV (2011+), NAAIM (free until 2026-08-01 — download now), AAII, ICI, FINRA margin debt, IG client sentiment.
• News & sentiment — GDELT (1979+), Alpaca's free real-time Benzinga websocket, Interactive Brokers bundled Reuters/DJ-commodities feeds, FinancialJuice squawk, plus DIY FinBERT/LLM scoring of collected headlines.
• Alternative data — NOAA weather/hurricanes/degree days, EIA-930 hourly electricity, Wikimedia pageviews, Kalshi and Polymarket event odds, Société Générale Trend Indicator (CTA proxy), the free LSEG FTSE Russell reconstitution calendar, and CME roll tools.
• Calendars & reference — ForexFactory machine-readable weekly feed, BLS/BEA iCal/JSON schedules, Federal Reserve Board FOMC dates, all of CME Group's holiday/expiration/margin/fee/settlement/price-limit pages, and pandas_market_calendars.

The free tier is a real research platform — daily bars, complete macro/energy/metals/positioning fundamentals, vol indices, calendars. What it cannot do: intraday history at scale, any order-book depth, machine-readable consensus, or clean live feeds.

Core stack ($100–1k/mo)

The primary recommendation. Everything in the free tier stays; these are the paid additions per category:

1. Price history — Databento pay-as-you-go (~$100–150/mo budget) + Norgate Data futures package ($270/yr ≈ $22.50/mo). Databento is the catalog's default primary source: per-contract tick-to-daily CME data since 2010 with explicit roll symbology, priced per GB so a targeted-symbol program stays in the tens-to-hundreds of dollars. Norgate adds cheap, clean, survivorship-aware daily history with documented FND-safe continuous contracts for long-horizon backtests.
2. Market depth — Sierra Chart package 12 + Denali CME depth bundle ($56 + $40.50 = $96.50/mo). The best price/performance live depth + MBO stack for discretionary study and short-horizon depth recording; historical MBP-10/MBO slices come out of the Databento budget above. (Alternative: Rithmic via a broker, ~$70/mo all-in, if you want MBO inside Bookmap/ATAS-style tooling.)
3. Volatility & options — free Cboe CSV backbone + Interactive Brokers non-pro feeds (~$16/mo) + ThetaData Options Value ($40/mo). IBKR is the cheapest live path to CME futures-options, OPRA, and CFE quotes (feeds at $1.55–$4.50 each); ThetaData buys four years of SPX/VIX options tick history for signal research. Raw CME options chains ride the Databento budget.
4. Macro — free agency stack + CME Group FedWatch API EOD ($25/mo). The macro category is structurally free; the FedWatch API is the best value-for-money commercial item in it, delivering historical Fed-path probabilities back to 2015 without building your own ZQ bootstrapping.
5. Energy fundamentals — free ($0). EIA/OPEC/Baker Hughes cover the official layer completely, and the subscription-only Tuesday API headline free-rides through calendars within minutes; every paid "edge" product in this category is institutionally priced.
6. Metals fundamentals — free primaries + optional GoldChartsRUs (~$200/yr ≈ $17/mo, unofficial). The primaries are free but publish point-in-time files only; GoldChartsRUs sells decades of stitched COMEX warehouse-stock and ETF-holdings history in one transaction instead of scraping forward from today.
7. Positioning & flows — free CFTC/SqueezeMetrics layer + Menthor Q Premium ($59/mo). Menthor Q is the cheapest credible dealer-gamma feed that explicitly covers futures contracts (ES/NQ/CL/GC) rather than only the SPX complex; SpotGamma Standard ($89/mo) is the upgrade if SPX-complex depth matters more than futures tagging.
8. News & sentiment — Dow Jones Institutional News via IBKR ($78/mo) + free Alpaca/Benzinga headlines with DIY FinBERT scoring ($0 + compute). $78/mo is the only retail-priced path to a genuine institutional newswire with commodity market talk; DIY scoring of timestamped headlines is the realistic route to futures-keyword sentiment features, since no retail vendor tags ES/MCL/1OZ natively.
9. Alternative data — Quiver Quantitative ($30/mo) + SerpApi ($25/mo). Quiver is the cheapest broad alt-data API (retail flow, WSB sentiment, congress trades); SerpApi is the reliable programmatic Google Trends route while the official API sits in alpha. Weather, electricity, attention, and event-odds layers stay free.
10. Calendars & reference — free ($0). ForexFactory's feed, the agencies' ICS/JSON schedules, CME's reference pages, and pandas_market_calendars fully cover this category; paid calendar APIs buy SLAs, not data.

Core stack total: ≈ $490–545/month (midpoint ≈ $515, dominated by the usage-elastic Databento line). That sits comfortably inside the $100–1k band and leaves headroom for one-time purchases when a strategy demands them — a Kibot all-futures 1-minute archive ($820), an HistoricalOptionData SPX bundle ($230–2,035), optionsDX chain files ($0–50), CQG Data Factory per-commodity-month buys ($7–27), or scaling Databento pulls for a depth-hungry backtest.

Institutional context

What does unconstrained money buy, and why should a retail-budget project care? Three reasons: to know what edge you are competing against, to know when privileged information arrives (much of it leaks on a fixed schedule), and to know what to buy first if the budget ever grows.

• Market data: institutions buy venue-complete history from CME Group DataMine (to 1972, MBO/PCAP), nanosecond book replay from Nanotick, cleaned multi-decade tick from TickData (CL from 1987, GC from 1984) and Portara (intraday from 1987, daily from 1899), and professional feeds from dxFeed or Trading Technologies (unofficially $775+/mo). The retail consequence: anything before 2010 at tick resolution, or venue-wide MBO, is effectively institutional-only.
• News and sentiment: Bloomberg Event-Driven Feeds, LSEG Machine Readable News/News Analytics (point-in-time to 1996/2003) and MarketPsych commodity sentiment (to 1998), and RavenPack define millisecond event trading and long-history sentiment factors — all contact-sales, realistically $10k–100k+/yr. Retail is structurally late to headlines; design strategies that do not race them.
• Energy "edge" data: Wood Mackenzie's Genscape Cushing report (twice weekly, two days before EIA), Kayrros daily satellite inventories with Cushing drone flights, Ursa Space's Monday SAR estimate, Kpler/Vortexa cargo flows (~$55k/yr unofficial), and the American Petroleum Institute's Tuesday bulletin. Knowing this calendar is itself useful: Tuesday-afternoon and pre-EIA price action reflects data you cannot see, and its headlines leak on newswires minutes later.
• Everything else: Haver Analytics/Macrobond one-stop macro with licensed ISM/Conference Board content, Econoday/Trading Economics enterprise consensus feeds, OptionMetrics/SpiderRock research-grade vol surfaces, EPFR/Vanda Research fund and retail flows, Nasdaq's LBMA-i OTC metals trade data (via Nasdaq), ICE IBA benchmark licences ($8k–30k/yr for LBMA price usage), and Context Analytics futures-tagged social sentiment. If the project institutionalizes, the first purchases that change research quality are: a real consensus/calendar API, OptionMetrics-grade vol history, and Cushing measurement.

Per-category picks

Category	Free pick	Core pick	Institutional pick
Price history	Stooq + CME Group settlements	Databento PAYG + Norgate Data	TickData / Portara + CME Group DataMine
Market depth	CME docs only (no real free depth)	Sierra Chart Denali + Databento MBO slices	CME Group DataMine PCAP / Nanotick
Volatility & options	Cboe CSVs + FRED + QuikStrike	ThetaData + Interactive Brokers live feeds	OptionMetrics / SpiderRock
Macro	FRED/ALFRED + BLS/BEA + Fed nowcasts	Free stack + CME Group FedWatch API ($25/mo)	Haver Analytics / Macrobond + Econoday
Energy fundamentals	EIA + OPEC + Baker Hughes	Same (free)	Wood Mackenzie Cushing + Kpler + American Petroleum Institute
Metals fundamentals	CME Group stocks + LBMA + SPDR Gold Shares/iShares	Free stack + GoldChartsRUs backfill	Metals Focus + LBMA-i via Nasdaq + CME Group DataMine registrar
Positioning & flows	CFTC COT + SqueezeMetrics DIX/GEX	Free stack + Menthor Q ($59/mo)	EPFR + Vanda Research + SpotGamma Institutional
News & sentiment	GDELT + Alpaca + DIY FinBERT	Dow Jones via IBKR ($78/mo) + DIY scoring	Bloomberg EDF / LSEG MRN + RavenPack
Alternative data	NOAA + EIA-930 + Wikimedia + Kalshi and Polymarket	Free stack + Quiver Quantitative + SerpApi	Kayrros / Ursa Space + Kpler/Vortexa
Calendars & reference	ForexFactory + agency ICS/JSON + pandas_market_calendars	Same (free)	Econoday / Trading Economics enterprise + TradingHours

Gaps and caveats

Data this project cannot practically get at the $100–1,000/month budget — plan strategies around these holes rather than pretending they don't exist:

• Execution is on Wealthsimple, and that bounds what data matters. The tradable universe is exactly the 14 target symbols, only the front two expiry months of each, in a margin account, at US$1.00/contract per side plus exchange/regulatory pass-throughs (as of 2026-07-12). Practical implications: depth/microstructure spend should target front-two-month contracts only; roll logic never needs back-month data; and the liquid non-Wealthsimple siblings (CL, GC, SI, MGC, SIL) are data assets — settlement sources and long-history backtest proxies — not tradable instruments.
• No COT positioning for the small contracts. Live queries confirmed no separate CFTC entries for MCL, QM, QO, QI, 1OZ, or SIC — positioning for every energy/metals contract we trade must be proxied by the full-size parent (CL, GOLD, SILVER).
• No deep history for the new contracts at any price. 1OZ launched 2025-01-13 and SIC 2026-02-09; no vendor can sell history that does not exist. Backtest on the parent/benchmark contracts (GC, SI, CL, ES) and trade the small ones, watching for their thinner books.
• No affordable machine-readable consensus. Bloomberg/Reuters-grade forecast panels and Econoday's enterprise feed are contact-sales; free calendars (ForexFactory, Investing.com) carry editorial consensus with ToS-restricted scraping. Surprise definitions at this budget lean on the Cleveland Fed nowcast and stored calendar values.
• PMI internals are paywalled. ISM reports and calendar are login-gated and S&P Global PMI history is subscription-only — only headline surprises are capturable free.
• The Tuesday API number arrives without you. The American Petroleum Institute bulletin is licensed via LSEG/ICE; retail sees the headline minutes later via calendars, never the full table.
• Cushing tank-level truth is institutional. Wood Mackenzie/Kayrros/Ursa Space measurement data (2 days ahead of EIA) is out of reach; the same goes for cargo-flow analytics (Kpler, Vortexa) and credible tanker intel (TankerTrackers at $10k/yr minimum).
• Order-book depth history is bounded. MBO exists only from mid-2017 (protocol limit), MBP-10 from ~2009–2010, and venue-wide MBO pulls are prohibitively bulky even on Databento — buy narrow per-symbol slices, and self-record anything your live Sierra Chart/Rithmic stack shows you (retention there is ~180 days at best).
• Scraping friction on key free sources. CME Group metals files, US Mint CSVs, and several fundamentals pages 403-block datacenter IPs; budget engineering time (headless browsers, residential egress) or accept aggregator risk via MetalCharts.
• No point-in-time news/sentiment archive. RavenPack/LSEG-grade scored news history is institutional; the DIY substitute (score Alpaca/Tiingo headlines with FinBERT, storing crawl timestamps) only accumulates from the day you turn it on.
• Dealer-positioning history is dashboard-locked. SpotGamma/Menthor Q/Volland sell current levels, not bulk historical exports; the only free backtestable series is SqueezeMetrics' SPX-only DIX/GEX file.
• No live dollar index. ICE's DXY is licensed and off the CME feed; the free Fed H.10 broad dollar publishes weekly — a live dollar factor requires buying DX quotes through a multi-exchange vendor.
• Licensing tripwires. LBMA benchmark prices used for production valuation trigger ICE IBA licences ($8k–30k/yr); most retail APIs prohibit redistribution and some (IQFeed, IBKR) prohibit even storing the feed; Open-Meteo's free tier is non-commercial.
• Social firehoses are gone at retail. X (Twitter) is pay-per-use ($5k per million posts read), StockTwits closed API registration, and Reddit commercial access needs approval — historical dumps and news-based sentiment are the workaround.
• A closing deadline: NAAIM exposure-index history goes subscription-only on 2026-08-01 — download the full free Excel history before then.