Skip to content

Changelog

  • Fix: parquet nullable columns: Observation parquet files now use Optional schema for nullable fields. Previously, the Go API inferred the parquet schema from the first row only and created non-nullable columns — null values were silently written as zero (0 for int, 0.0 for float), skewing downstream analysis. JSON was unaffected.
  • API: writeParquetTyped now scans all rows for the full column union and wraps nullable columns with parquet.Optional()
  • Python SDK: 7 new regression tests verifying nullable parquet columns produce NaN in DataFrames, not 0. No SDK code changes needed — PyArrow handles optional columns correctly.
  • 213 unit tests + 45 integration tests

  • Integration test suite: 45 parametric E2E tests against the live API + AWC, covering every public function x every output mode (list, DataFrame, CSV, parquet)
  • CI publish gate: integration tests must pass before PyPI publish on tag push
  • Fix: parquet bytes→str: DataFrame columns from parquet now return str, not bytes (PyArrow types_mapper)
  • Fix: climate dates tz-naive: climate observation_date index stays tz-naive (calendar dates), not UTC
  • Fix: LOB numeric index: LOB ts column correctly detected as numeric epoch, not parsed as date string
  • Fix: empty DataFrame NaN edge case: empty result sets no longer misclassify index timezone
  • Fix: DataFrame immutability: _apply_datetime_index no longer mutates input DataFrame
  • CI security: pypa/gh-action-pypi-publish pinned to commit SHA, publish step gated on tag ref
  • 206 unit tests + 45 integration tests

  • Domain-split modules: therminal.weather (WeatherHistory, WeatherLive) and therminal.markets (MarketsClient)
  • Canonical names: WeatherClient renamed to WeatherHistory, LiveClient renamed to WeatherLive
  • WeatherLive: Real-time METAR from AWC (aviationweather.gov), direct fetch with identical Observation schema to historical data
    • Batch support: live.current(["NYC", "ATL", "MDW"]) makes one AWC request
    • as_dataframe=True on current() and latest()
    • Relative humidity computed via Magnus formula (native Celsius, cross-checked against IEM for 18 stations)
    • Feels-like temperature via full NWS algorithm (wind chill + Steadman + Rothfusz + adjustments)
    • Precipitation mapped from AWC precip field
    • Peak wind gust/direction/time parsed from raw METAR PK WND remarks
    • Go-matched rounding ensures training/inference parity
  • Auto-pagination: JSON responses transparently paginated, as_dataframe=True uses parquet for full datasets (no 50K JSON cap)
  • TherminalConfig: ~/.therminal.toml config file with 4-layer resolution (file < env vars < kwargs)
  • Typed models: Frozen dataclasses with dict-style access (obs["temp_f"], obs.get(), obs.temp_f)
  • Security hardening: guarded float() on all AWC numeric fields, direction validation, truncated unbounded strings
  • Python 3.11+ required (dropped 3.10)
  • TherminalClient facade preserved for backward compatibility
  • 206 tests, 80% coverage

  • GET /api/v1/lob — Reconstructed order book snapshots from Kalshi and Polymarket delta data
  • Parameters: market, series, source (kalshi/polymarket), date/from/to, interval (1–3600s, default 60), levels (1–50, default 10)
  • Supports JSON, CSV, Parquet, and NDJSON output formats
  • Singleflight deduplication for identical concurrent requests
  • 4 concurrent LOB requests max (semaphore-bounded)
  • Result caps: 50K JSON, 500K CSV/Parquet/NDJSON
  • ?format=ndjson now supported on candles, observations, and LOB endpoints
  • Newline-delimited JSON — one object per line, streamable
  • Same 500K row cap as CSV/Parquet
  • No pagination, no forward-fill (returns raw sparse data like CSV)
  • client.lob() — LOB snapshots with full format support, DataFrame, save_path
  • LOBFeatures sklearn transformer — spread, mid, imbalance, depth, weighted_mid, pressure features with lookback aggregation
  • CLI: therminal lob MARKET --date 2026-03-01 --interval 60

  • ?resolution=1min on /api/v1/observations returns 1-minute ASOS data from NCEI (DSI-6406)
  • ~166M records, 2000–2026, 19 of 20 stations (NYC excluded — Central Park is not an airport ASOS)
  • Response fields: station_code, observed_at, temp_c, dewpoint_c, pressure1_inhg, pressure2_inhg, pressure3_inhg, precip_type, precip_in
  • ?units= rejected with 400 for resolution=1min (temperatures always integer °C)
  • ?type= silently ignored for resolution=1min
  • ?format=csv|parquet and ?columns= work normally
  • New report type ncei_final (priority 2.5) on /api/v1/climate — GHCN-Daily CF6 first-published values from NCEI superghcnd diffs
  • Source tagged as ncei
  • Added resolution parameter to observations()resolution="1min" for 1-minute ASOS data
  • CLI: therminal observations STATION --resolution 1min

  • ?format=json|csv|parquet on all data endpoints (candles, observations, climate)
  • ?columns=col1,col2 for column selection in CSV/Parquet output
  • Removed old /api/v1/download/* presigned URL endpoints
  • CSV streams with Content-Disposition header for browser download
  • Parquet uses Zstd compression
  • Added interactive API playground on every endpoint page
  • Added Data Sources page with full provenance details
  • Added llms.txt for AI agent integration
  • Replaced splash homepage with introduction
  • Fixed playground field alignment and prev/next button sizing
  • Added format and columns params to candles(), observations(), climate()
  • Removed download_parquet() (use format="parquet" instead)

Initial public release.

  • GET /health — API health check
  • GET /api/v1/series — List and retrieve event series
  • GET /api/v1/markets — List and retrieve prediction markets
  • GET /api/v1/candles — OHLCV candlestick data with forward-fill and aggregation
  • GET /api/v1/observations — Weather observations (METAR/SPECI)
  • GET /api/v1/climate — Daily climate reports
  • GET /api/v1/analysis/climate-gaps — Climate report gap analysis
  • GET /api/v1/download/observations — Bulk observation parquet download
  • GET /api/v1/download/candles — Bulk candle parquet download
  • GET /api/v1/download/climate — Bulk climate parquet download
  • Hot/cold query routing (Supabase 14-day cache + DuckDB historical parquets)
  • Forward-fill sparse candle data
  • Candle aggregation (1min → hourly, daily)
  • Unit conversion (?units=raw|metric|imperial)
  • Timezone conversion (?tz=UTC|station|<IANA>)
  • Presigned R2 URLs for bulk parquet downloads
  • Per-IP rate limiting (100 req/s)
  • therminal-py v0.1.0 published on PyPI
  • All endpoints wrapped with optional DataFrame support
  • CLI tool included (pip install therminal-py[cli])