Skip to content

pmlab 1.0.2stable API · MIT · Python 3.12+

Prediction markets, without lookahead.

pmlab is a plugin-based ML framework for Polymarket. Write four methods for a market family; pmlab does the rest: edge and cost arithmetic, walk-forward backtests, a hard GO / NO_GO gate before anything is promoted, paper trading, and settlement that waits until the truth is final.

pip install pmlab
Start the tutorial API reference GitHub
rolling origin · bundled sample 60 folds
  1. fold 1 0.101
  2. fold 12 0.086
  3. fold 24 0.106
  4. fold 36 0.085
  5. fold 48 0.089
  6. fold 60 0.087

traintest daynot seen

brier by fold, lower is better (0.075 to 0.117)

folds
60
trades
298
brier
0.0957
gate
NO_GO

Synthetic sample: 750 markets, 5 cities, 150 days, no network. It says nothing about real markets.

signals
  • Athens 07-31 <=24°C mkt 0.109 model 0.280 +0.167 YES
  • Istanbul 07-31 <=24°C mkt 0.111 model 0.011 +0.093 NO
  • Lisbon 07-31 31°C mkt 0.208 model 0.318 not traded: Lisbon failed the gate
  • Madrid 07-31 >=37°C mkt 0.086 model 0.549 +0.460 YES
  • Rome 07-31 25°C mkt 0.158 model 0.029 +0.122 NO
  • Athens 08-01 <=26°C mkt 0.112 model 0.015 +0.090 NO
  • Istanbul 08-01 30°C mkt 0.064 model 0.170 +0.103 YES
  • Lisbon 08-01 29°C mkt 0.179 model 0.238 not traded: Lisbon failed the gate
  • Madrid 08-01 >=36°C mkt 0.077 model 0.620 +0.539 YES
  • Rome 08-01 25°C mkt 0.121 model 0.204 +0.079 YES
  • Athens 08-02 <=26°C mkt 0.097 model 0.004 +0.086 NO
  • Istanbul 08-02 29°C mkt 0.144 model 0.022 +0.115 NO

It will not look ahead, or ship a loser.

Enforced in code and checked by the test suite, not left to convention. Each claim below links to the page that shows how.

  1. No lookahead.

    rolling_origin_eval trains only on rows from before the day it predicts. build_panel refuses a price seen at or after the decision it informs, and check_no_leakage raises when a feature gives the outcome away.

    fold  1  train 2025-04-01 → 2025-05-30  test 2025-05-31
fold 60  train 2025-04-01 → 2025-07-28  test 2025-07-29  (5355 rows)
    How the checks work

  2. A failed gate cannot ship.

    ChampionManifest.publish raises unless the gate says GO, and that is not configurable. On the sample the gate reads every city, one of them loses money, and publishing is refused. Scope the champion to the cities that pass and it publishes, with that scope recorded in champion.json.

    • Athens +33.72 ok
    • Istanbul +20.36 ok
    • Lisbon -10.98 negative_pnl
    • Madrid +26.41 ok
    • Rome +26.15 ok 
    ValueError: Cannot publish champion with NO_GO gate. Gate decision: NO_GO
    The champion registry

  3. Reproducible.

    Every backtest records its seeds, library versions, resolved config and a fingerprint of the data. pmlab reproduce re-runs it and verifies that the metrics come back.

    reproduce → reproduced  data match: yes  metrics compared: 5
    Reproducibility

  4. Frozen API.

    From 1.0 pmlab follows semantic versioning. What is public is compared with a checked-in snapshot on every change, and nothing public is removed before 2.0 without a deprecation warning that names the replacement.

    145 public names in 8 modules · 11 CLI commands · tests/api/public_api.json
    What is public

  5. Five rules, in code.

    A champion is never published with a NO_GO gate. Canonical training data is never overwritten without an explicit flag. Workspaces are never mixed: live forward evidence and training data stay apart. The champion's segment gate is always read from champion.json, the record made at promotion. A trade is never settled before MarketPlugin.is_truth_final says the truth is final.

    Architecture

Run it in a minute.

Offline, on a small synthetic sample that ships in the package: no network, no account, no order placed. The code is the README's, and CI runs it.

Quickstart

This runs offline on a small synthetic sample bundled in the package: 750 weather-temperature markets, no network and no account. The numbers say nothing about real markets.

from pmlab import (ChampionManifest, HoldoutGateResult, LGBMForecaster, TradePolicy,
                   build_panel, rolling_origin_eval)
from pmlab.datasets import load_sample

sample = load_sample()                       # 750 synthetic weather markets, checked against a checksum
resolver = sample.plugin()                   # the real WeatherTmaxPlugin, wired to the bundled file
found = [(day, spec) for day in sample.days for spec in sample.plugin(day).discover_markets()]

# One row per (market, bin), each at its own price. build_panel refuses a price seen at or after
# the decision, and check_leakage=True raises if a feature gives the outcome away.
panel = build_panel(
    [resolver.build_training_rows(s, "previous_evening", decision_time=f"{d}T12:00:00+00:00")
     for d, s in found],
    check_leakage=True,
)
# The forecast alone cannot tell a market's bins apart: add how far each bin sits from it.
bounds = {(s.market_id, b.label): [x for x in (b.lower, b.upper) if x is not None]
          for _, s in found for b in s.outcome_bins}
centre = [sum(bounds[k]) / len(bounds[k]) for k in zip(panel["market_id"], panel["outcome_label"])]
panel["feature_bin_dist"] = centre - panel["feature_fcst_tmax"]

# Walk-forward backtest: every fit sees only rows from before the day it predicts.
model = LGBMForecaster(random_state=7, n_jobs=1)
result = rolling_origin_eval(
    panel, model, stride=10, min_train_rows=2700,
    policy=TradePolicy.live(min_edge=0.05),  # what `pmlab record-trades` trades: best bin and side
)
gate = HoldoutGateResult.evaluate(result.trades)
print(len(result.trades), "trades | PnL", round(result.trades["realized_pnl"].sum(), 2), "|", gate.decision)
panel.to_parquet("panel.parquet")            # the command line below reads this file

# A gate that fails cannot publish a champion.
failed = HoldoutGateResult.evaluate(result.trades, required_segments=["Atlantis"])
try:
    ChampionManifest.publish(model, failed, "champion", plugin_family="weather_tmax")
except ValueError as err:
    print("refused:", err)

It prints the number of trades, the total PnL and the gate's decision, then the refusal. The same panel runs from the command line, and the tutorial continues from here: tuning, the gate per city, paper trading and the report.

pip install pmlab            # Python 3.12+; or: uv add pmlab
pip install "pmlab[tune]"    # + Optuna, for TunedForecaster
pip install "pmlab[track]"   # + MLflow, for MLflowTracker

Where to go.

Source and issues on GitHub · PyPI · Changelog · Roadmap · Deprecation policy. pmlab is research software and not financial advice: trading prediction markets can lose all the money you put in, and no pmlab command places an order.