Built for Prophet Hacks

SIBYL

Retrieval beats recency.
Every prediction market has a lag window.
We live inside it.

Python 3.12 FastAPI GPT-4o-mini Gemini 2.5 Flash Claude Sonnet 4 litellm Exa Search

https://api.sibyl.edycu.dev/

sibyl — live prediction
# Real-time forecasting with evidence retrieval
$ curl -X POST api.sibyl.edycu.dev/predict \
  -H "Authorization: Bearer •••" \
  -d '{"title": "Will Fed raise rates in June?"}'
 
# 180ms later
{
  "probabilities": {"Yes": 0.63, "No": 0.37},
  "confidence": 0.81,
  "model": "gemini-2.5-flash",
  "sources_retrieved": 4
}
the edge
8 min

The window between breaking news and a fully priced prediction market.
That's where every alpha lives. That's where Sibyl operates.

// retrieval-augmented · market-anchored · calibrated

the problem

Models are frozen.
Markets are not.

Maya holds a Kalshi contract on Fed rate hikes, priced at 42¢. The CPI report dropped 8 minutes ago. She knows the market is wrong — but she can't quantify by how much. Existing AI forecasters just re-prompt the same stale training data and return the same wrong answer.

Stale training data
LLM knowledge cutoff is months old. No access to today's CPI release, earnings call, or injury report.
No market anchor
Most agents ignore the prediction market price entirely — a free calibrated prior from thousands of traders they're throwing away.
One-size-fits-all routing
Running GPT-4 on an 85% confidence sports result is expensive overkill. No cost tiering means burning budget on easy questions.
the solution

Retrieval-augmented
forecasting.

Sibyl treats prediction as an evidence retrieval problem. The LLM reasons — the pipeline creates the edge.

01 Market Anchoring — uses the current prediction market price as a calibrated Bayesian prior, not a from-scratch guess.
02 Category-Specific Retrieval — routes each question to a specialized pipeline. Sports gets injury feeds. Economics gets CPI data.
03 Calibrated Ensemble — post-hoc Platt scaling trained on historical Prophet Arena data, plus cost-tiered model selection.
pipeline internal trace
Step 1 parse_event("Will Fed raise rates?")
Step 2 classify → Economics
Step 3 anchor → market_price=0.42
Step 4 retrieve(Exa) → 4 sources
Step 5 assemble_context(3.8k tokens)
Step 6 route → confidence=0.71 → gemini-flash
Step 7 reason → raw_p_yes=0.68
Step 8 calibrate(platt) → 0.63
 
→ {"Yes": 0.63, "No": 0.37}
how it works

8-Step Prediction Pipeline

End-to-end: raw event text → calibrated probability in <500ms

Event input Parser normalize Classifier GPT-4o-mini Market Anchor Bayesian prior p_market = 0.42 Evidence Retrieval Exa / Brave search 4 sources · 3.8k tok Assembler 4K token budget Model Router cost-tiered GPT-4o-mini high confidence Gemini 2.5 Flash medium confidence Claude Sonnet 4 close call 40–60% Calibration Platt scaling Output p(Yes) = 0.63 Retrieval layer AI / LLM layer Calibration layer avg 180–500ms · 100% completion rate
live demo

Try it now.
It's live.

Deployed at api.sibyl.edycu.dev — Bearer auth, OpenAI-compatible.

live request → response
① $ curl -X POST https://api.sibyl.edycu.dev/predict \
   -H "Authorization: Bearer 483d62…" \
   -H "Content-Type: application/json" \
   -d '{"②title": "Will SpaceX catch Super Heavy?",
      "category": "Technology"}'
 
{
  ③"probabilities": {"Yes": 0.71, "No": 0.29},
  "reasoning": "Recent flight success rate suggests...",
  ④"sources_retrieved": 5,
  "model": "gemini-2.5-flash"
}
field annotations
1
Bearer token auth — same token works for /predict and OpenAI-compatible /chat/completions.
2
Free-text event title — no rigid schema. Sibyl parses title, description, outcomes, and category from plain text.
3
Probabilities object — Prophet Arena native format. Also returns p_yes for backwards compatibility.
4
Sources retrieved — transparency into evidence chain. Zero sources = LLM-only fallback using calibrated market price.
● Live at api.sibyl.edycu.dev
feature 01

Market
Anchoring

Instead of asking "what's the probability?", Sibyl asks "how much should I update from what the market already says?" The market price is a calibrated prior from thousands of informed traders. We start there and adjust on fresh evidence.

Systematic edge over naive LLM baseline
Category Routing

5 specialized retrieval pipelines — each tuned to its domain's evidence patterns. Sports gets injury feeds. Economics gets macro data. Geopolitics gets Reuters.

Sports Economics Geopolitics Science Pop Culture
Cost Tiering

Route high-confidence events to GPT-4o-mini. Close calls (40–60% market) get Claude Sonnet 4. Full 2-week evaluation window estimated at $15–40.

GPT-4o-mini · fast Gemini Flash · balanced Claude · precise
why now · why us

Three forces converge
right now.

01 / timing
Prophet Arena is live

The first evaluation harness that scores AI agents on real prediction markets with real money at stake. Without it, we'd need 7 separate systems ourselves.

15 integration points used

02 / capability
Retrieval APIs exist

Exa, Brave, Tavily — production-grade web search SDKs with sub-200ms latency. The Prophet Arena SDK ships ai_prophet.search.SearchClient as a first-class primitive. This wasn't possible 18 months ago.

4 sources per prediction

03 / moat
LLM cost asymmetry

GPT-4o-mini is 40× cheaper than GPT-4 on high-confidence sports results. Claude Sonnet 4 demonstrably outperforms on ambiguous political questions. Cost-aware routing is a durable technical moat.

$15–40 total · 100% completion

traction

Shipped.
Not pitched.

48h

from zero to live deployment

delivered in the window
✓
8-step prediction pipeline — parse, classify, anchor, retrieve, assemble, route, reason, calibrate
✓
Dual endpoints — /predict + OpenAI-compatible /chat/completions
✓
Platt scaling calibration trained on 26 resolved Prophet Arena events
✓
Cost-tiered model routing — GPT-4o-mini / Gemini Flash / Claude Sonnet 4 via litellm
✓
Brier score 0.1644 vs ~0.201 baseline — measured edge of +0.0366 on 26 resolved Prophet Arena events
✓
100% completion guarantee — fallback to calibrated market price, no unanswered events
✓
Docker + live deployment at api.sibyl.edycu.dev with Bearer auth and Swagger docs
✓
CI pipeline + 118 passing tests — Ruff linting, Pytest coverage, GitHub Actions
the ask

Run Sibyl.
Beat the baseline.

Point the Prophet Arena evaluation harness at our endpoint. We built every integration point — from SDK retrieval to Platt calibration — so the harness just works.

API https://api.sibyl.edycu.dev/
Token 483d6297…cc2be0c1 (see README)
Docs api.sibyl.edycu.dev/docs
GitHub github.com/edycutjong/sibyl
License MIT · Open source
Most Technically Sophisticated Best Prophet Arena Integration
Edy Cu
edy.cu@live.com · Full-Stack & ML
api.sibyl.edycu.dev
Retrieval beats recency. Every time.
01 / 10