Methodology

How the forecasts are made, how they are scored, and everything we tested and refused to ship.

The event

A forecast asserts the probability of exactly one event: that the dividend- and split-adjusted total return of the stock, from the adjusted close on the price-basis date to the adjusted close on the 21st trading day afterward, is strictly greater than zero.

How it is scored

Resolution is count-based: exactly 21 observed trading days after the basis date, on the exchange calendar derived from the pinned price history — never a wall-clock date. Each forecast is written to an append-only ledger the moment it is issued and is scored automatically at its horizon. Rows are immutable, enforced by the database itself. A correction, if ever needed, is a new superseding row that carries a reason and points at the row it replaces; the original is never altered or removed.

The universe, and its cohorts

Our universe grew on 2026-08-12 from the S&P 500 to all US-listed common stocks on the major exchanges (NYSE, NASDAQ, AMEX, above a minimum size and share price). To keep the record honest, the calibration page scores two equity cohorts separately: the original S&P 500 as its own continuous series, and the extended universe (cohort us-extended-2026-08) as its own. The aggregate never blends across them, so adding thousands of names can't silently shift the number you've been watching. A name is issued a probability only once it has enough of its own history for a stable climatology; thinner-history names publish distribution ranges only — no directional probability — exactly as crypto does, and the newest names sit in a record-building state until they qualify.

Incident: stale-anchor forecasts (2026-08-12)

On 2026-08-12, the first extended-universe batch issued 1,404 forecasts on tickers that were already delisted or acquired — companies the market data vendor's screener still lists (for example PXD, Pioneer Natural Resources, acquired by ExxonMobil in May 2024, and RETA, Reata Pharmaceuticals, acquired by Biogen in 2023). The history gate checked how many price bars a name had, but not whether its most recent bar was near the batch date, so these forecasts were anchored to prices up to about three years stale. A second verification batch dated 2026-08-11 issued another 1,405 the same way, for 2,809 in total. The bug was a missing price-freshness gate; it was fixed on 2026-08-14 (PR #75), which now refuses to issue — and refuses to display — any forecast whose latest bar is more than ten days before the batch. Because the ledger is public and immutable, none of the 2,809 rows were deleted. Each is voided in place with a resolution marked excluded_reason='stale_anchor_bug_20260812'; every calibration score skips that reason, so the mistake can never move the record in either direction, and the affected pages now show a delisted state instead of a forecast. The count voided (2,809) equals the count issued.

The probability, and its uncertainty

The confidence interval and effective sample size come from a cluster bootstrap: forecasts issued in the same week share the market's realised path, so they are not independent. Resampling whole issuance-weeks rather than individual forecasts prevents the false precision that treating them as independent would produce. The effective sample size is reported alongside every probability, and when only one batch has resolved it is reported as one — not padded.

What we tested — and refused to ship

Eight candidate predictor families, each tested the same way, none beat the base rate out of sample. That record is the argument for the whole product: every page publishes a base-rate probability with a confidence interval and calibrated distribution ranges — never a stock-specific directional call. The table below is that record, each row dated to when the test was run.

Every candidate predictor family is auditioned in a walk-forward harness before it may touch the published probability. Models are refit on resolved-only history over a 2012–2021 development window, S&P 500 point-in-time, 1-month horizon. To clear the bar a family must earn a positive out-of-sample Brier skill score against both climatology and the price/macro baseline, with a calibration slope between 0.8 and 1.25. From the 2026-08-05 round on, the improvement must also survive a cluster bootstrap that resamples whole issuance-weeks (because overlapping monthly windows are not independent): its 90% confidence interval has to exclude zero, and range coverage has to stay between 85% and 95%. The 2022-onward holdout is opened only after a family clears the development gate, and is never used to search for one.

A negative Brier skillA single score for probability accuracy — lower is better; it rewards being both confident and right. score means the family did worse than simply quoting the base rateThe base rate from history — how often this kind of situation resolved up versus down across many past years, before any stock-specific adjustment.. Every family below is negative. We publish this because a track record of refusals is the honest version of a track record.

Skill is measured out of sample against the base rateThe base rate from history — how often this kind of situation resolved up versus down across many past years, before any stock-specific adjustment.; the interval is a week-clusterForecasts made on nearby days cover almost the same future stretch, so they are not independent; we count that overlap so we never overstate the evidence. bootstrap, and any surviving family also had to hold a calibrationWhether our probabilities match reality — of all the times we said an outcome was, say, three-in-five likely, did about three in five actually happen? slope near 1. None did — which is why every page publishes calibrated ranges and a base-rate probability rather than a stock-specific call.

Where that leaves the product

As of 2026-08-05, across eight families — price/macro analogues, a direct price/macro model, earnings surprises, insider transactions (naive and typed), an earnings-in-window re-score, and momentum/trend — none distinguished individual S&P 500 stocks from the roughly 59% one-month base rate by enough to register out-of-sample. Every skill score is negative and, where a confidence interval was computed, it includes zero, so no family cleared the gate and the holdout stayed sealed. So the product ships no stock-specific directional claim: probabilities are the universe base rate (climatology) with an honest confidence interval, alongside per-stock distribution ranges and typed descriptive panels. A future data source graduates to the probability only by earning a positive out-of-sample skill score in this same log.

Families not yet cleared for testing (each requires its own data download and audition entry before any product claim): options flow, institutional trading, and short-interest. Point-in-time analyst estimate revisions were attempted on 2026-08-05 and returned NEEDS-DATA rather than a rejection — the only reachable source gives final consensus values, not the consensus as it stood on each historical date, the same look-ahead flaw that made the earnings-surprise result conclusive; a proper as-of-date feed is required first. Congressional disclosures now carry a reconciled, dated tendency (audition 2026-07-27): purchase clusters (2 or more members buying within 90 days) preceded a modestly higher forward 1-month return — plus 45 bp per month in 2015-2021 and plus 28 bp per month in 2022-2026 versus the weekly-grid baseline. The tilt persisted out-of-sample but roughly halved after 2021. It is shown as a descriptive historical tendency only — not something to act on, and not an input to the forecast probability.

Crypto: a separate record, distributions only

For crypto the event is the same shape as for stocks, but on the calendar-day calendar (coins trade every day): the adjusted total return from the close on the basis date to the close 30 calendar days later, strictly greater than zero. Windows are 7 / 30 / 91 calendar days for the 1w / 1m / 3m horizons.

The crypto universe (v1 'majors') is the set of coins with continuous daily price history since 2017-08-01 that rank in the top 10 by current market capitalization. Two limitations we state plainly: this list contains only survivors, so its historical statistics are flattered by construction; and it is ranked by current — not point-in-time — market cap. A point-in-time universe arrives when a historical-rankings source does.

Before any crypto forecast was published, the same walk-forward discipline was run on crypto majors over a development window ending 2021-12 (the 2022-onward data was held out). The result, dated 2026-07-25: the distribution ranges are well-calibrated — the 90% drawdown and rise bands held about 93% of the time at every horizon, inside the 85–95% target. But the directional base rate did not predict the forward up-rate out of sample (calibration slope near zero or negative at every horizon). So crypto ships its distribution ranges and says plainly that its monthly direction is not something our harness can calibrate — no headline probability.

Crypto records never blend with stocks — separate universe, pool, and scoreboard.

Model panels — token transparency

The earnings/context panels are written by multiple language models, checked, and stored before display. We track what that costs. In 2026-07, generating them took 51 model calls — about 14K input and 6K output tokens. Signed-in visitors can see the full month-by-month breakdown on the usage page.

Verify the record yourself

The forecast ledger is append-only: a forecast is written once, never edited or deleted, and scored automatically at its horizon. To make that promise checkable rather than trusted, each day's forecast rows are hashed into a single daily root hash, published here with its date. Anyone can recompute the same root from the public data; if a single stored value had been changed after the fact, the root would no longer match.

Recipe (algorithm sf-ledger-v1): fetch a day's rows from https://api.tickercove.com/v1/ledger/<date>. For each row, take the listed fields in order, as the exact strings returned, and join them with a vertical bar to form the row's canonical string; a null value contributes an empty field. Take the SHA-256 of that string as lowercase hex — the row hash (the endpoint returns each row_hash so you can check per row). Sort all row hashes as strings, join them with newlines, and take the SHA-256 of the result: that is the day's root hash. Compare it to the value published below and to https://api.tickercove.com/v1/ledger/roots. They must match to the character.

Research tool, not investment advice — historical frequencies, not recommendations. Full terms.