Online Alternative-data Reinforcement Learning on Bitcoin Perpetuals

• perpetual means the contract never expires, unlike traditional futures which settle on a fixed date
• funding rate is a periodic payment between longs and shorts that keeps the contract price close to the spot price
• the system trades BTC-USDT perpetuals on Bitunix, a crypto derivatives exchange, using paper mode (simulated fills, no real capital at risk)

Each block polls its own data source on its own cadence, spanning two orders of magnitude from 200 milliseconds down to once every 30 seconds. The feature ring buffer handles the cadence mismatch so that on every tick, the runner pulls the latest cached value from each block regardless of when that block last refreshed. Any block showing all zeros is still warming up its z-score normalization window, which can take anywhere from a few minutes to several hours depending on its polling cadence.

One row per scalar across every feature block feeding the state vector. The block health light (first column) reflects the poll thread's last successful fetch; the scalarhealth light (next to each feature name) reflects the time since that specific scalar's value last changed — a scalar that sits frozen past 2× its expected period turns red even if the block itself is still being polled. Δ% compares the current value against the value immediately prior to its most recent change, so flat scalars don't show a misleading 0.0%.

Every node below is streaming from the running system. Click any node to open a detail panel with architecture, live values, and the full latent or feature vector. Scroll to zoom, drag to pan.

Fills from the runner, streamed live. The NAV columns form a continuous ladder — each open's NAV @ entry equals the prior close's NAV @ exit. ΔNAV is thecumulative_pnldelta between consecutive closes (restart-immune), so the column sums to the account's trade-to-trade realized PnL net of real fees, slippage, and funding. Hold is the time the position was open.

BTC/USDT candles with historical backfill from Bitget plus live ticks from the orderbook feed. Toggle Candle / Linear (a mid-price line); in Linear mode the Bid / Ask overlays draw the live best bid/ask. Scroll to zoom, drag to pan. Entry arrows use the tape's open colours (light blue L / light purple S); exit arrows are green/red by round-trip ΔNAV. Each round trip also draws a price-level segment from its entry price to its exit price, and the current open position shows a dashed entry line.

• 116 dims from 16 live feature blocks (normalized z-scores; slow blocks carry an explicit age dim), each on its own cadence contract
• 64 dims from a book CNN encoder consuming the L2 order book directly
• 64 dims from a GRU temporal encoder consuming a rolling 60-tick window (plus 64 each from the full-state MLP and cross-block attention encoders)

The four 64-dim encoder latents feed a standard continuous SAC head with a 1-dimensional action in the range minus one to one. The raw action is accumulated into a temperature integrator with hysteresis thresholds so the policy cannot churn.

The integrated temperature is what decides whether we open long, exit long, open short, or cover short. The live x-ray further down shows all of this flowing in real time.

Critic loss, actor loss, alpha loss, and the current entropy temperature from the learner process — averaged per second over the entire training history. If you are watching this live, you can see the SAC temperature and losses move in real time as the policy trains.

This solves the cold-start problem. The RL policy has sparse reward, so without auxiliary supervision the encoder latents would be random noise for hours. By the time the SAC head is ready to exploit those representations, they already contain forward-looking signal.

What is working:

• everything is wired up, gradients flowing, encoders learning
• the agent is actively taking positions without blowing through risk limits
• alpha (SAC entropy temperature) is decaying on schedule, meaning the agent is gradually trusting its policy over random exploration
• critic loss is stable rather than diverging
• the hold-triangle reward is visibly pushing the policy toward trade durations long enough to clear simulated costs

What I am explicitly not claiming:

• that this agent is profitable
• that the signal generalizes beyond BTC
• that the feature selection is optimal
• that transaction cost modeling is accurate beyond the 4 basis points baked into the reward
• that a few hours of live data lets me distinguish alpha from noise

• graduate out of burn-in: need rolling 200-trade Sharpe to clear 0.5 on paper, which is a few days of training away if the system is going to work at all
• touch real capital: the calibrated phase starts at $3 positions, which is where the transaction cost modeling actually has to be right
• feature ablations: figure out which of the seventeen blocks are actually doing work, which is the thing I was not able to answer during the capstone window
• extend to a small basket: carry the same stack to a few more symbols once the single-symbol case holds up
• learned risk gate: replace the hand-tuned temperature integrator so the policy decides its own trade cadence instead of inheriting mine
• position sizing: add an appendix on Kelly criterion and volatility targeting, which is out of scope for this research talk

• Dao and Gu, Transformers are SSMs (Mamba2), ICML 2024
• Gu and Dao, Mamba: Linear-Time Sequence Modeling with Selective State Spaces, 2023
• Haarnoja et al., Soft Actor-Critic, ICML 2018
• Cont, Kukanov, Stoikov, The Price Impact of Order Book Events, 2014
• Hyperliquid docs
• Polymarket Gamma API