Skip to content

Trade Lifecycle

This page traces a CFD position open from the client's HTTP POST to the Postgres INSERT. The same pattern applies to user-initiated closes, stop-loss/take-profit fires, and margin washouts — they all run through a Lua script followed by a flusher write.

Entry Point

The client sends:

POST /api/positions/open
Authorization: Bearer <jwt>

{
  "instrumentId": "<uuid>",
  "direction": "long",
  "lotSize": 1.0,
  "clientRequestId": "<uuid>",
  "clientPrice": 142.50,
  "slippageTolerancePct": 0.025
}

The server default tolerance is 2.5% (env: TRADE_SLIPPAGE_TOLERANCE_PCT; ADR-0020). The client may omit this field to accept the server default.

The handler lives in ftl-backend/internal/handler/. It validates the JWT with auth.NewJWTValidator, then calls positions.Service.Open.

Idempotency

Before calling the Lua script, the Go service builds the idempotency key:

idem:{userId}:{clientRequestId}

The Lua script performs SET idem:{userId}:{clientRequestId} 1 NX EX 86400 as its first atomic operation after input validation. If the key already exists, the script reads idem_result:{userId}:{clientRequestId} and returns the cached response with replayed = true. The Go handler sees replayed = true and skips non-idempotent side-effects (XP awards, notifications, activity feed writes).

Decision: Redis SET NX as the idempotency boundary over a Postgres unique index — the Postgres index on client_request_id is a partition-local fallback only. A globally unique constraint is impossible on a RANGE-partitioned table. Redis SET NX is globally consistent across replicas via a single Redis instance.

Error codes — POST /api/positions/open (ADR-0020)

Error HTTP status Meaning
PRICE_MOVED 409 Client price deviates > 2.5% from last_published_price. Mint a fresh clientRequestId before retrying.
ErrDuplicateInFlight 409 Replayed request with no cached fill (openPrice == 0 on a duplicate response). Retry with a new clientRequestId.
ErrCorruptPositionState 409 Zero-price open detected at the Go layer.
ErrInvalidSLTP 422 Stop-loss or take-profit level is invalid for the current price and direction.
Insufficient free margin 422 Not enough free margin to open the position.
Instrument inactive 422 Instrument is not currently tradeable.
ErrInstrumentFrozen 423 Instrument is frozen (full-time freeze or admin).
ErrCooldownActive 429 Minimum-hold cooldown (180 s) is still active on this position.

TRADE_SLIPPAGE_TOLERANCE_PCT controls the server-wide tolerance (default 2.5%).

If the api-server crashes after the Lua script commits to Redis but before the Postgres INSERT, the trade-outbox drainer (flusher.NewOutboxDrainer) retries the INSERT from trade_outbox. For CFD positions, flusher.NewPositionOutboxDrainer handles the same fallback from position_outbox.

Lua Script: position_open.lua

File: ftl-backend/internal/redis/lua/position_open.lua

Keys passed in:

Key Purpose
instrument:{id} Live instrument state
wallet:{userId} User balance
positions:{userId} Sorted set of open positionIds (score = opened_at ms)
dirty:instruments, dirty:wallets, dirty:positions Flusher pickup queues
idem:{userId}:{clientRequestId} Idempotency guard
last_open:{userId}:{instrumentId} 180s minimum-hold anchor

The script executes these checks in order:

  1. Input validation — direction must be long or short, lotSize > 0, positionId non-empty.
  2. Instrument state — reads live_base_price (live match price) or falls back to base_price. Rejects if is_active == false or frozen == 1.
  3. Idempotency claim — SET NX idem:... EX 86400. On collision returns the cached result. Instrument-state rejections (inactive, frozen, non-positive base price) are checked in step 2 — before this claim — so they never touch the idem key (bug #135 pattern). For any rejection that occurs after the claim but before the first durable mutation (slippage too wide / PRICE_MOVED, invalid SL/TP, insufficient margin, missing wallet), the script calls reject(), which DELs the freshly-claimed idem key before returning — so a same-clientRequestId retry re-evaluates cleanly instead of replaying an empty {status:"duplicate"} marker. The idem key is persisted (via cache_and_return) only on a successful open or close (ADR-0020).
  4. Slippage check — if clientPrice is supplied, compares it against last_published_price — the price last broadcast over the WebSocket — falling back to last_noisy_price, then the computed server price (ADR-0020). Both fields are fetched in the same instrument HMGET, so no extra Redis round-trips are needed. If the delta exceeds slippageTolerancePct (default 2.5%, env TRADE_SLIPPAGE_TOLERANCE_PCT), returns PRICE_MOVED. On acceptance, fills at clientPrice so the wallet debit matches what the user agreed to.
  5. SL/TP directional validation — for a long, stop_loss < current_price < take_profit. Inverted values would trigger the StopWatch immediately after open.
  6. Equity and free-margin check — iterates positions:{userId} (the user's open positionIds), reads each position:{id} hash to compute used_margin and unrealized_pnl. equity = balance + unrealized. If free_margin < margin_required, returns insufficient_free_margin.
  7. Mutations — writes position:{positionId} hash, ZADD positions:{userId}, updates net_position_imbalance on the instrument, sets last_open:{userId}:{instrumentId} with a 180s TTL (minimum hold), SADD to all three dirty sets.
  8. Pub/sub — PUBLISH price:{instrumentId} with new price, PUBLISH portfolio:{userId} with updated equity/margin snapshot.

The leverage is 1:10, contract size is 5 shares per lot. Margin required = (openPrice × lotSize × 5) / 10.

Dirty Sets → Flusher → Postgres

The Lua script adds three members to dirty sets: - SADD dirty:instruments <instrumentId> - SADD dirty:wallets <userId> - SADD dirty:positions <positionId>

The flusher (ftl-backend/internal/flusher/flusher.go) runs every FlushInterval (100ms in production). On each tick it calls flush(), which processes all three dirty sets.

For each dirty set, the flusher: 1. Atomically renames dirty:instruments → dirty:instruments:flushing:<ts> via RENAME. New writes accumulate in a fresh live key untouched by this cycle. 2. Reads all members from the snapshot key with SMEMBERS. 3. For each member, reads the Redis hash (instrument:{id}, wallet:{userId}, position:{id}) and writes to Postgres. 4. Requeues failed members back to the live dirty set so the next tick retries. 5. Deletes the snapshot key.

For instruments the Postgres write is: 

UPDATE instruments SET net_shares_sold = $1, updated_at = NOW() WHERE id = $2

For legacy positions (the buy/sell model): 

INSERT INTO positions (user_id, instrument_id, shares, avg_price)
VALUES ($1, $2, $3, $4)
ON CONFLICT (user_id, instrument_id)
DO UPDATE SET shares = $3, avg_price = $4, updated_at = NOW()

For CFD positions, the position_outbox table holds any row that failed the synchronous Postgres write. flusher.NewPositionOutboxDrainer drains it every 5 seconds.

On flusher startup, RecoverOrphanSnapshots scans for dirty:*:flushing:* keys left by a prior crash and merges them back into the live sets before the first tick.

Price Broadcast After Trade

The Lua script publishes to price:{instrumentId} inside the same atomic execution. Each ws-server replica runs a ws.Subscriber goroutine that subscribes to all price:* channels on startup. When a publish arrives, the subscriber decodes the JSON, marshals a PriceUpdate struct to msgpack, and calls hub.Broadcast. The hub iterates the subscription map for that instrument's numeric index and calls writeWithDeadline (100ms timeout) on each connection.

Portfolio state is published to portfolio:{userId} in the same Lua execution. The ws-server routes this via hub.BroadcastToUser to all connections subscribed to that user's portfolio channel.

Sequence Diagram

sequenceDiagram
    participant C as Client
    participant API as api-server
    participant RD as Redis
    participant WS as ws-server
    participant Browser as Browser (other users)
    participant FL as flusher
    participant PG as Postgres

    C->>API: POST /api/positions/open {instrumentId, direction, lotSize, clientRequestId, clientPrice}
    API->>API: Validate JWT, build Lua keys
    API->>RD: EVALSHA position_open.lua (8 keys, 11 args)
    Note over RD: Lua atomic:<br/>1. Validate inputs<br/>2. Check instrument active/frozen<br/>3. SET NX idem key<br/>4. Slippage check vs last_published_price (fallback: last_noisy_price → server_price)<br/>5. Compute equity + free_margin<br/>6. Write position:{id} hash<br/>7. ZADD positions:{userId}<br/>8. Update net_position_imbalance<br/>9. SADD dirty sets<br/>10. PUBLISH price:{instrumentId}<br/>11. PUBLISH portfolio:{userId}
    RD-->>API: {status:"opened", positionId, openPrice, marginRequired, equity}
    API->>PG: INSERT INTO cfd_positions (sync, may fail → outbox)
    API-->>C: 200 {positionId, openPrice, marginRequired}

    Note over RD,WS: Async — Redis pub/sub backplane
    RD->>WS: PUBLISH price:{instrumentId}
    WS->>Browser: msgpack PriceUpdate frame (binary, ~25 bytes)

    RD->>WS: PUBLISH portfolio:{userId}
    WS->>C: JSON portfolio frame (equity, usedMargin, freeMargin)

    Note over FL,PG: Async — flusher 100ms tick
    FL->>RD: RENAME dirty:instruments → dirty:instruments:flushing:<ts>
    FL->>RD: SMEMBERS dirty:instruments:flushing:<ts>
    FL->>PG: UPDATE instruments SET net_position_imbalance = ...
    FL->>RD: RENAME dirty:wallets → dirty:wallets:flushing:<ts>
    FL->>PG: UPDATE wallets SET balance = ...
    FL->>RD: RENAME dirty:positions → dirty:positions:flushing:<ts>
    FL->>PG: INSERT INTO cfd_positions ON CONFLICT DO UPDATE

Legacy Trade Path

The original BUY/SELL spot trade (POST /api/trades) follows the same pattern through ftl-backend/internal/redis/lua/trade_execute.lua. That script uses position:{userId}:{instrumentId} hashes instead of per-position UUIDs, and records the fill in the trades table (range-partitioned by day). The idempotency key, dirty sets, pub/sub publish, and flusher drain are identical.

During the ADR-0004 rollout both endpoints are live. The CFD path is the preferred path for new frontend work.

Known issues (Sportmonks upstream)

These are upstream data gaps, not trade execution bugs:

  • Goal and card event feeds are empty for some fixtures. Sportmonks does not supply goals and cards for every fixture on the current plan. Live price moves driven by match events will be absent for those fixtures.
  • Per-player long-ball, aerial, and tackle stats are absent for U18 competitions. Only fixture-level team totals are available for under-18 fixtures.
  • Kazakhstan U18 squad returns 0 players. The squad is absent from the Sportmonks data plan. Instruments are created but have no playable squad data.

These affect live price moves but not trade execution correctness.