Output format
The structure of tick-level simulation data returned by Pulse.
Pulse outputs data in Pulse Format — a normalised, exchange-agnostic representation of limit order book activity. The same schema works across any supported venue, so analysis code written for one exchange transfers directly to another without modification.
Every simulation produces two synchronized DataFrames: L2 (order book snapshots) and Ticks (order events). They are always the same length and share a sequence index — every row in L2 has a corresponding row in Ticks at the same position.
The default file returned by get_sim_data(sim_id) is sim_data.parquet, which contains both DataFrames. See Download data for the full list of output files.
import polars as pl
sim_id = client.simulation.list_cached()["simulations"][0]["example_sim_id"]
df = client.simulation.get_sim_data(sim_id)
# L2 and Ticks share the same index
l2 = df.filter(pl.col("bid_price_1").is_not_null())
print(df.columns)L2 — order book snapshots
Each row is the state of the order book after the corresponding event in Ticks has been applied. HKEX Securities provides 10 price levels on each side.
| Column | Type | Description |
|---|---|---|
| sequence | Int64 | Shared monotonically increasing index, aligned with Ticks |
| timestamp | Datetime[ns] | Event time at the highest resolution available (nanoseconds) |
| bid_price_1 … bid_price_10 | Float64 | Bid prices from best (1) to 10th level |
| bid_size_1 … bid_size_10 | Int64 | Total volume resting at each bid level |
| bid_count_1 … bid_count_10 | Int64 | Number of individual orders at each bid level |
| ask_price_1 … ask_price_10 | Float64 | Ask prices from best (1) to 10th level |
| ask_size_1 … ask_size_10 | Int64 | Total volume resting at each ask level |
| ask_count_1 … ask_count_10 | Int64 | Number of individual orders at each ask level |
The pre-computed l2_by_second.parquet file gives you the same columns sampled once per second — useful for most analysis without processing tick-level data. See Download data for the schema.
Example: bid-ask spread
sim_id = client.simulation.list_cached()["simulations"][0]["example_sim_id"]
df = client.simulation.get_sim_data(sim_id)
spread = df["ask_price_1"] - df["bid_price_1"]
mid = (df["ask_price_1"] + df["bid_price_1"]) / 2
print(f"Mean spread: {spread.mean():.4f}")
print(f"Mean mid: {mid.mean():.4f}")Example: book depth at a snapshot
sim_id = client.simulation.list_cached()["simulations"][0]["example_sim_id"]
df = client.simulation.get_sim_data(sim_id)
# First snapshot
row = df.row(0, named=True)
print("Bid side:")
for i in range(1, 11):
print(f" L{i}: {row[f'bid_price_{i}']} x {row[f'bid_size_{i}']} ({row[f'bid_count_{i}']} orders)")
print("Ask side:")
for i in range(1, 11):
print(f" L{i}: {row[f'ask_price_{i}']} x {row[f'ask_size_{i}']} ({row[f'ask_count_{i}']} orders)")Ticks — order events
Each row captures a single market event. Ticks is aligned row-for-row with L2 — join on sequence to link an event to the book state it produced.
| Column | Type | Description |
|---|---|---|
| sequence | Int64 | Shared index, aligned with L2 |
| timestamp | Datetime[ns] | Event timestamp at the highest resolution available (nanoseconds) |
| message_type | Int64 | Event classification — see message types below |
| side | Int64 | 1 = BID, −1 = ASK (passive/resting side — see side convention) |
| price | Float64 | Price level of the order |
| size | Int64 | Always positive — meaning depends on message type |
| order_id | Utf8 | Exchange identifier linking related events for the same order |
Message types
| Value | Name | Description |
|---|---|---|
| 1 | PLACE | New limit order submitted to the book |
| 2 | TRADE | Aggressive order executed against a resting order |
| 3 | AMEND | Existing resting order modified |
| 4 | CANCEL | Resting order removed from the book |
Size semantics by message type
size is always positive but its meaning differs by event:
| Message type | size means… |
|---|---|
| PLACE | Original quantity submitted to the book |
| TRADE | Quantity executed in this fill (size of the aggressive order) |
| AMEND | Remaining quantity of the order after the amendment — the order's new absolute size, not the change |
| CANCEL | Quantity removed from the book — the order's remaining quantity at the moment of cancellation, never 0 |
Some exchanges do not include sizes in CANCEL and AMEND messages. Pulse back-fills these from the preceding PLACE or AMEND record. Rows where price or size cannot be determined are flagged with _to_drop = True and can be excluded via df.filter(~pl.col("_to_drop")).
Side convention
side always reflects the passive (resting) order, not the aggressor. For a TRADE, this is the side of the resting order being consumed — not the side of the incoming order that triggered the fill.
| side value | Meaning |
|---|---|
| 1 | BID — the resting order was on the buy side |
| −1 | ASK — the resting order was on the sell side |
Example: filter by event type
import polars as pl
sim_id = client.simulation.list_cached()["simulations"][0]["example_sim_id"]
df = client.simulation.get_sim_data(sim_id)
MESSAGE_TYPES = {1: "PLACE", 2: "TRADE", 3: "AMEND", 4: "CANCEL"}
# All trades
trades = df.filter(pl.col("message_type") == 2)
# Cancels on the bid side
bid_cancels = df.filter(
(pl.col("message_type") == 4) & (pl.col("side") == 1)
)
print(f"Trades: {len(trades)}, Bid cancels: {len(bid_cancels)}")Example: reconstruct trade flow
import polars as pl
sim_id = client.simulation.list_cached()["simulations"][0]["example_sim_id"]
df = client.simulation.get_sim_data(sim_id)
trades = df.filter(pl.col("message_type") == 2)
# Volume on each side
buy_vol = trades.filter(pl.col("side") == -1)["size"].sum() # aggressor hit the ask
sell_vol = trades.filter(pl.col("side") == 1)["size"].sum() # aggressor hit the bid
print(f"Buy-initiated volume: {buy_vol:,}")
print(f"Sell-initiated volume: {sell_vol:,}")
print(f"Net order imbalance: {buy_vol - sell_vol:,}")