Data before models: an AI-readiness checklist for trading and risk teams

When an AI pilot on a trading desk fails, the post-mortem rarely blames the model. It blames the data: trades that didn't match the confirmations, curves with unexplained overrides, settlement history nobody could trace. The model did what it was told with what it was given.

So before anyone picks a model, I'd check the data. Here's the checklist I use, a 30/60/90-day plan to get from "we should do AI" to a pilot you can defend, and the questions your risk committee is going to ask.

The five tests

For each data domain, ask five things:

• Complete: is everything there, including amendments, cancellations, and the awkward deal types?
• Timely: is it available when the use case needs it, not just at month-end?
• One source of truth: is there a single authoritative copy, or several that disagree?
• Access-controlled: can you give a model, and the people using it, exactly the access they need and no more?
• Auditable: can you show what the data was at a point in time, and who changed it?

Score each domain red, amber, or green against each test. You'll end up with a 5-by-5 grid. That grid is your readiness assessment.

Trade data

• Are amendments and cancellations stored as events with timestamps, or do they overwrite the original trade?
• Do booked trades match confirmations, and is the break rate tracked?
• Are proxy bookings (deals booked as something they're not) flagged anywhere?
• Can you reproduce the trade population as of any past date?

If the answer to the last one is no, any model trained or tested on history is learning from a version of the book that never existed.

Reference data

• Do counterparties, books, products, locations, and calendars each have a named owner?
• Are there duplicates, like the same counterparty under two names or two location codes for one delivery point?
• Is reference data history kept, or only the current value?

Reference data is the most overlooked domain. It's also where most "the model is confused" problems start.

Curves and market data

• Is every curve point tagged with its source: broker quote, exchange settle, derived, or manual?
• Are manual overrides logged with a reason and an approver?
• Is curve history stored as it was used on each day, not as it was later corrected?

If curves are corrected in place, you can't explain yesterday's P&L, and neither can a model.

Settlements

• Is each invoice line traceable to trades, volumes, and prices?
• Are disputes and their resolutions recorded in a structured way, with a reason code?
• Are shadow spreadsheets in settlements inventoried, with what each one checks?

Settlement exception triage is one of the better early AI use cases. It only works if past resolutions are recorded, not just done.

Registry and environmental data

• Do internal REC positions reconcile to registry accounts by serial range, not just by quantity?
• Are eligibility attributes and status changes stored as data with an audit trail?
• Is there a clear record of what was retired, for which program, and when?

If you run REC programs, this domain often has the best documentation and the worst integration. Fix the integration first.

The 30/60/90-day plan

Days 1-30: inventory and baseline. - Fill in the 5-by-5 grid with the people who use each data set every day. - Pick one or two candidate use cases where a person stays in the loop and the output is easy to check. - Record a baseline for each: hours spent today, error or break counts, and turnaround time. Without one, you can't show the pilot worked.

Days 31-60: fix the data the pilot needs. - Fix only the domains the chosen use cases depend on. Don't try to fix everything. - Assign owners to the reference data involved. - Set up access: a read-only, logged path from the data to the pilot environment. - Write the control: who reviews outputs, how, and what happens when the output is wrong.

Days 61-90: pilot-ready. - Run the use case in parallel with the current process, not instead of it. - Log inputs, outputs, and reviewer decisions for every case. - Compare against the baseline. Decide to scale, change, or stop based on the numbers you recorded, not on how the demo felt.

What your risk committee will ask

Have written answers before you walk in.

1. What decisions does this support, and who remains accountable for them?
2. What data does it use? Who owns that data, and how is its quality checked?
3. Does any confidential, counterparty, or personal data leave our environment? Where is it processed and stored?
4. How is the output reviewed before anyone acts on it?
5. What happens when it's wrong? How would we detect that, and how fast?
6. What's logged, and for how long, so we can reconstruct what the model saw and produced?
7. How do we know it keeps working? Who monitors it, and what triggers a review or a shutdown?
8. Does it fit our existing model-risk, change-management, and vendor-risk policies, or do those need updating first?

If you can't answer question 6, you're not ready. In a trading business, "we can't reconstruct what happened" is the answer that ends the conversation.

Use it this week

Build the 5-by-5 grid. One hour, with one person from trading ops, one from risk, one from settlements, and one from IT. Don't argue about the colors. Write down the disagreements. They tell you where the data problems are.

I'm putting together a more structured version of this readiness review. Newsletter subscribers will hear about it first. In the meantime, reply to any issue and tell me which square on your grid is the reddest.

Drafted with AI assistance from my research notes and outline; reviewed, edited, and fact-checked by me. The opinions are mine.