Tom Xiao

About

Is this number right, and who is allowed to see it?

Seven years of working on that one question, first for analysts, now for AI agents. Each step below was pushed by the problem the previous one exposed.

7+ years on data platforms10 articles, one covered on the AWS Big Data Blog

2018–2020 · Stage 1

Data Analyst

a consumer platform

I started by answering business questions and building the reports people relied on every week. The analysis was sound, but I kept finding that a wrong answer rarely came from the analysis itself. It came from how a metric was defined, or from the data upstream of it.

LearnedMost wrong answers come from definitions and upstream data, not from the analysis.

so → I moved upstream

2020–2021 · Stage 2

Data Engineer

a consumer platform

I became a data engineer on the same platform. I built a layered warehouse across the business domains and generated the scheduling from the dependencies between SQL files, so nobody had to maintain run orders by hand. I also worked on a change-data pipeline that crossed cloud providers.

LearnedScale comes from automation and conventions, not from people watching jobs.

so → I went where a number must be exact

2021–now · Stage 3

Data Engineer

a financial platform

In a financial platform, a number that is nearly right is still wrong. I worked on the shared data platform and its warehouse. Two overlapping platforms became one, now running on Kubernetes. I designed and built a reconciliation platform, a live risk-feature service and a mapping that ties each security together across price providers.

LearnedCorrectness is a property of the system: data contracts, gates and human approval.

Reconciliation platformData on EKSAll stage 3 work

so → the users changed: agents began reading the data

Now · Stage 4

AI data engineer and agent engineer

The people asking for data are no longer only people. Agents read it too, and they sound equally sure when they are wrong.

As an AI data engineer I design what lets an agent work on real data: a semantic layer with the business definitions and the meaning of each value, identity injected by the gateway instead of chosen by the model, queries that run with the user's own permissions, and generated SQL that runs only after the user confirms. That is in production.

As an agent engineer I designed and built the AI portfolio assistant for an investment app, and it is live. I also built a public multi-agent harness, and a pipeline that tests new AI models on real tasks.

LearnedThe model explains. The code counts.

Agents on your dataAI portfolio assistantAll stage 4 work

Based inHong Kong
Works withPython, SQL, Trino, Hive, Airflow, Kubernetes, AWS
SpeaksMandarin, English, Cantonese
ElsewhereGitHubYouTubeXLinkedIn

Have a number that has to be right?

I reply within two working days.