The short version.
Bank of America was moving a large Hadoop data estate and needed to be certain nothing broke on the way. As a contract application architect I owned the validation — testing HDFS migrations against their integrity benchmarks, then hunting down the silent failures and stale lineage that a move like this exposes. Alongside it I built Python and JavaScript tooling that gave trading operations about 30% better visibility into their own data. Careful, checkable work on numbers people traded on.
A migration you can't afford to get wrong.
Moving a data platform at a bank is not a weekend project. Trading operations depend on the numbers being right, and a migration is exactly the moment integrity quietly slips — a dropped row here, a broken lineage there, discovered weeks later when someone acts on a figure that was wrong the whole time. The migration couldn't proceed on faith. It needed proof at each step.
On top of that, the data feeding executive and trading views carried failures nobody had traced yet, and onboarding a new engineer into systems this tangled took months. Both were drags on a team that couldn't afford either.
Prove every move. Then make the data usable.
I ran migration-ready testing and validation for the HDFS moves, holding each step against the integrity benchmarks so the team could migrate knowing the data survived intact. Where the pipelines were failing silently, I worked across five-plus cross-functional pods to surface the breaks and refresh the lineage that had gone stale.
Then I built the tooling that turned all that clean data into something operations could see — custom Python and JavaScript that raised executive trade-data visibility by roughly 30%. And because the systems were hard to learn, I wrote down how they actually fit together, which pulled new-developer ramp time from months down to weeks.
Numbers the trading floor could trust.
The migration went through with its integrity intact, the silent failures got names and fixes, and trading operations came away with a view of their data that was about 30% clearer than before. The onboarding work compounded quietly — every engineer who joined after got up to speed in weeks instead of months.
This is the enterprise end of what I do, and the through-line to the smaller work is exact: data you can trust is the precondition for everything else, including any AI agent you eventually point at it.
What people ask about this one.
- What does data migration validation involve?
- Proving the data that lands after a move is the same data that left — row counts, integrity benchmarks, lineage. On the Bank of America engagement that meant testing HDFS moves against those benchmarks so the migration could proceed without silently corrupting anything downstream.
- Why does trade data need special handling?
- Trading decisions run on it, so a wrong or stale number isn't a reporting nuisance — it's a bad call with money on it. The work here was making the numbers visible and trustworthy fast enough for operations to actually use them.
- How does this connect to AI and automation work?
- Every AI system is only as good as the data underneath it. The discipline of validating a migration and cleaning up lineage is the same groundwork that makes an AI agent safe to point at production data later.
See the other case studies or how an engagement starts.