Data engineering
Why a transfer can break your player database
A missing player history can be a join error. That matters when the next recruitment decision depends on it.
The transfer is complete, but the database has split the player’s career. His new club appears in the dashboard; his academy matches have vanished from the profile. In this hypothetical case, the files still exist. The fault lies in the way identities and team membership were joined. Investigate that link before interpreting the missing history as a lack of experience.
One player, more than one team registration
Earlier team
Registration dates and appearances
Receiving team
Registration dates and appearances
Tayo Stats’s June 2026 account of building Afriskaut describes resolving player and team identities before profiling players. It is a practitioner’s account, not an independent evaluation. Its relevance here is specific: reliable identification comes before a convincing scouting profile. A career must remain connected when the provider, spelling, or club changes.
The player and the registration are different records
Separate the person from their spell at a team. A player identifier should connect the career; a registration record should explain which team and competition an appearance belongs to. When a system uses a player’s name and current club as the join key, a transfer can split one career into unrelated records. Names also change spelling across providers, so a tidy-looking text match is insufficient verification.
Keep the provider’s original identifier and an explicit mapping to the club’s own identifier. Unmatched records should remain visible for investigation rather than disappearing from the analysis. A missing value can mean that an event was not supplied, that the competition was not covered, or that the join failed. Those explanations support different recruitment decisions.
A successful import is only the first check
Beggs’s chapter on harvesting football data explains how to bring files into an analytical environment. The engineering work continues after that import: establish the table’s unit of observation, preserve the source, and check that combining tables has not duplicated appearances. If one appearance becomes several rows after a join, totals can increase without any change in the player’s actual performance.
Use checks with a football meaning. Reconcile the player’s appearances and minutes against the source before and after a transfer; inspect whether competition and season labels survive; and flag unexpected team changes. These checks should run again when a provider changes its export. A pipeline that reports success while silently dropping an academy season has failed the reader even if its code completed.
More samples are not more independent evidence
Tracking volume is not a sample of separate performances
- One second
- 10samples
- One minute
- 600samples
- 90 recorded minutes
- 54,000samples
Tracking files add another distinction. Beggs illustrates a player recorded ten times per second over 90 minutes. The calculated sample count in the figure describes the volume of readings, not separate performances or independent demonstrations of ability. Nearby observations belong to the same movement, and an uninterrupted recording is an assumption that should be checked.
For an aspiring data engineer, a small project that follows one player across a transfer and exposes these failures is useful evidence of judgement. Publish the schema, a reproducible transformation, a deliberately failing check, and the corrected output. A recruiter should be able to understand what would have gone wrong without needing to read every line of code.
Before the next data refresh
- Separate stable player identity from time-bound team membership.
- Reconcile appearances and minutes before and after combining sources.
- Make missing coverage and failed identifier matches visible to the analyst.