Football data systems / Operations
When the refresh succeeds and the report is wrong
A completed pipeline proves that the code ran. Publishing a football report requires a separate decision about the data it produced.
The coach opens the opposition report before training. The page says it has just refreshed, but a delayed fixture export has removed the opponent’s latest match. Nothing crashed. In this hypothetical case, the system has completed its instructions and still failed its reader.
A failed candidate does not replace a validated report
Transform and check
Fixture coverage · Appearance keys · Player mappings · Reconciled totals
Publish the candidate
Record the release date, coverage, and definitions used by the report.
Hold publication
Investigate the failure and tell the reader what is unavailable.
Last validated release
Remains identifiable by its date and coverage. Display a warning if a refresh is held; confirm whether the older report still answers the current question.
A correction returns through the same checks. Replaying a source revision must not create another copy of the same appearance.
Treat publication as a controlled step after transformation. An incomplete candidate release should not silently replace a checked report. Keep the last validated release accessible, show its date and coverage, and explain when it is unsuitable for the current question. Older information can also mislead; availability is not permission to conceal staleness.
Test the football assumptions
A unique appearance key should not occur twice. A player identifier should resolve to a known person. The expected fixture list should reconcile with the received matches. Totals of appearances and minutes should agree with the source after transformations. These checks address different failures; one passing total cannot establish that every player’s records are intact.
dbt’s built-in tests cover uniqueness, missing values, allowed values, and relationships between tables. Use those where they fit, then add checks for fixture coverage and reconciled football totals. A test query identifies records that violate an assertion. It cannot decide whether the assertion represents the club’s football question.
Make a provider correction safe to replay
Suppose the provider changes a player’s minutes after a match. Reloading the export should replace the affected record according to a documented rule, rather than append another copy of the appearance. Preserve the source revision and record which downstream metrics were rebuilt. Run the same input again and check that it does not change the result or multiply rows.
Test failures deliberately before relying on the controls. Introduce a duplicate appearance, remove an expected fixture, and break a player mapping in a test dataset. Each should trigger the intended response. Agree which failures block publication and which produce a visible warning; make that distinction part of the operating procedure.
Give the failure an owner
An alert needs a destination and a response. Name who investigates the source, who decides whether the existing report remains usable, and who tells the coaching staff what has changed. Keep technical details in the incident record and put the consequence on the report: a missing fixture, an older release, or a comparison temporarily unavailable.
The SSAC24 panel describes analytics becoming an operational responsibility as people come to depend on it. That is the useful transition for an aspiring engineer to demonstrate. A portfolio can show the failed candidate, the publication hold, the corrected release, and the reader’s notice. The visible achievement is a report somebody can use with an accurate understanding of its limits.
Before scheduling the report
- Define the expected fixtures, checks, and blocking failures with the analyst.
- Prove that duplicate records and missing coverage stop a candidate release.
- Show the validated release date, coverage gaps, and person responsible for recovery.