All work
Retail & FMCG · Method + synthetic demonstration
FMCG Data Cleansing & Enrichment Framework
Repeatable rules for cleansing and enriching product data — demonstrated here with synthetic data.
Prototype / ExperimentSynthetic demonstration
Context & challenge
Cleansing is usually done once, by hand, and then lost. It needs to be a repeatable, explainable process.
Objectives
- Codify cleansing stages
- Make every correction explainable
- Produce reproducible quality metrics
Solution approach
- Seven stages from profiling to reporting
- Rules expressed as code with reason codes
- Metrics computed directly from the dataset
Architecture
- Raw records
- Profile
- Standardise
- Match
- Validate
- Exceptions
- Reconcile
- Report
Simplified and anonymised. No private infrastructure details are published.
Important technical decisions
- Every changed value records its rule and original value
Operational considerations
- Rule ownership
- Threshold tuning
Results
Results pending verification. We only publish outcomes that have been confirmed.
Lessons learned
- Explainability is what makes cleansed data trusted.
Visual demonstration
An interactive, synthetic demonstration related to this work is available in the laboratory.
Open demonstration