Data Analytics: Foundations to Practice · Data Cleaning and Preparation
Why Data Cleaning Consumes So Much Time
Real-world data almost never arrives ready for analysis, and understanding why explains why this unglamorous stage deserves genuine respect rather than being rushed through.
Data collected from real-world business systems, customer forms, or external sources rarely arrives in a format immediately ready for analysis; it typically contains missing values, inconsistent formatting, duplicate records, and genuine errors, all of which must be identified and addressed before any of the analytical techniques covered elsewhere in this course can produce trustworthy, reliable results.
Key Takeaways
- Real-world data rarely arrives analysis-ready; it typically contains missing values, inconsistent formatting, duplicates, and errors requiring attention first.
- A widely cited estimate holds analysts spend roughly 80% of project time on collection and cleaning, though the exact figure varies by project.
- This time investment is necessary, not wasted, since sophisticated analysis applied to poorly cleaned data produces misleading 'garbage in, garbage out' results.
- Project timelines should honestly account for how much time cleaning genuinely takes, rather than treating it as a minor, compressible afterthought.