Data Analytics: Foundations to Practice · Descriptive Statistics Fundamentals
Identifying and Handling Outliers
An outlier can be a genuine data quality error or a genuinely important, real signal, and treating every outlier the same way regardless of which it is produces two opposite kinds of mistake. This chapter covers how to tell them apart.
An outlier is a data point that differs substantially from the rest of a dataset, commonly identified through a rule like falling more than a set number of standard deviations from the mean, or falling outside a set multiple of the interquartile range covered in an earlier chapter of this module; these rules provide a consistent, repeatable starting point for flagging candidates, though the specific threshold chosen is itself a judgment call rather than a fixed, universal law.
- An outlier is commonly identified through rules like standard deviations from the mean or a multiple of the interquartile range, though the specific threshold is a judgment call.
- An outlier can be a genuine data quality error or a real, legitimate, important event; treating every outlier the same way produces two different kinds of mistake.
- The appropriate response is investigating a flagged outlier's context before deciding, rather than applying a blanket automatic removal rule.
- Whatever decision is made about an outlier should be documented transparently, so stakeholders understand which data points were excluded and why.