Big Data Quality, Then and Now

A decade ago, just before the beginning of the data science hype cycle was the big data hype cycle. At that time I had the privilege of sitting down with Ph.D. Statistician Dr. Thomas C. Redman (aka the “Data Doc”).  We discussed whether data quality matters less in larger data sets, if statistical outliers represent business insights or data quality issues, statistical sampling errors versus measurement calibration errors, mistaking signal for noise (i.e., good data for bad data), and whether or not the principles and practices of true “data scientists” will truly be embraced by an organization’s business leaders. This episode is an edited and slightly shortened version of that discussion, which even though it is from ten years ago, I think it still provides good insight into big data quality, then and now.   Extended Show Notes: ocdqblog.com/dbp   Follow Jim Harris on Twitter: @ocdqblog   Email Jim Harris: ocdqblog.com/contact   Other ways to listen: bit.ly/listen-dbp