The Harvard Business Review article by Michael Li is probably behind a pay wall but it's a great, short note on two important questions to ask a data analyst. Maybe I just like it because I agree with him - it's something I constantly tell my students - but I think it's good advice for anyone. If you have access to HBR, then stop reading this blog post. Go read the article and judge it for yourself.
If you don't have HBR access here's an even shorter version.
1) How was the data collected? I tell my students that statistical calculations, even complicated ones, are fairly straightforward and no one can really argue with your calculations. All of the debate in statistical analysis is about the data.
2) What's the margin of error? If you understand the difference between a point estimate and a population parameter, then you know that people regularly present point estimates as "the" number. That's a mistake. A point estimate is just an estimate. Therefore, the uncertainty behind it needs to be openly acknowledged with a margin of error.
Don't take my word for it. Go read Dr. Li's article.
