5.4 Biased and Unbiased Point Estimates

A sample is unbiased if the estimator value (sample statistic) is equal to the population parameter. For example, if the sampling distribution mean (x̅) is equal to the population mean (𝝁) or if the average of our sample proportions (p)is equal to our population proportion (𝝆).. 

A sampling distribution has a minimum amount of variability (spread) if all samples have statistics that are approximately equivalent to one another.  It is impossible to have no variability, due to the nature of random sampling. However, a larger sample size will minimize variability in a sampling distribution.  

Bias is how skewed (also how screwed) the distribution is. Specifically, if an entire distribution is on the left side of our population parameter, it is skewed to the left. If a sample is equally spread out around the mean, then there is no bias. 

The more spread out a distribution is, the more variability it has. The standard deviation of the sampling distribution is the estimator of the population standard deviation. If the standard deviation of the sampling distribution is equal to population standard deviation, it is said that the standard deviation of sampling distribution is the consistent estimator. High variability can be fixed by increasing your sample size, but if your sample does have high bias, there is no statistical way to fix it. 

A good illustration for bias and variability is a bullseye. Bias measures how precise the archer is (how close to the bullseye), while variability measures how consistent he/she is. See the illustrations below for different circumstances regarding bias and variability: