Sanderson M. Smith

STATISTICAL SIGNIFICANCE: NOTE TO NON-AP STATISTICS STUDENTS

Below the ==== line is an e-mail note to all students in the NON AP STATISTICS course we have introduced at Cate School. We still have AP STATISTICS, but we felt a definite need for a course that was not AP.

Remember... I am addressing NON AP students.

I am attempting to give them a "feel" for the phrases "statistical significance" and "quality control."

I've even tossed in "type I error" and "type II error.

The PREMISE that the students are working with is embedded in the note.

My students have started to do a simulation.

Without going into great detail, each student has five dice (4 white, 1 red). They are randomly selecting one of the die from a container... and recording the color.

OK, I'm going to stop here and let you read what is below the ==== line.

The activity above (with the dice) may make some sense after you read what is below.

All of my students have e-mail access, so they have (or will) see this note.

Comments, suggestions, criticisms, etc. are welcome.

===================================

Hi STATers...

We are beginning another project.

This relates to the concept of QUALITY CONTROL and the concept of STATISTICAL SIGNIFICANCE

Here is the situation...

A company manufactures GIZZMOS.

Production standards are such that 20% of the GIZZMOS have some minor flaw that must be corrected (or adjusted) before they are sold.

Periodically, random samples of 25 GIZZMOS are chosen and thoroughly inspected in an attempt to determine if production standards are being met.

Based on the number of defective items in the sample, a decision will be made relating to the production process.

OK, THINK, THINK, THINK...

In the sample, one would expect (dangerous statistical word if not properly defined, but I'll use it here) to see about (.2)(25) = 5 defective items if production standards were being met.

Now, if all 25 items were defective, this would certainly suggest that something is wrong with the production process. It would probably make sense to stop production (a costly decision) and attempt to find the problem you are sure exists.

Suppose just 3 items out of the 25 were defective. This would probably not be a concern. That is, you don't have strong evidence to suggest that something is wrong with the production process. In this situation, you actually have less than the expected number of defective items.

OK, now what about gray areas? You expect to see 5 defective items.

Suppose you get 6 defective items in the sample? (Is this significant?)

Suppose you get 7 defective items in the sample? (Is this significant?)

What about 8?

What about 9?

DO YOU GET THE POINT? At what point (number of defective items) do you cross the threshold between no concern and concern?

There is no set answer to the question.

In real life situations, someone has to make a decision as to when CONCERN sets in.

The individual could make two types of errors:

TYPE I: Everything is OK, but it is concluded that something is wrong.

TYPE II: Something is wrong, but it is concluded that everything is OK.

Remember now... a decision will be made based on the examination of a random sample of GIZZMOS.

And, whatever decision is made, there is a chance that it will be wrong. That is, there is always the possibility of making a TYPE I ERROR or a TYPE II ERROR.

In real life, one must truly use MATH POWER to come to an intelligent decision.

(NOTE: No matter how "intelligent" the decision is, it could still be the wrong one. Basically, all one can do is attempt to minimize the probability of making a wrong decision.)

One would have to be intelligent enough to know that...

* Shutting down production is a costly decision. Is it necessary to shut down?

* Not shutting down production when a problem exists could lead to tremendous future expenses.

(Perhaps not a good analogy, but what would happen if you never changed the oil or had routine maintenance on your car?)

If (for instance), we find 8 defective GIZZMOS in the random sample of 25 GIZZMOS, what should be done?

OK, good people. Somebody has to make decisions like this in real life situations.... and you can bet your pet panda's petunias that MATH POWER (and specifically STATISTICAL POWER) is needed to make intelligent decisions. Note that I said "intelligent" decisions, not necessarily "correct" decisions. A fact that often confuses statistical novices is that the most intelligent of statistical decisions will not necessarily be the correct one. (If you understand this last statement, you are on your way to STATISTICAL POWER.)

May MATH POWER (and STATISTICAL POWER) shine on you always...

==================================
OK, remember the five dice (4 white, one red)? Mix them up and randomly choose one of them. Record the color. Do this 25 times and you have simulated choosing a random sample of size 25 from a population in which 20% of the items are defective.
==================================

"If I have seen farther than others, it is because I have stood on the shoulders of giants

-ISAAC NEWTON (1642-1727)

"The golden age of mathematics - that was not the age of Euclid. It is ours!"

-C. J. KEYSER

Copyright © 2003-2009 Sanderson Smith