How do auditors work out what sample size to use? It is actually a complex calculation. And consequently, in my experience, auditors use rules of thumb – like 10%. These rules of thumb cannot hope to give an adequate estimate of the needed sample size and consequently auditors either under-sample or over-sample. Often these samples are vastly too small - causing inappropriate decisions - or much too large - a wasted effort.

What sample size should you take?

The diagram below answers the following commonly asked question:

An auditor is sampling a stack of N = 10,000 invoices to decide if he/she should accept the work of the clerk involved. She/he has previous experience that the error rate has been 0.3 percent, and she/he is willing to accept this rate 5 percent of the time. However, she/he is not willing to accept the lot if the error rate is more than 1.5 percent more than 5 percent of the time. What sampling plan should the auditor use?

Use the sliders to set up your own sampling plans. Definitions etc are below. )

To veiw the calculator, click the button below

Some definitions

You are also asked if you have a preference for not accepting bad lots or not rejecting good lots. That is, in the event of the sample being just too small to be fully representative of the lot, you would like to err on the side of accepting it even if there were errors, or rejecting it when it was good. Both choices have costs. Rejecting a lot when it is good can cause unnecessary political flack and rework. Protecting for this risk also increases the sample size. Whereas accepting the lot when it is bad means systematic errors are not found and corrected early - which can have considerable costs later.

Try all the sliders and the radio buttons to see the effect on the sample size and acceptance number for different scenarios.

Notice that the sample size is independent of the lot size N.

Sequential sampling plan

Sequential sampling plans allow you to bail out early. It allows you to proceed item-by-item. For example, in the example presented, if the auditor has examined 244 items and not found a defect, she/he can accept the lot. If she/he has examined 378 items and found only 1 error, she/he can accept the lot. Similarly, if she/he has examined only 25 and already found two defects, she/he can reject the lot. If she/he has examined 159 items and already found 3 defects, she/he can reject the lot.

The graph gives you the shape. If you enter the red-shaded region, you should reject the lot. If you enter the green-shaded area, you should accept the lot. The exact cut-points (which would be hard to read from the graph) are shown to the right.

Once again, try the sliders to see the effect.

Operating Characteristics (OC) curve

The OC curve shows the sampling plan that best fits your requirements of alpha, beta, AQL and LTPD. The OC curve shows how well the sampling plan discriminates between good and bad lots. People usually want a plan that gives a high probability of acceptance if the lot is good and low probability of acceptance if the actual quality is poor. A sampling plan that discriminates perfectly between good and bad lots would have a vertical OC curve. Unfortunately, the only plan that achieves that level of discrimination is 100% inspection. The justification of acceptance sampling turns on the balance between inspection costs and the probable costs of passing bad lots or rejecting good ones.

Note

The theory on which this calculator is built is set in a world of no more than 3 sigma. You will recall from the material on six sigma that at 3 sigma, you would be expecting 1 error in 741. At 4 sigma, the expected error rate is in the order of 1 in 31,574. At 5 sigma, the expected error rate is 1 error in 3,488,556. The sample size necessary to sample adequately at 4 or 5 sigma mean that if you are intending to use inspection sampling to find errors, you believe yourself to be in a 1, 2 or 3 sigma world. I have see auditors advised that they need only take 10 samples. Such advice indicates a 1 sigma world. When your company moves to a 4, 5 or 6 sigma world, sampling by inspection is not feasible or practical. You need to completely rethink you audit strategy. Audit the systems being used to eliminate errors rather than look for errors. In a 5 sigma world, if you insist on any form of inspection audit, you prove that you do not understand the statistics involved. You also hold back progress towards 5 or 6 sigma by insisting on inspection.

References

The technique used is a form of 'Acceptance Sampling' as developed and used extensively in Quality Control in manufacturing.

Buffa & Sarin (1987) Modern Production/ Operations Management. pp408-420.

Engineering Statistics Handbook

Cooke, Craven & Clarke (1981) Basic Statistical Computing. (for the Poisson probability and reverse chi square algorithms).

Abramowitz & Stegun (1972) Handbook of Mathematical Functions (for the equation for the reverse Poisson p959).