1. Introduction
In order to utilise a result to decide whether it indicates compliance or non-compliance with a specification, it is necessary to take into account the measurement uncertainty. Depending on the circumstances, and particularly on the risks associated with making a wrong decision, the probability of an incorrect decision may be or may not be sufficiently small to justify a decision of non-compliance.
2. Scope
This guide is applicable to decisions on compliance with regulatory or manufacturing limits where a decision is made on the basis of a measurement result accompanied by information on the uncertainty associated with the result. It covers cases where the uncertainty does not depend on the value of the measurand, and cases where the uncertainty is proportional to the value of the measurand.
This guide assumes that the uncertainty has been evaluated by an appropriate method that takes all relevant contributions into account.
3. Reference
-ISO/IEC 17025 : 2017 General Requirements for Testing & Calibration Laboratories
– The Eurachem/CITAC published a Guide “Use of uncertainty information in
compliance assessment” in 2007 on the subject on Decision Rule and Compliance;
– ILAC G-8:03/2009 “Guidelines on the reporting of compliance to specification” attempts to provide some clarity by writing a guide on how to look at making a “pass/fail” conformity assessment, and how to present the “conformance/non-conformance” statements;
– The JCGM 106:2012 “Evaluation of Measurement Data- the Role of measurement uncertainty in conformance assessment” is another guideline on how to deal with this problem with more details and suggestions on different ways of interpreting results.
4. Definition
“Decision rule” under Terms and Definitions Clause 3.7 of ISO/IEC 17025:2017, which states that it is a “rule that describes how measurement uncertainty is accounted for when stating conformity with a specified requirement”.
ISO/IEC 17025:2017 and Decision Rule
The current version of ISO/IEC 17025:2017 has the following clauses:
Clause 7.1.3 : “When the customer requests a statement of conformity to a specification or standard for the test or calibration (e.g. pass/fail, in-tolerance/out-of-tolerance), the specification or standard and the decision rule shall be clearly defined. Unless inherent in the requested specification or standard, the decision rule selected shall be communicated to, and agreed with, the customer.”
Clause 7.8.6.1 : “When a statement of conformity to a specification or standard is provided, the laboratory shall document the decision rule employed, taking into account the level of risk (such as false accept and false reject and statistical assumptions) associated with the decision rule employed, and apply the decision rule.
NOTE Where the decision rule is prescribed by the customer, regulations or normative documents, a further consideration of the level of risk is not necessary. “
Clause 7.8.6.2 c) “ the decision rule applied (unless it is inherent in the requested specification or standard). “
Appendix A A.2.3 : Measurement standards that have reported information from a competent laboratory that includes only a statement of conformity to a specification (omitting the measurement results and associated uncertainties) are sometimes used to disseminate metrological traceability. This approach, in which the specification limits are imported as the source of uncertainty, is dependent upon:
— the use of an appropriate decision rule to establish conformity;
STATEMENTS OF COMPLIANCE ACCORDING TO ISO/IEC 17025
Clause 7.8.6.1
In this issue of conformity, one therefore must make an educated discussion with his customers or regulators during the acceptance of a test request or contract review, It is a risk to both parties concerned, which must be duly assessed and decided upon when the reported measurement is found not within or below/above specification stipulated.
Following situations maybe considered :
When Customer provides a specification or standard for reference a statement of conformity to a specification or standard – It is the responsibility of the laboratory to inform the customer about the decision its decision rule on making a statement of compliance to prevent the probability of “false accept” when a product or material should fail and “false reject” when this product or material should pass before taking up the job.
For the marginal results due to effect of Uncertainty of measurement and other factors a confusion may occur in reporting the statement of conformity and it is called “Guard Band” (see Figure 1). The use of Guard Band to limit may be discussed with the customers that will be comfortable to both parties, and also ensure that its test procedure employed can meet the specification limit required. It may even need to change or modify its test procedure (e.g. by lowering its method detection limit) to cater for its decision rule to be applied.
Figure 1 Graphic presentation on a stringent Acceptance zone and a ‘relaxed’ Rejection zone for a specification with an upper limit
Figure 2 : Acceptance and rejection zones for simultaneous Upper and Lower Limits
Decision Rules :
A decision rule ultimately relies on the outcome of the ever popular hypothesis or significance testing based on the distribution(s) of test statistic and sets a risk level that is mutually acceptable.
1 In order to decide whether or not to accept/reject a product, given a result and its uncertainty, there should be
a) a specification giving the upper and/or lower permitted limits of the characteristics (measurands) being controlled
and
b) a decision rule that describes how the measurement uncertainty will be taken into account with regard to accepting or rejecting a product according to its specification and the result of a measurement.
2 The decision rule sets up for a well-documented method of unambiguously determining the location of the acceptance and rejection zones, ideally stating or using the minimum acceptable level of the probability that the measurand lies within the specification limits. It should also give the procedure for dealing with repeated measurements and outliers.
3 Utilising the decision rule the size of the acceptance or rejection zone may be determined by means of appropriate guard bands. The size of the guard band is calculated from the value of the measurement uncertainty and the minimum acceptable level of the probability that the measurand lies within the specification limits.
4 In addition, a reference to the decision rules used should be included when reporting on compliance.
Choosing Acceptance and Rejection zone limits
The size of the guard band, g depends upon the value of the uncertainty, u and is chosen to meet the requirements of the decision rule. For example if the decision rule states that for non-compliance, the observed value should be greater than the limit plus 2u, then the size of the guard band is 2u.
If the decision rule states that for non-compliance that the probability P that the value of the measurand is greater than the limit L, should be at least 95%, then g is chosen so that for an observed value of L+g, the probability that the value of the measurand lies above the limit L is 95%. Similarly, if the decision rule is that there should be at be least a 95% probability that the value of the measurand is less than L, then g is chosen, so that for an observed value of L-g, the probability that the value of the measurand lies below the limit is 95%. In general the value of g will be a function of or a simple a multiple of u where u is the standard uncertainty. In some cases the decision rule may state the value of the multiple to be used. In others the guard band will depend upon the value of P required and the knowledge about the distribution of the likely values of the measurand.
STATING COMPLIANCE OR NON-COMPLIANCE WITH SPECIFICATION
a. When a specification describes an interval with an upper and lower limit, a statement of compliance or non-compliance are made where the ratio of the expanded uncertainty interval to the specified interval is reasonably small and fit for purpose (meaning that the laboratory should be able to meet the needs of the customer).
b. When compliance with a specification is made it should be clear to the customer which coverage probability for the expanded uncertainty has been used. In general the coverage probability will be 95 % and the reporting shall include a remark such as “The statement of compliance is based on a 95% coverage probability for the expanded uncertainty.” This means that the probability that the measurement is below the upper specification limit is higher than 95 %, i.e. approximately 97.5 % for symmetrical distributions. A lower limit is treated similarly. Other values for the coverage probability for the expanded uncertainty should be agreed between the laboratory and the customer. Coverage probabilities for the expanded uncertainty higher than 95 % might be chosen while lower values should be avoided.
c. The following approach for a certain upper specification limit is recommended. (A lower limit is treated similarly):
(i) Compliance: If the specification limit is not breached by the measurement result plus the expanded uncertainty with a 95% coverage probability, then compliance with the specification can be stated. This can be reported as “Compliance” or “Compliance – The measurement result is within (or below) the specification limit when the measurement uncertainty is taken into account”;
(ii) Non-compliance:
Situation 1. If the specification limit is exceeded by the measurement result minus the expanded uncertainty with a 95% coverage probability, then non- compliance with the specification can be stated. This can be reported as “Non-compliance” or “Non-compliance – The measurement result is outside (or above) the specification limit when the measurement uncertainty is taken into account”;
Situation 2. If the measurement result plus/minus the expanded uncertainty with a 95 % coverage probability overlaps the limit or Guard Band (g), it is not possible to state compliance or non-compliance. The measurement result and the expanded uncertainty with a 95 % coverage probability should then be reported together with a statement indicating that neither compliance nor non-compliance was demonstrated. A suitable statement to cover these situations would be “It is not possible to state compliance”. In this case, it is similar statement “It is not possible to state compliance using a 95 % coverage probability for the expanded uncertainty although the measurement result is below the limit”. If shorter statements are reported it should not give the impression that the result complies with specification.
Sanchita Bhattacharya, CEO
Consultrain Management Service, Kolkata