Prima Facie Reliability 3
Canada is a member state of the OIML - the International Organization of Legal Metrology - the Organisation International de Métrologie Légale. The OIML mission statement is as follows:
“The mission of the OIML is to enable economies to put in place effective legal metrology infrastructures that are mutually compatible and internationally recognized, for all areas for which governments take responsibility, such as those which facilitate trade, establish mutual confidence and harmonize the level of consumer protection worldwide.” - OIML B 15:2011
The Canadian representatives at OIML are officials at Measurement Canada. Canada's primary legal metrology statute is the Weights and Measures Act, which is, or appears to be, consistent with Canada's OIML obligations.
One would expect that any Position Paper prepared by the Alcohol Test Committee of the Canadian Society of Forensic Science would be consistent with the Weights and Measures Act and with international scientific literature, particularly documents prepared by the OIML, of which Canada is a member state. Science in Canada should not be any different from science in the rest of the world. What is relevant to a reliable scientific determination in Canada should not be any different from what is relevant to a reliable scientific determination elsewhere on the planet. Unfortunately the transparency, metrological characteristics protection, fraud protection, and durability, that OIML speaks about in its scientific literature, don't seem to be incorporated into the ATC Position Paper. The Weights and Measures Act, BIPM, and OIML documents aren't mentioned in the Position Paper.
The vocabulary in its 2013 version includes a definition of "audit trail". The "audit trail" described in VIML sounds a lot like COBRA data in an 8000C:
OIML V 1:2013
continuous data file containing a time stamped information record of events, e.g. changes in the values of the parameters of a device or software updates, or other activities that are legally relevant and which may influence the metrological characteristics
It appears that the OIML, as of 2013, considers that "changes in the values of the parameters of a device" and "software updates" are "legally relevant" and "may influence the metrological characteristics" of an instrument.
The activity of "calibration" or "re-calibration" involves a change in a parameter of the device, specifically the "calibration curve".(1) Such activity influences the metrological characteristics of the instrument. It has an impact on the reliability of the measurement result.(2) Please don't confuse the word "calibration" with "control check/accuracy check/verification".(3) Control checks or accuracy checks don't change the metrological characteristics of an instrument. A verification doesn't change the metrological characteristics of an instrument, but a failed verification may trigger a re-calibration which changes metrological characteristics. Evidence of a failed verification, previously, is likely relevant to the issue of malfunction at time of use - an eventual finding by a Court that police did not take an instrument out of service for re-calibration when they should have done so, resulting in unreliability of the metrological characteristics at time of use.
At the very least, the audit trail would seem to be likely relevant to the identification of software updates and changes to the parameters of the instrument, both of which affect the reliability of the measurement result.
However, the metrological characteristics of an instrument, over its measurement range, change over time. Significant "drift" in accuracy and precision over time is a very real phenomenon and affects reliability.(4)
Every Intoxilyzer 8000C imported into Canada has a Certificate of Calibration by the manufacturer, that states that the instrument complies with OIML standard R 126.
OIML R 126 applies to instruments such as the Intoxilyzer 8000C but not ASDs:
"This Recommendation applies to quantitative breath alcohol analyzers that render a measurement
result of alcohol concentration in exhaled human breath for the purpose of establishing compliance
with national policy for fighting against alcohol abuse.
These types of quantitative breath alcohol analyzers are referred to by some national authorities as
“evidential” breath alcohol analyzers and serve to provide the principal means by which a definitive
alcohol measurement is obtained.
These devices are not to be confused with those that provide a preliminary result, or do not
quantitatively indicate a measurement result (i.e. pass/fail devices), or which do not provide a
sufficiently accurate result to definitively establish a breath alcohol concentration (often referred to as breath alcohol “screening” devices)."
OIML R 126 in its Terminology section defines, inter alia, the following:
measurement error (VIM 2.16 1)
measured quantity value minus a reference quantity value
automatic checking facility
internal device or process that checks whether the breath alcohol analyzer is suitably adjusted. Such a device may include internal checking elements (for example signal stability or temperature stability) or additional external elements to be connected to the instrument such as optical or electrical filters or a cylinder with a test gas of known concentration
change in the instrument indications of the same alcohol concentration which occurs during a stated period of time at a given mass concentration of alcohol in air
MPE Maximum Permissible Error
EUT Equipment Under Test
significant fault (OIML D 11, 3.10 )
difference between the error (of indication) and the intrinsic error greater than the value specified in
this Recommendation. Significant faults are only relevant to electronic measuring systems
intrinsic error (OIML D 11, 3.7 )
error of a measuring instrument, determined under reference conditions
OIML R126 makes it clear that evidentiary breath testing equipment needs to always be capable of accurately measuring BAC across a range of values. On the Intoxilyzer 8000C that manufacturer specifies that it is capable of measuring from 0 to 600 mg/100mls with an accuracy of =/-3 mg/100 mls at each value. OIML R 126 states:
The breath alcohol analyzer shall be capable of measuring all mass concentrations in the range 0.00 mg/L to at least 2.00 mg/L. However, in the measuring mode, the breath alcohol analyzer may indicate 0.00 mg/L for mass concentrations equal to or smaller than a given value defined under the
responsibility of national authorities. Such a masking function shall be able to be cancelled in
The instrument shall fulfill the requirements of this Recommendation for the complete specified
A greater upper limit of the measuring range may be defined by the manufacturer.
The breath alcohol analyzer shall indicate when its upper limit of measurement is exceeded.
It is important to note that the ATC 2012 Position Paper only speaks about control tests, at time of use, at 100 mg/100 mls, not across the range of values identified by OIML.
OIML R 126 limits maximum permissible errors across the range of values:
5.2 Maximum permissible errors (MPEs)
The following MPEs shall apply within the rated operating conditions (specified in 5.8).
Maximum permissible errors for breath alcohol analyzers in service
The maximum permissible error, positive or negative is 0.030 mg/L or 7.5 % of the reference value of
mass concentration, whichever is the greater.
If the upper limit of the measuring range is greater than 2.00 mg/L, the maximum permissible error
shall be: reference value x 3/4 - 1.35 mg/L for all mass concentrations greater than 2 mg/L.
Please notice that OIML does not limit the concept of maximum permissible errors to those at the 100 mg/100 mls level but rather across the range of values. Please also notice that the maximum permissible errors at 100 mg/100 mls is significantly less than 10 mg/100mls (Ontario). The table below provides a calculation of OIML R 126 MPEs at various values commonly used in annual inspections of 8000Cs in service in Ontario:
Please notice that the concept of "drift" in OIML relates to a time period that is not limited to a few seconds (the 8000C self-diagnostics test) or 17 minutes (the truncated 20 mg/100mls agreement rule - what our colleagues in the US call .02 agreement).
The drift measured under reference conditions as defined in 11.4.1 at 0.40 mg/L shall be less than
0.020 mg/L in two months.
OIML R 126 states that instruments must comply with these rules over a specified duration of time. In military standards there are rules about reliability of equipment over a specific mission life. OIML R 126 states the expectation that the manufacturer, will specify a mission life, for these instruments and the instrument is expected to maintain its metrological qualities across the range of values over time, not just a matter of minutes. The national authorities are then expected to set a verification period. Instruments should be durable until the next required verification (perhaps similar to our annual inspection) across the range of values.
The provisions in 5.2, 5.4, 5.5, 5.6, 5.8 and 5.10
shall be met durably.
The breath alcohol analyzer shall be designed to maintain stability of its metrological characteristics
over a period of time (to be specified by the manufacturer) which shall be at least as long as the
The verification period is defined under the responsibility of the National Authorities (subsequent
The problem of durability is an important reason why maintenance, re-calibrations, and other changes to metrological characteristics need to be transparent. Lack of maintenance or re-calibration, after a failed, negligent, or missing verification also need to be transparent. The phenomenon of drift in accuracy and precision renders such transparency necessary.
Fraud is an important reason why hardware or software changes to metrological characteristics need to be carefully controlled and transparent.
OIML R 126 makes it clear that the instrument's software is "legally relevant" and instrument software needs to have protections against fraud:
The whole software of the breath alcohol analyzer should be considered as legally relevant.
In the event of a software separation as described in 184.108.40.206 of OIML D 31:2008 , the whole
software is considered as legally relevant.
Fraud protection (D 31:2008; 220.127.116.11 )
The software shall be secured against unauthorized mod
ification, loading, or changes by swapping the memory device. In addition to mechanical sealing, technical means may be necessary tosecure measuring instruments having an operating system or an option to load software.
Software protection comprises appropriate sealing by mechanical, electronic and/or cryptographic
means, making an unauthorized intervention impossible or evident.
Only clearly documented functions are allowed to be activated through the user interface,
which shall be realized in such a way that it does not facilitate frau
For the type approval procedure, the manufacturer of the measuring instrument shall declare and
document all program functions that can be activated through the user interface. No hidden functions shall exist. The manufacturer shall state the completeness of the documentation of these functions.
Parameters that fix the legally relevant characteristics of the breath alcohol analyzer shall be
secured against unauthorized modification. For the purpose of verification, the current parameter
settings should be able to be displayed or printed.
OIML R 126 specifically contemplates that data may be stored electronically:
Storage of data
The breath alcohol analyzer may store measurement data for further applications under legal metrological control. In such a case, the requirements defined below apply (18.104.22.168 to 22.214.171.124).
The measurement value stored shall be accompanied by all the relevant information that is necessary for future legally relevant use.
The data shall be protected by software means to guarantee the authenticity, integrity and, if necessary correctness of the information concerning the time of measurement.
The software shall check the time of measurement, authenticity, and integrity of the data.
If an irregularity is detected, the data shall be discarded or marked unusable.
Confidential keys employed for protecting data shall be kept secret and secured in the breath alcohol analyzer. Means shall be provided whereby these keys can only be input or read if a seal is broken.
OIML R 126 requires an Instruction Manual:
An instruction manual for users shall be made available for each individual instrument.
The instruction manual shall be in the official language(s) of the country (or another accepted
language according to national legislation) and easily understandable.
It shall include
a) operating instructions,
b) maximum and minimum storage temperatures,
c) rated operating conditions,
Strangely, in Ontario, no one seems to pay much attention to any Operator's Manual for the 8000C. Instead police follow their own practices loosely based on the "recommendations" of the Alcohol Test Committee or the Centre of Forensic Sciences. Under OIML R 126, (and CMI claims R 126 compliance for the 8000C) there should be an Instruction Manual in existence that everyone can refer to.
Quaere: What is the manufacturer's specified period of time for the 8000C - the
"period of time (to be specified by the manufacturer) which shall be at least as long as the verification period."?
Quaere: What is the national authority's verification period for the 8000C - "The verification period is defined under the responsibility of the National Authorities (subsequent verifications )."
Quaere: Why isn't the manufacturer's period specified in an Instruction Manual?
Quaere: If an "annual inspection" to manufacturer's specifications is only a "recommendation" then it appears that in Canada our national authority is not complying with Canada's OIML obligations. If, in Canada, there is no need for such a mandatory verification period, is science in Canada different from the rest of the planet? Is "durability" of metrological characteristics across the full range of values not a concern in Canada?
(1) VIM 4.31 and 2.39 Vocabulaire International de Metrologie, published by the BIPM, International Vocabulary of Metrology, definition of "calibration curve". A "calibration" is an "operation that, under specified conditions, in a first step, establishes a relation between the quantity values with measurement uncertainties provided by measurement standards and corresponding indications with associated measurement uncertainties and, in a second step, uses this information to establish a relation for obtaining a measurement result from an indication".
(2) R. v. Lam 2015 ONSC 2194 (CanLII) para 31.
(3) VIM 2.39 and 2.44 " Verification should not be confused with calibration."
(4) See the reference under Authorities in R. v. St-Onge Lamoureux to an article by Brian Hodgson " “The Validity of Evidential Breath Alcohol Testing” specifically his definition of "reliability".
Note: The images and discussion in this blog are not for use by the public in Court. They are designed to stimulate discussion among criminal law lawyers. They are not evidence. The author is not a scientific expert.