Reliability: .02 Agreement
Tip 49: It is very possible to have 02 agreement of results, yet the results are not accurate or not reliable. It is important to remember that the BAC readings on an approved instrument are measurement results. The calibration check readings that are read on an approved instrument are also measurement results. Whether one is considering 02 agreement between two BAC readings (20 mg/100 mls truncated) or ± 10 mg/100mls of target value agreement between two (or three) cal. check readings, symmetry of results may be deceptive.
Consider the beauty, symmetry, and precision of the design and construction of these arches. Close comparison of the arches reveals that they are exactly parallel and symmetrical.
If you look at the results in the context of a series of more than two arches the symmetry is even more striking.
However, notwithstanding beautiful design in a building, or symmetry in breath testing results, a poor foundation can result in inaccuracy. It isn't enough to simply assume that symmetry / precision imply accuracy. Architecture and breath measurement results need to be assessed in their full scientific context, if the results are ultimately to be considered reliable.
The underlying foundation of a building or the foundation of an approved breath instrument, that is, its calibration, may also shift over time. Shifts in the calibration of an instrument that happen over time are called "drift". No one builds a building or an Intoxilyzer® with the intention that it will not be vertically true or evidentially true. The designers of the building, the builders of the tower, the manufacturer of the instrument, and the Alcohol Test Committee may all have performed their duties well, but the foundation may have shifted over time. There may have been a laissez faire approach to maintenance, quality assurance, or quality control at some point in the process. An assessment of reliability at the time of subject tests analysis, requires full appreciation of a whole system of alcohol and other standards that are themselves reliable. Symmetry is not helpful in assessing reliability unless the alcohol and other standards are traceable to higher known and reliable standards.
The following two videos depict an Intoxilyzer® 5000EN (US model) evidentiary breath instrument where the calibration of the internal standards, themselves, not just the instrument calibration, has shifted over time. Notice that the instrument repeatedly fails its "internal standards" checks yet eventually (after 11 restarts) seems to work very well, without a further internal standard "FAILED" flag. This instrument is obviously not "reliable" and its calibration is suspect, yet if restarted enough times, eventually it appears to perform normally. If you don't know the past history of 11 internal standard fails, you appear to have a perfectly functioning instrument that is quite capable of producing symmetrical cal. checks and symmetrical subject tests.
As with the beautiful bell tower at the church in Pisa, apparently normal and symmetrical results do not necessarily imply accuracy and reliability in relation to the real external world. Use of external standards (including a plumb line) always proceeds from known to unknown, not the other way around. You don't use symmetry in a building to establish what is plumb and you don't use an Intoxilyzer® to establish the reliabilty of the simulator or the alcohol standard.
After 10 starts the instrument continues to flag internal standards "FAILED".
This experiment was performed at my office in Mississauga, Ontario using an Intoxilyzer® 5000EN and a Guth 10-4D.