2012 ATC Position Paper Built Upon Hypothesis Without Empirical Evidence
To obtain admissions that the opinion of the Crown expert (and indirectly the 2012 ATC Position Paper) that single point control test(s) at time of use are adequate to establish reliability is based upon a hypothesis that has no empirical study to support the hypothesis.
To challenge the assumption by the Crown expert (and indirectly the 2012 ATC Position Paper) of continued linearity of response in aging instruments.
To challenge the Crown to produce any empirical study that supports the Crown expert's (and the ATC's) hypotheses.
To ultimately suggest that the 2012 ATC Position Paper is not based on science, but rather policy
Q. All right, let’s talk about the Intoxilyzer 8000C then. The Intoxilyzer 8000C – your understanding is that the software at the factory has to learn what each of – the relationship between the instrument response and the indication at each of 50, 100, 150, 200, 300. That’s how calibration is done on an Intoxilyzer 8000C, right? A. Yes, it’s tested at a variety – I believe it’s four different levels, including zero, and that the software then
during the calibration is modified to get the correct result.
Q. A calibration curve is created in the software of the instrument?
A. That’s correct.
Q. That relates instrument response off the detector to the indication on the screen?
A. That’s correct.
Q. So a calibration curve is created. So on a new instrument that’s being evaluated, whether it’s a 5000EN or an Intoxilyzer 8000C that’s a new instrument that’s being evaluated, one would expect that the calibration curve is going to be a good one otherwise it wouldn’t have been coming from the manufacturer for evaluation, right?
A. Correct. Q. But here’s the problem, let’s assume that someone like you, I know it wasn’t you with respect to the Intoxilyzer 8000C, writes up a report – at least I don’t think it was you... A. No. Q. ...With respect to the Intoxilyzer – I think it was Ms. Martin? A. That’s correct.
Q. Submits a report to the Alcohol Test Committee and makes certain recommendations, a full report and later on
there’s Ministerial approval, right?
Q. But here’s the problem. That results in something called type approval, approval of a type of instrument as an approved instrument by the Minister. The question is how do we make sure that a given Intoxilyzer 8000C, which comes off the factory floor a year later or two years later, how do we make sure, how do we verify that it is properly calibrated to manufacturer’s specifications in accordance with the manufacturer’s specifications? Do we have a practice in Canada in about what happens to new instruments that are put into service by police services? Is there some recommendation by the Alcohol Test Committee with respect to that?
A. Yes, they should be inspected upon they being received and then periodically inspected thereafter.
Q. Right and that’s to make sure that they conform manufacturer specifications and that’s so that we know that they’re calibrated at 50, at 100, at 150, at 300, right, and other things? A. Yes, although there’s not necessarily – we don’t specify how they do that inspection. Q. Right. You don’t specify it but the problem with respect to any kind of analytical instrument is that it needs to be calibrated and we know to know that it’s calibrated if it’s to be reliable, right? A. Yes. Q. And so that’s why you have specific recommended standards of the Alcohol Test Committee. Let me put it this way, if an Intoxilyzer 8000C is being used out in the field by a particular police officer, it’s reliability, it’s calibration doesn’t come from something that the qualified technician does... A. That’s...
Q. ...On the day of the subject test. Its calibration comes from what the manufacturer did at the date of the calibration back a number of years before. A. That’s correct. Q. We are counting on the calibration curve, in other words the relationship between the response at the end of the detector and the indication; we are counting on that calibration curve not having changed, right?
A. But we are also verifying the calibration, albeit only at that...
Q. Only at one data point, namely 100 milligrams per 100 mills.
A. That’s correct.
Q. So my question to you is, if an instrument has not been calibrated properly in the first place, if we don’t ever see the Certificate of Calibration from the manufacturer from five years ago or six years ago or seven years ago when it came from the manufacturer, we don’t have any of that information, how can we ever make a determination that there has been a reliable - that the indication on the instrument is reliable if the test result is 150 or 160 if the only checking that we’ve done is at 100 milligrams per 100 mills?
A. Well in general breath-testing instruments that have been used in Canada demonstrate that they have a linear calibration curve and so any change in calibration of the instrument should be uniform across that calibration curve. That gives us the ability to put together a procedure that only uses that single calibration checkpoint.
Q. All right, so now we get to the meat of the real issue of all the questions that I’m asking you. You’re saying that any change should be uniform because the calibration curve essentially becomes linear...
Q. ...As a result of the calibration empirically. I’m asking you as a scientist. You know what I mean by empirically?
Q. You’ve put forward a hypothesis to suggest that by in large, evidentiary breath test equipment that’s used in Canada – the response, the relationship between response at the
end of the detector and what comes off of the indication is a linear response. That’s the hypothesis that you’ve just put forward? A. Yes. Q. Right? All right, let’s test that empirically. I know, and I want to suggest to you the same Terry Martin that we just talked about, when she did her evaluation for the Intoxilyzer 8000C wrote a paper right after that evaluation where she tested the instrument and made – reached the conclusion that the response was linear, right?
Q. That was a new instrument with a new calibration certificate just like the one – like the 5000EN that you received for evaluation, right?
Q. No one at the Centre of Forensic Sciences, no one in the Alcohol Test Committee, no member of the Canadian Society of Forensic Sciences has every empirically tested the hypothesis that you’ve just put forward in an aging instrument, in an instrument that’s five years, six years, seven years old, right? A. Not that I’m aware of or no one’s – I’m not aware of a study that’s been published showing that but it certainly has been tested and is part of our training and it’s also been tested by manufacturers, which is why their internal test only measures the calibration at one point, again at 100 milligrams of alcohol in 100 millilitres of blood unless by statute a jurisdiction is going to use another point. Q. Okay, now before we talk about the internal test procedure, you said that you have this hypothesis that any change over time - and let’s just go back to a paper by Brian Hodgson, you know who he is?
A. Yes. Q. You know that he wrote a paper that the Supreme Court of Canada relied upon in a case called St. Onge Lamoureux? A. Yes. Q. And you’re very familiar with the paper in which he defined what accuracy is, what precision is and he defined what reliability is. He also defined specificity, right? A. Yes. Q. And he referred to reliability as referring to significant drift – or I’m sorry, significant drift in accuracy and precision over time, right? Have I got that roughly right? A. Sounds like a good...
Q. Sounds like a good...
Q. ...Definition. You’d agree with that definition with what reliability is?
A. Yep. Q. So here’s the question, if reliability relates to drift in accuracy and precision over time, how can a court reach any conclusion whatsoever with respect to one data point calibration check without any knowledge of the original date and certification of calibration, the date of any re-calibration of the instrument and without any information as to if anybody has verified that calibration at any other data points other than 100? Why on earth would a court assume that the response is linear? A. Well, without having any evidence I suppose that the court couldn’t. They would need the evidence of an expert. ...
Q. You’d agree with me that reliability increases with any kind of measuring instrument with frequency of calibration or short calibration interval? That’s a general concept across science. A. Well, obviously yes but then relative to how long that actual estimate keeps its calibration... Q. And again, you’d agree with me no published studies, certainly none that you’re aware of, that has tested length of time that an instrument keeps it calibration? No empirical studies in Canada, no empirical studies in the United States on that subject. You’re not aware of any that are published? A. You know, none comes to mind but I haven’t turned to my mind to that for 20 years. Q. Right. So here’s the problem, you say – you draw the inference from the hypothesis that you’ve proposed, that any change should uniform right across the measuring interval scale so therefore you put together a technical program, a technical set of recommendations that says let’s have – and this is what the Alcohol Test Committee has done, let’s have all of the police services across Canada run at least one or more control tests at one data point when they’re running an evidentiary breath test. That’s the reason for that procedure?
A. See, I don’t know that it was – that the basis was just the hypothesis. Obviously there must have been testing of instruments reliability over time and the change in calibration. Q. I wanna suggest to you that there are no such empirical studies and I mean I’m challenging you, I’m challenging the Crown to produce them but I wanna suggest you to there are no such empirical studies. It’s an assumption that’s been made and as a result of that, I think you said earlier, as a result of that that they put together a program, a package, a set of norms, a set of technical norms for qualified technicians to follow, right? A. Yes, I’d agree with that.
Q. It’s because of that assumption.
A. But I also can’t – I mean I can’t say that it’s just an assumption. I can’t think of any published studies right now, but even if there were no published studies I can’t discount that – our understanding of the linearity of the instruments wasn’t – hasn’t been, in fact, studied in any number of forensic laboratories.
[after a recess and further cross-examination]
Q. Yes. A. ...So that fulfills that section. In my opinion, the use of just one calibration check at a concentration of 100 milligrams of alcohol in 100 millilitres of blood is sufficient to determine if the instrument remains in calibration.
Q. I want to suggest to you that’s a technical opinion based on a norm of your employer. It’s not a scientific opinion and you don’t have an empirical... A. No.
Q. ...Research to substantiate that. A. Actually do, which I’ve completely forgotten about because it actually relates to comparison analysis and I know of several papers that – comparing blood tests to breath testing and in those, while there is always going to be difference between the breath test and the blood test...
Q. Yeah. A. ...Based on the time that the tests occurred, as well as the fact that breath testing in North America produces results that are systemically low between 10 to 12 percent... Q. Yes. A. ...So in correcting those two factors, these comparison studies have shown that there is good correlation between blood and breath results from the same individual from the same incident and moreover – more importantly is that there was the difference between the two did not vary by concentration. In other words, there was no evidence that breath-testing instruments got worse as the blood concentration changed over the occurrence and most if not all of these tests were done with instruments that had been in the field for some time. Some could have been just re-calibrated; others certainly would have been in use for a variety of times...
Q. Do you have a copy of that study? A. Not with me, no. It’s one by Jim Wigmore and I forget who the other author was. Then there’s a paper by Hodgson who also looked at the relationship and there is one by Cowan and I don’t know the age of the instrument he used but he performed simultaneous analyses, blood and breath, using the Intoxilyzer 8000C and showed that in 100 percent of cases the breath result was lower than the blood result and it – then he wasn’t doing a linearity analysis in that particular case. Q. All right, let’s talk about Wigmore and Hodgson. Do you have knowledge of whether the instruments in that particular case, it may have been older instruments, but when had they last been recalibrated? A. That’s information – in the Centre for Forensic Science study wasn’t available. I don’t know Mr. Hodgson’s study makes any statement about when the instrument he was using was last calibrated. Q. So those studies don’t support the hypothesis that it doesn’t matter the length of time between calibration and the breath testing result doesn’t matter in terms of reliability of the instrument. It does not support that hypothesis. A. I would disagree. Q. Well, except that we have no information. You’re saying that these studies support that hypothesis but we have no information about how recently before the testing had been done, the simultaneous testing had been done, of the length of time before that instrument had been recalibrated. I mean, instruments that are out in the field – I’m sure that your instruments at the Centre of Forensic Sciences from time to time go back for recalibration?
A. Very rarely, yes. Q. But on occasion they do?
A. Yes. Q. But the point is that nobody’s done an empirical study to determine whether that linear relationship lasts over a long period of time? A. I can’t say with certainty whether there has been a specific study of that or not. Q. All right. A. And I don’t have access here to my Alcohol Test Committee files to see if in fact early on at some point if the Committee didn’t actually perform that or other labs from which those individuals came had in fact done such studies.