• Stephen Biss

Scientific Reliability of the Measurement Result

THE COURT: Can I – can I just intervene just for a moment, Mr. Biss, just to clarify where you’re going with this? The Crown isn’t required to establish anything other than the device used falls within the legislation and I appreciate you’re going back to first principles here, and talking about how we measure, how the device analyses. Quantification of weights and measures. Accuracy and certainty in terms of outcome. But isn’t there a certain presumption here that Parliament has been prepared to accept in referencing this particular instrument as an approved instrument? There’s an assumption, is there not, that the instrument, assuming the qualified technician determines it to be in proper working order, is capable of giving an accurate result and aren’t I, who’s charged with the responsibility of addressing your request for additional information here, bound by those considerations which I think are presumptions within the context of the Criminal Code provisions that govern utilisation of this machine, this instrument, for this purpose? Or am I missing the boat here? MR. BISS: No, you’re not missing the boat Your Honour, and that’s a very – very normal, conventional, usual way to look at the problem, that I run into when I talk to other colleagues in the defence bar. The problem is that in

2008 I guess it was, we had Bill C-2. And C-2 changed the whole system so that instead of the defence, in defending against the presumption in 258(1)(c), instead of the defence presenting evidence that contradicted the measurement result of the Intoxilyzer 8000C or the Intoxilyzer 5000C that we moved to a new era where the Court needs to – or more correctly, the defence needs to raise an issue having to do with function of the instrument or operator error, or both. And it appears that the case law has developed, through R. v. St‑Onge Lamoureux, and through R. v. Lam that what the defence needs to do is to cast doubt on the scientific reliability of the measurement result. In other words, we have to – as I – I remember Justice Tuck-Jackson one time confronting me on this and saying, well, Mr. Biss, connect the dots. Well, I’m trying to connect the dots to show that what I think is a problem with the function of this instrument, there may be operator error as well, and that’s why the video is so important, and which we don’t have,

and that’s why the – the variation in the – in the test record, the interlineation in the test record is so important. We don’t have a video in relation to that. But from the perspective of function the difficulty is that an instrument isn’t an instrument anymore if it doesn’t have its calibration. Now in R. v. Jackson, the Court – the Ontario Court of

Appeal was dealing with a new instrument that was not yet due for its first periodic inspection. In this case, we have an instrument that the certificate of calibration reads 2009.

My client was tested in 2015. I’m going to suggest to the Court that there’s a big difference in that situation. But the point is, the two issues the Court is ultimately is going to have to deal with that I’m going to have to make submissions at the end of the day on, are has there been a change in the function of this instrument, i.e. a malfunction, having to do with, among other things, calibration of the instrument and I’m going to continue in my cross-examination, I want to continue in my cross-examination to ask Mr. Palmentier questions about those periodic inspections and what they show. I’m going to suggest that there is a problem with function. But the issue then, that I need to address is the scientific reliability of the instrument and its – I’m sorry. That’s a mistake. Not the scientific reliability of the instrument, but the scientific reliability of the measurement result using the instrument. They’re two different things. Now, we have, we’re very, very fortunate in this case, and I don’t know of any other cases that have really looked at what Brian Hodgson had to say, in the report that was used by the Supreme Court of Canada in St‑Onge Lamoureux and was – and of

course, Mr. Hodgson gave evidence at the time, when C-2 or its predecessor were being contemplated, to talk about what reliability means, specifically instrumental reliability.

Instrumental reliability has to do with significant change in accuracy and precision over time. And I’m going to be cross-examining Mr. Palmentier about that. I expect that Mr. Palmentier will have a different perspective maybe on what ‘over time’ means. He may have a different perspective on what the word ‘significant’ means. The word – what the word ‘significant’ means may ultimately be a trial issue. But if we’re dealing with the issue of relevance, which I think is the key here, the relevance is: what is the scientific reliability of an instrument that is starting to express its function in a way that may or may not be a malfunction. But my – what I need to focus on is not just that. Not just whether there’s a function or malfunction of this instrument, but I’ve got to address what’s the scientific reliability of that instrument. So, we know that this instrument – it’s measurement results shortly after it was calibrated, and brought into Canada, then applying Jackson, and looking at the Jackson kind of fact situation, we’ve got a certificate of calibration from the manufacturer. We have a certificate of the traceability of that calibration through NIST and back through the rest of the calibration hierarchy. And the word calibration hierarchy

is there in the – the document that has all the definitions in it. The point is that initially these instruments, we can assume them to be reliable. That’s quite apart from there being a legal presumption. We can infer their reliability because they’ve got a certificate of calibration. They have traceability. And that’s why – that’s the reason for me doing the references to the paper clips. The point is, that measurements need to be traceable to the original measurement standards through a calibration hierarch. And the B-I-P-M deals with that extensively as to how you do that. But the point is, that that measurement reliability may change over time. THE COURT: Well, I – I’m just going to make this suggestion. Unless I’m misunderstanding Parliament has determined that this particular instrument is reliable. It’s approved, and it’s reliable for the purposes in which these sections apply. MR. BISS: Yes.

[Quaere: does this mean the presumption can never be rebutted - contrary to St-Onge?]

THE COURT: The sections of interest in this case. How they reached that determination, I’m sure, you know, we’ve touched on some of that. You’ve touched on the calibration process with respect to this particular instrument which is a process that the witness, just by virtue of his experience in training others to use this instrument is familiar with but he’s not an expert in calibration. But I think we can start with the presumption that Parliament has

determined that this is a reliable instrument for the purpose designated. Now what you’re contending, I gather, and where the experts disagree, yours as opposed to the Crown’s expert witness, is with respect to the issue of reliability over time and whether historic information of incidents that have occurred over that period of time involving other test subjects might compromise the reliability of the test involving your client. Your client is the only one I’m interested in. Not the theory of this device generally, or the history of it or why Parliament determined what they have. The Court of Appeal has already opined on this, so – and I authorised you to proceed in this fashion because I recognise that there were two factual distinctions that were of significance in the Court of Appeal. Otherwise, I’m bound by what Justice Watt has determined there.

[in Jackson]

You know that. And so, you have to somehow persuade me that this is a distinction with a difference. And I appreciate you have great scientific knowledge. In fact, I’ve learned more about this device just over the course of our interactions in the past few days than maybe I’ve learned sitting here for now, almost 15 years. But some of it is presumed, and that’s all I’m saying, for the purposes of the application here, we’ve got to focus, I think, or we’re not going to get to the finish line, on why you say, assuming, as Parliament has, a reliable instrument, properly calibrated,

certified by those capable of doing both of those tasks, over time, as maybe – maybe unreliable and the test result in issue in this case may not be relied upon, and that I shouldn’t be certain of that, and that I can infer that from previous test results where errors occurred or error messages or the fact that recalibration had occurred or was about to occur. Why do the experts differ in that regard? Why do you say, through your expert witness, for whom I have a lot of respect and have known over many years, and have had testify in similar cases many times in the past, why do two individuals with similar scientific experience have such a marked difference in opinion with respect to that – that issue? And you know, it’s not a question of preferring expert opinion here. It’s a question of whether the evidence creates – is capable of creating some doubt, or warrants disclosure of additional information. But I think we’ve got a – it would be helpful to me, at least, if you narrow the focus to your client. And I appreciate there’s a record here to be developed. I get that, and I’m certainly trying to grant you whatever leeway may be required, but I think we have to start with the presumption, especially in view of the Court of Appeal’s existing review of this subject, that the machine is reliable. It’s capable, certainly when it is placed in use initially, and I don’t think you’re disputing this,

provided it’s been properly calibrated and the manufacturer’s specifications have been met, and the appropriate testing has taken place, that it’s capable for its designated purpose. And we don’t have to debate that. Parliament has said that and we can’t go behind the legislation. I know you’re not endeavouring to do that. So what about – and we’re really talking about now, and the difference between this circumstance and the one before that was already determined, involving many of the same issues, is that this is an older machine. This instrument has been in use for a long period of time. Does it make a difference? Is the reliability suspect over time? I’ve had one expert tell me all of these previous error messages and concerns that were referenced in great detail compromise the reliability of the test results in your client’s case. And I have another expert, who I’ve heard over a period of now two days, who says that’s not so. So, help me understand that because that’s really, I think, where the – and you referenced it. That’s where the difference of opinion lies. Well, I suppose informed scientists can disagree just like you know, lawyers disagree. That’s what we hear every day in here, lawyers disagreeing. Well, is one wrong, one right? Well, sometimes it’s a matter of interpretation. But we’re dealing with scientific principles here and I was always under the belief, well, there’s a reason you

end up as a judge in the criminal Court. It generally means you’re not the greatest scientist or else you would’ve been – you would’ve become a chemist or maybe work at the Centre of Forensic Sciences or you would be doing something else. Now, I have some colleagues who – who have science degrees, having said that, but most of us weren’t that great at science. So explain to me in a way I can understand, how does this reliable machine, which goes through all of these manufacturer’s specifications, employment of international standards, recognition by Parliament, that we utilise on a daily basis in this Court to quantify blood alcohol concentration readings that we rely on in order to either question the net result or in more often than not, accept the quantification of the blood alcohol concentration level, absent some legal misadventure, and there are many of them, how over time does it become unreliable? And why is it that your expert says it does and the resident expert today says that’s not the case? That’s historic data of no moment. That’s the issue. So, help me understand that, because I think we can start with the presumption of reliability, based on what you’ve told me and how you define that term. Unless I’m missing that point, I think you’re trying to tell me in slow measured terms, I get that, probably a good way to go, that you know, it may start out with all the bells and whistles as far as

reliability is concerned, independently assessed and confirmed, that over time that’s not the case.

[the full answer to this question has to do with "drift" (see Hodgson definition of reliability above and OIML R126 on drift), changes in "linearity" over time, and "uncertainty growth" v. "calibration interval")

MR. BISS: I’m going to do that, Your Honour. But just before we go there, I just ask Your Honour to consider the difference between approval of type or kind, and an individual instrument. And just think about that for a moment. We know that the Intoxilyzer 8000C was evaluated and then memo went from the Alcohol Test Committee to the Minister, who then approved the Intoxilyzer 8000C as an approved instrument. In order to do that approval there was – there were actually two instruments that were submitted for evaluation; one at the Centre of Forensic Sciences in Toronto, and one to the R.C.M.P. Reports were drafted up, they’re submitted to the Minister. The Minister then says this is an approved instrument. The question of whether or not Intoxilyzer 80 dash 0-0-4-6-3-2 is identical to the instruments that were submitted for evaluation is another story. Now, yes, initially I want to suggest to you, it’s safe – pretty safe to assume because we know these instruments have a calibration history at the manufacturer, we’ve got this document here.

[the certificate of calibration from the manufacturer] The question is, and ultimately the question is, is the instrument that is being used in 2015, does it have the same metrological characteristics or similar metrological characteristics

[see OIML R126 in 2012]

to the new machines that were

evaluated by the Centre of Forensic Sciences and by the R.C.M.P. on which the Alcohol Test Committee made its final decision? They were new instruments. And in a moment I’m going to spend some time with Mr. Palmentier talking about not just specifications for the instrument, but also the evaluation process and what the Alcohol Test Committee found when they did that evaluation. And then ultimately I want to compare that with the instrument in our case. Now when Your Honour says presumptions, yes, of course, there’s the section 258(1)(c) presumption. The section 258(1)(c) presumption makes use of something called an approved instrument. That takes us a long way in most cases. I’m suggesting this case is different. And the difference is that it’s only a – 258(1)(c) is a presumption and it is a presumption that can be rebutted and yes, it is challenging to rebut it these days, but our focus in that rebuttal has got to be on function or operator error, combined with, looking at the Lam case, combined with effect on the scientific reliability of the measurement result. And that’s – I – my submission is that’s exactly what I’m focusing on. THE COURT: Well, just so I’m clear, are you suggesting that this – this particular Intoxilyzer 8000C may not have been reliable from the outset? Is that.... MR. BISS: No. No.

THE COURT: All right, well then.... MR. BISS: And – and the reason why I’m doing that is because I have to face the reality of Jackson. THE COURT: Well that – well, that’s what I’m really just asking you to focus on. You – you were talking about – the difference here, in this case, and this is why we’re having this discussion otherwise it would’ve been much easier, certainly for yours truly, to have simply said at the outset, as I was invited to by the Crown, well, Jackson applies, I’m not interested in the distinction. It’s a distinction without a difference. Jackson applies. Court of Appeal’s already opined on this. You’re – you know, it’s very interesting to review the history of the utilisation of the device or the instrument, but it’s of no significance to the subject test. But I recognised and you were persuasive in that regard, and I think it’s reasonable for you, because of the differences here, we have a – an older instrument. Does that make a difference?

[see concept of "uncertainty growth" by NASA]

And that wasn’t before the Ontario Court of Appeal, and as you referenced, you may have been waiting for a circumstance where the deficiencies that have been identified in that case were manifest. Well, this is – this is that circumstance, but I think we have to focus on the parameters that the Court of Appeal has already referenced as being of significance, and I appreciate you’ve got a much broader

period of time to deal with, and you’ve got history, at least some of it, that has been reviewed, but again, I have to understand whether this makes a difference and why. And it’d be helpful for me to understand from the perspective of the current expert why his view is so different from that advanced by your own. Now I think I understand it because the focus is on the instrument’s ability to confirm its own reliability,

[a "policy" not a "scientific opinion", in the 2012 Position Paper]

in effect, and you take exception to that. So I’m going to let you develop this, but I think we have to – at least I’m urging you to sort of, keep your eye on the ball here, in terms of the limitations inherent in my role, and those limitations are governed in large measure, by the Court of Appeal decisions already been made. I’m going to have to be persuaded that there’s a distinction to be drawn here. And – and given the ruling already in place, there’s a limited purview for that to occur. We’re having this – we’re having this evidentiary hearing because I think there is a distinction to be drawn, the net result of which I’m – you know, that’s why we’re having this interesting hearing but again, I’m not endeavouring to restrict you other than I’m asking you to focus on the things that – you have broad knowledge in this area – that might be found compelling within the context of limitations inherent in the ruling that’s already been issued on this subject.

MR. BISS: All right. I – I’m going to move into something else then, Your Honour. THE COURT: Okay. MR. BISS: And I want to focus on – perhaps on the difference of opinion between Mr. Palmentier and Mr. Kupferschmidt. THE COURT: Okay.

#argument #uncertaintygrowth

13 views0 comments