VCMERS Webinar

Results of the 2018 3D Virtual Comparison Microscopy Error Rate Study (VCMERS)

Webinar length: Approximately 40 minutes

Summary:

3D Virtual Comparison Microscopy (VCM) is a powerful tool for microscopic examination which presents a firearms examiner with a highly detailed visualization of a toolmark surface. We hypothesize that over the next several years VCM will prove increasingly useful and will begin to see use in many crime laboratories. To support the use of VCM within the laboratory it is critical to establish the performance of VCM. This presentation will teach the audience about VCM and will present the results from the 2018 VCM Error Rate Study (VCM-ERS) completed as a collaboration between Cadre and the Royal Canadian Mounted Police (RCMP) National Forensic Laboratory Service. The VCMER Study was designed to investigate error rates and to identify the specific topographic features used by examiners when reaching source conclusions via VCM. The study focused exclusively on virtual comparison microscopy of cartridge cases. Forty test sets each containing three cartridge cases (two “known” and one “unknown”) were created. The selected test sets represent a variety of tool manufacturing/finishing processes and class characteristics. The sets include both cartridge cases that are well marked and those that are minimally marked. The sets range in difficulty, from easy to hard, to represent the variability experienced in real casework. The 3D surface topographies of all scans were collected using Cadre’s TopMatch-3D scanning system at a resolution of approximately 1.5 micrometers per pixel. Each participant was randomly and blindly assigned sixteen different test sets to evaluate. For each test set, participants were asked both to reach a source conclusion (utilizing the 5 Point AFTE range of conclusions) and to annotate areas of similarities and differences that were used when reaching their conclusion.

Results:
A total of 107 participants completed the study, including 63 in the United States. Participants reached 1712 source conclusions. In this presentation we will describe overall study results and error rates. Results will be broken-down by test set and toolmark class. Presented results will include our summary annotation maps, which are visual topographic overlays indicating the percent of participants that utilized each portion of the 3D surface when reaching their conclusion. Initially introduced in 2016, these annotation maps provide unprecedented insight into the examiner decision process and highlight the degree of consistency demonstrated by examiners. The results of this study strongly support the use of 3D Virtual Comparison Microscopy as a viable alternative to traditional light comparison microscopy within the discipline of firearm examination.

Presenters:

Ryan Lilien, MD/PhD. Ryan's research expertise focuses on the use of advanced scientific computing and statistical models to solve interdisciplinary research problems. Ryan earned an MD/PhD from Dartmouth Medical School and Dartmouth's Department of Computer Science. Ryan was faculty at the University of Toronto cross-appointed between Computer Science and the Faculty of Medicine. He has received research funding from the Gates Foundation, NIJ, NIST, and Canada’s National Sciences and Engineering Research Council. He is now located in Chicago and serves as Cadre Research's head of research and development while maintaining an adjunct appointment at the University of Toronto. Ryan leads development of the TopMatch-3D High-Capacity system (a 3D imaging and analysis system for firearm forensics and virtual microscopy). He’s presented his group’s steady progress on developing and validating the system at recent national and regional AFTE meetings. Ryan is also currently a member of NIST's OSAC (Organization of Scientific Area Committees) Subcommittee on Firearms & Toolmarks.

Todd Weller, MS. Todd has been a Criminalist for over 17 years. He worked for the Oakland Police Department for 16 years and is now in private practice. He has performed casework in drug analysis, DNA, crime scenes, and for the past nine years has worked in the firearms identification specialty. He graduated from the National Firearms Examiner Academy (NFEA) in 2009. Todd has a BA in biochemistry/molecular biology from Dartmouth College and a MS in forensic science from the University of California, Davis. For his master's thesis, Todd collaborated with NIST to study confocal microscopy on test fires from consecutively manufactured firearms. Todd is currently the Chair of NIST's OSAC Subcommittee on Firearms & Toolmarks.

If you're interested in attending please complete the form below.