Title: Scenario testing methodology for the assessment of screening technologies

Authors: Ray Johnson; Marcus Butavicius; Kathryn Parsons; Agata McCormac; Jadranka Sunde; Benjamin Hall

Addresses: Weapons System Division, Defence Science and Technology Organisation, P.O. Box 1500, Edinburgh SA 5111, Australia. ' Command, Control, Communications and Intelligence Division, Defence Science and Technology Organisation, P.O. Box 1500, Edinburgh SA 5111, Australia. ' Command, Control, Communications and Intelligence Division, Defence Science and Technology Organisation, P.O. Box 1500, Edinburgh SA 5111, Australia. ' Command, Control, Communications and Intelligence Division, Defence Science and Technology Organisation, P.O. Box 1500, Edinburgh SA 5111, Australia. ' Weapons System Division, Defence Science and Technology Organisation, P.O. Box 1500, Edinburgh SA 5111, Australia. ' Weapons System Division, Defence Science and Technology Organisation, P.O. Box 1500, Edinburgh SA 5111, Australia

Abstract: A range of advanced screening technologies has been developed to detect substances or objects that can pose a threat if carried on commercial aircraft. However, outside of technical testing in a laboratory setting, very little testing has been conducted to estimate their performance in an airport setting. This paper presents protocols developed by DSTO to assess the performance of security devices to detect contraband concealed on a person. These protocols are classified as scenario evaluations because they test the devices in a controlled environment that mimics crucial aspects of the airport environment (e.g., baggage and mock passengers) using trained operators. Such evaluations provide a range of data including reliability rates and timing estimates as well as detection and error rates which are very difficult to determine in either technical or full operational evaluations. The ability to detect and recognise dangerous objects at a safe distance is a very important task in a number of other defence, police and security applications and these protocols may be useful in other environments.

Keywords: security devices; human detection performance; scenario evaluation; evaluation methodologies; scenario testing; testing methodologies; screening technologies; substance detection; object detection; threats; commercial aircraft; airports; evaluation protocols; DSTO; Defence Science and Technology Organisation; Australia; contraband; controlled environments; baggage; luggage; mock passengers; trained operators; reliability rates; timing estimates; detection rates; error rates; dangerous objects; safe distances; object recognition; police; knowledge; computers.

DOI: 10.1504/IJKL.2011.043889

International Journal of Knowledge and Learning, 2011 Vol.7 No.1/2, pp.25 - 38

Published online: 31 Jan 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article