The involvement of the general public in the testing of security features and banknote configurations can provide valuable information about the usability and security of new banknote designs. Perception studies provide quantitative information through observation of users during authentication tasks and involve a level of objective rigour that is not captured in the subjective assessments that arise from consultations and focus-group tests. Andrea Firth and Erik Balodis discuss the relatively short and inexpensive process by means of the tests on the ‘Canadian Journey’ series of banknotes.
A good banknote
The Bank of Canada has defined a “good bank note” to be one that is difficult to copy or imitate but is easy to use in terms of authentication and denomination by a user.
Difficult to copy
The Bank of Canada has defined ‘difficult to copy’ according to:
• the investment required to start up a counterfeit operation;
• the rate at which the counterfeits could be produced once the operation is functional;
• the quality of the counterfeits produced.
Counterfeits are grouped into three broad conceptual grades as summarised in table 1. Level 1 counterfeits can be produced with little financial investment and require little time to fabricate. The quality (or deceptiveness) of the counterfeits can be determined through perception testing (discussed below), but in most cases is crude. Level 2 counterfeits are produced with comparatively greater financial investment on the part of the counterfeiter and have a reduced production rate when compared with Level 1 counterfeits. Often, counterfeits of this level are more convincing than Level 1 counterfeits. Level 3 counterfeits have a low production rate and make use of specialised equipment and materials, often producing the highest quality counterfeits.
Easy to use
The Bank of Canada has defined the ‘ease of use’ for a banknote or feature in terms of the ‘intuitiveness’ of usage and the amount of ‘interaction’ required for usage. Security features can be mapped onto scales for intuitiveness and interaction – this allows an assessment of the suitability of features or banknotes for particular user groups.
A feature intended for the general public should be intuitive to use (i.e. requires no training) and require very little interaction (i.e. time and effort) to authenticate. A feature requiring a small amount of training and time to use may be most suitable for a retail cashier. Features requiring a moderate amount of training and time/effort are most suited to bank tellers. And finally, features that require a significant amount of time and specialised training to authenticate can be considered as forensic features targeted at law enforcement.
Perception study experiments to design better banknotes
To design a better banknote and to measure the security and ease of use of features and banknote configurations, the Bank of Canada conducts perception study experiments. This process is relatively short (it can be completed in a matter of weeks), and inexpensive.
The perception studies involve the use of participants from the general public who perform authentication tasks on decks of samples that contain counterfeits. The bank initially tests security features in isolation and follows up with full note design testing. Three grades of counterfeits are manufactured by the Bank of Canada for use in these tests (as discussed earlier).
A comparison is made between the levels of effort invested in fabricating the counterfeits versus the ease with which human users are able to detect the counterfeits.
The Bank’s approach to these studies is to try to predict what may happen in circulation rather than to recreate realistic circulation scenarios, as these may be varied, complex and difficult to capture in a laboratory setting. The approach involves identifying the key variables that most consistently predict the successfulness of a security feature once it is implemented and issued.
The key variables are:
• the authentication accuracy of security features;
• the length of time required for authentication.
These variables can be measured very accurately in a lab setting. The measured results are then used to make decisions on the suitability of security features for particular user groups by correlating the data to a usability quadrant.
Canadian Journey banknote tests
To provide a baseline of information, the ‘Canadian Journey’ series of banknotes were tested, as the Bank of Canada has several years of circulation history with these notes. The holographic stripe, the watermark, the see-through number and the colour shifting thread were tested in isolation in focused feature tests (see sub paragraph a below)).
This series of tests was then followed up with whole note testing (see sub paragraph b) of the Canadian Journey $100 banknote. Each participant was asked to examine samples organised into three sample blocks (see sub paragraph c). In addition to the Journey series banknotes, new security features and banknote configurations were tested to evaluate their efficacy relative to each other and to the Journey features.
a) Focused feature tests
The simplest experiments involved the examination of security features in isolation. To isolate security features on a banknote, the banknote samples were placed in a Bristol board mask which occluded all areas of the note other than the feature of interest. For see-through features such as the watermark or puzzle-number, windows were placed on both sides of the mask to allow for viewing of the feature in transmission (see figure 1).
b) Whole note tests
Whole note tests, in which all security features were available for use in the authentication task, were also conducted. This allowed for testing of security features and suites of features within the context of the entire banknote. For the ‘Canadian Journey’ banknote tests, it was possible to obtain counterfeits from the Royal Canadian Mounted Police. For new banknote designs, counterfeits are fabricated in-house for testing.
c) Sample blocks
Each participant was asked to examine three hundred samples for a security feature or banknote configuration. The samples were organised into three blocks, each of which contained 20% of one level of counterfeit. 20% counterfeit loading was chosen to be the lowest loading possible (in actual circulation, the ‘loading’ of counterfeits is less than 0.005%) that retained statistical power in the results. The three blocks were randomised between participants to remove order bias.
Participants
Two types of subjects were chosen. In one series of experiments only university students were selected to participate in the experiment. This group was selected for their uniformity, high turnout rates (assured by a monetary incentive) and ease of recruitment. The group provided for good uniformity in the data collection, so that different results were significantly different and distinctions between security features could be relied upon.
A second group of subjects was also chosen to ensure that the results obtained from the first group were not biased by the comparatively narrow demographic profile. The second group was recruited from the general public and resembled an inhomogeneous reality-based sample.
Experiment
Authentication rates
In controlled experiments, participants were asked to observe a sample and identify whether it was genuine or counterfeit. The response was then recorded and tabulated as a ‘hit’ (a correctly identified counterfeit) or a ‘false alarm’ (a genuine identified as counterfeit). The ‘hit rate’ and ‘false alarm rate’ were determined as the proportion of all samples that were hits or false alarms. The ‘corrected detection rate’ is the difference between the hit rate and the false alarm rate and is an indication of the overall accuracy of the authentication task.
Intuitiveness scores
In order to establish the intuitiveness of security features or whole notes, the effect of training on the corrected detection rates was tested by providing a group of participants with training on a particular feature or whole note and comparing the results with those of another group that received no training. Intuitive security features and notes exhibited no difference between trained and untrained groups with high corrected detection rates in both groups.
Features or notes that exhibit low detection rates, regardless of training, can be considered poor while those that exhibit a dramatic increase in detection rates following training can be considered non-intuitive (and perhaps suitable for bank tellers or law enforcement) but useable.
Interaction scores
Speeded response times were recorded and tracked accurately via RFID tagging for each sample. Participants were asked to authenticate the sample as quickly and accurately as possible. The time recorded represents the interaction score for the feature/banknote.
Usability Quadrant
To obtain quantitative values for intuitiveness and interaction, the effect of training described above (the difference in corrected detection rates between trained and untrained groups) and the time required for authentication were converted to a numerical scale of 0 to 10 in the intuitiveness and interaction dimensions. These two quantitative dimensions were organised in a Cartesian plot to create a usability quadrant.
Usability quadrants are common in many marketing fields and are well suited to representing the results of these tests. Users can be mapped onto the interaction and intuitiveness scales and, consequently, the usability quadrant (see figure 2). Quadrant I (low intuitiveness, high interaction) was designated as the general public quadrant. Quadrant II (high intuitiveness, high interaction) was designated for cashiers and tellers.
Quadrant III (low intuitiveness, high interaction) was designated for law enforcement. The final quadrant (Quadrant IV) has been designated as the trigger quadrant and corresponds to the subliminal use of information users receive quite quickly, but not very overtly and not necessarily accurately. At present, we do not have experimental data to populate this quadrant with features, nor do we have information that indicates whether or not a reliably authenticated security feature or banknote design can reside in this quadrant.
Results and discussion
The information obtained on the Canadian Journey series of banknotes is very valuable in establishing the relative security of various features on the series and in providing a baseline of data against which uncirculated features can be tested. Features that performed poorly in the tests were determined to do so for a variety of reasons, amongst which manufacturing variability and poor overall perceived quality were most notable. The results of our experiments are consistent with real-life experience regarding the relative security of the features on the Canadian Journey series, which gives us confidence in the methodology’s ability to help guide our selection of future security features.
Part of the whole note tests involved configurations that were taken into full production testing as part of the new note series. Our data indicate that these configurations were easily authenticated at the general-public level and the configuration exhibiting the best performance (the polymer design eventually unveiled in June 2011) was selected as the final configuration.
Regarding new security features, our data indicate that a number of new features on the market, such as volume holograms and micro-optic effects are potentially good quadrant I features, provided careful design considerations are made. It remains to be tested whether these features perform as well in whole note tests. It is important to note that these tests were conducted in a Canadian context and it is therefore possible that these results may differ in different circulation environments.
Perception studies for counterfeit threats
Perception studies can also be used to determine the threat posed by a newly seized counterfeit. By conducting these tests on counterfeits taken from circulation and identifying the authentication rates and time to authenticate, these notes can be placed within a threat zone (blue, orange or red as shown in figure 3). The counterfeit’s location on the threat zone indicates the amount of time at which the counterfeit can be reliably detected and removed from circulation. The time required for accurate detection can be mapped onto the usability quadrant (see figure 2) to give an indication of the user level at which the counterfeit under consideration can be accurately detected and removed from circulation. This information can be used to inform the Bank of Canada on potential responses to the counterfeit threat (education campaign, public announcement, etc.).
Summary & conclusions
The involvement of the general public in the testing of security features and banknote configurations can provide valuable information about usability and security that can be used to make decisions on new note designs. This method provides quantitative information through observation of users during authentication tasks and therefore involves a level of objective rigour that is not captured in the subjective assessments that arise from consultations and focus group tests. The process is relatively short and inexpensive and generates useful information for pre-issue decisions on security value by providing supplementary user-based information to the experts and committees that make design decisions.
Dr Andrea Firth has been involved in the investigation of various aspects of banknote security since 2005. She is a Scientific Adviser at The Bank of Canada where she works on currency technology research and development. She has a background in chemistry and materials science.
