Not until long ago, an unmanned aircraft’s ability to identify people from the air sounded like science fiction. With overwhelming technology advancements in both software and hardware domains, an affordable method for face recognition from live video on drones is now available, opening up a wide range of paramilitary and civic-commercial applications.
In 2002, the blockbuster ‘Minority Report’ was released. Directed by Steven Spielberg and starring Tom Cruise, the science fiction film is set in Washington D.C. in 2054, where police utilise psychics to arrest criminals before they commit their crime. The film displays the use of highly innovative technologies including facial recognition for targeted advertising, and tiny spider robots for eye scan identification.[1]
Wearing my critic’s cap for a moment, the idea of predicting future crimes wasn’t completely original.
It appears in the Bible several times, long before Hollywood was even conceived, for example, in Deuteronomy 31 in the prophecy of Moses in the days before his death. Nevertheless, the idea of using a facial recognition system in a shopping mall in order to identify customers and show them targeted ads (also known today as ‘Smart Signage’) was quite remarkable and predicted the future use almost perfectly. In fact, malls and chain stores around the globe are using facial recognition software these days to count visitors, identify repeat customers and display advertisements based on age and gender estimation of the software.
Spider robots
Another interesting concept in the film were the small spider robots, which could identify humans. Although it was a step in the right direction in predicting the future use, nowadays it comes across as a bit clumsy and ineffective, especially compared to how face recognition can be performed by drones these days. The spider robots had to climb a human’s body with their metal feet to literally look someone in the eye in order to identify their target. The way the eye scan was performed was rather invasive and assumed that the target would cooperate while standing still, until the spider robot was satisfied. I don’t see too many suspects allowing any robot to climb all over them. Do you?
On the other hand, a drone that can perform face recognition makes use of a far more elegant, fast, efficient and non-invasive technology capable of identifying a person half a mile away from the air, provided of course it is equipped with the right gear, including a high-end camera, GPS, radio control and Wi-Fi. So how has reality exceeded fiction in such a relatively short period of time?
Technology revolution
The short answer to this question is technology. In the last few decades technology has grown at an exponential pace, empowering brilliant innovations in both hardware and software domains. For example, a rough, non-scientific comparison between the latest six-core Intel® Core™ i7-8700K processor and a 35-year-old Commodore 64 shows that the processing speed has multiplied by more than 20,000(!). With this tremendous increase in computing power came the ability to run complex mathematical and artificial intelligence (AI) models, as well as to process video in real time, which essentially allowed computer vision technologies including facial recognition to mature.
You might not have noticed it yet, but face recognition nowadays is being used almost everywhere: in shops, supermarkets, hospitals, classrooms and workplaces, and even in the latest iPhone as an unlocking feature. Although it looks like the recognition happens instantly, the actual process is composed of several steps which must take place before any face can be identified.
Facial recognition process
The first step in any facial recognition software is the detection of a face. That means the software must know that there is a face in the frame before the face can be identified. Most recent face detection algorithms are created by AI models, (aka ‘deep learning’). This is done by feeding millions of faces to the computer until it learns what a face should look like.
Once a face is detected, it has to go through a normalisation process, since not all frames were created in the same way, or better yet, not captured alike. Faces differ in size and resolution because the distance to the camera varies, and different cameras and lenses are used. Normalisation makes all faces comparable.
The next step is template generation. After a face is normalised, a mathematical model analyses the face and turns the pixels into a text string which represents the face. This process is also referred to as ‘feature extraction’. The subsequent matching process that compares the face from the camera to the facial images in a database is therefore much faster and more efficient, since it compares text and not actual photos.
Unlike what many people think, face recognition is not a matter of a simple ‘yes’ or ‘no’, but rather of obtaining a similarity score. Each face in the database gets a score between 0 and 100, which tells us how much it resembles the facial image from the camera. If the score exceeds a certain threshold that has been set by the user, an identification is declared. Naturally, a threshold trade-off is created between the recognition rate and the miss (rejection) rate: if we set the threshold too low it will allow us to identify low-quality images but at the same time it will increase our miss-identification rate. Setting the threshold too high can eliminate the false identifications but will only allow us to identify high-quality facial images. The solution is usually to use vendor’s best practice or engage in trial and error until you find your right threshold.
Facial recognition requirements
Notwithstanding the above, even the most accurate algorithm doesn’t ensure a high accuracy with your facial recognition system. Back in 2015 I gave a presentation during a video analytics conference. The presentation title was ‘Taking the Magic Out of Face Recognition’ and its purpose was to debunk a number of common myths surrounding face recognition. I also listed a number of requirements that you have to have in place if you want to identify people in video surveillance:
- Face: face recognition as its name implies actually needs a face for the purpose of recognition. People wearing ski masks or burkas can’t be identified.
- Lighting conditions: good lighting conditions are crucial for an accurate identification. Leading commercial facial recognition technologies still can’t handle the sun’s backlight or too dark/bright images successfully.
- High-quality reference photos: high-quality reference photos are necessary, as these are the ones that are enrolled into the database. You can think of them as high-resolution passport photos, which can be obtained by using any smartphone.
- Camera location: the camera’s location needs to be considered carefully. You can’t mount a camera 4 meters high and expect it to identify people from a short distance, simply because the camera will capture more hair and less face.
- Face size: if all prerequisites are met but the face in the video is too small then identification will fail. Most commercial technologies require at least 30 pixels between the eyes for each face in the video.
Drones anyone?
Drones, formally known as Unmanned Aerial Vehicles (UAVs), are aircraft which can perform autonomous pilot. They can easily reach difficult or dangerous locations and collect images through aerial photography. Their popularity was powered by a series of technology advancements[2], including the following:
- miniaturisation of electro-mechanical parts (some smaller than 0.01 millimetre);
- gyroscopes which serve as stabilisers;
- GPS availability;
- weight reduction for all components including a lightweight and energy-dense lithium battery;
- high-torque brushless motors.
Along with their commercialisation, drone prices decreased significantly, allowing anyone to buy a magnificent toy for under USD 100.
It is true that the army has been using UAVs for almost a century for purposes such as bomb detection, air strikes and surveillance operations, but new-generation drones started offering other affordable applications in many areas:
- filming and journalism – providing stimulating and refreshing shooting angles;
- rescue operations – locating lost persons in challenging terrains;
- healthcare – sending medical supplies to unreachable locations;
- shipping and delivery – delivering parcels and groceries via air;
- law enforcement – public safety and monitoring criminal or terrorist activity.
An insider’s view
To get an insight into the use of face recognition by drones, two drone experts were interviewed. The first is Tim Ketels, CEO of AerialSolutions, an industrial UAV systems company located in Belgium.
“Face recognition software opens up new perspectives for drone-related security applications”, says Tim. “Live video feed from aerial drones can be used by face recognition software to gather and compare faces from different watchlists. Think about tracking down fugitives or unwanted individuals in crowded places.” Tim’s company partnered up with a large security company to create a drone division that can deploy drones in a matter of minutes, in combination with a tethered power supply, with a mobile camera post that can be moved to different locations and stay in the air for longer periods of time.
Tim continues: “The ability to use facial recognition software on fixed cameras in combination with aerial cameras is a huge advantage to track people over large areas, for example during mass events. Our guess is that the drone market will grow exponentially in 2019-2020 when European legislation comes into effect.”[5]
Next in line is Ronen Keidar, a drone’s engineer and the CTO of the Israeli company Sky Sapience, whose expertise is tethered hovering UAVs, as a replacement for tactical balloons. Ronen explains: “We develop heavy-duty drones mainly for the paramilitary market, such as police, border control and anti-riot units. Unlike the standard drones, our tethered hovering system can carry a lot of weight including a 10-kilo camera lens which enables us to identify a suspect from half a mile away, from a height of 100 metres. Since our drone is connected to the ground via power cord, it has an unlimited power supply, and the communication to ground control can’t be intercepted. These capabilities allow us to collect intelligence over much longer periods of time.”
The tethered drones are also used in commercial venues such as football stadiums, in order to identify known hooligans and to track trouble makers in the crowd. However, Ronen doesn’t ignore the capabilities of untethered drones: “Their manoeuvrability and capability to track a suspect can serve as a complementary measure to our system.”
So what’s next?
Face recognition incorporated in drones clearly creates a unique and valuable combination, mainly in the security arena. The use of this powerful application will become widespread as more decision makers will realise its advantages. As technology continues to progress, we can also expect to see an increase in the use of drones in the civic-commercial market.
References
1 Loughrey, C. (2017). Minority Report: 6 predictions that came true, 15 years on, The Independent. [Accessed 4 April 2018].
2 Gao, Y. (2015). Quora question: why have drones suddenly become a big thing? Newsweek. [Accessed 4 April 2018].
3 Mowat, L. (2016). EUROPE’S NO-GO ZONES: List of 900 EU areas where police have ‘LOST CONTROL’ to migrants, Daily Express. [Accessed 4 April 2018].
4 Brown, J. (2016). Drone uses: the awesome benefits of drone technology. My Drone Lab. [Accessed 4 April 2018].
5 European Commission Press Release (2017). Aviation: Commission is taking the European drone sector to new heights. [Accessed 4 April 2018].
Moshe Greenshpan is an international face recognition expert and founder and CEO of two face recognition companies: Face-Six LLC and Skakash Ltd. He is a keynote speaker on the subject of face recognition, and holds an MS in Finance from Baruch College, City University of New York.
