How facial recognition software works

What really is facial recognition technology? From unlocking phones to locating missing persons, find out how biometric tech is already changing how we live and work.

Facial recognition software is what makes it possible for devices to recognise faces. Put simply, facial recognition systems create ‘maps’ of people’s features from scanning a photo or video. They then check for any matches that have been purposely stored in other databases to speed up identification.

This software is used across various aspects of our lives, from unlocking your personal mobile phone using facial recognition, to more advanced cases such as locating missing persons. This biometric technology is helping to create safer societies but is often not fully understood.

How does facial recognition software actually work?

In essence, facial recognition software learns what’s unique about your face so that it can find it again. Some systems are more advanced than others, but most involve these four stages:

  1. Feeding the database. First, the system is ‘fed’ with faces, so it has something to match against. To manage secure entry at a bank, for example, it might contain photos of staff. For security uses, it could mean a live video feed from a major event.
  2. Creating templates. Next, the algorithm detects facial features and creates a unique template – a ‘face print’ – for each face in the database. It’s a little bit like a fingerprint, in that it creates a record of a pattern based on reference points.
  3. Finding faces that match. The system then compares these templates with other image sources to find a match. These sources could be another database or even live video.
  4. Alerting users. The software alerts users to possible matches so they can check identity and act. Because every match gets either confirmed or rejected, the AI powering the system then learns to do it better.

How is automatic facial recognition used by the police?

The use of automatic facial recognition (AFR) varies between forces. For example, it can be used to identify individuals on their known ‘watch list’ using real-time CCTV. It can also help officers to compare crime scene stills with those in their custody systems. In other forces, it’s used to tackle specific types of crime, like gun violence.

AFR has proven a useful tool for officers. It can match up to three million images per second, narrowing a search in a way that can’t be done manually. Yet it’s not without its critics. Privacy is still an issue, although many systems will auto-delete unused data. Older algorithms can fall short on accuracy too, struggling with gender and race. The US National Institute of Science and Technology publishes annual ratings to help compare the performance of different systems. It’s a welcome development, given how quickly new vendors are joining the market.

Where else is AFR used?

AFR is already in use by governments and industry in every region. Here’s just a few examples of how this technology is changing our lives:

  • Tackling problem gambling. By helping casinos refuse entry to those who want to be excluded.
  • Finding Australian WWI veterans. By comparing images from 1916 with existing photos.
  • Improving customer service. Like adding an extra layer of security at cash points.
  • Speeding up security. By replacing manual checks at busy transport hubs like airports and train stations.
  • Cutting queues. By making it easier to monitor public spaces so staff can ensure safe crowd control.

What’s next for facial recognition software?

Looking ahead, the software has huge potential. It already helps to find missing persons or suspects in a crisis. The AI is learning constantly, to the point where it’s not fooled by hats, sunglasses, or face coverings. Potential data sources – like smartphones and body-worn cameras – are growing each day. And it can help confirm identity without physical contact.

But its success will depend on keeping the faith of the public. So, we need to strike the right balance between safety and privacy. Firstly, by taking active steps to reduce bias and by only choosing to deploy technologies developed by responsible suppliers. Secondly, by making sure that its use is proportionate. And lastly, by being clear about where we’re using it and why.

Our product NeoFace® Watch is the world’s most trusted facial recognition software. Performing consistently across age, ethnicity and gender, it combines the best feature-matching and AI technologies.