30. April 2020

SCCH: Pattern Detection – How AI Increases Safety

Computers have one advantage over humans: they can work on a task for hours without a break and with the same level of precision. Researchers at Software Competence Center Hagenberg (SCCH) exploited this fact to improve protection of buildings in security research (KIRAS). As scientific partners on the SKIN project (protection of the shell of critical infrastructures) they succeeded in reducing the error-proneness of video surveillance of public buildings by approximately 65% per camera and day on behalf of the Federal Ministries of Internal Affairs (BMI) and Defence (BMLV) and the Vienna Centre for Societal Security (VICESSE) under the leadership of PKE Holding AG. The project findings are incorporated in the lead project, “Connecting Austria”, and the EU project ALOHA.

 Surveillance of public buildings such as embassies, museums and offices has so far taken place using cameras. The cameras send images to screens in a control room where the actions of people are monitored. Unexplained movements in the vicinity of the building set off an alarm. The task of the SKIN project, which is funded by the KIRAS security research programme of the Federal Ministry of Transport, Innovation and Technology, was to find ways in which artificial intelligence can be used as an assistance system to detect activity in the vicinity of a building and so support security staff.

Generating knowledge from data

Software Competence Center Hagenberg conducts research into methods of analysing image data on the basis of geometric models and concepts. The research findings are used for movement analysis in medicine, object and person tracking in sport and quality inspection in industry. From mid-2014 to 2018 three researchers worked on the SKIN project, supported by Dr Bernhard Moser, research director at SCCH.

“We are working here in the field of artificial intelligence (AI) with deep learning, that is the progressive learning of interconnected systems,” explains Dr Moser. “The aim is knowledge-based event detection. The basis of this is large amounts of video data in combination with technologies that take into account observed or predictable conditions such as the weather or typical movement patterns. The aim is to train the network so that it learns to recognize predefined situations and not to set off the alarm at every movement of a tree.”

 Currently, surveillance systems too often trigger the alarm so that a member of staff checks whether an observed movement poses a threat to the building’s security. Because Austria is one of the safest countries in the world most of these incidents do not pose a real threat. For the security guard, however, this is tiresome: he has to respond to every single incident, even if it is highly likely that it is a false alarm.

Behavioural analysis can help here by deducing and qualifying movement patterns from the videos. For this, the researchers in Hagenberg need thousands of items of data relating to every movement and event that occurs around a building, ideally over the course of a year: a huge range of patterns of moving and stationary traffic on the road and in the car park, including anomalies such as refuse collection vehicles, pedestrians, cyclists, children’s prams and buggies – in other words, where and how vehicles and people move – as well as carrier bags being blown past and trees moving in the breeze. To this end, the firm of PKE supplied the SKIN project with data from CCTV. Relevant privacy issues were cleared up by PKE with the project partner VICESSE and taken into consideration in the system’s design.

Data-driven modelling

“We do not process any data that identifies individuals,” says the data scientist, describing the strategy adopted by SCCH. Instead, only anonymized movement data is used. This makes it possible to see which direction a person is going in without drawing conclusions as to the person’s identity. Using video data from the premises to be monitored and universal models of how and where people move (upright, mostly on defined paths such as footpaths) so-called movement maps are created by means of event recognition. The most common paths represent normal, non-threatening movements. The researchers can derive predictions using data-driven modelling. They then combine these predictions with facts about the environment: to do this, the system user determines relevant data such as the main entrance, peak traffic times, a nearby school with public access, entrance to an inner courtyard for refuse collection, footpath or exit. Movements such as those of trees are blocked out. To increase the accuracy of the predictions the experts in Hagenberg use this knowledge to construct a system based on probability and to train it using deep learning: it detects a movement that goes from A to B and learns on the basis of observation data gathered over many years how this movement will most probably continue. As long as the movement continues according to the pattern that the system has learned, no alarm is raised. This has made it possible to reduce the number of alarms by 65% in the prototype of a new security system for PKE Holding AG.

The methods used also allow subsequent event searches in the video archive without the need to view individual videos and without the search criteria having to be defined during recording. “This presents no difficulties whatsoever because every situation is stored as a pattern,” explains the researcher: “The video footage is split into segments as it plays. This makes it possible to search for every image showing a bicycle and a second, red bicycle, for instance. On the other hand, the movement pattern allows a search for a person who turns round at a particular spot.” Work on this has already started at PKE.

“From our perspective, SKIN was a successful project in every respect,” says Moser. “It also led to publication of a scientific paper – a key aspect for our competence centre. It has given our personnel experience with this type of data and shown them more finely tailored methods. These methods are now being incorporated in the lighthouse project CONNECTING AUSTRIA, for example, which is examining the ideal connection of energy-efficient and automated freight traffic from the motorway into the town, and in projects such as “RAILEye – AI and the blind spot on rail vehicles” and the EU projects ALOHA – in association with PKE – and TRESSPASS for predicting behaviour of individuals at defined boundaries and in airports.”

Photo 1: The pixels show movement patterns: the more frequently the recorded movements follow a particular trajectory the more accurately the data can be grouped to form primary movement patterns and the more easily deviations can be detected.

Photo 2: The findings of the aforementioned data from AI are among the things displayed clearly and ergonomically on a control console to security personnel along with alarms and error messages requiring their assessment and decisions.

Photos: reproduction free of charge; © SCCH © PKE