Update 05-2021 – Visual Analytics hits like Covid-19
One of the most promising real-time applications now emerging is the use of advanced analytics and artificial intelligence in video. Sometimes called visual analytics or computer vision, these two powerful technologies combine to open up new possibilities – from improving security to product quality – for growing numbers of organizations.
Visual Analytics help freight railroads
Security cameras, of course, can spot and recognize intruders in real time, and trigger responses. One major freight railroad employs cameras planted along tracks to inspect train-car wheel-trucks for wear and tear as they roll by. Around the world, many thousands of freight wagons are being moved every day, and accurate identification, racking and accounting processes are vital to ensure the timely arrival and dispatch of freight shipments.
In Brazil, most existing technology for counting and identifying wagons relies on radio-frequency identification. Although this provides fast and accurate results, installing specific RFID hardware on each wagon increases both the installation and maintenance costs considerably. Looking at the major advances that have been achieved through so-called ‘computer vision’, which is the automatic extraction, analysis and understanding of information from a single image or a sequence of images, it seems clear that rail freight operations can now make use of alternatives that can drive down costs and enhance operational efficiency.
advanced analytics developed by researchers from the Department of Informatics of the Federal University of Paraná in Curitiba and and the Connect Rail subsidiary of Boslooper Railway Technology make use of the unique identification number granted to each wagon in a given fleet, which is typically painted on the front, rear and sides. The partners have developed an automated tool to read the wagon numbers from cameras installed at certain control points, usually at a freight yard or maintenance depot. This tool has been designed to eliminate as far as possible the costs of RFID tagging while meeting the high reliability and efficiency requirements of freight railways.
How Visual Analytics works.
From a conceptual point of view, the operation of the system is quite straightforward. Firstly, the tool identifies from each image whether a train is passing through the control point or not. If a train is located accordingly, the wagon numbers are then detected in each frame and processed by an Optical Character Recognition algorithm. The processing can be performed either locally by a unit on the ground, or in the cloud if there is an internet connection capable of transmitting the images to the servers. While camera-based systems for reading wagon numbers have been around for many years, the problems in correctly identifying and interpreting the numbers limited the extent to which they could be deployed — leading railways to invest in RFID as a more secure option. However, it is now possible to harness advances in image processing to overcome such problems. The UFPR-Connect Rail algorithm makes use of widely adopted Convolutional Neural Network technology to find the regions containing the vehicle numbers in each image and process them rapidly.
To support the detection tool, only two off-the-shelf cameras are required at each control point. One is installed on each side of the track so that the system can identify every wagon, even in cases where the number on one side is illegible.
While clearly the bodysides of the vehicles where the numbers are displayed are likely to become dirty or potentially damaged because of the nature of long distance rail freight operations, we believe that the cost to the operator of maintaining, cleaning or repainting the relevant panels is minimal compared to the costs of alternative technologies.
Previous attempts in the North American freight sector to develop camera technology to identify wagons were hindered by the dirt caused by locomotive emissions, but advances in engine technology and regulatory restrictions mean that in Brazil today, the main challenges of obscured numbers are posed by inadequate wagon maintenance.
To validate the system, a proof of concept trial was conducted using more than 15 000 images extracted from videos of approximately 1 000 different wagons in trains operated by Rumo. These images were acquired under various weather conditions at four different locations in the state of Paraná between November 2019 and February 2020.
Through the detection and tracking of the wagon numbers in consecutive frames of those videos, UFPR-Connect Rail’s algorithm correctly counted the number of wagons that passed through the control points, regardless of the type of camera used, the condition of the wagon or the quality of the images. The research team from UFPR designed and applied heuristic rules to ensure that no wagon was counted more than once. Initially, the OCR algorithm correctly read 97∙3% of the numbers. Brazilian wagons are identified by an alphanumeric code containing exactly three letters and seven digits. The algorithm uses a rejection mechanism to check if the recorded number matches the expected format; here the system achieved a recognition rate of 99∙7% by rejecting those images where the wagon number was not in a visible condition or there was some other obstruction or artefact in the image
The tool then generated a mosaic that summarized the identification number of each wagon and its respective position in the formation of every train that passed through the control points. This could then be displayed as an easily understood visual ‘panorama’ of each train with every wagon clearly marked as a single block. The next step in the process sees this visual mosaic sent to a server or database in the cloud together with a .csv file containing the numbers of all the wagons identified; the two files added together are less than 1 MB. If there are any detection problems, which are highlighted by the tool’s automated alert system, an operator can compare the mosaic with the information contained in the .csv . is in turn provides operating staff with important information about the condition and formation of trains in real time, and can be used to highlight problems such as delayed consignments or illegible identification numbers.
Through the use of a high-performance Nvidia Titan XP processing unit, the tool proved capable of analyzing 16 frames per second. This means it could be deployed on embedded devices potentially located on site and still perform its functions in a matter of seconds or minutes.
Following this proof of concept trial, ConnectRail is keen to commercialize this and similar computer vision tools that offer significant potential for cost savings while meeting the stringent reliability and efficiency requirements of railway applications. The tool has subsequently been formally registered with Brazil’s National Institute of Industrial Property.