Thermal Sensors for Vehicular and Pedestrian Safety

A recent AAA study that measured pedestrian detection systems found that existing automatic emergency braking (AEB) systems are ineffective at night and are often challenged during daylight hours too. As 20 automotive manufactures – representing 99% of the U.S. car market - look ahead to 2022 to meet their commitment of AEB as a standard feature, there remains significant performance gaps in making standard AEB systems effective in all types of driving conditions.

Where Prevailing AEB Systems Fail

Current AEB or pedestrian detection systems rely on the use of radar and, in some cases, visible-light (RGB) cameras. However, there are several common conditions in which these sensors can have difficulty detecting a pedestrian, in particular. The aforementioned AAA study tested several production AEB systems and describes many such scenarios. In nighttime conditions, the vehicle under test (VUT) traveling at 25 mph hit the soft pedestrian target (SPT) 100% of the time. During daylight tests, the VUT driving 20 mph struck the stationary SPT 60% of the time showing that even in optimal conditions these systems have an alarmingly high rate of failure and need to be improved. 

Thermal imaging on the Embarcadero. (Image: Flir)

Improving Pedestrian Detection with Thermal Sensors

One technology gaining momentum as an effective tool to improve AEB systems is thermal sensing, or the ability for the vehicle to detect heat energy. Enabling vehicles to detect heat (or infrared) energy vastly increases awareness when used in conjunction with existing sensing technologies on vehicles today, including radar and RGB cameras. More specifically, thermal sensors are particularly adept at detecting and classifying pedestrians – whom drivers least want to hit – in all types of low-visibility conditions.

Because thermal cameras “see” heat, they perform well in both daylight and in darkness. Furthermore, thermal sensing is a passive technology, meaning in cluttered or busy urban environments they won’t interfere with one another. Thermal sensors can see through sun and headlight glare, most fog, and up to four times farther than a typical production car’s headlight in darkness.

Given the thermal camera’s unique capability, automotive research group VSI Labs developed a proof-of-concept automatic pedestrian detection system that fuses radar and thermal camera data to detect and estimate the distance of a pedestrian in front of a test vehicle. The VUT is programmed to automatically stop if a pedestrian is in the path of the vehicle at a proximity that it determines to be an emergency-stop distance.

Testing the Approach

Initial testing of a thermal-and-radar-infused AEB system occurred in December 2019 at the American Center for Mobility (ACM) near Detroit, Michigan. The test design was based on the Euro NCAP standard for AEB, but not all testing requirements were met due to winter weather. The outside air temperature was colder than the specified testing range while the roadways had snowy, wet, or slick surfaces along with cold wind that interfered with test fixtures, making the test more arduous than required.

The program featured three test cases in both daylight and darkness, giving six datasets and 35 total test runs using an adult Euro NCAP Pedestrian Target (EPTa). The three test scenarios included:

1. EPTa stationary in the middle of the VUT’s lane in Car-to-Pedestrian Longitudinal Adult 50% (CPLA-50) tests. 
2. EPTa crossed in front of the VUT from the roadside in Car-to-Pedestrian Far Side Adult 50% (CPFA-50) tests.  
3. EPTa crossed in front of the VUT from an obstructed position in Car-to-Pedestrian Far Side Adult Obstructed 50% (CPFAO-50) tests. 

In all runs for all test cases, the AEB system successfully brought the VUT to a stop before reaching the EPTa.

Additional testing has since occurred in July at ACM, where weather permitted the VSI Labs team to complete the testing with full Euro NCAP standards. This time the test included an AEB system that merges radar and thermal sensor data with visible camera data. The more robust AEB system was also pitted against four existing AEB-enabled production vehicles. Initial results have further reinforced the effectiveness of adding thermal to prevailing AEB systems. Full results and analysis are expected this fall.

Reversing the Trend of Rising Pedestrian Deaths

Pedestrian detection and AEB systems are critical to improving road safety, but improvements are needed to reverse the troubling trend of rising pedestrian deaths. According to preliminary data from the Governors Highway Safety Association (GHSA), pedestrian deaths in 2019 were the highest in 30 years. GHSA predicts that there were 6,590 pedestrian fatalities in 2019, a 5% increase from 2018. These numbers highlight a trending safety issue the automotive industry must address now.

Although the automotive industry’s commitment to making AEB a standard feature is commendable, more work must be done to make these systems as effective as possible. Lives depend on it. 

Chris Posch is the director of Engineering for Automotive Products, FLIR Systems