A Reality Check for Autonomous Cars: Safe at What Level?

Advanced driver-assist systems promise to provide significant safety benefits. But some of the initial applications have had the opposite effect, cautions a new study by the Insurance Institute for Highway Safety.

The report, titled Reality Check, identified several potential problems—ranging from irksome to dangerous—with the five vehicles IIHS assessed with Level 2 autonomous driving technologies.

The tests evaluated a BMW 5 Series equipped with Driving Assistant Plus, Mercedes-Benz E-Class with Drive Pilot, Tesla Model 3 and Model S with Autopilot, and Volvo S90 with Pilot Assist on a test track and public roads. They all were equipped with adaptive cruise control (ACC), lane-keeping assist and automatic emergency braking (AEB) systems.

In earlier tests, IIHS gave each of the vehicles’ a top rating of “Superior” for their AEB systems when coupled with front collision avoidance systems. The cars didn’t fare as well in the latest tests.

During track tests at 31 mph with the ACC turned off and the AEB on, both of the Tesla models failed to stop in time and hit stationary obstacles. With the ACC and AEB activated, all of the vehicles stopped in time—although the S90 was faulted for braking more abruptly than the others—to avoid the obstacles at close, medium and far following distances. The vehicles also stopped in time when the lead vehicle changed lanes to reveal a stationary object about four seconds in the path directly ahead.

The results were even more varied during the real-world tests on public roads. Except for the Tesla Model 3, each of the other vehicles failed to respond to stopped vehicles. In one case in which the E-Class was traveling at 55 mph with ACC and active lane-keeping engaged, the driver was forced to manually hit the brakes to avoid hitting a pickup truck stopped at a traffic light.

Testers also cited cases of “overly cautious braking.” During 180 miles of driving, the Model 3 unnecessarily braked 12 times, which IIHS says were caused by a mix of tree shadows, oncoming vehicles in adjacent lanes and vehicles crossing the road far ahead.

Evaluations for the lane-keeping systems focused on two conditions—curves and hills—both of which proved to be problematic. All five systems provide steering assistance that centers the vehicle within clearly marked lanes and use a lead vehicle as a guide when traveling at lower speeds or when the lane markers are blocked.

The two Tesla Models performed nearly flawlessly in the lane-keeping curve tests. But IIHS says the systems on the other three cars didn’t provide enough steering input on their own to consistently stay in their lane without driver input.

During the hill tests, IIHS notes the systems that rely on seeing road markings to keep vehicles in their lanes could become “flummoxed” as a vehicle crested a hill and lane markers down the road became obscured. While the E-Class and Model 3 performed well in these tests, the others struggled.

The 5 Series was the worst, failing to stay in its lane in all 14 hill test runs. The Model S only stayed in its lane in five of 18 runs, sometimes swerving back and forth and rarely alerted drivers to take control, according to IIHS. During slow-moving traffic with the lane marks obscured, some of the vehicles inadvertently continued to follow a lead vehicle into the exit lane when it was the car’s intended direction.

Based on these concerns and data from recent real-world accidents involving vehicles with Level 2 technologies engaged, IIHS is developing tests procedures and a ratings program for various ADAS technologies. The institute also plans on conducting further research to better evaluate new systems.

But the safety group doesn’t expect fully autonomous vehicles to be ready anytime soon. Concluding that none of the technologies is 100% reliable, IIHS urges drivers to remain attentive with their hands on the wheel at all times.