Is Tesla Backtracking On Its Camera-Only Autopilot Strategy?

About a year ago, Tesla announced that it was dropping the use of radar for Autopilot and Full Self-Driving on the Model 3 and Y. The argument was that it could perform the same function with just cameras. But a few weeks ago, a Tesla Model Y was spotted in Canada with a LiDAR system integrated into its roof. Could there be a change coming to Tesla’s camera-only Tesla Vision plan?

One of Elon Musk’s many famous pet hates over the years (although not disliked as much as hydrogen) has been LiDAR, which he considers an unnecessary expense for autonomous driving. But when Tesla started removing radar as well to create the camera-only Tesla Vision, this raised a few eyebrows. The removal started in the US market in July 2021, and since April 2022, the Tesla Model 3 and Y cars sold in Europe have not included a front radar, either.

The theory is that human beings don’t have radar, they just use their eyes. With proper training, an autonomous system should be able to drive using cameras alone too. These can be used for object recognition and depth perception, although this requires computation. There could be advantages as well, such as the fact that amalgamating data that is all in the same domain (vision) is more efficient and simplified than trying to do this with lots of different sensor types. With the latter, heterogenous inputs need to be matched so that the system can interpret each object it “sees” correctly. Having lots of different types of sensors could even introduce conflicts and hesitation, which would be dangerous in a safety system.

However, the sighting of the LiDAR-equipped Model Y in Toronto has led some to ask whether the Musk disdain for this sensor technology might be waning. The advantage of LiDAR is that it captures accurate 3D depth measurements directly. Vision cameras, in contrast, only capture 2D information, so complex algorithms are required to generate a 3D depth measurement. Radar also captures distance directly, but with a lower resolution than LiDAR. In other words, it’s great for generally spotting if there’s a slowing or stationary object ahead of your vehicle in an adaptive cruise control system, but it won’t be so useful for a more active level of autonomy that might attempt to steer your car around that object. On the other hand, radar has a much longer range than LiDAR.

There has been no official word from Tesla what the car in Canada was or its purpose. However, unofficial commentators have argued that Tesla does in fact use LiDAR to train its vision cameras, and this is what the Toronto vehicle was doing. Another eagle-eyed observer saw a Model 3 with a similar LiDAR rig on its roof in Skokie, Illinois, in October 2021. The high 3D depth resolution of LiDAR makes it a good check when training other systems, but an automotive LiDAR sensor allegedly costs over $1,000. A radar sensor costs $50-100 in comparison, and an automotive camera just $10-20. You can see why Tesla would like to make do with cameras instead of stumping up for expensive LiDAR.

There’s still a lot of controversy over how safe any level of autonomy is, however, and how safe it could be in the future. The Toronto LiDAR sighting comes amid renewed concerns over Tesla safety after the NHTSA’s Office of Defects Investigation upgraded its research into Autopilot to an “engineering analysis”. This was instigated by 16 crashes involving Tesla vehicles with Autopilot engaged, which had resulted in 17 injuries and one death. The NHTSA has now reviewed more than 100 Tesla crashes involving a version of Autopilot during its investigation.

In the long run, autonomous systems have the potential to drastically improve vehicle safety. Over 46,000 people die every year in crashes on US roadways, or 124 per million inhabitants. In the UK, it was 1,516 deaths in 2020, or 23 per million inhabitants. In the EU-27 countries, it was 42 per million inhabitants, with Romania the most dangerous and Norway the least. Any technology that can reduce these numbers should be welcome, and any technology that can reduce the costs associated even more so, as this will leader to wider adoption. But they need to be very reliable to be worth using, so if Tesla is deploying LiDAR to improve its systems, that’s a good thing, and if the NHTSA investigation is anything to go by, it’s essential that Tesla does this too.