Tesla FSD 10.9 Review Update; Three Averted Crashes

Previously, I published a review of Tesla FSD 10.8 where I gave it an “F” from my perspective as a robocar industry consultant. The system, while called a “beta” isn’t yet at that level. Here’s a new video review of FSD 10.9, examining how it’s improved, but also discovering extremely major new flaws immediately on an alternate route. On a short trip to the grocery store, it made various minor mistakes and required 3 interventions to prevent collisions at the same intersection, something that shouldn’t happen in 50 years of driving for a self-driving car.

The new video shows you all of this, and discusses just how you grade a self-driving car, and what Tesla needs to do to compete. A large part of what it needs to do is follow the “Tesla Master Plan” from 2006, where Elon Musk outlined his strategy of starting with expensive niche cars, and later scaling up with more affordable cars. Musk has reversed that and is trying to develop self-driving with low-cost sensors.

While the video contains recorded examples of the issues on the review drives, if you prefer text, a transcript is provided below.

10.9 Review

In the earlier review, on a simple suburban loop I found it making mistakes, some serious, at every intersection and complex point on the route. It didn’t get a good score. Later, I got 10.9 and took it out on that loop, and on a new drive. So you’ll find me negative again. So negative that some people think I dislike Tesla. Actually, not only do I own a Tesla and some Tesla stock, I want all robocar teams to deliver this technology as soon as they can. I am critical when I don’t think Tesla is on the best path to success and wide deployment.

First, the good news. It did better on a few of the areas it had problems with 10.8.

The most changed was the right turn with a dedicated lane. Previously, on a red light it would just stop and sit there, not seeing it had its own lane. Now it sees the bollards and figures that out. You may have seen another recent video of 10.9 where the car actually hit a large bollard.

In a left turn near Vallco, 2 times out of 10 the earlier version moved into the left turn lane for no reason and then turned left, once right through the red left arrow. Both my trips through here it briefly tried to enter the left turn lane again, but changed its mind and went straight. This may or may not be an improvement or just the disturbing randomness of the system.

With the problematic left turn lanes entering Apple HQ it seemed to do better. On 10.8 it would usually wait until the last moment to figure out it had to get right and do so, but a couple of times it didn’t figure it out and got into a no-win position. In one case it went through the red arrow to turn left until I intervened. On these two passes it seemed to identify that it should get over a bit sooner. Of course, if it had a map, as I outline in my video on mapping seen above, it would just have been driving in the right lane to start, so it would not need to make the last minute lane change, which is trouble when there is traffic.

The result was worse on a new drive, a simple trip to a Safeway.

At the first right turn, it blocked the crosswalk and just never had the confidence to make use of long gaps to make the turn. I nudged it.

Not long after it stopped hard for no reason well ahead of the light.

First Intervention

Next was the scary part, where there were 3 times I had to intervene to prevent a collision. 3 times at the same intersection on a short round trip. The first has an “excuse” of sorts, but begins with a moderately complex turn onto El Camino Real, the main road of Silicon Valley. The right thing to do is to get into the 2nd lane from the left, which is marked on the road as the lane to go across the first street in the intersection. The Tesla enters it, but once it starts moving, it immediately veers right into the wrong lane, then crosses Fremont and quickly crosses two lanes to make the left.

As it is doing so the light turns yellow, so it stops — badly, with the car past the crosswalk and into the intersection. I manually back the car up. The guy behind me backs up to give me room.

Now I am stopped at the red light. Having intervened, I must release the brake and re-activate FSD. The result is terror. The car lurches forward, ignoring the red light and his heading into cross traffic. I react quickly to intervene.

After that, I waited until the light turns green before I re-activate and it makes the turn OK.

It is not clear what happened. Did activating FSD when stopped at a light somehow signal the car to go, in spite of both a red light and speeding cross traffic in front of it? There’s no way that’s right.

Buying a Tesla in the modern era

The drive passes the Tesla store and service center. It is interesting to note that I didn’t buy my Tesla this store in walking distance of my house — I was in Korea at the time, when the new model went on sale and I ordered it online. On delivery day, a driver brought it to my home right from the factory, and it arrived just as I was watching a SpaceX Falcon 9 launch go up right from my driveway. Yes, you can see them regularly from Silicon Valley and I watch most of them. We made the delivery driver wait a few minutes. It was a very Elon Musk day. The driver returns to the factory in an Uber — a very different and very nice car buying experience.

At the store, it enters a left turn lane, and decides to go when the lanes are clear, but then stops, in the middle of the road because a right turner (who had his signal on) is going to use the same mall entrance. Fortunately there is no traffic so I let it resolve. Once in the parking lot it gets erratic — as I might expect — so I disengage and park.

Second Intervention

After shopping, it’s darker and time to go home. The trip on El Camino is driven well, and we get back to the intersection of the previous prevented collision. Now we’ll get 2 collisions in short order. It moves to the right turn lane, which has many cars in it but it drives like there is nobody, zooming at high speed to the rear car, even though it sees it, and I intervene.

Third Intervention

Around the corner is a challenge. There is a short stretch to another intersection, and after that intersection, the 3 lanes suddenly merge to 2. Locals know this is always a tough merge, but the Tesla doesn’thave a map and doesn’t see the signs. When the 2nd light turns green with lots of traffic, it can’t see that the lane vanishes ahead. The map it builds is wrong. I thinks the lane continues. Suddenly the car in front merges left revealing that the lane ends immediately. The Tesla is still speeding, I have to hit the brakes as there is nowhere to go.

3 averted crashes in just a few miles of driving, plus several other mistakes. While the update has improvements, as we should hope it should, it’s still in extremely poor condition.

How to review self driving

There were several who disagreed with my prior review, where I gave FSD an “F.” In truth, it’s not even a close call, and it continues to get that grade. Any of these actions would result in immediate interruption of a driving test even for a teen-ager on a learner’s permit. To get a “C” you need to drive 25 years without a mistake like these. To get a B would require driving a whole human lifetime, because that’s the record of a typical human, around one police-reported accident per lifetime. An A is reserved for being much better than a human. Tesla FSD still has a long way to go before it can even think about getting a “D.” Make major mistakes right out of the driveway and you’re a long, long way.

Is it a “Beta?”

Tesla calls this a beta, but it’s no beta yet. It’s a prototype, and Tesla is still doing major rewrites on it and plans more, which immediately disqualifies it from beta status. But we grade it on what it’s a prototype of, namely a full self driving system. If Tesla wants to use that name, it asks to be judged by it.

It’s a shame because it could be much better. The mistake at the merge, and several others, would be easily averted with maps, which, as explained in my video on Tesla and maps (see the description) are cheap, scalable and robust against changes in the road. It’s much more scalable to work in just one city and grow than to drive very badly in every city. The cause of the other mistakes is unclear, but one thing that is true is no car with good software and a LIDAR would make them.

As I said before, I want robocars out on the road as soon as we can make them safe. I wish Tesla were following a better path to that. Instead, it has made a longshot bet on hoping for a breakthrough in computer vision whose date of arrival nobody can name. It is trying to take the cheap, longshot bet rather than the effort that starts more expensive but gets cheaper later.

Tesla should follow the “Tesla Secret Master Plan”

That’s very odd. In 2006, Elon Musk laid out what he called the “Tesla Master Plan.” He described how he would start with a very expensive, specialty sportscar, then make an expensive luxury sedan and build the brand and get the experience. Then he would release an affordable car — the model 3 — and eventually an economy car. Start expensive first, then do cheap later. He carried out that plan to make the best cars and trounce the old-world car industry.

For unknown reasons, when it comes to self-driving, he has reversed that strategy. In 2016 he declared that they would try to make self-driving work with the sensors of 2016, mainly low-cost cameras and a radar he later abandoned. The route that starts expensive and gets cheaper later — LIDAR and maps — was not for Tesla. I want them to go back to the master plan, expensive first then scale later, and use their other advantages to be a real player in this space, and give the other competitors — the software companies, not the car OEMS — a run for their money.

Maybe Tesla will get the breakthrough it is depending on and show the world. It’s not impossible. But it’s probably not the way to bet.

Read/leave comments on this article