Skytrax’s Ratings Of Best Airlines Excludes U.S. Airlines, And Pricing

Skytrax is a service that collects consumer information and uses this to publish an annual list of the best airlines in the world. Their most recent ranking does not include any U.S. airlines in the top 10, and not surprisingly the top 10 includes all airlines that fly large wide-body airplanes and fly long distances. Rankings like this are common and it’s not sure what value they provide to consumers, since their results are quite different from decisions flyers actually face when choosing an airline. If you are flying from Washington to Boston or from London to Amsterdam, what good is it to know that Skytrax thinks Qatar Airways is the top airline in the world?

I’ve written before that surveys like this should consider value received, not just product integrity. It’s hard to get this from customer surveys, because price is often ignored when comparing product features. If I am ranking seat comfort or movie selection, for example, does it matter what I paid to sit in that seat to watch that movie? Once I am in the seat, the price isn’t relevant anymore. Understanding how these surveys are created and what they really show requires a bit more digging.

Skytrax Methodology

Skytrax dedicates a detailed page on their site to the methodology used for their survey. They survey millions of travelers around the world, and customers are asked to rank dozens of factors on a scale from one to five. The features ranked are in four categories: cabin service, onboard product, ground and airport service, and for this year the additional category of Covid-19 sensitivity. For each of the four categories, they list dozens of attributes for ranking, and some are quite detailed. For example, in the ground and airport category, there are at least four attributes related to baggage. Within the onboard product category, one parameter is the onboard magazine. It’s not clear from their methodology page whether “NA” is an acceptable answer.

No where in the methodology does it ask about the price paid for the flight or the overall value received. Interestingly, they do ask about the price of onboard food and beverages along with the quality of these. It’s not clear why they don’t ask this about the baggage, though they do ask to rank the baggage policies. If Skytrax is willing to ask customers their separate rankings for quality of meals, quantity of food, and selection of meals, it’s not a stretch to ask if the customer felt they got good value for their flight.

Why U.S. Airlines Don’t Make The Cut

Skytrax’s top 10 include eight Asian airlines, one European airline, and Qantas from Australia. The common feature of the top ten is that they all fly primarily long-distance routes. A Delta Airlines flight from Atlanta to Louisville would have different customer expectations from a Delta flight from Atlanta to Tokyo. If asking random customers from each airline, the likelihood of getting a long distance ranking from an airline that mostly flies these routes is increased greatly. Another way to say this is that if they only surveyed the long-distance flyers on Delta, American, or United, they likely would get different responses on the quality of food, onboard seat comfort, movie selection and the like. Incidentally, the first U.S. airline on the list is Delta, who shows up at number 30. JetBlue is close behind at number 32.

Flying Air France from Paris to Lyon is an experience not significantly different from flying a short flight in the U.S. Yet, the Skytrax surveys for Air France, the lone European airline in the top 10, include mostly flyers flying them to North America or Asia. This bias is not overstated. The worldwide fleet of Airbus aircraft shows over four narrow-body aircraft for every wide-body they have sold. The same is true for Boeing. With under 20% of all flights in the world being flown by wide-bodies, what does it say about a ranking that gives all the top spots to wide-body airlines? It’s like looking at only Michelin four and five star restaurants to choose the best place to eat dinner. This may be true, but it is far from the best value list as many would see higher value at lower price points. U.S. airlines fly a lot of short domestic routes, and comparing these to long-distance flights on other airlines is an apples to oranges view.

Watch What People Do, Not What They Say

People say all kinds of things, and then when they act they often do different things. How many people who claimed they would move to Canada if Donald Trump was elected President actually did that in 2016? We all know that many people say they want to lose weight, especially at the beginning of the year. Companies like Jenny Craig and Nutrisystem bank on this and advertise for this effect. While some people of course do lose weight when they choose to, most say the words but don’t follow it up with the actions. Most flyers say that they want wifi while in flight, but when there is even a nominal charge the take rates are miniscule.

Companies, including airlines, need to watch what people do, not what they say. Rankings of the most profitable airlines in world regularly show mostly narrow-body, short-distance focused airlines in the top 10. At least one prominent airline investor has stated that he’d never invest in an airline the he really enjoyed flying. Companies like Walmart are often criticized and ridiculed, yet they are regularly among the biggest retailers on the planet. Nordstrom is known for astonishing feats of customer service, like taking a tire in return when they don’t even sell tires. Yet Nordstrom earns but a small fraction of what Walmart earns, even when they make money. Surveys like Skytrax completely ignore this reality, and are comfortable publishing data that has almost no relation to how how consumers actually make purchasing decisions

Moving To A Value-Based Ranking

Value-based rankings would give consumers quality information on which to base future purchases. Is a certain car priced at $60,000 really twice as good as another car priced at $30,000? It may be three times better in which case it is a much better value. But comparing the two cars on features without considering their price gives consumers almost no information, and worse may actually result in misinformation. People fly for different reasons, and the biggest group of flyers are there for fun and leisure. What they value and what they will pay are different than what business travelers would choose, especially when those travelers aren’t directly paying the ticket price. That’s why a Skytrax survey that states categorically the best airlines in the world is not helpful to anyone except the winning airlines so they can use this in their marketing.