An AI Model Tested In The Ukraine War Is Helping Assess Damage From The Hawaii Wildfires

Using satellite imagery by Planet, a Microsoft AI tool compared the pictures before and after the fire and made maps to help organizations like the Red Cross assess the damages.

On August 7, 2023, the day before the Maui wildfires started in Hawaii, a constellation of earth-observing satellites took multiple pictures of the island at noon, local time. Everything was quiet, still. The next day, at the same, the same satellites captured images of fires consuming the island. Planet, a San Francisco-based company that owns the largest fleet of satellites taking pictures of the Earth daily, provided this raw imagery to Microsoft
MSFT
engineers, who used it to train an AI model designed to analyze the impact of disasters. Comparing before and after the fire photographs, the AI model created maps that highlighted the most devastated areas of the island.

With this information, the Red Cross rearranged its work on the field that same day to respond to the most urgent priorities first, helping evacuate thousands of people who’ve been affected by one of the deadliest fires in over a century. The Hawaii wildfires have already killed over a hundred people, a hundred more remain missing and at least 11,000 people have been displaced. The relief efforts are ongoing 10 days after the start of the fire, which burned over 3,200 acres. Hawaii Governor Josh Green estimated the recovery efforts could cost $6 billion.

Planet and Microsoft AI were able to pull and analyze the satellite imagery so quickly because they’d struggled to do so the last time they deployed their system: during the Ukraine war. The successful response in Maui is the result of a year and a half of building a new AI tool that corrected fundamental flaws in the previous system, which didn’t accurately recognize collapsed buildings in a background of concrete.

“When Ukraine happened, all the AI models failed miserably,” Juan Lavista, chief scientist at Microsoft AI, told Forbes.

The problem was that the company’s previous AI models were mainly trained with natural disasters in the U.S. and Africa. But devastation doesn’t look the same when it is caused by war and in an Eastern European city. “We learned that having one single model that would adapt to every single place on earth was likely impossible,” Lavista said.

Lavista’s team at Microsoft spent the last year and a half building a new AI tool that was easier to train with fewer bits of data, so instead of having one model for all disasters, researchers would quickly have one new tailor-made model for each event.

When the before and after images from the Hawaii wildfires arrived on the Microsoft computers, researchers quickly started manually labeling up to 200 instances in which buildings appeared in the before picture but not in the after one.

This helped the AI system learn to compare every pixel in each set of images to see whether the corresponding three square meters that each pixel represented had changed before and after the fire. For example, if a pixel in the photography of a forest changed from green to gray in a matter of hours, the three square meters that used to be trees have most likely been burned.

With this new system, Microsoft was able to train a new AI model with fresh data from Maui within a matter of hours. The model estimated that out of a total of 2,810 buildings in a specific area of Lahaina, one of the most impacted areas of the island, at least 1,722 buildings were damaged. The tool also quantified the percentage of damage: 1,205 buildings had suffered between 80% to 100% damage.

The data was initially shared with the Red Cross. Shortly after, Planet and Microsoft decided to make the information available to every humanitarian organization that reached out to them.

The AI imagery and maps have sped up the Red Cross’s preliminary damage assessment, helping the organization better understand the overall impact of the disaster, said Kasie Richards, senior director of Situational Awareness and Decision Support at the American Red Cross in a statement to Forbes. “The volume of information that can be synthesized would typically take us several days to process,” she said. “But this model has allowed us to operate in a more condensed period of time.”

One of the biggest advantages of satellite imagery is that it provides real-time updates. Besides the 200 satellites that take daily images of the Earth, Planet has a second constellation of 21 satellites that the company’s engineers can use to manually take closer pictures, at roughly 50 centimeters per pixel, to examine in more detail a damaged region. This has allowed them to monitor active fires such as those in the center of the island.

At the moment, the AI tool is handled by engineers in a central office, not by workers in the field. The next goal is for humanitarian workers to be able to interpret the maps and update them dynamically based on what they see on the ground, Andrew Zolli, the chief impact officer at Planet, told Forbes. “This is really the next exciting chapter, people on the ground collecting the information as they’re responding,” said Zolli.

Beyond aiding in initial disaster relief efforts, Zolli hopes the model will help public and private organizations in the longer-term reconstruction of Maui. “It’s helpful for resilience, rebuilding and reconstruction, insuring against the next disaster and reducing the risk of the next event.”