Upgrade Code Bug Caused Downtime

Sui Network released its first network outage analysis report which found that a bug in its upgrade code caused the downtime. The report also shared that the bug was fixed within 2.5 hours. The network outage took place on the morning of November 21, 2024, between approximately 1:15 and 3:45 am PT, which halted all transaction processing as validators were stuck in a crash loop.

Sui took to X to share the report, stating, “The rapid, collaborative response from Sui’s validator operators is an incredible testament to the strength of the Sui community.”

Earlier today, Sui experienced its first network outage due to a bug in congestion control code, which had been recently upgraded to allow for better shared object utilization.

Sui contributors quickly deployed a fix restoring normal network activity in 2.5 hours. The rapid,…

— Sui (@SuiNetwork) November 21, 2024

The network outage report claimed, “An assert! in congestion control code (described below) erroneously caused validators to crash if the estimated execution cost was zero”. 

Sui Network Outage Report Findings:

The network had recently upgraded their congestion control system to improve shared object utilization by more accurately estimating the complexity of a transaction. The code for the new mode, ‘TotalGasBudgetWithCap’ reportedly had a bug that caused this issue.

According to the report analysis, the issue with the code was identified, and the fix was implemented through PR #20365. Later, the update was deployed to Mainnet in version 1.37.4 and to Testnet in version 1.38.1. Following a swift and positive response from validators, the Sui network was restored within just 15 minutes of the fix’s release.

Taking lessons from the last outage which caused a dip of 5% in SUI’s market price, Sui stated that its efforts will focus on strengthening testing systems to simulate a broader range of adversarial transactions to prevent similar issues in the future. Additionally, improvements to build workflows will be made by Sui to fast track debug and release binaries, as a significant portion of the outage was due to delays in the release build process.

SUI is currently trading at $3.64 after recording a 1.35% surge in the past day. Its market cap also remains strong, standing at $10.37 billion, after the 24-hour market volume increased by 13.89% to $1.85 billion

Also Read: Solana Developers Silently Patch a Critical Security Bug