Toyota’s “insufficient disk space” blunder brings car manufacturing plants to a halt

Why it matters: Recurring security incidents have highlighted the poor state of Toyota’s IT operations. Even when an external threat is missing, the Japanese giant is working in self-sabotage mode with potentially significant consequences on its finances and daily operations.

Toyota suffered a recent “production order system malfunction” in its domestic manufacturing plants, an issue that forced the company to put everything offline for a few days. Unlike past incidents, however, this time there was no security attack or unwanted data exposure. Toyota was simply relying on very bad IT practices and was forced to sacrifice a good chunk of its growing domestic output as a consequence.

Apologizing to customers, suppliers and other interested parties, Toyota has now confirmed that the system malfunction happened because “some multiple servers” that process part orders suddenly became unavailable. Routine maintenance work was performed on those servers on August 27, a day before the malfunction event. Toyota says that while the maintenance procedure was deleting and organizing some data that had accumulated in the database, an “insufficient disk space” error occurred.

The system suddenly crashed, and the error was replicated in the “backup function” because both servers were running on the same system. The double error forced Toyota to halt production at all of its 14 manufacturing plants in Japan, the BBC reports. It was just a one-day disruption, but Toyota experienced an output loss of around 13,000 cars.

The part ordering system and the crashed databases were seemingly restored on August 29, after IT managers transferred the data to a server with larger storage capacities. Manufacturing operations were resumed the following day, and countermeasures are now in place to avoid any new issues with database operations. Toyota was eager to reassure the public that the system malfunction wasn’t caused by a cyberattack.

بخونید:  Neo Semiconductor 3D X-DRAM را راه اندازی کرد، اولین فناوری جهان برای تراشه های حافظه DRAM 3D

According to internal sources quoted by Reuters, Toyota’s database blunder will be financially expensive as domestic manufacturing output increased by 29% in the first half of the year. Toyota business was growing for the first time in two years, but the company was seemingly unable (or unwilling) to properly manage its database backups on secondary servers away from production environments.

Going forward, the Japanese carmaker is assuring a complete review of its “maintenance procedures” to prevent a database error like the one experienced this August. The company wants to deliver “as many vehicles as possible” to its customers, but it also should strengthen its IT and security practices a bit. In February 2022, Toyota’s Japanese factories were shut down because of a supply chain attack. Hackers compromised the company in 2021 through a US subsidiary, and they did the same at least three more times in 2019.