PDA

View Full Version : Failed Windows XP Upgrade Downs 60,000 UK Gov't PCs


KingFish
28 Nov 2004, 1:00am
Most of the desktop computers in the UK's Department for Work and Pensions were paralyzed for four days on Monday, when a failed upgrade took them offline. The outage, covering 75 percent to 80 percent of the DWP's 80,000 PCs, is one of the largest in the UK government's IT history.

And possibly one of the most costly. According to staff reports, the outage occurred on Monday afternoon, disconnecting staff e-mail, benefits processing, and Internet and intranet connectivity. According to one, a limited network upgrade from Windows 2000 to Windows XP was taking place, but instead of this taking place on only a small number of the target machines, all the clients connected to the network received a partial, but fatal, "upgrade."

Another source says that the DWP was trialing Windows XP on a small number ("about seven") of machines. "EDS was going to apply a patch to these. Unfortunately the request was made to apply it live and it was rolled out across the estate, which hit around 80 percent of the Win2K desktops. This patch caused the desktops to BSOD and made recovery rather tricky as they couldn't boot to pick any further patches or recalls. I gather that [Microsoft Corp.] consultants have been flown in from the U.S. to clear up the mess." EDS is also thought to be flying in fire brigades.
ouch :(

Source: eWeek (http://www.eweek.com/article2/0,1759,1732672,00.asp)

FAH_WW
28 Nov 2004, 1:44am
Serves them right for doing it on a live setup before testing... Stupid stupid stupid!

I betya somehow my tax money will be covering that extra cost... GRRRR

KingFish
28 Nov 2004, 2:29am
From what I gather it was an accident as the "upgrade" wasn't supposed to happen at all on the affected computers. It'll be quite a mess to clear up.

FAH_WW
29 Nov 2004, 4:28am
I'm actually surprised XP patches were able to run on W2K boxes - normally there are some checks built in.

So far I've been pretty lucky with upgrades - only one SP2 install went nutz. Took a re-install from scratch to get it fixed again. All the other rigs behaved themselves :)

primesuspect
29 Nov 2004, 4:44am
It sounds like they were going from Win2K --> WinXP ... What a disaster...

john (Guest)
2 Dec 2004, 9:48pm
Sounds like a PXE scripting on the loose to me!