Disaster strikes Norwegian government web portal

Joakim Larsen 21 Mar, 2012 at 12:50am ET Article published in Tech

Altinn is a web service run by the Norwegian government, on which citizens can find, fill out and deliver forms electronically. Every year Norwegian citizens can also log in to check their tax results. Since the birth of Altinn, the public has complained that the service is too slow, and every year the server has crashed due to high traffic from people wanting to check their taxes.

2012 has proven to be no different. The tax results were published at around 6:00 AM local time on Tuesday the 20th. By 9:00 AM over 200,000 people had tried to log on, and as a result the server crashed.

This was the status until noon, where traffic evened out and the server was stable again. Logging in is fairly simple: you type in your social security number and a personal password, and you receive a pin-code that you need to type in. At 6:17 PM local time, every single user who tried to log in went right past the login screen, and found themselves logged in as Kenneth, a 36 year old man from Oslo.

Users then had access to all financial data of this unfortunate person over two years back in time, in addition to the financial information of his wife and the company he worked for. Altinn shut down some 15 minutes later, and has been down since.

It is not known how many people got access to this information, or if any data were copied or downloaded. According to Jørgen Ferkinstad, communications director for Altinn, Kenneth had logged in and his information got stored in the server’s cache memory.

It is unknown how long Altinn will be down, and what is being done to prevent this from happening again. Kenneth had at 8:00 PM contacted his lawyer, and refused to give any statement. Brønnøysundregisteret, the company responsible for the web portal, were assembled for a crisis meeting at 11:00 PM. To make matters worse, DNV, a Norwegian company responsible for quality assessment and certification, published a report in the beginning of 2012, stating:

“Altinn is a rushed solution, testing has been lackluster at best, the service has very few options for future upgrades and the overall quality is considered to be below average. Furthermore, we question the competence and preparation of the publisher to manage such a complex system as Altinn.”

According to this report, there were no plans for backup, and the service was not built to handle the scale of the requests seen on Tuesday morning.

Comments

21 Mar 2012 ~ 7:07am LoopyChew What's the frequency, Kenneth?
21 Mar 2012 ~ 8:05am JBoogaloo Wow! I'd love to see how this all pans out.
21 Mar 2012 ~ 8:31am Cliff_Forster Nicely played @LoopyChew
21 Mar 2012 ~ 8:58am AlexDeGruven Wow... That's kind of crazy.

I'm surprised that they continue to try to maintain without appropriate capacity considering the complaints they've gotten right from the start.

Edit: The PragtasticKenneth pic is classic.
21 Mar 2012 ~ 9:06am Rudd-O GOOBERMENT!
21 Mar 2012 ~ 10:48am Vicar oh boy, what a load of bulls
21 Mar 2012 ~ 11:40am Ilriyas That has to be, by far the funniest thing I've heard all morning.
21 Mar 2012 ~ 12:40pm Jokke You know we got too much money when the development and launch cost one BILLION Norwegian Kroner ($173 681 978), paid by the end user, the taxpayers..... I feel sort of cheated..
21 Mar 2012 ~ 12:41pm UPSLynx Oh my word, that PIC.

Kenneth gon' be pissed.
21 Mar 2012 ~ 5:40pm primesuspect I played that Price is Right sound about seven times in a row and laughed every time.
21 Mar 2012 ~ 6:37pm UPSLynx I really hope the person playing the tuba in that bit is a fat old white guy with a big beard.
22 Mar 2012 ~ 4:03am wow Kenneth is going to be rich
22 Mar 2012 ~ 5:51am Pelotard You should note that in Norway, your income statement is a public record (at least as far as I know; I live in Sweden where this is definitely the case, and Norway has a very similar system). This means that anyone can walk into the tax office and demand to see any piece of paper, including anyone's tax returns, and they're not allowed to ask who you are or why you want to see it. The reason for this (which to most Anglo-Saxons seems like having a nude picture of yourself posted on your garage door) is that the taxman is regarded, in a very real sense, to be a servant of the people. That is, the people - every single citizen - employ and own the authorities, and is entitled to inspect their behaviour on a whim. "By and for the people" isn't just an empty phrase.
22 Mar 2012 ~ 12:44pm hojo Why does kenneth look like weev?
22 Mar 2012 ~ 2:40pm ot @wow No Kenneth is goiong to have a huge tax bill... he earned the most this year than in any other previous year.
23 Mar 2012 ~ 11:35am primesuspect Hey @jokke any updates on this?
23 Mar 2012 ~ 12:58pm Jokke @Pelotard: Yes, the tax records are (somewhat) public. You get to see a person's income before tax, how much was paid in taxes, and how much value (estates, cars, savings etc) that person has. HOWEVER, these data are all from the pre-correction phase. This mean that in cases where people document that they get xxxx amount of money returned, it will not show on these data. Also, the data Altinn managed to display on "Kenneth" were way more comprehensive than anything you will ever see from checking a persons tax record.

Update: It is estimated thet 1500-2000 people got access to "Kenneth"'s information during the 17 minutes before the site was shut down. It's determined that server caching caused the problem. Altinn was back up at 11am, local time on Friday. Caching has been disabled on this server which means it will perform even worse than before. Companies, the biggest usergroup of Altinn's services has got an extension on the deliver date for their taxes. At one point thursday evening, it was decided to print every tax statement and send them to the recipients by snail mail. This will cost 2 million NOK.
It is unknown whether this will have any repercussions on the people responsible.
26 Mar 2012 ~ 11:14am AnonymousPunter ...so the "server" caching Kenneth's data, was actually the F5 BigIP load balancer (. I'm no fan of Accenture, the distinguished vendor behind the unscalable Altinn-behemoth, but let's give "credit", where "credit" is due - this was a case of the BigIP-boxes performing "queue management" (because the back end system couldn't handle the load) failing under the pressure. Caching was never supposed to be enabled on these boxes - it "kicked in" under very high load. It was an unknown bug in the BigIP-software - off-the shelf components, used in lots of large installations. "Performance" (if you can apply such a word to Altinn) won't be affected by this caching being disabled.
9 Apr 2012 ~ 5:33pm Jokke The company responsible is now looking for 29 new employees in the ICT-sector, among them technical project managers, senior architects, and test managers. Something for currently unemployed/IC-members looking for new jobs?
9 Apr 2012 ~ 6:02pm Tushon Too bad I don't speak Norwegian and can't qualify as a senior architect or anything similar :P
9 Apr 2012 ~ 6:19pm Jokke Bah, most of the people working there are probably foreigners anyways. At least you speak English, which is more than can be said for most of them.