On Saturday around 3pm ET we saw a massive spike in traffic. One of our articles was featured prominently on Google News which resulted in more than a six-fold increase over our normal peak. We’ve been battling some stability issues with our server related to MySQL and this traffic spike brought the site to a standstill. We became aware of the outage around 7pm and quickly fixed it.
Around 2:30am that same night, the site again ground to a halt due to a similar problem. This time we were not able to remedy it and had to wait for it to sort itself out, which it did around 8am the following morning. Due to a coinciding power outage at our office, we were not able to restart the IRC server until well past noon (I eventually drove to a nearby cafe to fix other issues as well). We regained power at the office around 7pm but only recently got back online after replacing a hard drive in the domain controller that died in the outage.
Icrontic is growing very rapidly and our server’s previously-ignorable idiosyncrasies have turned into a problem that needs to be addressed. On top of that, we just had a rough weekend at Icrontic HQ. We’ll be investigating the stability issues tonight and tomorrow to both find a solution and reduce database load. Thanks for your patience while we work through these growing pains.
Articles RSS