SecurityFocus.com Forum Poster Believes MSBlaster may have caused Blackout.....
Omega65
Philadelphia, Pa
SecurityFocus.com Forum Poster Believes MSBlaster may have caused Blackout....
<i>I believe that the outage was caused by the MSblaster, or its
mutation, which was besieged upon the respective vulnerability
in certain control and monitoring systems (SCADA and otherwise)
running MS 2000 or XP, located different points along the Grid.
Some of these systems are accessible via the Internet, while
others are accessible by POTS dialup, or private Frame relay and
dedicated connectivity.
Being an old PLC automation and control hack let me say that
there is a very good plausibility that the recent East Coast
power outage was due to an attack by an MBlaster variant on the
SCADA system at the power plant master terminal, or more likely
at several of the remote terminal units "RTU". SCADA runs under
Win2000 / XP and the telemetry to the RTU is accessible via the
Internet......</i>
<i>I believe that the outage was caused by the MSblaster, or its
mutation, which was besieged upon the respective vulnerability
in certain control and monitoring systems (SCADA and otherwise)
running MS 2000 or XP, located different points along the Grid.
Some of these systems are accessible via the Internet, while
others are accessible by POTS dialup, or private Frame relay and
dedicated connectivity.
Being an old PLC automation and control hack let me say that
there is a very good plausibility that the recent East Coast
power outage was due to an attack by an MBlaster variant on the
SCADA system at the power plant master terminal, or more likely
at several of the remote terminal units "RTU". SCADA runs under
Win2000 / XP and the telemetry to the RTU is accessible via the
Internet......</i>
0
Comments
Now, it takes approximately 11,000mW EACH from four powerplants in Michigan to supply the state. 44,000mW is a LOT of power...There are only a couple times this amount of powersupplies for the section of the U.S. that powers the great-lakes states. And this is not an entirely isolated sector. What were to happen if the GLS (Great Lake States) sector relied upon its neighboring New England sector to provide additional power and this sector suddenly lost 1/5th of its power. New England is more dense than the GLS sector...1/5th of its power is quite a lot. Now, say that power is gone, the New England plants suddenly shoulder an additional burden they weren't designed to bear, and they shut down. Then the GLS plants, which rely on the New England plants and the NE plants on GLS, shut down because New England is toast and the whole of the New England system is suddenly shouldered by the GLS systems...A cataclysmic cascade effect occurs simply because the system is too outmoded to isolate itself in the event that an entire sector fails. It's the domino theory, and the system is old enough to support that theory.
So to recap:
-Niagra goes down
-Other New England plants shoulder a burden they can't take
-New England goes down
-Great Lake States plants are connected to New England
-Great Lake States shoulder the new burden from New England
-Great Lake States plants can't shoulder it
-Great Lake States plants go down
-To the south and the west are several newer plants (Nuclear), and isolation stations in effect, they stay up because the technology is 20 years newer.
People who uselessly extrapolate based on events of similar times, but unrelated pretense annoy me.
People who believe canned news stories with simple explanations for a complex problem annoy me....
It applies quite well. The logic I have used is quite sound.
One lightning strike can incapacitate when 999 more can't. As in regards to a forest fire...A thousand, ten thousand strikes a year in forests, and only one or two cause devastating fires. Is it so unlikely that a similar ratio, predicated upon similar logic applies here? The problem is not complex, nor is the solution in terms of need.
One news story warrants verification, two warrants marginal credibility, twenty three warrants belief.
Save me the modification of my own words for the purpose of a sarcastic bite. It has a much greater effect when you don't steal someone else's statements to produce a mild insult.
A lightning strike in Ohio dosen't knock out power in Canada, NY, PA, Deleware and Maryland and cause Nuclear Power plants to shut down. You need a wide ranging effect to affect that many places and people.
Personally I suspect a Solar event or an Earth Magnetic Field shift - Not Lightning.
Feel free to be sarcastic - it's well known that no one elses opinion or thoughts matters as much as yours does..
If one strike of lightning manages to take out a plant just ONE time, power needs to be shuffled to other plants damn fast. But it can't shuffle it like that, the system is too old, power grids are still coming out of the isolated privitization of the 60s and 70s when the systems were created.
The NRC and other power regulatory commities didn't mandate free interlinking of power until the 80s, when the infrastructure was almost entirely mature. New interlinks would have to be created at the cost of billions. Lyndon B. Johnson mandated the creation of these interlinks in the outage of 69, but we're only beginning to see the amount of capital produced necessary for power companies to develop the techniques for deprivitization.
So what happens when a powerplant fails, and several others are dependent? A crucial card from the house has been pulled and one whole floor of the card house falls over. The floors of the card house below the collapsed must now bear the burden, and the sudden, unexpected force of collapse...Those go down too. And it keeps going until it hits the very wide and stable base...The south and the west, where power plants and infrastructure are 50% newer than the great lakes and new england. It's the direct result of deprivitization inherent to new wiring projects, new powerplants...It has enough interconnects and substations that it knows how to bear the load, store it, and redistribute it with little interrupt in customer service.
When they build it new, they adhered to Johnson's plans and the plans of major power companies to protect their cash flow. Why would they not build it with these failsafes? It means their customers are happy and paying.
But what reason does any company with an existing grid have to switch? There's been no failure in almost 35 years, it's cost prohibitive to a serious degree...Why change?
So they leave it. And the cripling dependencies remain.
I'll ignore the last comment.
How hard is it to see that existing systems are overworked, and the failure stopped at the new systems that were designed to take a higher-than-existing demand?
CNN: Blackout 2003 (by the numbers)
CNN: According to NERC 3 transmission lines went down in Ohio.
If the Power loss started in Ohio, Why did NY lose power( NY is connected to Ohio THROUGH PA)? And why did 21 PowerPlants in other areas shut down instead just supplying their local areas? Or is NY state & NYC power largely supplied by Ohio and Canada (I doubt it)? The fact that 21 Plants shut down because because of (possibly) a lightning strike taking out 3 transmission lines or a whole Power Plant is on the face of it Ludicrous.
Several years ago Power Suppliers in Cali colluded to create a phony Electrical crisis ( which led to a blackout ) to drive up prices. Several major Power Plants were shut down for maintence 2months ahead on time (In Aug BTW) to create a low power situation which led to a bloackout and "emergency" Power plant shutdowns. It's happening all over again. Fool me once, fool me twice, fool me thrice....
*
In three minutes, 21 power plants shut down, including 10 nuclear plants (acted fast didn't they - why didn't they cut power to the affected areas and supply their local customers...PA did.)*9,300 square miles covering the U.S. and Canada were without power
*PENNSYLVANIA:
No major problems reported after 100,000 customers, mostly in northwestern counties, initially lost power. Most recovered electricity shortly after nightfall Thursday, (By the Ohio border - the cutoffs worked here & PA systems are 30+ years old)
*VERMONT:
A quick shutdown of transmission lines from New York averted major outages in Vermont. A small area near the Canadian border lost power briefly, but it affected only a few thousand customers. (worked here too)
*NEW JERSEY:
Electricity was restored to nearly all affected customers, with fewer than 5,000 customers still without power, from the peak of 1 million homes and businesses. Commuter railroads and buses had limited to full service.
I can hardly believe that the power system can be that vulnerable. There has to be another explanation to this.
We get you Thrax, we've heard what you have said and we understand what you have said, some of us may just not agree with you.
Everyone is entitled to their opinions, but try and be a little more, how shall I put it... less confrontational. We know what you think caused it, you might be right, you may not be right.
Polite debate is fine, but I don't want this to be the first News thread to have to be closed due to people being too forcefull with their opinions.
Sit back, relax, have a drink mate. We're all friends here, lets try and show it every now and again shall we.
SPINNER
I tend to take what he says as the truth.
CBS.com: California Energy Crisis A Sham
"(CBS) State officials say California's energy crisis -- its blackouts and sky-high power prices that cost California billions -- was manufactured by key power companies that hoarded energy supplies to make more money, reports CBS News Correspondent Vince Gonzales.
Overall, the companies kept more than 30 percent to 50 percent of their power off the market. During some of the worst moments of the crisis, they held back even more -- anywhere from 55 to 76 percent of production -- all in an effort, whistleblowers told CBS News more than a year ago, to cut the power supply and drive up prices."
NY Times: California Power Failures Linked to Energy Companies
"NATIONAL DESK | September 18, 2002, Wednesday
California Power Failures Linked to Energy Companies
By JOHN M. BRODER (NYT) 885 words
Late Edition - Final , Section A , Page 22 , Column 5
ABSTRACT - California's Public Utilities Commisson reports that widespread power failures during state's energy crisis of 2000 and 2001 could have been avoided if five independent energy companies had not withheld electricity they were capable of producing; does not directly accuse companies--Duke, Dynegy, Mirant, Reliant and AES/Williams--of deliberately trying to drive up prices, but says investigations are continuing into possible price manipulation and collusion among companies; photo (M) "
09/18/02: Another report, more evidence of collusion
"Wholesale prices for power leaped to as high as $1,500 a megawatt-hour from $40 per megawatt-hour, causing electricity bills to jump, forcing two major utilities into bankruptcy and plunging the state into debt as it spent billions of dollars buying electricity on the deregulated market. The state entered into long-term energy contracts at the inflated prices of 2000-2001, one factor that has led to the state's $24 billion budget deficit.
The utility commission found that on each of the days when power was interrupted all of the five main electricity generators had capacity that went unused. For example, on May 8, 2001, there were two-hour blackouts caused by a shortage of 400 megawatts of power in Northern and Southern California. But Duke Energy had about 1,000 megawatts of available capacity that was not used that day, the commission report said. "Thus, Duke alone had more available and unused power than the total amount of power that was needed to avoid the blackout that day," it said. "
[url=]California Power Failures Linked to Energy Companies [/url]
By John M. Broder
New York Times
" LOS ANGELES, Sept. 17 -- Widespread power failures during California's energy crisis of 2000 and 2001 could have been avoided if five independent energy companies had not withheld electricity they were capable of producing, a study by state regulators said today.
The investigation by the California Public Utilities Commission said the five companies -- Duke, Dynegy, Mirant, Reliant and AES/Williams -- had withheld power from their California plants.
This contributed to the "unconscionable, unjust and unreasonable electricity price spike that California experienced during the energy crisis," the report said.
The commission did not directly accuse the companies of deliberately trying to drive prices up. Officials said investigations were continuing into possible price manipulation and collusion among the companies. "
============================
Now that all of a sudden Politicians are calling for Modernzation (I thought deregulation was supposed to handle that..) Guess who's going to pay for it....
Carry on...
But:
- it hasn't happened before
- you'd think they'd have measures to prevent it
- they reported even Thursday night that the National Weather Service said there was no lightning near Niagra
I'm not saying it's impossible but unless you have info that we don't, I wouldn't call anything Teh Cause.
I'll get to the specifics in a moment (long-ass post and I'm dead tired after being out on the Trouble Truck for the last 72 hours).
A detailed first-hand look of events on Thursday, through my eyes at the time, can be found over @ IcronticForums.com I suggest you read it for some indepth information that will pertain to this article.
//Note: I should mention that I work for Hydro One Networks Incorporated, the company responsible for the transmission & distribution of electricity to Ontario's 12 million customers. Because of this position, I've had access to information that is public knowledge, but the media chose not to pass on to you.
==============================================
What began as a handful of commonplace, summertime "trips" -- brief transmission line shutdowns, usually due to ebbing voltage caused by anything from a bird hitting the lines to a power overload -- set off the biggest outage in U.S. history. Fifty million people lost power in eight states and parts of Ontario.
Though some problems began earlier, the actual blackout did not begin until shortly after 4 p.m. EDT. Some key moments:
-- 2 p.m. FirstEnergy Corp.'s Eastlake Unit 5, a 680-megawatt coal generation plant in Eastlake, Ohio, trips off. On a hot summer afternoon, "that wasn't a unique event in and of itself," says Ralph DiNicola, spokesman for Akron, Ohio-based FirstEnergy. "We had some transmission lines out of service and the Eastlake system tripped out of service, but we didn't have any outages related to those events."
-- 3:06 p.m. FirstEnergy's Chamberlin-Harding power transmission line, a 345-kilovolt power line in northeastern Ohio, trips. The company hasn't reported a cause, but the outage put extra strain on FirstEnergy's Hanna-Juniper line, the next to go dark.
-- 3:32 p.m. Extra power coursing through FirstEnergy's Hanna-Juniper 345-kilovolt line heats the wires, causing them to sag into a tree and trip.
-- 3:41 p.m. An overload on First Energy's Star-South Canton 345-kilovolt line trips a breaker at the Star switching station, where FirstEnergy's grid interconnects with a neighboring grid owned by the American Electric Power Co. AEP's Star station is also in northeastern Ohio.
-- 3:46 p.m. AEP's 345-kilovolt Tidd-Canton Control transmission line also trips where it interconnects with FirstEnergy's grid, at AEP's connection station in Canton, Ohio.
-- 4:06 p.m. FirstEnergy's Sammis-Star 345-kilovolt line, also in northeast Ohio, trips, then reconnects.
-- 4:08 p.m. Utilities in Canada and the eastern United States see wild power swings. "It was a hopscotch event, not a big cascading domino effect," says Sean O'Leary, chief executive of Genscape, a company that monitors electric transmissions.
-- 4:09 p.m. The already lowered voltage coursing to customers of Cleveland Public Power, inside the city of Cleveland, plummets to zero. "It was like taking a light switch and turning it off," says Jim Majer, commissioner of Cleveland Public Power. "It was like a heart attack. It went straight down from 300 megawatts to zero."
-- 4:10 p.m. The Campbell No. 3 coal-fired power plant near Grand Haven, Michigan trips off.
-- 4:10 p.m. A 345-kilovolt line known as Hampton-Thetford, in Michigan's thumb region, trips.
-- 4:10 p.m. A 345-kilovolt line known as Oneida-Majestic, in southeast Michigan, trips.
-- 4:11 p.m. Orion Avon Lake Unit 9, a coal-fired power plant in Avon Lake, Ohio, trips.
-- 4:11 p.m. A transmission line running along the Lake Erie shore to the Davis-Besse nuclear plant near Toledo, Ohio, trips.
-- 4:11 p.m. A transmission line in northwest Ohio connecting Midway, Lemoyne and Foster substations trips.
-- 4:11 p.m. The Perry Unit 1 nuclear reactor in Perry, Ohio, shuts down automatically after losing power.
-- 4:11 p.m. The FitzPatrick nuclear reactor in Oswego, New York, shuts down automatically after losing power.
-- 4:12 p.m. Units 5-8 of the Bruce Nuclear station in Douglas Point, Ontario, Canada, shut down automatically after losing power. Units 1-4 have been offline since 1997 for refurbishment.
-- 4:12 p.m. Rochester Gas & Electric's Ginna nuclear plant near Rochester, New York, shuts down automatically after losing power.
-- 4:12 p.m. Nine Mile Point nuclear reactor near Oswego, New York, shuts down automatically after losing power.
-- 4:13 p.m. Units 5-8 of the Pickering Nuclear station in Pickering, Ontario, Canada, shut down automatically after losing power. Units 1-4 have been offline since 1997 for refurbishment.
-- 4:13 p.m. Units 1-4 of the Darlington Nuclear station in Bowmanville, Ontario, Canada, shut down automatically after losing power. I live maybe 3 minutes from this location.
-- 4:14 p.m. Units 1-8 of the Nanticoke Coal-Fired station in Haldimand County, Ontario, Canada, shut down automatically after losing power.
-- 4:14 p.m. Units 1 & 2 of the Beck I Hydroelectric station in Niagara Falls, Ontario, Canada, shut down automatically after losing power.
-- 4:14 p.m. Units 1 & 2 of the Beck II Hydroelectric station in Niagara Falls, Ontario, Canada, shut down automatically after losing power.
-- 4:15 p.m. FirstEnergy's Sammis-Star 345-kilovolt line, in northeast Ohio, trips and reconnects a second time.
-- 4:16 p.m. Units 1-4 of the Lennox Gas & Oil-Fired station in Napanee, Ontario, Canada, shut down automatically after losing power.
-- 4:16 p.m. Units 1-4 of the Lakeview Coal-Fired station in Toronto, Ontario, Canada, shut down automatically after losing power.
-- 4:16 p.m. Units 1-4 of the Lambton Coal-Fired station in Sarnia, Ontario, Canada, shut down automatically after losing power.
-- 4:16 p.m. Oyster Creek nuclear plant in Forked River, New Jersey, shuts down automatically because of power fluctuations on the grid.
-- 4:17 p.m. The Enrico Fermi Nuclear plant near Detroit shuts down automatically after losing power.
-- 4:17-4:21 p.m. Numerous power transmission lines in Michigan trip.
-- 4:25 p.m. Indian Point nuclear power plants 2 and 3 in Buchanan, New York, shut down automatically after losing power.
==============================================
The 3 major Nuclear Generating Stations (NGS) in Ontario were forced to immediately shut-down generation activities due to not having an energized grid to shunt their power out. A generating station can succumb to considerable damage if there is nowhere for the generator to shunt its electricity. Therefore, for safety reasons (and to prevent damage to both the entire Transmission & Distribution System and the generation stations themselves), they decided to perform emergency shutdown procedures to stop generating electricity.
Why was the grid shut down? All of the trip-outs (as CNN likes to call them) are caused when a high-voltage transmission line (115 KV to 500 KV) intersect with a grounding object, such as a tree, a car or an idiot carrying a ladder and tapping a power line (don't laugh, it happens more often than you think). Each of these transmission lines carry 3 "phases" of power; red, white & blue phase. Each of these phases are identical. There is no difference between any of the phases, it is simply how they are color coded for easy identification of which areas of the grid they feed.
Anyways, with all the trip-outs happening, generating stations were not able to shunt their generated electricity anywhere, resulting in a steady "Demand" for electricity, but a declining "Supply" of electricity.
Think of it as a see-saw. Demand has to match Supply. They MUST balance. When all of a sudden the Transmission lines trip-out, the generators can't send their power anywhere. All of a sudden, the "Supply" side of the see-saw flies up high. In order to preserve the Transmission & Distribution Network from being completely overloaded and damaging sensitive transformer equipment (some of which did get damaged during this little incident), the Demand side of the see-saw must balance out to the Supply side. HENCE, certain parts of the grid are turned off to balance out Supply & Demand.
As the timeline above shows, as the "Supply" got smaller and smaller with the different power stations kicking out of the grid, the "Demand" side had to equally shed usage. To attempt to balance the network, the demand (customers & businesses) had to be severed from the grid. Once all the power stations went offline, there was 0 supply. Hence, to balance, the grid had to have 0 demand.... and there's why the entire system went offline. Balance. Blame the scales.
Power-wise, Ontario is doing a little better today. We've got all of our Thermal (Oil, Gas & Coal) plants running at 105% of nominal capacity, generating a total of 9,500 MW (megawatts, or millions of watts, or MW) and all of our HydroElectric (Water) plants running at 110% of nominal capacity, generating a total of 8,000 MW. Even with 17,500 MW, Ontario is importing power to keep the province energized. On a weekend, Ontario consumes between 19,000 MW and 21,000 MW. During a typical weekday, Ontario consumes between 20,000 & 23,000 MW. On an insanely hot weekday where everyone has their AC units cranked, Ontario consumes between 22,000 MW & 26,000 MW.
Our power interconnects with Quebec & Manitoba are helping to stabilize the power grid while we attempt to SAFELY re-start our Nuclear Generating Stations (NGS).
You can't just simply hit a button and electricity is generated out of an NGS. During emergency shutdown procedures, the CANDU reactor (Canadian Deuterium Uranium) is "poisoned" by both biproducts of the reaction (when terminated) and by a certain chemical injected into the Heavy Water moderator (H3O) that controls & cools the reaction. This moderator must be completely (and I mean meticulusly removed) and re-filled before the nuclear reaction can be re-started.
It will be at least another 36 hours before the main NGS, Darlington, has all 4 of their units back online, running at 25%. As for Pickering & Bruce, they will be well into the week before we get all 8 units up and running at 25% (which is considered the absolute minimum for energizing the grid out of each area). It will take up to 2 weeks for these reactors to get running back at full tilt once again (95% of nominal). These 3 NGS's are responsible for the generation of almost 11,000 MW of Ontario's electricity and their current offline situation prevents Ontario from returning back to normal.
It's been a long hard 72 hours, working and living out the back seat of a trouble truck (those cherry picker power utility trucks you see working on poles and such). The team I work for out of the Hydro One Peterborough, Ontario office started a normal shift on Thursday, August 14, 2003 and have been around-the-clock since then. The team was assigned to replace any of the faulty transformers throughout Eastern Ontario (a district running from Petawawa to Peterborough to Bowmanville to Ottawa for you Geographic types) that were damaged during the grid failure on Thursday. I think we've changed a total of 2,300 transformers at the main Transmission & Distrubtion transformer stations (see attached graphic for details) with a crew of 150 lines guys.
As of 5:00 PM Eastern Standard Time on August 17, 2003, Hydro One had re-energized 98% of the power grid throughout Ontario and were ready to restore the other 2% (mainly around Toronto) this evening.
I'm just happy I get to come home and sleep now
So basically several Transmission line Trips in Ohio lead to Grid instability & Power Plant shutdowns in Ohio, Michigan, Canada, NY and NJ (PA cuts out and stays up).
I can understand Ohio losing Power, it's sad that the system is (supposedly) so antiquated that Michigan, Canada and NY couldn't save themselves in time.
That's exactly what I mean. With no-where to send power, they have to shut down the power plant or else risk seriously damaging the generation equipment inside.
You Rule.
Thank you for keeping us so well informed. I feel as if we have knowledge here that the mass media doesn't, and it's because of you. Thank you.
That really shouldn't be possible. As I saw in another thread or at addaboy, they don't run Windows, they run Linux, or if they do run Windows, they are heavily isolated and under a firewall.
Mission critical systems at any of the generating stations and transmission/distribution centres throughout all of Ontario DO NOT run Windows. They are all SCO/Unix based systems that are NOT accessible across the internet.
I am not permitted to reveal how telemetry is monitored from off-site locations, but I can tell you this. It's NOT across public networks (ie the IntarWeb).
The IT systems of NY, NJ, OH, VA and CT... that's another story that I don't have information available on.
Not a problem at all Prime. Hope things are better down in your neck of the woods, as I heard that Detroit had some "problems" this weekend with the water situation and rampant looting.
The Darlington area (where I live) was one of the last areas in Ontario to be re-energized (I did it myself). We didn't get power back until nearly 6:30 PM Eastern Standard Time today, after having it go out at 4:13 PM EST on Thursday.
I came home to a nice fridge/freezer & deepfreeze of rotting milk, cheese, red meats and a whole host of stinky sh*t. Oh well, didn't need that stuff anyways.
I was about to take the sandals I had just bought and mercilessly beat them down. It's kinda sad people take so much advantage of something like electricity and then bitch about it when it goes offline for a few hours. :banghead:
When you read posts like the detailed events SimGuy posted, you realize how deep this stuff is. I bet most people here don't even know what the "little grey garbage can" is on their poles in front of their houses.
Yeah, uh, I meant people in my city.. yeah, uh.. not Short-Media..
...
****.
/me also knows what the "little grey garbage cans" are.
Thanks for the inside knowledge SimGuy!
HAHAH Best quote.. EVAR
Those "little grey garbage cans" have the potential to kill you/severely injure you and your neighbours if your not careful. Don't forget those "little green dumpsters" that sit on the front-lawns of people living in subdivisions. They are the same as those "little grey garbage cans."
One day out in the field, we were changing one of those little grey garbage cans when we saw a squirrel running down one of the lines (keep in mind this is a 44 KV distribution line, 3-phase).
All of a sudden, the little critter stops on the connector line crossing one of the little grey garbage cans to the recloser on the blue phase when all of a sudden we see it start to chew the blue phase line.
Normally, if an animal punctures the outside casing of a single line, the line will still operate normally. However, seeing that he was standing on the connector and chewing on the blue phase line, he essentially became a conduit for the 44 KV of electricity.
BOOM. Squirrel explodes and the transformer arcs blue and bursts into a full-blown electrical fire.
All because of a god damned squirrel we had to replace 2 transformers and re-string a section of the blue phase.
I agree we could all use a new, modern power grid with consolidated & reinforced phases, but that costs money. In times of financial insecurity and razor-thin margins, deregulated power companies (like Hydro One) do not have the operating budget to upgrade the transmission & distribution lines all at once.
This takes time.