Oops, I Just Unplugged San Diego: Let's Fix it so Just One Person Can't Put Much of SoCal in the Dark Again

Categories: Environment

homer d'oh fox.JPG
Fox
Updated at the bottom: The California Independent System Operator will join the party, er we mean, investigations. First posted at 7:09 a.m.

When the power went out yesterday for a large swath of Southern California only days before the anniversary of 9/11, our thoughts naturally took us to a possible act of terrorism.

After all, the San Onofre nuclear power plant seemed to go off line about the same time people from Ensenada to South Orange County saw their lights go out. Was there an attack on SoCal's main radiation station? But Al-Qaeda ain't got nothing on human error.

No, the punchline to a night without a/c during one of the hottest weeks of the year:

It only took one employee to set the whole thing off.

APS, the Arizona company that owns the 500 kV transmission line near Yuma that feeds San Diego and environs said in a statement:

The outage appears to be related to a procedure an APS employee was carrying out in the North Gila substation ... Operating and protection protocols typically would have isolated the resulting outage to the Yuma area. The reason that did not occur in this case will be the focal point of the investigation into the event, which already is under way.

Hope so.

We can just see the guy, plump like Homer Simpson, wearing a hardhat, and flipping the wrong switch. Do'h. As many as 5 million people were effected.

Here's the thing. Dude (and you know women don't make mistakes like this) probably made an honest uh-oh. But how in the hell is it possible for one person to take one of America's largest metropolitan areas -- a major city, parts of a different country, parts of a different state, too -- off-line in almost an instant?

If a hapless American worker can do this, just imagine what a terror-minded MacGruber could do with some chewing gum and a paper clip.

Should we -- oh, maybe -- design a fail-safe here?

And, for criminy, why, as SDG&E officials explained, are there only two lines feeding all of San Diego and its neighbors, from South O.C. to Ensenada? Snip-snip and you're a caveman again. Really?

The outage knocked out SoCal's only nuclear power plant and could have put air traffic in jeopardy, as the FAA's airspace controllers for L.A. and Southern California are based in San Diego.

Let's not just point fingers here. Let's fix this. We have some electrical tape we can donate to the effort.

Update: Look out, Homer, because the feds are getting involved in this.

Today the Federal Energy Regulatory Commission (FERC) said it's joining hands with the North American Electric Reliability Corporation (NERC) for a joint inquiry into the Great SoCal Blackout of 2011.

Gerry Cauley, president and chief executive officer of NERC:

NERC and FERC are partnering to conduct a full inquiry into the cause of this event. Partnering brings together the expertise of both organizations, and emphasizes the importance placed on reliability of the bulk power system.

Somebody's going to get FERC'd.

Update No. 2: The California Independent System Operator stated this afternoon it would launch its own joint task force " ... in close coordination with the Western Electricity Coordinating Council (WECC)" to look into the blackout. According to a Cal ISO statement:

The task force will conduct a full investigation in a thoughtful, collaborative and transparent fashion to guard against it happening again.

Cal ISO CEO Steve Berberich:

We now turn our focus to root-cause analysis to investigate the reason for the series of events that triggered the widespread power outage.

[@dennisjromero/djromero@laweekly.com]

My Voice Nation Help
51 comments
Miguel Barbosa
Miguel Barbosa

"Should we -- oh, maybe -- design a fail-safe here?" Enough said to the bone. I wonder how many other things don't have a plan B, ridiculous.

dudicaldude
dudicaldude

I live in South Orange county and I had power.  Hahahaha.  Too bad for you!

Wxx544
Wxx544

(and you know women don't make mistakes like this)really...man hatermale,female...sometimes sh** happends

walter conell
walter conell

 I can't help it- this is funny. I feel sorry for the man who did this and I hope his career isn't over because of it. Any one of us can make mistakes- right? I also hope no one was seriously hurt during this ordeal.

Joe Cattleman
Joe Cattleman

Having worked in the electrical industry for many years, I think this sounds like a typical case of a person working under pressure from a supervisor who blindly ignores proper workplace safety habits and forces his underlings to work on systems live.   I was in situations like this and fortunately nothing I worked on resulted in any problems that resulted in major inconvenience or danger to human life. 

douglas
douglas

I bet you're union too (in response to Sam above)

Guest
Guest

Anyway who still believes a plane hit the Pentagon or that the BBC lost the tape of their broadcast proclaiming the collapse of Building 7 before it happened is too hopelessly ignorant to waste time on.  Wake up, people, the only terrorists were Dick Cheney and George Bush Laden.

Ellenkurek
Ellenkurek

Lamestream TV talking head said 1.4 million affected; alternative report says 6 million were.  Who are you going to believe?

sam l
sam l

They took 1.4 from S.D. city proper stats, the 6 million from the entire area affected.[yes I am refraining from a ww2 joke]

sam
sam

Dam-ned moon ruined a good night of a rare view of the night sky from here. No low clouds coming in [fairly rare] and that moon up there blew it. More than half the sky was flooded out by the glow of the moon shine.

Ellenkurek
Ellenkurek

Can you say "Coronal Mass Ejection"?

Vigilant Satyr
Vigilant Satyr

Spoken as someone that has very little technical knowledge about large scale power distribution systems. When you lose a big supply the rest of the system takes time to compensate and during that time you can get some big fluctuations on the system. The power distribution system is pretty good at "steady state," expected changes, and slowly changing conditions, but sudden changes can wreak the kind of havoc that we saw Thursday.

sam l
sam l

The cascading failure is the result of the wrong people running the operation. Someone is seriously lacking in the supervisory management department. But then again, was the contractor [if it was the real cause] one of those that worked their way up from the bottom or just another 'lowest bidder' winner? AZ is a union free *"right to work"* state,,, a union man would never have done that.

* another misnamed thought like 'workers compensation'.

Afaf
Afaf

It takes multiple failures of redundant systems for such an event to occur. Not much is said for 100 years of near flawless energy delivered to the entire country 24hrs a day, 7 days a week, 365 days a year - only the very few - once in a decade events - are the focus. People are funny that way. It's too bad people don't demand 100% up-time for other products and services.

Killuminati
Killuminati

They're also "testing" a new grid "system" for absolutely no reason.  Could be the start of the mishaps for the country's electric grid

Theodore R. Smith
Theodore R. Smith

I pay for 99.95% uptime SLAs for my web servers. I would pay the same for electricity if such was an option:  Say 99.5% uptime or the week of the outage free.  That's 3 hours 36 minutes.  This would encourage the same redundancy in the markets as there exists on the Internet. It would be optional and there'd be a premium, so most of the time it'd be a nice premium for already-cash-strapped energy providers.

douglas
douglas

You mean you're not getting your electricity at better than 99.5%?  You should get off solar and wind, and get you power from the utility company.  They've been doing better than 99.5% for decades.

And for all you conspiracy theorists, yes, stuff happens, unanticipated events occur and circumvent what everyone thought were good precautions.  Or maybe your life has been an unusual example of perfection leaving you thinking that imperfections are some sort of evil plot...

AnEngineerWhoWorksWithIt
AnEngineerWhoWorksWithIt

It typically cost $10,000 for a web server.  It typically cost $1,200,000+ for a substation power transformer.  I know those prices because I have purchased both.  So you can buy 120 backup web servers for every 1 backup transformer you buy.  And thats just one component of the system.  Do you really think you can compare the two industries?

Gumby
Gumby

...and those people are making me lol

Ohmganesha
Ohmganesha

This shows you there is no "War on Terror."

Al Qaeda would just walk right across the border and take down our entire electrical grid with a pair of wire cutters.

sam l
sam l

It wouldn't even take wire cutters to do it.

Theodore R. Smith
Theodore R. Smith

I am totally going to quote you on that to other people! What a succinct and truthful way of saying it!

Gumby
Gumby

Great! You just had to give the CIA another idea!

Porter
Porter

And yet, the rubes are still buying this baloney story. TSA needs to feel up little boys and girls, and grandma an grandpa but forgot to check what was important...or

At this point, the rubes will swallow whatever lie they're told. Sad, I am assuming most people on this forum went to school when "1984" was required reading in high school, or is that on the banned book list nowadays?

Anyway, isn't it amazing that whatever the "party" (no matter which side of the coin) tells the lies that  the people chomp it down whole without a thought?

Theodore R. Smith
Theodore R. Smith

When the lower class rubes wake up, it will be with hostility in their eyes and animosity in their hearts. They will exact a hundred years worth of vengeance upon the most visible and accessible targets they have been inculcated by The System into perceiving as their afflictors:   Upper middle class, particularly white, intelligent people. This is why you should have at least one form of lethal protection.

sam l
sam l

Teddy, we were a fine example of how you behave in a black out. Nobody noticed that part of it. I'll be uploading a video soon to the web taken that night. The funny part of it, I reched for the camera and by the time I had it recording, the one house of kids to one side was roaring it up like never before, the ones behind us the same thing with their kids in the back, and the young adult renters on the other side, were playing guitars and partying it up real good. Someone in the hood found some roman candles,,, they are not recommended in a black out,,, for indoor use. Plenty of candles like you are told, batteries, radio [we were gathered around it, missing a shadow though] a puzzle book, regular book with at least two candle power and you'll be fine. Just break that addiction to teevee and it will be A-O-Kay. I wished the moon wasn't as big. HEY NEXT TIME YOU PULL A POWER FAILURE,,, do it on a no moon night please. Cheated me out of a chance to see the real full star'd night sky from here.

automan
automan

I was hoping for a story about a bird pooping on a transformer or something.... scary.

Porter
Porter

Ah, what a mishap and what great timing to stir up hysteria on that particular day in this particular 10th anniversary month.

Well, does this "mistake" justify the idiotic waste of money on the entire completely incompetent and thoroughly useless and worthless HLS scam?

I realize we all live in the altered reality of the post 9/11 world where logic, reason, and memory have been replaced with fantasy and thrill but the infrastructure grid was supposedly checked and re-checked by this useless HLS years ago.

Maybe some people think that electricians who work on high voltage lines are high school drop out types just doing manual labor but this is NOT the case, especially the engineers who designed and have subsequently inspected the every line, switch, and everything else many times over the years.

We are suppose to believe that every single one of the men working on this project from blueprint to construction to the maintenance through through the years-that this "vulnerability" was NEVER noticed by anyone including HLS?

Folks, I have three magic beans and I want to make you a special deal........

gerryhiles
gerryhiles

Ooops!  I tried posting a direct link to actual footage on the day of 911, from the BBC and other TV channels, but it has gone into "moderation" and might never appear.

A way around this is to go, via Google or whatever, to A&E for Truth, so as to see the "smoking gun" of World Trade Centre Seven.

I suppose I should mention that I am in Australia, but we all live in the same sort of world today ... NWO by stealth? ... so, now, what happens anywhere impacts on nearly everyone globally, e.g. when the US consumer economy fails, China will lose the major export market and Australia will lose being able to export raw materials to China.

In brief: We are all in shit state and just waiting for that all to hit the fan ... meanwhile distracted by ever-more unjustifiable and insane wars. Iraq, Afghanistan, Pakistan, Yemen, Libya, Syria, Iran ... the list becomes endless since WW2 failed to stop wars.

Investigator Gadget
Investigator Gadget

You say Ooops - yet claim to be the one that worked on complex systems

hummmmm

he dit it! he did it!

Tosheba
Tosheba

"Women don't make mistakes like this..."  Shuddup.  

AnEngineerWhoWorksWIthIt
AnEngineerWhoWorksWIthIt

Here's the thing.  I agree we need to improve.  But don't underestimate the complexity of what is involved here.  Look around.  Virtually everything going on around most of you except for most transportation vehicles is powered by electricity.  That takes a LOT of energy.  Its all getting there via those innocent looking wires.  When something goes wrong and that energy starts to get into somewhere it doesn't belong a lot of damage can happen very quickly.  And replacing damaged major transformers, circuit breakers, etc can take a long time and be very expensive. For that reason the entire electrical system, which needs to be very robust also needs to sit on a knife edge and be ready turn itself off at any instant to minimize damage when things do go wrong.  Its a delicate balance that tens of thousands of people work at every day to maintain.  Once in a while all the safeties don't work quite right and we get a this.  Its steadily improving and needs to continue but don't think for a minute its as simple as throwing a couple more rolls of electrical tape at it.  And remember who pays for all of this.  Do you want your energy bills even higher?

gerryhiles
gerryhiles

As a person - now retired - who used to work with complex systems, what you say is spot on, whilst a journalist taking cheap shots is just on an ego trip.

As for that bit about "and you know women don't make mistakes like this"!  Where the hell did that come from?

Of course they don't, very few do the sorts of work involved in building and maintaining power grids, etc..

It is actually offensive for Dennis Romero to effectively slander  - before any enquiry - whomever and whatever led to the outage, with this reference to the cartoon character Homer Simpson; who has NO resemblance with any man I ever worked with.

Imagine the uproar if I, any man, pounced on a female who made a mistake and called her "Barbie".

If, IF one man did actually make a mistake; then, Dennis, I hope he sues you for defamation ... for bringing in a verdict before a proper inquiry.

Not sure if, you,  AnEngineerWhoWorksWIthIt, would agree with all I have written, but we do work off the same page of all men who have been "at the coal face", so to speak ... unlike some journalist who (it seems) cannot live in an office without 24/7 AC and artificial lighting ... poor dear!

Theodore R. Smith
Theodore R. Smith

What this shows us is that any rational person with any sort of means (and especially every man with a woman and/or children) should NEVER assume the power grid *can* be fully functional over the long-haul.  Hurricane Ike taught Houston that, yes, the power and water and roadways *can* be out for 6+ weeks. No, no one is going to save you; and yes, if the wind is blowing, it's already too late to get prepared.

Californians: Use this as a wake up call.  Our modern system is precariously balanced between fragile high-capacity electrical lines, our ability to secure tremendous amounts of affordable, if not cheap, fossil fuels, the ability to import food and water, the willingness of the government to stay out of our business, and the complacency of the burgeoning underclass to stay in their hovels and not react violently and in mobs against those they think have it better (You and me).

sam l
sam l

Year wake up call to those that operate the sewage pumps that demand constant power to keep the human waste out of the ocean. Once again the beaches are contaminated and there shall be no fishing for a few weeks. Imagine relying on that ocean in the event of a major power failure and for a lot longer time frame, when all the fish are contaminated.

Power back ups for these things are mandated until a shut off can be installed to block the spills.

Julian
Julian

9-11 wasn't done by Muslim terrorists. It was done by Zi-nist terrorists. (Just trying to keep things straight here.)

Vigilant Satyr
Vigilant Satyr

"(Just trying to keep things straight here.)"

Not doing a very good job of it I'm afraid. In the 10 years since the attacks happened, has there been a single shred of direct evidence exposed that proves attacks were not perpetrated by Muslin terrorists? No there hasn't. NOT A SINGLE SHRED.

sam l
sam l

Has there been in the ten years since one shred of evidence to back up the official liars claims that 19 lone nutts did it?? There is ample proof that a second flight impacting the wtc area was NOT a 767 or a 757. The liars buried that plane part in a land fill.

gerryhiles
gerryhiles

I do not know if I will get away with posting a direct link concerning 911, so I will split this post by firstly saying that I am a member of A&E for Truth ... and what about WTC7?

If this posts, then I will try a direct link to ...

Porter
Porter

Keep trying and doing! Unfortunately, judging by this nonsensical story and most responses, critical thinking has been replaced with the usual Pavlovian programming. 

Ranford
Ranford

Damn temps! Guess we should have kept our American guys and not hired all the wetbacks we did! Damn, oh well, ok back to work ya dirty messicans!

Pseedie
Pseedie

awkward day at work for the guy today, don'tcha think?  [= 

James Downing
James Downing

The same exact thing happened in Miami a few years ago

Torgen Johnson
Torgen Johnson

It doesn't take a terrorist to create a nuclear disaster at San Onofre.  A large earthquake, control room fire, computer hacker, mechanical failure, loss of cooling, broken pipe weld, tsunami triggered by submarine landslide offshore, human error, etc.  Whistleblowers from San Onofre have for years been telling the public about serious safety issues there.  Last night could have be a taste of what a nuclear disaster triggered blackout would have been like, only you would have been scrambling for an escape from the radioactive plume in the dark and in bumper to bumper traffic with difficulty reaching loved ones on overloaded phone systems. San Onofre has the second worst safety record of all 104 nuclear reactors in the U.S. and 4,000 tons of spent nuclear fuel on site, 4X more than at Fukushima Daiichi.

Towelie
Towelie

Don't get me wrong, I love the doom and gloom along with the double posting. Next time the tits melt down, do me a favor and stand right next to them Torgen.

Who the fuck is named Torgen?

Theodore R. Smith
Theodore R. Smith

Towelie, your post makes you out to be a bully. Do you really want to risk being associated with such an uncalled for and negative comment?

Torgen Johnson
Torgen Johnson

It doesn't take a terrorist to create a nuclear disaster at San Onofre.  Whistleblowers from that plant have for years been telling the public about serious safety issues there.  Last night could have be a taste of what a nuclear disaster triggered blackout would have been like, only you would have been scrambling for an escape from the radioactive plume in the dark and in bumper to bumper traffic with difficulty reaching loved ones on overloaded phone systems.

Jake D
Jake D

Great article. Love your style. Hopefully some major changes are made so we're all better prepared. I have a generator...but no gas. Whoops. Emergency flashlight...still in it's packaging with no charge, batteries, or sun to power it. Whoops. Pool full of water...no filter. Whoops. Sometimes you don't know what you need...til you don't have it.

Now Trending

From the Vault

 

General

©2013 LA Weekly, LP, All rights reserved.
Browse Voice Nation
  • Voice Places Los Angeles

    Voice Places

    Find everything you're looking for in your city

  • Happy Hour App

    Happy Hour App

    Find the best happy hour deals in your city

  • Daily Deals

    Daily Deals

    Get today's exclusive deals at savings of anywhere from 50-90%

  • Best Of

    Best Of...

    Check out the hottest list of places and things to do around your city