Politics, Religion, Science, Culture and Humanities > Science, Culture, & Humanities

Electricity (including EMP, electro magnetic pulse, CME)

<< < (2/20) > >>

ccp:
I am not sure I can access to article from the link. 

Crafty_Dog:
Meltdowns Hobble NSA Data Center
Investigators Stumped by What's Causing Power Surges That Destroy Equipment

 

    By
    SIOBHAN GORMAN
 




Chronic electrical surges at the massive new data-storage facility central to the National Security Agency's spying operation have destroyed hundreds of thousands of dollars worth of machinery and delayed the center's opening for a year, according to project documents and current and former officials.
[image]

There have been 10 meltdowns in the past 13 months that have prevented the NSA from using computers at its new Utah data-storage center, slated to be the spy agency's largest, according to project documents reviewed by The Wall Street Journal.

One project official described the electrical troubles—so-called arc fault failures—as "a flash of lightning inside a 2-foot box." These failures create fiery explosions, melt metal and cause circuits to fail, the official said.

The causes remain under investigation, and there is disagreement whether proposed fixes will work, according to officials and project documents. One Utah project official said the NSA planned this week to turn on some of its computers there.

Electrical surges at the new data-storage facility central to the National Security Agency's spying operation have destroyed hundreds of thousands of dollars worth of machinery and delayed the center's opening for a year. Siobhan Gorman reports. Photo: Getty.

NSA spokeswoman Vanee Vines acknowledged problems but said "the failures that occurred during testing have been mitigated. A project of this magnitude requires stringent management, oversight, and testing before the government accepts any building."
Washington Wire

    Failures Follow History of Electrical Troubles

The Utah facility, one of the Pentagon's biggest U.S. construction projects, has become a symbol of the spy agency's surveillance prowess, which gained broad attention in the wake of leaks from NSA contractor Edward Snowden. It spans more than one-million square feet, with construction costs pegged at $1.4 billion—not counting the Cray supercomputers that will reside there.

Exactly how much data the NSA will be able to store there is classified. Engineers on the project believe the capacity is bigger than Google's largest data center. Estimates are in a range difficult to imagine but outside experts believe it will keep exabytes or zettabytes of data. An exabyte is roughly 100,000 times the size of the printed material in the Library of Congress; a zettabyte is 1,000 times larger.

But without a reliable electrical system to run computers and keep them cool, the NSA's global surveillance data systems can't function. The NSA chose Bluffdale, Utah, to house the data center largely because of the abundance of cheap electricity. It continuously uses 65 megawatts, which could power a small city of at least 20,000, at a cost of more than $1 million a month, according to project officials and documents.

Utah is the largest of several new NSA data centers, including a nearly $900 million facility at its Fort Meade, Md., headquarters and a smaller one in San Antonio. The first of four data facilities at the Utah center was originally scheduled to open in October 2012, according to project documents.

In the wake of the Snowden leaks, the NSA has been criticized for its expansive domestic operations. Through court orders, the NSA collects the phone records of nearly all Americans and has built a system with telecommunications companies that provides coverage of roughly 75% of Internet communications in the U.S.

In another program called Prism, companies including Google, Microsoft, Facebook and Yahoo are under court orders to provide the NSA with account information. The agency said it legally sifts through the collected data to advance its foreign intelligence investigations.

The data-center delays show that the NSA's ability to use its powerful capabilities is undercut by logistical headaches. Documents and interviews paint a picture of a project that cut corners to speed building.

Backup generators have failed numerous tests, according to project documents, and officials disagree about whether the cause is understood. There are also disagreements among government officials and contractors over the adequacy of the electrical control systems, a project official said, and the cooling systems also remain untested.

The Army Corps of Engineers is overseeing the data center's construction. Chief of Construction Operations, Norbert Suter said, "the cause of the electrical issues was identified by the team, and is currently being corrected by the contractor." He said the Corps would ensure the center is "completely reliable" before handing it over to the NSA.



But another government assessment concluded the contractor's proposed solutions fall short and the causes of eight of the failures haven't been conclusively determined. "We did not find any indication that the proposed equipment modification measures will be effective in preventing future incidents," said a report last week by special investigators from the Army Corps of Engineers known as a Tiger Team.

The architectural firm KlingStubbins designed the electrical system. The firm is a subcontractor to a joint venture of three companies: Balfour Beatty Construction, DPR Construction and Big-D Construction Corp. A KlingStubbins official referred questions to the Army Corps of Engineers.

The joint venture said in a statement it expected to submit a report on the problems within 10 days: "Problems were discovered with certain parts of the unique and highly complex electrical system. The causes of those problems have been determined and a permanent fix is being implemented."

The first arc fault failure at the Utah plant was on Aug. 9, 2012, according to project documents. Since then, the center has had nine more failures, most recently on Sept. 25. Each incident caused as much as $100,000 in damage, according to a project official.

It took six months for investigators to determine the causes of two of the failures. In the months that followed, the contractors employed more than 30 independent experts that conducted 160 tests over 50,000 man-hours, according to project documents.

This summer, the Army Corps of Engineers dispatched its Tiger Team, officials said. In an initial report, the team said the cause of the failures remained unknown in all but two instances.

The team said the government has incomplete information about the design of the electrical system that could pose new problems if settings need to change on circuit breakers. The report concluded that efforts to "fast track" the Utah project bypassed regular quality controls in design and construction.

Contractors have started installing devices that insulate the power system from a failure and would reduce damage to the electrical machinery. But the fix wouldn't prevent the failures, according to project documents and current and former officials.

Contractor representatives wrote last month to NSA officials to acknowledge the failures and describe their plan to ensure there is reliable electricity for computers. The representatives said they didn't know the true source of the failures but proposed remedies they believed would work. With those measures and others in place, they said, they had "high confidence that the electrical systems will perform as required by the contract."

A couple of weeks later, on Sept. 23, the contractors reported they had uncovered the "root cause" of the electrical failures, citing a "consensus" among 30 investigators, which didn't include government officials. Their proposed solution was the same device they had already begun installing.

The Army Corps of Engineer's Tiger Team said the contractor's explanations were unproven. The causes of the incidents "are not yet sufficiently understood to ensure that [the NSA] can expect to avoid these incidents in the future," their report said.

Write to Siobhan Gorman at siobhan.gorman@wsj.com

A version of this article appeared October 8, 2013, on page A1 in the U.S. edition of The Wall Street Journal, with the headline: Meltdowns Hobble NSA Data Center.

Crafty_Dog:
http://www.familysecuritymatters.org/publications/detail/american-blackout-a-real-life-nightmare-nearer-than-you-think?f=must_reads

DougMacG:

--- Quote from: Crafty_Dog on November 04, 2013, 06:24:43 AM ---http://www.familysecuritymatters.org/publications/detail/american-blackout-a-real-life-nightmare-nearer-than-you-think?f=must_reads

--- End quote ---

I think this is part of the argument against ethanol, wind, solar subsidies.  By definition, you pay more (willingly) for a source other than lowest cost power, that will work when the grid is down.  Undermining and distorting the free market is not how you bring down those prices.

My experience with a gasoline generator is that you run out of gas very quickly and stations require electricity to pump gas.  Wind and solar tend to small in output and weather dependent.  In this part of the country where we have severe winters and natural gas pipelines to nearly everyone, a natural gas backup system seems far more useful.  Blackouts have tended to happen during the air conditioning season or as the result of storms.  But if a grid attack or failure happened in winter, most people don't seem to realize their natural gas furnace requires electricity to operate.  

Crafty_Dog:


Assault on California Power Station Raises Alarm on Potential for Terrorism
April Sniper Attack Knocked Out Substation, Raises Concern for Country's Power Grid
By Rebecca Smith
Feb. 4, 2014 10:30 p.m. ET

SAN JOSE, Calif.—The attack began just before 1 a.m. on April 16 last year, when someone slipped into an underground vault not far from a busy freeway and cut telephone cables.

Within half an hour, snipers opened fire on a nearby electrical substation. Shooting for 19 minutes, they surgically knocked out 17 giant transformers that funnel power to Silicon Valley. A minute before a police car arrived, the shooters disappeared into the night.

A sniper attack in April that knocked out an electrical substation near San Jose, Calif., has raised fears that the country's power grid is vulnerable to terrorism. WSJ's Rebecca Smith has the details. Photo: Talia Herman for The Wall Street Journal

With over 160,000 miles of transmission lines, the U.S. power grid is designed to handle natural and man-made disasters, as well as fluctuations in demand. How does the system work? WSJ's Jason Bellini has #TheShortAnswer.

To avoid a blackout, electric-grid officials rerouted power around the site and asked power plants in Silicon Valley to produce more electricity. But it took utility workers 27 days to make repairs and bring the substation back to life.

Nobody has been arrested or charged in the attack at PG&E Corp.'s PCG -0.41% Metcalf transmission substation. It is an incident of which few Americans are aware. But one former federal regulator is calling it a terrorist act that, if it were widely replicated across the country, could take down the U.S. electric grid and black out much of the country.

The attack was "the most significant incident of domestic terrorism involving the grid that has ever occurred" in the U.S., said Jon Wellinghoff, who was chairman of the Federal Energy Regulatory Commission at the time.

The Wall Street Journal assembled a chronology of the Metcalf attack from filings PG&E made to state and federal regulators; from other documents including a video released by the Santa Clara County Sheriff's Department; and from interviews, including with Mr. Wellinghoff.
Related

Q&A: What You Need to Know About Attacks on the U.S. Power Grid

The 64-year-old Nevadan, who was appointed to FERC in 2006 by President George W. Bush and stepped down in November, said he gave closed-door, high-level briefings to federal agencies, Congress and the White House last year. As months have passed without arrests, he said, he has grown increasingly concerned that an even larger attack could be in the works. He said he was going public about the incident out of concern that national security is at risk and critical electric-grid sites aren't adequately protected.

The Federal Bureau of Investigation doesn't think a terrorist organization caused the Metcalf attack, said a spokesman for the FBI in San Francisco. Investigators are "continuing to sift through the evidence," he said.

Some people in the utility industry share Mr. Wellinghoff's concerns, including a former official at PG&E, Metcalf's owner, who told an industry gathering in November he feared the incident could have been a dress rehearsal for a larger event.

"This wasn't an incident where Billy-Bob and Joe decided, after a few brewskis, to come in and shoot up a substation," Mark Johnson, retired vice president of transmission for PG&E, told the utility security conference, according to a video of his presentation. "This was an event that was well thought out, well planned and they targeted certain components." When reached, Mr. Johnson declined to comment further.

A spokesman for PG&E said the company takes all incidents seriously but declined to discuss the Metcalf event in detail for fear of giving information to potential copycats. "We won't speculate about the motives" of the attackers, added the spokesman, Brian Swanson. He said PG&E has increased security measures.
View Graphics

Utility executives and federal energy officials have long worried that the electric grid is vulnerable to sabotage. That is in part because the grid, which is really three systems serving different areas of the U.S., has failed when small problems such as trees hitting transmission lines created cascading blackouts. One in 2003 knocked out power to 50 million people in the Eastern U.S. and Canada for days.

Many of the system's most important components sit out in the open, often in remote locations, protected by little more than cameras and chain-link fences.

Transmission substations are critical links in the grid. They make it possible for electricity to move long distances, and serve as hubs for intersecting power lines.

Within a substation, transformers raise the voltage of electricity so it can travel hundreds of miles on high-voltage lines, or reduce voltages when electricity approaches its destination. The Metcalf substation functions as an off-ramp from power lines for electricity heading to homes and businesses in Silicon Valley.

The country's roughly 2,000 very large transformers are expensive to build, often costing millions of dollars each, and hard to replace. Each is custom made and weighs up to 500,000 pounds, and "I can only build 10 units a month," said Dennis Blake, general manager of Pennsylvania Transformer in Pittsburgh, one of seven U.S. manufacturers. The utility industry keeps some spares on hand.

A 2009 Energy Department report said that "physical damage of certain system components (e.g. extra-high-voltage transformers) on a large scale…could result in prolonged outages, as procurement cycles for these components range from months to years."

Mr. Wellinghoff said a FERC analysis found that if a surprisingly small number of U.S. substations were knocked out at once, that could destabilize the system enough to cause a blackout that could encompass most of the U.S.

Not everyone is so pessimistic. Gerry Cauley, chief executive of the North America Electric Reliability Corp., a standards-setting group that reports to FERC, said he thinks the grid is more resilient than Mr. Wellinghoff fears.

"I don't want to downplay the scenario he describes," Mr. Cauley said. "I'll agree it's possible from a technical assessment." But he said that even if several substations went down, the vast majority of people would have their power back in a few hours.

The utility industry has been focused on Internet attacks, worrying that hackers could take down the grid by disabling communications and important pieces of equipment. Companies have reported 13 cyber incidents in the past three years, according to a Wall Street Journal analysis of emergency reports utilities file with the federal government. There have been no reports of major outages linked to these events, although companies have generally declined to provide details.

"A lot of people in the electric industry have been distracted by cybersecurity threats," said Stephen Berberich, chief executive of the California Independent System Operator, which runs much of the high-voltage transmission system for the utilities. He said that physical attacks pose a "big, if not bigger" menace.

There were 274 significant instances of vandalism or deliberate damage in the three years, and more than 700 weather-related problems, according to the Journal's analysis.

Until the Metcalf incident, attacks on U.S. utility equipment were mostly linked to metal thieves, disgruntled employees or bored hunters, who sometimes took potshots at small transformers on utility poles to see what happens. (Answer: a small explosion followed by an outage.)

Last year, an Arkansas man was charged with multiple attacks on the power grid, including setting fire to a switching station. He has pleaded not guilty and is undergoing a psychiatric evaluation, according to federal court records.

Overseas, terrorist organizations were linked to 2,500 attacks on transmission lines or towers and at least 500 on substations from 1996 to 2006, according to a January report from the Electric Power Research Institute, an industry-funded research group, which cited State Department data.

An attack on a PG&E substation near San Jose, Calif., in April knocked out 17 transformers like this one. Talia Herman for The Wall Street Journal

To some, the Metcalf incident has lifted the discussion of serious U.S. grid attacks beyond the theoretical. "The breadth and depth of the attack was unprecedented" in the U.S., said Rich Lordan, senior technical executive for the Electric Power Research Institute. The motivation, he said, "appears to be preparation for an act of war."

The attack lasted slightly less than an hour, according to the chronology assembled by the Journal.

At 12:58 a.m., AT&T fiber-optic telecommunications cables were cut—in a way that made them hard to repair—in an underground vault near the substation, not far from U.S. Highway 101 just outside south San Jose. It would have taken more than one person to lift the metal vault cover, said people who visited the site.

Nine minutes later, some customers of Level 3 Communications, LVLT +10.00% an Internet service provider, lost service. Cables in its vault near the Metcalf substation were also cut.

At 1:31 a.m., a surveillance camera pointed along a chain-link fence around the substation recorded a streak of light that investigators from the Santa Clara County Sheriff's office think was a signal from a waved flashlight. It was followed by the muzzle flash of rifles and sparks from bullets hitting the fence.

The substation's cameras weren't aimed outside its perimeter, where the attackers were. They shooters appear to have aimed at the transformers' oil-filled cooling systems. These began to bleed oil, but didn't explode, as the transformers probably would have done if hit in other areas.

About six minutes after the shooting started, PG&E confirms, it got an alarm from motion sensors at the substation, possibly from bullets grazing the fence, which is shown on video.

Four minutes later, at 1:41 a.m., the sheriff's department received a 911 call about gunfire, sent by an engineer at a nearby power plant that still had phone service.

Riddled with bullet holes, the transformers leaked 52,000 gallons of oil, then overheated. The first bank of them crashed at 1:45 a.m., at which time PG&E's control center about 90 miles north received an equipment-failure alarm.

Five minutes later, another apparent flashlight signal, caught on film, marked the end of the attack. More than 100 shell casings of the sort ejected by AK-47s were later found at the site.

At 1:51 a.m., law-enforcement officers arrived, but found everything quiet. Unable to get past the locked fence and seeing nothing suspicious, they left.

A PG&E worker, awakened by the utility's control center at 2:03 a.m., arrived at 3:15 a.m. to survey the damage.

Grid officials routed some power around the substation to keep the system stable and asked customers in Silicon Valley to conserve electricity.

In a news release, PG&E said the substation had been hit by vandals. It has since confirmed 17 transformers were knocked out.

Mr. Wellinghoff, then chairman of FERC, said that after he heard about the scope of the attack, he flew to California, bringing with him experts from the U.S. Navy's Dahlgren Surface Warfare Center in Virginia, which trains Navy SEALs. After walking the site with PG&E officials and FBI agents, Mr. Wellinghoff said, the military experts told him it looked like a professional job.

In addition to fingerprint-free shell casings, they pointed out small piles of rocks, which they said could have been left by an advance scout to tell the attackers where to get the best shots.

"They said it was a targeting package just like they would put together for an attack," Mr. Wellinghoff said.

Mr. Wellinghoff, now a law partner at Stoel Rives LLP in San Francisco, said he arranged a series of meetings in the following weeks to let other federal agencies, including the Department of Homeland Security, know what happened and to enlist their help. He held a closed-door meeting with utility executives in San Francisco in June and has distributed lists of things utilities should do to strengthen their defenses.

A spokesman for Homeland Security said it is up to utilities to protect the grid. The department's role in an emergency is to connect federal agencies and local police and facilitate information sharing, the spokesman said.

As word of the attack spread through the utility industry, some companies moved swiftly to review their security efforts. "We're looking at things differently now," said Michelle Campanella, an FBI veteran who is director of security for Consolidated Edison Inc. ED -0.37% in New York. For example, she said, Con Ed changed the angles of some of its 1,200 security cameras "so we don't have any blind spots."

Some of the legislators Mr. Wellinghoff briefed are calling for action. Rep. Henry Waxman (D., Calif.) mentioned the incident at a FERC oversight hearing in December, saying he was concerned that no one in government can order utilities to improve grid protections or to take charge in an emergency.

As for Mr. Wellinghoff, he said he has made something of a hobby of visiting big substations to look over defenses and see whether he is questioned by security details or local police. He said he typically finds easy access to fence lines that are often close to important equipment.

"What keeps me awake at night is a physical attack that could take down the grid," he said. "This is a huge problem."

Navigation

[0] Message Index

[#] Next page

[*] Previous page