Red Storm at Sandia National Laboratories/NM
Mysterious Lybian Glass and Supercomputers?
April / May 2007
Volume 5 Number 2
The connection between a piece of Libyan desert glass set in jewelry from King Tut's tomb and a state-of-the-art supercomputer at Sandia National Laboratories is only one of the surprises resulting from a new generation of supercomputers that is revolutionizing modern science. The origin of the Libyan desert glass had remained a tantalizing mystery because it was not clear that it could have been formed by a comet or asteroid. Sandia physicist Mark Boslough enlisted the 100+ Teraflop computational power of Sandia's Red Storm supercomputer to simulate the effects of a comet or asteroid and demonstrated that the enormous heat required to fuse glass from sandstone could have been produced by a comet or asteroid in a low atmosphere event.
High Performance Computing, or HPC, activity at Sandia produces remarkable computing resources such as Red Storm as part of the Department of Energy's National Nuclear Security Administration Advanced Simulation and Computing Program. These resources are directed at simulating the complex physical processes that need to be understood to maintain the nation's nuclear stockpile without the requirement for nuclear testing. Using this simulation capability, new replacement parts for the stockpile are designed and tested in a virtual domain under hostile, abnormal and normal conditions.
Red Storm is a massively parallel processing (MPP) supercomputer that recently joined the ranks of the ASC supercomputers monopolizing world computing records.
Red Storm was uniquely designed by Sandia and constructed by Cray, Inc., to address the highly complex, real-world computing problems that particularly characterize the simulations required by an engineering laboratory such as Sandia. Red Storm allows modeling and simulation of complex problems in nuclear weapons stockpile stewardship that were only recently thought impractical, if not impossible. Other ASC researchers at Los Alamos and Lawrence Livermore are also finding it a valuable resource.
Red Storm is partitioned to support classified and unclassified operations with the physical throwing of a secure switch, similar to the way a railroad switch can divert a train from one track to another. As Sandia's Jim Tomkins says, "That's important because we have a whole community here that does science. We can allocate part or even the whole machine to a science problem and move to DOE interests and do secure work."
Its high-performance Input/Output system facilitates connecting with external Sandia networks and storage. Red Storm's architecture is scalable from one cabinet to several hundred and today boasts more than 25,000 processors. New memory, multi-core processors, and new interconnects may lead to a doubling of its capability, faster solution time, and reduced cost for the performance provided.
Sandia's collaboration with Cray supports commercialization of the technology. This not only increases national competitiveness in supercomputing, but results in a wide user knowledge base to detect and fix errors and problems.
The ASC computing resources offer different advantages for different types of applications. Red Storm specializes in MPP problems that involve considerable interaction and coordination of the high-volume AMD processors running an application. The problems it addresses tend to be more engineering based than theoretical, but it can contribute in both environments.
Red Storm was constructed of commercial off-the-shelf parts supporting the custom Cray designed and IBM-manufactured SeaStar interconnect chip. The interconnect chips, one of which accompanies each of 12,960 AMD OpteronTM compute node processors, make it possible for Red Storm's processors to pass data to one another efficiently while applications are running. The interconnect is also key to the three-dimensional mesh that that allows 3-D representations of complex problems. The mesh interconnect is another unique feature. A simple design permits information to pass more directly from processor to processor without having to pass through many levels of processors in a complex hierarchy.
Cray's Catamount software runs the application with a user-friendly Linux system serving as the user interface.
Red Storm excels in the three High Performance Computing Challenge benchmarks that are particularly relevant to the kinds of problems it was designed to address: STREAM, PTRANS, and RANDOM.
—€ STREAM measures sustainable memory bandwidth. Red Storm's higher memory bandwidth keeps the processors from being starved for data and makes the Central Processing Unit more efficient.
—€ PTRANS is a useful measure for the total communications capacity of the internal interconnect. This high score means that data can flow freely between the 10,368 Red Storm processors without bottlenecks or congestion.
—€ RANDOM indicates the performance in moving individual data elements as opposed to long arrays of data. Red Storm has the ability coordinate the many interactions of various processors.
Red Storm has proven itself an able contender among the world's most powerful supercomputers. Recently upgraded, Red Storm was judged second fastest in the world on the TOP500 high-performance Linpack test, and best in the world in two of six new categories set up to analyze performance more accurately. The value of a resource such as Red Storm, however, is not its standing in the competition for greater computing power, but its functionality and ability to support the work for which it was built. For example, one researcher was able to do an entire year's worth of calculations in only a month using Red Storm.
Another Sandia-developed supercomputer is the Thunderbird Linux Cluster, developed in collaboration with Dell and Cisco Systems. The lab uses Thunderbird to perform a broad range of weapons simulations, including modeling of radiation effects on semiconductor electronics, assessing weapon-response safety in extreme thermal and impact environments and quantifying uncertainties in weapon performance. The level of detail being modeled in these assessments was not practical without the new level of scalable capacity provided by Thunderbird.
With its 4,480 commodity compute servers with an Infiniband message-passing interconnect, Thunderbird is the largest cluster of its type in the world and achieved sixth place on the TOP500 list of supercomputers.
The lab's HPC resources are key to the success of many ASC Program activities, which involve similarly sophisticated computing at Los Alamos and Lawrence Livermore national laboratories. The wide array of resources provided at Sandia extends far beyond the supercomputers that attract media attention to the equipment that supports the many specialized needs of researchers at Sandia. The nation's investment in high performance computing keeps the U.S. computer sector more than competitive in the world marketplace for advanced computing.
High-speed networks, cyber security and authentication services, long-term data storage, and world-class visualization capabilities all play a part in advancing the state of scientific knowledge and engineering excellence that form such a great part of Sandia's reputation.
Philip Brittenham is a technical writer assigned to the Creative Arts Department of Sandia National Laboratories.
Copyright © 2012 | Innovation America