SpyderByte.com ;Technical Portals 
 The #1 Site for News & Information Related to Linux High Performance Technical Computing, Linux High Availability and Linux Parallel Clustering
Home About News Archives Contribute News, Articles, Press Releases Mobile Edition Contact Advertising/Sponsorship Search Privacy
Research and Services
Cluster Quoter (HPC Cluster RFQ)
Hardware Vendors
Software Vendors
HPC Consultants
Training Vendors
Latest News
News Archives
Search Archives
Featured Articles
Cluster Builder
User Groups
Golden Eggs (Configuration Diagrams)
Linux HPC News Update
Stay current on Linux related HPC news, events and information.
LinuxHPC Newsletter

Other Mailing Lists:
Linux High Availability
Beowulf Mailing List
Gelato.org (Linux Itanium)

Linux HPC Links
Cluster Monkey (Doug Eadline, et al)
HPCWire (Tabor Communications)
Scalability.org (Dr. Joe Landman)

Beowulf Users Group
High Performance Computing Clusters
Thinking Parallel
The Aggregate
Cluster Computing Info Centre
Coyote Gultch
Robert Brown's Beowulf Page
FM.net: Scientific/Engineering
HPC User Forum
Mobile Edition

Linux Cluster RFQ Form
Reach Multiple Vendors With One Linux Cluster RFQ Form. Save time and effort, let LinuxHPC.org do all the leg work for you free of charge. Request A Quote...

Latest News

Low Fees, No Fluster, With Today's Linux Clusters
Posted by Terry Shannon, Wednesday April 30 2003 @ 04:45PM EDT

HPTC Computational Horsepower at Popular Prices is Here Today

Twenty years ago, VAXclusters were all the rage at Fermilabs, Los Alamos National Labs, a number of US Department of Defense three-letter agencies, not to mention a goodly number of large corporations requiring mass quantities of computational horsepower with excellent scalability and reliability.

But the times, they have a-changed. For a variety of reasons, not the least of which was marketing malfeasance of Digital and the rise of Sun Microsystems, VMS found itself edged out of the HPTC market. One by one, VMS systems were displaced by UNIX systems in commercial and government markets alike. DEC's AltaVista, the first --and in its day, the best--search engine on the planet--was powered by Digital UNIX. Celera Genomics relied on Digital TruCluster technology until IBM came in and made Celera an offer it couldn't refuse: free top-of-the Regatta boxes running AIX. Amazon.com, an erstwhile Digital UNIX marquee account, ditched its enterprise-class AlphaServers and went with HP enterprise servers. And eBay, for better or worse, runs on Sun UE10K high-end enterprise servers using (when they are up and running) the Solaris OS.

Enter UNIX, Exit VMS

Digital, even after it was Compaq-ted, adapted to the trend and begin promoting Alpha and Tru64 UNIX and TruCluster technology for HPTC apps. Thanks to Wes Melling and a coterie of marketing geniuses at DEC and Compaq, VMS was relegated to "crazy aunt in the basement" status. For a time, Tru64 and Alpha fared well in the HPTC space. So well that Compaq displaced HP as the Numero Uno HPTC vendor in terms of market share. Big Wins at the ASCI-Q project at Los Alamos National Labs, APAC and VPAC in Australia (where Compaq displaced Sun, whose $1M USD UltraSparc II-based UE10K enterprise servers couldn't stay up and running long enough to pass initial acceptance tests thanks to issues with zinc whiskers, cosmic rays, and bad karma), commercial database and CRM behemoth Acxiom, helped Compaq achieve Number One status. Not surprisingly, the fastest processor on the planet, the best clustering in town, and the most cluster-centric UNIX in the industry combined to create this success story--a story so successful that Oracle's Larry Ellison licensed key components of Ttru64 UNIX and TruCluster technology to beget Oracle 9i RAC (Real Application Clusters), otherwise known as Oracle Parallel Server Done Right.

HP, Intel, and Compaq Converge on HPTC and Linux

Meanwhile, HP was marshalling its HPTC resources with HP-UX enhancements and the Superdome enterprise server, which proved to be a high-end server endowed with plenty of whoop-ass once the TPC benchmarks were run properly on the initially lackluster machine. IBM expanded its niche in the HPTC realm, trotting out SP2 machines that could not only dethrone the world's best human chess player, but also could play a leading role in commercial and defense applications. And microprocessor maven Intel Corporation made no bones about the fact that it had its eye on the HPTC prize as well.

Nearly two years ago, Compaq bit the bullet and made what turned out to be the correct decision by consigning Alpha to early retirement and replacing the 64-bit chip with Intel's brand new, low-performance Itanium I processor (not to be confused with the very capable Itanium 2 "McKinley" processor. A great hue and cry, and much wailing and whining erupted in Usenet newsgroups, immediately erupted. This bothered Compaq and Intel not in the slightest, as those who whine the most, buy the least. By opting to take Alpha out of the running in the 64-bit microprocessor horserace, Compaq was able to get much closer to Intel in the HPTC arena. Over time, this will prove to be a major coup for both firms.

Out of the Cathedral and Into the Bazaar

Meanwhile, a sea change was occurring in the HPTC community. The faithful began a slow but rapidly increasing migration from the cathedral to the bazaar, thanks to the unexpectedly high growth rate of Linux, the development of Beowulf clustering technology for Linux (which I will discuss in a follow-up article), and the emergence of low-cost, high-bandwidth cluster-ready interconnects like Myrinet as well as some of the forthcoming economy-class fabric interconnects. (Thanks to its obsession with limited-functionality "Wolfpack" clustering, Microsoft missed the boat on this phenomenon in a big way!)

A sea change was occurring in the way customers used computers, too. Genomics,. Proteomics, protein folding, meteorological modeling, and oil and gas exploration initiatives heretofore impossible due to limits in computational horsepower became viable. And the nuclear winter aftermath of the dot.com bombing of Y2K rendered incumbent HPTC approaches unaffordable to a significant number of potential adopters.

Not Quite "Cycles For Nothing and MIPS For Free"... Yet

The migration from the cathedral to the bazaar changed all that. Today, the cost of a pair of gas-guzzling SUVs--about $100K USD--will get you a cluster of Linux-based PCs that'll give a $1M USD enterprise server a run for its money. And unlike Windows NT and its successors--which have been on the scene for about as long Mister Torvaalds OS Opus- Linux has earned its stripes as a stable, scalable, and reliable OS that will run on a multitude of platforms ranging from an iPAQ handheld to a Superdome or Marvel or Regatta enterprise server. Of greatest interest to the HPTC on a shoestring community is Intel-based hardware, which often sells for less than the commercial applications that run on the Intel-Inside boxes in question.

Complementing a price-and-performance "offer you can't refuse," a legion of unpaid Open Source developers, and a rapidly maturing Open Source UNIX clone, potential adopters are at the cusp of acquiring cycles for nothing, and MIPS for free. Add to this the proliferation of capable commercial Linux system and cluster management tools, and Linux is ready for prime time in the life sciences, HPTC, and even some sectors of the commercial marketplace. Is it any wonder that HP a year ago changed its Intel-Inside ProLiant tagline from "Industry-Standard ProLiant Servers Running Windows" to "Industry Standard ProLiant Servers Running Windows AND LINUX?"

Not Quite Ready For Prime Time.. Yet

To be sure, Linux in mid-2003 has yet to achieve full-fledged bet-your-business operating environment status. The OS remains at Ground Zero in a free-for-all, Open Source is incompatible with a Linux Czar or even a benevolent despot (although early Linux proponent Jon "maddog" Hall can arguably be called a benevolent Linux canine), hence the OS lacks lacks a single, authoritative, internationally-recognized commercial support and marketing organization. (HP, IBM, Red Hat and others are working on this issue, and in HP's case the point men are Shane Robison and Martin Fink). Linux has yet to make the grade in parallel file processing, job scheduling, and performance analysis tools and suites, but these products are waiting in the wings.

As for security--an issue near and not so dear to the hearts of Microsoft OS users--Linux is not yet running nuclear reactors nor does it serve as the OS playbook for Armageddon in the military environment, but it should be noted that the US National Security Agency is developing SELinux, or Security Enhanced Linux, to render the OS capable for mission-critical national defense applications. (A former employee of the NSA's Army subsidiary, this author puts much greater faith in the NSA's project than in Microsoft's "Security is Job One" platitudes!)

Linux Cluster Growth: No Limits

Not surprisingly, the rough edges on a decade-old OS are not proving to be a showstopper in the HPTC, life sciences, and pharmaceutical markets. Linux may not be nirvana, but being obtaining an audience with the CEO and asking for a few million dollars for name-brand hardware is tantamount to entering Dante's sixth circle of Hell. Market research indicates that four out of five major pharmaceutical companies and almost two-thirds of life science companies rely on clustered and distributed computing platforms, many of which are Linux-based. And those who have opted for Linux are reaping substantial economic rewards. Rivals and wannabes have taken notice: Framingham, MA, USA-based market researcher and box-counter IDC predicts the cluster market will grow ~35 percent per annum to over $4B USD by 2005 and that Linux will become the dominant choice in the cluster environment, growing from ~$225M USD two years ago to nearly $1.5B USD by 2005. A billion five could be what Microsoft makes in less than a week, but it certainly isn't chump change!

In the second portion of this article, I'll step through Beowulf clustering, hardware-specific issues (yes, you can run Linux on an Intel 386 box if you can find one, or even on your iPAQ PDA... try that with Solaris or MVS!), and management and tool issues. Stay tuned!

(c) 2003 by Terry C. Shannon, IT Consultant and Publisher, Shannon Knows HPC


Want ALL the details on Itanium2, AlphaServer and Superdomehigh performance computing? Easy: subscribe to SKHPC. Send an email to terry@shannonknowshpc.com for subscription information.

< Event Reminder: HPCS 2003 starting May 14th | c|net: What's 64-bit computing to Linux? >




Cluster Monkey

Golden Eggs
(HP Visual Diagram and Config Guides)
CP4000 32x DL145G2 GigE Opteron, Dual Core
CP4000 64x DL145 GigE Opteron
CP4000 102x DL145 GigE Opteron
CP4000 32x DL145 Myri Opteron
Rocks Cluster 16-22 DL145 Opteron
Rocks Cluster 30-46 DL145 Opteron
Rocks Cluster 64-84 DL145 Opteron
LC3000 GigaE 24-36 DL145 Opteron
LC3000 Myri 16-32x DL145 Opteron
LC3000 GigaE 16-22x DL145 Opteron
LC2000 GigaE 16-22x DL360G3 Xeon
DL365 System 2600Mhz 2P 1U Opteron Dual Core
DL360 G5 System 3000Mhz 2P 1U EM64T Dual/Quad Core
DL385 G2 2600Mhz 2P Opteron Dual Core
DL380 G5 3000Mhz 2P EM64T Dual/Quad Core
DL140 3060MHz 2P IA32
DL140 G2 3600MHz 2P EM64T
DL145 2600MHz 2P Opteron
DL145 G2 2600MHz 2P Opteron Dual Core
DL360 G4 3400MHz 2P EM64T
DL360 G4p 3800MHz 2P EM64T
DL380 G4 3800MHz 2P EM64T
DL385 2800MHz 2P Opteron Dual Core
DL560 3000MHz 4P IA32
DL580 G3 3330MHz 4P EM64T
DL585 2800MHz 4P Opteron Dual Core
Montecito 2P-16P, rx2660-rx8640 (multi-system diagram)
rx2660 1600MHz 2P 2U Montecito Systems and Cluster
rx6600 1600MHz 4P 7U Single & Cluster
rx3600 1600MHz 2P 4U Single & Cluster
rx2620 1600MHz 2P 2U Single & Cluster
Superdome 64P base configuration
Integrity Family Portrait (rx1620 thru rx8620), IA64
rx1620 1600MHz 2P MSA1000 Cluster IA64
rx2620 1600MHz 2P MSA1000 Cluster IA64
rx4640 1600MHz 4P MSA1000 Cluster IA64
rx7620 1600MHz 8P 10U Systems and MSA1000 Cluster
rx8620 1600MHz 16P 17U Systems and MSA1000 Cluster
MSA30-MI Dual SCSI Cluster, rx3600, rx6600 and rx2660
MSA30-MI Dual SCSI Cluster, rx1620...rx4640
MSA1500 48TB, SCSI and SATA
Dual Core AMD64 and EM64T systems with MSA1500

Appro: Enterprise and High Performance Computing Whitepapers
Is Your HPC Cluster Ready for Multi-core Processors?:
Multi-core processors bring new challenges and opportunities for the HPC cluster. Get a first look at utilizing these processors and strategies for better performance.

Accelerating Results through Innovation:
Achieve maximum compute power and efficiency with Appro Cluster Solutions. Our highly scalable clusters are designed to seamlessly integrate with existing high performance, scientific, technical, and commercial computing environments.
Keeping Your Cool in the Data Center:
Rethinking IT architecture and infrastructure is not a simple job. This whitepaper helps IT managers overcome challenges with thermal, power, and system management.

Unlocking the Value of IT with Appro HyperBlade:
A fully integrated cluster combining advantages of blade and rack-mount servers for a flexible, modular, scalable architecture designed for Enterprise and HPC applications.
AMD Opteron-based products | Intel Xeon-based products

Hewlett-Packard: Linux High Performance Computing Whitepapers
Unified Cluster Portfolio:
A comprehensive, modular package of tested and pre-configured hardware, software and services for scalable computation, data management and visualization.

Your Fast Track to Cluster Deployment:
Designed to enable faster ordering and configuration, shorter delivery times and increased savings. Customers can select from a menu of popular cluster components, which are then factory assembled into pre-defined configurations with optional software installation.
Message Passing Interface library (HP-MPI):
A high performance and production quality implementation of the Message-Passing Interface (MPI) standard for HP servers and workstations.

Cluster Platform Express:
Cluster Platform Express comes straight to you, factory assembled and available with pre-installed software for cluster management, and ready for deployment.
AMD Opteron-based ProLiant nodes | Intel Xeon-based ProLiant nodes

Home About News Archives Contribute News, Articles, Press Releases Mobile Edition Contact Advertising/Sponsorship Search Privacy
     Copyright © 2001-2007 LinuxHPC.org
Linux is a trademark of Linus Torvalds
All other trademarks are those of their owners.
  SpyderByte.com ;Technical Portals