The cloud is great for HPC: Discuss High-performance computing (HPC) environments are expensive. Government research facilities and commercial laboratories spend hundreds of thousands building out large, monolithic supercomputers and then jealously guard their compute cycles. This approach to HPC is restrictive. It creates a rarified environment in which only the …
It would have been really nice if the warning disclaimer was at the top of the article, not at the very end - although something did start to smell fishy about 1/4-way through the text already.
Obviously, one could use the cloud for HPC. For some situations, mostly involving very spiky demand and very granular budget controls, it even makes economic sense to use it. In many other cases, it doesn't.
For an academic researcher, the killer argument against putting all your eggs in the cloud basket is very simple: what happens when my current grant runs out, and I spent all my money on the cloud? To have a realistic chance of getting another grant I need to show some preliminary results. To do that, I need some facility - and cloud needs paying up front, while with an in-house facility it is usually possibly to defer the running cost for a while (though obviously not forever). What happens to my data if it is in the cloud, and my grant runs out so that I can no longer pay the ransom upkeep money? What happens with the commercially-licensed software I might need, which is typically much more expensive for off-premises use?
The bottom line is that yes, there are some very important HPC niches the cloud could fill, but that's what they are: the niches. Or, as Sir Terry would put it, the nitches.
Oh Dear,
Sounds as if they've realised the pitfalls of over provisioning, I guess capacity planning went straight out of the window when building these facilities, though don't despair, a quick conversion into luxury apartments powered by Alexa should do the trick ;-)
Consider how AWS behaves relative to an HPC application compiled with "-xAVX" on a SandyBridge or later AWS node (this causes the compiler to use the "AVX" vector instruction set in building the executable program). Here are some notes from my own experience:
When running "on the bare metal", well-written CFD applications typically see an 80% or better performance boost. The WRF weather model only sees about a 30% improvement with "-xAVX", showing that the design and/or coding is rather sloppy.
However, on AWS, WRF compiled with or without "-xAVX" causes virtually identical performance (within 2%) -- performance that is significantly worse than the "bare metal" performance, by the way. This indicates that the virtualization is killing the potential improvement from the more powerful processor and its (more-powerful) instruction set.
FWIW.
"You may want to try throwing a set of GPUs at a particular computational fluid dynamics problem to see if that architecture can handle the workload in a more effective way. If it doesn’t deliver the gains you expected, then you haven’t sunk capital into a hardware investment"
Yada, yada, yada. I take the point, effectively renting unusual hardware, but GPUs are a poor example, in academia (if not Industry) they have moved into the mainstream and any self respecting central University service will offer nodes including them. So how about FPGAs or Xeon Phi's? You offer them, great ... how much ?!?
The cloud being of use to academic HPC currently is a myth. High throughput computing, yes there is a place. But at the moment not HPC.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
