Re: I hate Agile as well
In and of itself it doesn't, and frankly the "traditional" VMware-and-SAN architecture gives a much, much better experience in that respect. Tools are better, functionality is more extensive, skilled people are far easier to find.
The scalability side isn't just smaller scaling units (i.e. one cheap machine vs one massive array), it's that the scaling tends to be in all dimensions simultaneously. You buy compute, storage and bandwidth as a unit because, well, frankly, they are a single unit. This means they should, in principle, scale together. It also means that, again in principle, you can reduce those 3 functions down to 1. No more running a procurement on networks, compute and storage to get some apps going. It's all one "hyper-converged" system.
It's many of the same ideas that underpin the Hadoop ecosystem, just applied to more general purpose workloads. Frankly none of the tools are quite there yet, and I suspect whatever becomes the market leader will come out of the K8s/Docker ecosystem rather than some hot new startup with proprietary tech.
The big elephant in the room with "HCI" is that they're trying to present a similar experience to the entirely abstract VMware-on-SAN experience, where broadly every machine is equal because you're hitting the same storage array with the same random profile from the same compute array over the same (probably highly constrained) network. HCI kit is going to end up fragmented and specialised (wot, GPUs on all your machines?) so the abstraction becomes leaky, and this makes scheduling and provisioning much more complicated. No one's nailed that yet.