Re: POWER8 disappoints
@MadMike
I have done comparisons between Oracle Mx and Tx chips and IBM Power x chips. I've also looked at the server design and resilience etc.etc. Been doing it for a while. Oracle chips do well in benchmarks, but it simply doesn't work through to reality, except for some limited applications. If you want to run huge instances (one of the reasons why Oracle push Solaris containers rather than LDOMs) doing a single workload (such as a huge BI machine), you might be able to make use of the performance. However, if you want to do more normal workload (such as OLTP), using a lot of partitions (or LDOMs), performance falls away rapidly. Cache size is one reason, but there are others. Using a lot of LDOMs, containers (to a lesser extent) and really working the threads up (again to a lesser extent) causes cache thrasing as the cache simply isn't big enough for the speed of the cores. Power chips have far fewer problems in this area and have much larger cache sizes which is one of the reasons.
If you want to run large numbers of partitions (or LDOMs) or generally anything that switches between threads etc. a lot, the Power chips do much better in real life. Yes, the benchmark figures are good, but don't translate into real life performance under many circumstances. We, run Power servers with VP to PP ratios of up to 10 to 1 with really good performance. Tx and Mx chips simply won't do this. It's been a well known problem since the beginning of these chip lines. The Mx chips lost cores in order to increase the cache amount for each core specifically to try and address this problem. And it worked, to a point.
The M7 chip design looks more like a T7 design, primarily due to the very large core count and low cache quantities per core. If it had been launched as a T7, I would not have been surprised at all and would have expected to see a M7 launched slightly later with fewer cores and bigger cache per core. But, that isn't what Oracle have done.
Also, have a look at the server designs and their resilience etc. You find an interesting story. I was very surprised to find out some time ago that a T3-2 server would DELIBERATELY reboot itself if a socket failed!! That's not resilience. This was to reconfigure all the I/O onto the remaining socket. However, if you design the implementation correctly, all I/O would be mirrored across the two sockets anyway, so I/O would be maintained. It actually rather seems like Oracle are putting resilience more into their software stack and not their hardware. In the event of a hardware fault, they expect to loose the hardware and simply failover to another instance through software.
The above is one solution to a problem, but it should always bee borne in mind that software is normally the least reliable part of the stack and therefore deliberately using that for resilience is arguably not the best. As a last resort, fine, but hardware surviving faults is a good starting point first.