Revealed: Blueprints to Google's AI FPU aka the Tensor Processing Unit

Not so fast?

""The comparison doesn't look quite so rosy next to the current-gen Tesla P40 GPU, which advertises 47 INT8 TOP/s at 250W TDP; compared to the P40, the TPU is about 1.9x faster and 6.5x more energy-efficient," Johnson wrote."

Which is still a pretty decent margin really. Given that Google has been actually using these things since 2015, while the P40 was only announced a few months ago, that's still a pretty favourable comparison. Compared to contemporary competition, the TPU is way ahead; compared to modern competition, the TPU is still way ahead, just not by quite so much.

Was Myles Dyson on the development team?

Bah!

Yesyesyesyes, quantizing the whoozit, gottit.

But the important question remains: whose engrams were used?

So about par with the (soon I hope) new Parrallella chip.

Ish . Possibly. Who knows - it should be back from fab soon?

NVIDIA FTW?

Though I have great hopes for ASICs like the TPU, and for the many FPGA-based ANN accelerators, as well as for upcoming 1K+ core designs like Adapteva's, the bottom line is support for the common ANN packages and OpenCL.

In that regard, the GPU will reign supreme until one of the other hardware solutions achieves broad support. Only AMD and NVIDIA GPUs provide "serious" OpenCL support, and between the two, NVIDIA is preferred in the ANN world.

An important previously-mentioned caveat is that most of the ASIC and FPGA accelerators aren't much help during training. For now, you're still going to need CPU/GPU-based ANN support.

Can it do anything else?

I mean, it sounds just great for people who have a workload that is 99% machine learning according to the particular algorithm for ML that Google happen to be using (*), but just as GPUs look great but can't do anything that isn't embarrassingly parallel, so the TPU looks great but is even more specialised. (* Specifically, the one they were using back in the distant past when the chip was designed. I doubt the algorithm designers have sat on their laurels since then.)

It reminds me of the dedicated hardware (in non-server CPUs) for AVC and HEVC which are many times faster than using either GPU or CPU for the same job, but I'm not aware of anyone managing to turn those to any other task. (At least those algorithms have been adopted as standards and consequently have something of a shelf-life to justify baking them into the chip.)

Worse, for Google, if it is only a small integer factor faster than a GPU then it will face pretty stiff competition from FPGA-on-chip if and when that gets some traction from OS and application writers. An FPGA is dedicated hardware that you can change when you think of a new algorithm.

Major thing there... price.

A top of the line Tesla is going to run you $4,000+ MSRP. If going in house to develop their own ASIC saves them money at the order of magnitude they need (thousands of units)... then by all means do it. Maybe Nvidia might be forced to lower the price of their Tesla / Quadro cards to actually earn a place in these large data centers again.

Re. price

There was I thinking that £350 for a used GTX1080 card was cheap.

Seems that folks using these for Bitcoin mining realized early on that custom hardware was better, however the cards themselves have many other uses such as medical imaging, machine learning, data analysis, password bruteforcing, etc.

Serious gamers moved on a long time ago but for certain applications (eg machine learning) an array of cheap used cards on budget boards (eg quad core AM3+) will still do more work than one £xpen$ive card when suitably retrofitted with better cooling and updated thermal management such as heat pipes just for the RAM not to mention the latest software custom designed to squeeze every last IOPS out of the overclocked GPUs.

I once did this to my not-2-day-old RS480 and got about a 11% performance boost with no overheating.

Some other folks are leveraging used laptop boards with on-chip GPUs as they are quite compact but this approach is significantly more complicated due to limitations in clock speeds with a laptop.

Even replacing the base clock does not help here as the limit is in the interconnects.