Why, you're no better than an 8-bit hustler: IBM punts paper on time-saving DNN-training trick

Our own vision uses it...

I heard our own eyes only use high resolution and color fidelity on the very focal center, while the rest just picks black-and-white patterns, motion detection... *while our brains fill the gaps*...

So image classification could use the same principle our eyes do...?

The same principle is already used in compressing MP3 with VBR, so why not apply it to speech recognition too?

Image recognition for specific cases will even drop the color entirely, and use black-and-white patterns for the processing... things like checking if a bottle has a cap correctly placed on it. I heard Coca-Cola will check with a camera for their specific Pantone color when printing their cans, otherwise they don't use color whatsoever. Applying the same principles to neural networks should be the next logical step.

No loss in model accuracy?

That sounds a lot like the typical advertising "There is no better <x>", which they intend us to read as "This is clearly the best" while those who stayed awake in rhetoric might discern it as "This is not really any worse than the rest of the crap".

I like the analog stuff, though. At last an explanation for occasionally wildly odd AI results. "It works just like your brain", which is so simple/obvious that even Uncle Phil can understand, after a few too many pints.

There's $<x> in those <y>

A lot of early geophysical data tapes used very weird floating point formats, and for some it was as little as 6 or 8 bits of 'precision' with wonky small exponents affixed.

It was good enough precision to find last century's oil. This century they want to find your face in a crowd. Congratulations, you're resources.

codecs

A-law on mu-law codecs use funny 8-bit floating-point formats

So if you use lousier numbers, you can even more vastly overfit patterns you don't understand without the errors getting noticeably worse.

Couldn't be because they stank already in really important ways, right?

How's that turtle==gun stuff doing these days? Everything I see complaining about issues with NN's was identified in the 90's or earlier, by people who said it would only get worse with more layers, more overfitting, and a lousier squash function (relu) with...math to prove it. Guess what. GIGO is still a thing.

Previously

When I last tried speech recognition, 8-bit audio which sounded quite clear to me did not get recognized. It had to be 16-bit, which I think is officially CD quality.