Total recog: British AI makes universal speech breakthrough

Re: Positive

Non AI based methods for this have been around for a while too.

Ages ago, worked on project where digital images from slides of cells stained with reagents where assessed by cancer specialists as to how likely to be cancerous.

Variety of tissue types and staining reagents used.

We developed algorithms (per tissue type / reagent combination) - a bit of pattern recognition, image analysis (heavily informed by what clinicians said they used as "key signs" to guide them)

These were tried on old & a new set of images, and see how well algorithms compared to clinicians on new images.

A few refining cycles and good agreement.

This was then used, not as auto diagnosis system, but in real world to flag if "automated" system and a doctor gave markedly different assessments. Then task was to see if system error or human error - flagged images scheduled for check made by multiple different clinicians.

Inspecting biopsies (and any similar is this likely to be cancerous inspection) is slow, tedious & mentally fatiguing, human error can occur (especially after several hours) - having automated systems to help can only be a good thing.

AC obv

Re: Positive

Technology in this area is moving forward and will simply be better than humans in a lot of areas and we need to deal with that.

Well, yes there is that. We need a lot more Dr Tony Robinsons in this world if we are going to create an AI that is truly beneficial to society, and not just another Googlebot or Siribot (but obviously not in Scotland or Tyneside).

Speaking of the chocolate octopus, how long will it be before Google buy Speechmatics?

Re: A.I. for P.M.

An AI would be more human....

Sure, this is pattern recognition and just part of AI.

Our brains are pattern matching machines, there is no general reasoning without abstraction of patterns, combined with rules and storage that is our 'mental model'.

Cognition and reasoning are the next stage up and there is work on that area e.g https://opencog.org/ this needs more theory but also better processing power, it mainly academics at the moment but money will flow to commercialise/privatise eventually

my impression still is that what is named AI is just pattern recognition, even if sophisticated pattern recognition

I don't think that's an accurate characterization. There's a great deal of research being done in machine learning that goes far beyond pattern recognition. You might want to have a look through the archives at the morning paper.

For example, researchers have built NLP-processing stacks that can do semantic algebra. You can pose a query to the engine like "king - man + woman" and it will respond with "queen", having learned (without supervision) to associate vectors of concepts with various words. This has also been done with images; there's a paper demonstrating a network that given an image of someone's face can synthesize an image of them wearing sunglasses - and if you give it an image of someone wearing sunglasses, it can produce an image without them.[1,2]

Broadly speaking, a lot of the work done with generative networks can't reasonably be described as pattern matching.

It's also worth noting that ML-based applications which are not, under any reasonable definition, pattern matching have been around for decades. Even calling something like document summarization "pattern matching" is a stretch.

[1] Obviously that requires some "guesswork" on the part of the network, since it's extrapolating the features it can't "see", such as eye shape and color. But it does an uncannily good job. Again, this is unsupervised; the network stack derived a model from a large corpus of facial images and learned the differences between "with sunglasses" and "without sunglasses".

[2] I've forgotten which paper did the sunglasses thing, but here's a nice TMP post on some of the interesting things being done with DNNs.

Re: Gulag Archipelago

It probably was translated to English from Russian.

I must get the newer version of First Circle. The difference between that establishment and certain US companies, is that you can leave. Though they might sue you afterwards. They will insist they own all your ideas. On the plus side they don't physically torture or shoot you for "wrong-think".

Re: So how does it work?

I don't think so. Viterbi decoders (someone correct me if I'm wrong here) are basically MLE (Maximum Likelihood Estimation) processes for Markov models (HMMs or MEMMs or otherwise).

This article (and some other bits I've skimmed about SpeechMatics) mentions deep recurrent neural networks. Like a Markov model, a neural network has a bunch of hidden nodes with weighted edges between states; but where an MM's edges represent probabilities of transitions between states, the edges in a NN represent signaling thresholds. So with a NN you can have multiple nodes "active" at any step, whereas an MM has only one current state.

Then add back-propagation (the bit Hinton's talking about), which is what makes it a recurrent neural network, and you have the weights on the edges constantly changing - as opposed to the typical MM which is trained once and static from then on.

The Viterbi algorithm is basically a tweak to the Forward algorithm - where the latter finds the total probability of a given output, Viterbi finds the most likely output (given current visible state, hidden state, and input vector). I think in theory it should be possible to convert any RNN to a MM (and thus to a Viterbi decoder), but you'd have a combinatorial increase in in the number of states.

Viterbi decoders are still used in ML, for example in the simpler stages of natural language processing such as part-of-speech identification (for many applications, not universally). But there's too much ambiguity in natural language to do much NLP with them. A lot of simple sentiment analysis used to be done with HMMs or MEMMs, for example, but it can't cope with sarcasm, references to other subjects, etc. (There was a good paper in CACM a few years ago about a new approach to implementing Rhetorical Structure Theory for sentiment analysis that demonstrates some of the issues.)

One of the things that's interesting to me about SpeechMatics is that they're using a deep RNN stack rather than deep convolutional neural nets. CNNs were the Next Big Thing for a while, with e.g. Google pushing them heavily. (Extremely simplified: CNNs use a convolution layer - basically a signal-shape matcher - rather than back-prop.) But there's so much research in ML that deep-learning stack architecture has become really complex, with dozens of layers that mix various kinds of networks with mixing layers and such.

And now we have things like generative adversarial networks, where one network produces a stew of real and forged data, and another network tries to distinguish the two, and they learn from each other. Interesting stuff.

Re: Looking good!

Yes it can definitely say that’s language that is