I worked for a fintech company a while back, where regulatory compliance on explaining how significant, financially life-changing decisions by algorithms was imperative. At some point the company 'adopted AI' and I asked a question about how such an 'AI' working for the company could explain its (evolving) decision making process to a regulator.
I received no answer from the AI guys, but I suspect it would be along the lines of "Well, it all started with training data set n-458493, I thought the answer was 2, but I was told it was 3, so I adjusted one of my neurons to give 2.6 in future. Then training data set n-458492 came along, I thought the answer was 2.6, but I was told it was 1, so I adjusted two of my neurons...."
Biting the hand that feeds IT
