Real world data set with accents?
As local accents die out; as a population we're becoming more gutter 'Essex'. Then due to uniformity, the data set can shrink, and the accuracy of translation should go up. I hope.
"Yent nevr gunna git Rowl - Yent!" trans. "I don't think that you will ever understand the Rothwell based accent". Source: Rothwell & Desborough, Northamptonshire very local accent. (So local that even BBC Radio Northampton, based 15 miles away, took the p*ss out of it!)
In the 1970's and 80's when I went to the local school, it would take newly imported teachers about a month to understand us properly. The only careers advice I was ever given was "get some elocution lessons, if you want to work more than 5 miles away!"
Sadly now a very diluted accent as the town has grown with outsiders and the youth has learnt to speak from the TV and Internet rather than talking with their parents.