Not seeing the problem here.
I'm not seeing the problem here at all.
When you get right down to it, Cloud = (usually large scale) server farm + hype. Any "cloud" improvements should amount to being able to move VMs among machines better & more easily, use pools of storage better & more easily, and deploy VMs more easily, all things that are good to have even if you don't consider your physical servers you are running VMs on to be a "cloud".
vGPU itself is admittedly pretty useless for servers, but also doesn't take away from everything else being developed. But, any improvements in latency that may be being done with an eye towards desktop virtualization will still reduce the latency on server VMs as well.