Re: RAM
"The reason accessing GPU memory over a PCIe bus (which is faster than TB4 though still far slower than the M1 Mac's memory bandwidth, and still quite problematic for latency) isn't a problem for GPGPU because when the CPU needs to access GPU memory it isn't latency sensitive"...
And we've never had multiple levels of cache, memory, disk - tiered storage works for memory as well as archives.
The memory on the M1 is stupid fast, clearly no comparison with anything external - but even DDR4 3200 was only 25GB/s, well under the speed available with TB4.
I'd love to have some idea of the intrinsic latency of TB4, but google just shows conversations about audio interfaces, and the latency is so far below the limits of human audio perception that it's been basically irrelevant for many years now. Memory is certainly not something you'd want to run over many metres of fibre based TB (given that we're adding at least* two nanoseconds per foot), but I expect it to provide a pretty good "next best" when mounted close (basically directly attached).
Again, clearly the onboard memory will have lower latency, but that's not necessarily the only relevant benchmark - and slightly slower, but reusable, memory is something that could reasonably catch on - particularly because we now have an interface that is basically universal. A laptop with some extra memory when docked...
* At least - it depends on the medium, but probably closer to 3 because the speed of light is slower in a fibre than in free space, and the signal gets to go both ways.