Ghost of DEC Alpha is why Windows is rubbish at file compression

Re: Obvious bull

Let's not confuse algorithm and file format. The language used seems very loose to me. The algorithms are simply the methodology taken to transform one byte stream to another. It stands to reason that different architectures will be better at some algorithms than others because of the various sizes of caches and buses involved. Some lend themselves to larger dictionaries and better parallelism than others. There's no reason other than priorities as to why they haven't switched to something more suited to x86 in newer versions.

Re: Obvious bull

So one scenario is file is created on a x86 box and then updated on an Alpha box, So the Alpha system has to be able to compress in the same way, while still meeting to meet the performance criteria.

Why would that matter? If the OS is doing the compression then it will present the compressed file to any external programs as uncompressed. Another box accessing the file over the network would see it as uncompressed (network compression technology is different). And this still doesn't really back up the fairly flimsy assertion that Alpha's weren't up to the job. I certainly don't remember this being an issue when the architectures were being compared back in the day. It's certainly not a RISC/CISC issue. At least not if the algorithm is being correctly implemented. But I seem to remember that many of the performance "improvements" (display, networking, printing) in NT 4 were done especially for x86 chips which suck at context-switching. Coincidentally NT 4 was seen as MS abandoning any pretence of writing a single OS for many different architectures.

I'm not a whizz at compression but I think the common approach now is to use a container approach as opposed to trying to manage compression on individual files in place.

Re: Obvious bull

Well, uh. The Alpha (like the Itanium) was a pure 64 bit chip. There was no such thing as a 32 bit Alpha. A 32 bit build of NT for the Alpha doing the equivalent of the x32 ABI, maybe...

Re: Obvious bull

Maybe I am wrong but I recall original NT being for... i98x cpus or something like that (not x86 and not alpha). X86 alpha mips ppc were added later.

I used NT 3.51 and 4 on my desktop(x86) for a few years before switching to linux in 1998.

The article doesn't seem to mention who actually uses ntfs compression. I've only seen it used in cases of emergency where you need some quick disk space. I seem to recall patch rollbacks are stored compressed too (I always tell explorer to show compressed in different color)

Even back when I had NT4 servers 15 years ago, never used compression on em. Saw a spam recently for Diskeeper, brought back some memories.

Re: Obvious bull

"I mean why does Windows Explorer take 30 minutes to unzip a file that 7Zip can manage in 30 seconds??"

And why does it unzip first into a temp folder then copy into the destination? What's wrong with putting it straight into the target? Saves space and time and avoids yet more crud in the temp folder.

Re: Obvious bull

I think that's a point of confusion here. Chen was talking about "file system compression" but the article repeatedly says "file compression" which is a very different beast. As written, it makes it sound as if LZMA may only work on some platforms which is silly.

Re: What's cheap

Spinning rust is cheap, network bandwidth may not be, SSD certainly isn't cheap.

One recent Windows clever idea is to supply the entire operating system pre-compressed. Much space saved.

Your monthly patches, however, aren't compressed. So the disk fills up with operating system files anyway.

Maybe they will get around that by reinstalling the entire operating system from time to time, but calling it an update. Or maybe they already have.

Re: Obvious bull

Read the first article of that series (there's a link in the Chen post). On-the-fly file/filesystem compression needs to works differently from an "archive" format like zip. You have time constraints, and you need to be able to read/write blocks of the file without the need to decompress and then re-compress it fully. Try random access to a zip (or rar) compressed file...

Re: Obvious bull

Yes, that must be why I can't use Zip or Rar files on a different PC than the one that created them.

Stop using floppy disks.

Re: Obvious bull

@patrickstar:

Funny how none of the assume-everything-MS-does-is-crap crowd has actually read the article.(...)

This is not a design decision you can reasonably comment on without having a full view of the actual, intended, and possible uses of NT on various architectures in various configurations at that time - which you don't, and MS presumably did.

Well, assume everything MS does is crap and you arrive at the conclusion that they also didn't have that full view.

Re: bottleneck in the memory access pathways.

"the first Alpha chips also had a bottleneck in the memory access pathways. Once the data and code had been loaded into cache it went blindingly fast (for the time),"

Citation welcome. See e.g. McCalpin streams memory performance benchmark(s) from the era.

Bear in mind also that in general, early Alpha system designs (with the exception of 21066/21068, see below) could be based on 64bit-wide path to memory, or 128bit. 128bit was generally faster. The presence or absence of ECC in the main memory design could also affect memory performance.

You *may* be thinking of the 21066/21068 chips. These were almost what would now be called a "system on chip" - a 21064 first-generation Alpha core (as you say, blindingly fast for its time), and pretty much everything else needed for a PC of the era ("northbridge","southbridge", junk IO), all on one passively-coolable 166MHz or 233MHz chip. Just add DRAM. Even included on-chip VGA. In 1994. I'll say that again: in 1994.

Unfortunately it had a seriously bandwidth-constrained DRAM interface, which was a shame.

The 21066/21068 were used in a couple of models of DEC VMEbus boards and the DEC Multia ultra-small Windows NT desktop, which was later sold as the "universal desktop box", because it could run NT (supported), OpenVMS (worked but unsupported), or Linux (customer chooses supported or not). The 21066/21068 weren't used in any close-to-mainstream Alpha systems, not least because of the performance issues.

Alternatively, someone may be (mis?)remembering that early Alpha chips also didn't directly support byte write instructions and the associated memory interface logic, which meant that modifying one byte in a word was a read/modify/rewrite operation. I wonder if this is confusing the picture here.

The compilers knew how to hide this byte-size operation, but in code that used a lot of byte writes, the impact was sometimes visible, which was why hardware support was added quite quickly to the next generation of Alpha designs, and DEC's own compilers were changed to make the hardware support accessible).

As for Mr Chen's original Alpha-related comments: (a) it's hearsay (b) it's somewhat short of logic (at least re hardware constraints), and even shorter on hardware facts.

References include:

Alpha Architecture Handbook (generic architecture handbook, freely downloadable) see e,g,

https://www.cs.arizona.edu/projects/alto/Doc/local/alphahb2.pdf

Alpha Architecture Reference Manuals (2nd edition or later, if you want the byte-oriented stuff)

Digital Technical Journal article on the 21066:

http://www.hpl.hp.com/hpjournal/dtj/vol6num1/vol6num1art5.pdf

Re: The biggest problem with the alpha chip was yield

@cageordie: It depends. These days everyone sort of expects a new node to just work and to get 20%+ yields even from the first wafers. And 35% isn't bad as far as yields go.

Plus, even cutting edge process technology doesn't always mean starting with nothing, although it depends on the specific design.

Compare AMD's 4770 and Evergreen series to Nvidia's Fermi.

AMD first did a midrange chip to learn and understand TSMC's 40 nm process node and got good yields before going for larger chips.

Nvidia decided to go for a large chip as their first 40 nm design and got lousy yield, basically all of their GF100 chips effectively yielded 0% because they needed to fuse off portions of each manufactured chip. Starting with a large chip meant it was harder for them to understand the process, and the problem repeated when working on smaller GF104/106/108 chips. It still didn't bring them down at all.

Re: The biggest problem with the alpha chip was yield

"That's just how development goes at the cutting edge, and that's what the Alpha once was."

Correct. As I mentioned earlier, the 21066/21068 "Low Cost Alpha" chips had a disappointing memory interface performance. One of the reaons for that was packaging, Chip and board designers hadn't got round to surface mount chips with high pin density (this was in the early Pentium era) and to keep costs down the package had to be small, which kept the number of pins unrealistically low.

Today if someone were to try a similar trick with SMD packaging, there'd be no priblem. Heck every smartphone for years has used SMD packaging on its SOC. But back then it wasn't an option.

Re: So why not create a new v2 compression scheme?

and not to forget the eye watering cost of hard drives in those days as well

Re: So why not create a new v2 compression scheme?

"You queue up the I/O and wait for DMA to deliver the data and then re-schedule the thread. Meanwhile, there's 101 other things the CPU can be doing"

For an interactive system "wait" means poor user experience.

Processor performance has improved much faster than disc IO. A modern system spends huge amounts of time hammering the disc - that's why we all upgrade to SSDs whenever we get the chance.

Re: So why not create a new v2 compression scheme?

Both factors meant that the CPU was probably kicking its heels whilst the I/O happened, so reading less data and burning the wasted cycles on decompression was a win.

This was, of course, true of Windows up to at least XP.

It was almost certainly not the case with VMS (I used it, but was not familiar with the code). However, Unix was around from before 1978 (when I first met it) and most definitely could do proper DMA transfers and multi-threading of tasks where the hardware permitted. I know: I wrote tape and disk drivers for it (in the 80's).

And I am bloody sure that Alpha assembler could do bit twiddling with little more pain that x86. I have written assembler for both. C compilers might have been less good at it.

The Alpha architecture allows you to write and load microcode routines at run time - so you could implement the bit twiddling instructions yourself and load them when your program loaded. Great for implementing database joins, etc. Of course, you have to know what you are doing to write microcode. This might be the real problem.

Re: So why not create a new v2 compression scheme?

"...the disc access probably wasn't a simple case of "send a few bytes to the controller and wait"..."

Ah, yes, the joys of "Programmed IO" (PIO) -- "rep insb", "rep insw", "rep outsb" and "rep outsw"; your single threaded processor would be tied up shunting sectors through a port, byte by byte.