Dell soups up low-end Data Domain deduper Down in the Dell there is a new Data Domain box – a smallish, entry-level one – the DD3300. It is a 2U rackmount chassis with 3TB or 4TB SAS disk drives providing up to 32TB of usable capacity, said to be 1.6PB after deduplication (logical capacity) and data can be ingested at 7TB/hour using Data Domain Boost. If a backend …
I would be curious to know what software technology Dell used for this Storage technology, since it is generally understood that ZFS is the indisputable OS File System, and large capacity storage software like Ceph are the standard, while Dell is predominantly a Windows software only vendor.
‘Indisputable’, only in circles not familiar with the alternatives. Isilon’s OneFS, Nimble’s CASL and Nutanix are just three of the file systems that would make ZFS a very disputed file system OS ... DD doesn’t use any of these. From the little I’ve seen DD looks Linux based, might be ext4 but I don’t think it’s ZFS, which has some challenges, poor snapshots, performance dropping when over 70% full, complex to manage, certainly disputed (not that I am saying Ext4 is perfect). It all depends on what you want your file system to do, for many uses ZFS may be adequate, but it’s not Elvis.
I migrated my org off of ZFS(Nexenta on OpenSolaris) for backup/dedupe onto HP StoreOnce (I believe filesystem is IBRIX but dedupe was in house HP labs stuff). Averaged 6:1 dedupe on ZFS(July 2014 from my records here), shot to about 37:1 on StoreOnce (both systems accessed over NFS). Current dedupe rates across a few shares:
share 1 20TB user data 39:1
share 2 2TB user data 22.6:1
share 3 781GB user data 12:3.1
share 4 500Gb of data 33:1 (not much active data here)
share 5 8.3TB user data 62.4:1 (I think inactive data here too)
share 6 13.5TB user data 13.6:1
This is on the smallest HP storeonce appliances (1U box / 5.5TB raw). HP's marketing stuff references 20:1 ratios a lot so I think we get pretty good ratios all things considered, the last share has a ton of unrelated things stored on it so dedupe is lower, there is a limited number of shares allowed, and the dedupe applies at the share(file system?)
typical data retention above is 60 days. StoreOnce is far from perfect, NFS performance is not good, maybe get 5-10% of the throughput of the super accelerated VTL stuff(for bigger backups have to mount NFS with sync mode which slows it even more). But it works for our needs.
ZFS dedupe on linux at home I tried recently(debian 8), completely unusable, on a 4-disk raid 10 on 3ware throughput was under 1MB/sec(32GB of ram quad core intel Xeon). No idea why. I created a 100GB volume on the 3ware(LUN level not LVM or anything) because I had no need to run dedupe on the entire array(especially knowing ZFS's memory needs for dedupe). I was hoping to replace rsnapshot with zfs dedupe, but after about an hour I gave up and went back to rsnapshot (still on ZFS but just no dedupe). Without dedupe performance easily 30-50MB/sec(don't care about going much faster than that).
I have no doubt zfs dedupe on linux can work fine, not sure what the issue on my side was, wasn't important enough to try to fix.
Not sure if DD's NFS is much better or not, seems everything is optimized for VTL.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
