Big names unleash 1,000-node Hadoop stampede Hadoop, the distributed big-data processing system named after a toy elephant, now has a group of suppliers behind it. The tech giants are collectively aiming to develop the popular big data beast with a 1,000 node cluster – dubbed the Analytics Workbench – as a laboratory. EMC's Greenplum, Intel, Mellanox Technologies, Micron …
This may be a newsfeed (press release) article, but I disagree with your comment:
Relevant news is several companies (including EMC, Intel, Seagate, VMware etc) have donated materials to this project.
Other relevant news is that results will be available to open source communities.
Both of these are significant developments and of great interest to those of us interested in Hadoop or even just large scale data processing.
A 1000 node Hadoop cluster isn't all that big, in terms of those who are actively using the technology to store and analyze petabytes of data. 1000 nodes == ~250 production nodes (3x redundancy + 1x control/fail-over), which if each has 24TB of disc (12x2TB in a 2U enclosure), you have 6PB of production data. Not a small Hadoop cluster, for sure, but not the biggest by any stretch. So, I think this is probably a decent sized cluster for research, and many/most production systems, but not nearly big enough to model installations such as Google, Facebook, et al.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
