Posts by Platypus

Efficient in what way?

This might be power efficient, but rushing to buy new servers could be very inefficient in terms of other resources that go into them. Especially all those rare-earth metals. If the cost of disposal is considered, it might already be more environmentally sound to continue using older servers, and this will only become more true as clean power becomes a higher percentage of the total. This isn't just an appeal to conscience, either. In another ten years or so I expect the cost of disposal will be *required* as part of the purchase cost. Then it will also be more *expensive* to keep buying new servers. Power efficiency is a positive at the time of initial purchase, but after that I think too much focus on it is misguided.

Once they got into it, I'll bet they found that they had a much lower percentage than they thought of what goes into a real filesystem. It's easy to get something that kinda-sorta does the very basics, but there's a *long* tail of obscure features and edge cases. I used to be on the filesystem team adjacent to the LVM folks are Red Hat, so they should have known better than to make such facilities claims.

Also, like most gift shops, this one is full of cheap crap that nobody in their right mind would really buy. Between the fake reviews, counterfeits, and the invasion of the generic drop-shippers, the quality of the shopping experience on Amazon has taken a nose-dive in the last year or two.

Good luck

For all of its good points, Ceph is very demanding wrt the hardware it's running on. In particular it's an atrocious CPU hog. Running it on systems with underpowered CPUs is not something I would want to be involved in. TBH this seems more like a publicity stunt than a serious offering.

Disclosure: I'm still technically a maintainer for Gluster, though I no longer work on it day to day.

Dr. Peuto's name seems to be spelled correctly in the story, but not in the headline.

My first processor was the 6502, but I always kind of wished I'd have a chance to try both the Z80 and 6809. Maybe I'll find a simulator and putter around a bit. Low-level programming on the old 8-bit processors with their paucity of registers and addressing modes etc. always felt more like puzzle solving than most of what I've done since. Got a second taste of it with the 88K, which was an early exposed-pipeline RISC, and it's something I still remember fondly.

Misalignment of incentives

Startups, tech or otherwise, do not exist to create jobs. They exist to make money. They are *funded*, sometimes massively, to make money. If they can do so with greater automation, they will. Even for those startups that create physical things, it's more cost-effective to use pre-manufactured parts (including microcontrollers) and automated assembly than to hand-craft everything from small pieces. It's more cost-effective to sell on the internet than fight for space in a kazillion brick-and-mortar stores. Many logistics and even HR systems can be outsourced. So it's no surprise that even startups that intend to grow rapidly employ what once would have been regarded as a skeleton crew.

The thing is, if startups hire fewer people each, then maybe there should be more startups. Unfortunately, the funding models aren't geared for that. They want to fund a few unicorns not many minnows. Tax laws, accounting requirements, insurance requirements, and regulatory systems are increasingly hostile to smaller businesses, as they each tend to exact a fixed cost per company that's in the noise for a larger company but significant for a smaller one. (BTW I'm way over on the progressive end of the spectrum and support most of these things in principle, but even I recognize that their implementation has had ill effects.) If we want to undo the over-centralization (and over-financialization) of our economy and go back to a time when regular people were creating jobs for other regular people, we need to stop putting startups in the shadiest part of the garden and letting the giants stomp all over the fresh shoots.

The silver lining is that those who were laid off might be better able to apply their talents elsewhere, and might ultimately be paid more for it. There was a time when Veritas did innovative things. That time is long gone. For the last $forever they've seemed intent on *strangling* innovation. When they've wrung the last dollar out of products that were innovative twenty years ago, they'll just fold the whole thing up and they won't be missed.

So Facebook takes this suggestion and hires a bunch of editors, who at some point inevitably turn out to all be Trumpians who brand anything they don't like as fake news. Or Bernians who do likewise from a different direction. Then the same people continue to lambaste them for doing The Wrong Thing because it was *never* about doing what's right or protecting democracy or anything like that. A lot of it is competitors in the information business doing what competitors do, along with a big dose of Tall Poppy Syndrome.

Dealing with fake news doesn't mean giving any one group editorial control - not Facebook itself, not El Reg, sure as hell not any government. It means allowing multiple rating or collaborative-filtering services to flourish, and giving users a *choice* of which ones to trust, much as we do for spam filters and ad blockers. It's a market, not a planned information economy. Facebook's role is to help users find anti-fake-news filters *they* trust, and to honor the results as they're displaying an individual user's feed.

Google's all about network neutrality when it comes to routing, but I guess not when it comes to names. "You must share but I must own" is a pretty crappy attitude.

Mage, you're either woefully misinformed yourself, or deliberately misinforming others. Google has brought in or copied everything? Besides the obvious exceptions of the PageRank concept and map/reduce, here are plenty more exceptions.

https://opensource.google.com/

Facebook hasn't innovated technically? More exceptions.

https://code.facebook.com/projects/

Never heard of Spanner or Borg/Kubernetes or Cassandra or HHVM or Open Compute? That's just ignorance. Look, I'm sensitive to the issues of privacy and market dominance and so on, but the specific claim here is that these two companies are bad for *innovation* and that's clearly false. Name a company you like better. Let's see if they contribute as much to innovation. Highly unlikely.

It's refreshing to see a little skepticism about such vague claims by storage upstarts. Without an actual apples-to-apples comparison - not their NVMe gear vs. someone else's spinning disks as usually happens - it's impossible to say whether they have anything or not. Maybe they really have come up with some great innovation. If so, I'd wonder why they're not bragging about the patents they've already filed. Absent that, this looks like a pitch for investors/acquirers rather than customers. If they're still standing and still independent a year from now, maybe we can have a discussion about the technology then.

Bikeshedding

Isn't this true of other groups as well? We all fight over the issues with which we're most intimately and constantly familiar. Emacs vs. vi is our version of angels dancing on the head of a pin. Another metaphor is the infamous bikeshed. Nobody wants to argue about the design of a nuclear power plant, because that's complicated and hard and requires a lot of knowledge, but everyone has an opinion on what color the bikeshed at that plant should be. Perhaps the general tendencies of programmers - highly focused, introverted, a bit brittle - make this somewhat worse than elsewhere, but mostly it just seems like human nature.

Third Mover Advantage

Coho ran into an all-too-common problem for storage startups: storage customers are hard to evangelize. They have pressing, immediate problems. They want solutions to those problems, and ideally solutions that fit into their current paradigm. Getting them to look forward to the *next* set of problems is really really hard. Coho had looked ahead, seen a problem on the horizon, developed some interesting technology to address it ... and then found themselves too far ahead of the customers to get any revenue out of all that. Like other technologies (*ahem* CDP) it will be not the originators but some late-arriving copycats who hit the right market window to benefit. With luck, some of the people who actually had the vision and made the efforts will get to ride the gravy train a few years from now, but in my experience that happens all too rarely.

Is it just me, or does the idea of running an internet-accessible memcached server already seem insane?

Own your availability, own your security

I've seen way too many cases where the preinstalled firewall crap at a cloud provider interfered with the operation of the distributed systems I was installing. Often the tools and documentation available to resolve the issue were miserable too. I did not appreciate it. I'm perfectly capable of locking down my own system, without making it unusable, all by myself. IMO it's perfectly reasonable for a provider to avoid the complexity and cost and aggravation associated with trying to do what any competent Linux administrator can and should do themselves.

At a previous job we had a similar - but not identical - problem with a machine in Boulder. In our case there was one more step. Because of the thinner air, we got less cooling. The warmer temps made the PSU less efficient, causing brownouts which manifested as transient errors on our internal communications links. The fix turned out to be a slight adjustment the the ratio between temperature and fan speed. I was the guy on-site, but kudos to the hardware folks back on the east coast for figuring it out.

Are they also asking for an investigation into White House staffers using Confide? Of course not, because this isn't about infosec or policy. It's purely a matter of attacking the other team and defending your own.

Yawn

As far as I can tell, this is just EC2 with features removed to enable a simpler pricing model. The fact that many of these features become available again through VPC peering suggests that it's a separate (someone else's?) data center. But the price isn't really going to destroy Digital Ocean etc. Looking at the 2GB level, which is the lowest they all have in common and is what really constitutes a starter system:

* Digital Ocean - $20/month for two cores and 40GB SSD

* Linode - $20/month for one core and 24GB SSD

* Vultr - $20/month for two cores and 45GB SSD

* Lightsail - $20/month for one core and 40GB SSD

Lightsail is below median for cores, at median for storage, all for exactly the same price. Without benchmarks - especially storage benchmarks which IMX have shown a 2-3x difference between providers or even instances within one provider - it's hard to know which is really the better deal. The real take-away here seems to be that Amazon was feeling pressure at the low end.

NT, or XP?

I think the proper analogy is XP, not NT. NT was a new architecture, separate from the legacy 3.x/95/98 codebase. XP was the reunification of these divergent streams. Similarly, Android represented a bit of an architectural departure with its unique JVM-based userspace. Andromeda will represent the reunification of that with the more traditional architecture of ChromeOS (so traditional that I'm running full Ubuntu in another window on this Chromebook right now).

Still trying to get a handle on what Andromeda will mean for us Chromebook users, BTW. *That* would be an interesting story to delve into.

"The Realm Platform works with Java, Objective-C, and Swift"

So of course the image shows PHP. Yeah, I know, it doesn't matter and only a geek-pedant would notice or comment on it. Still, perhaps not the best design choice.

It's misleading to say Red Hat Gluster Storage will be available this summer, or to imply that it's just now competing with Portworx et al. RHGS has been available for years, since before some of those others issued their first press release - let alone wrote their first line of code. It's just the new version that's coming.

So you finally admit that there's such a thing as open-source storage, but only to shoot it down with more mentions of proprietary competitors. :sigh:

Looks like Dunning/Kruger to me

As with many things, the first level is easy but then things get much harder. Can I build a simple database? Sure I can. Can I build a fully SQL-compliant database with a sophisticated query planner and good benchmark numbers? Not without some help. Can I build an interpreter for a simple language? No problem. Can I build a 99.9% gcc-compatible compiler that spits out correct high-performing code for dozens of CPU architectures? Um, no. Similarly, building a very simple storage system is within reach for a lot of people and is a great learning exercise. Then you add replication/failover, try to make it perform decently, test against a realistic variety of hardware and failure conditions, make the whole thing maintainable by someone besides yourself . . . this is still a simple system, no laundry list of features to match (let alone differentiate from) competitors, but it's a lot harder than a "one time slowly along the happy path" hobby project.

I'm not saying that the storage vendors deserve every dollar they charge. I'm pretty involved with changing those economics, because the EMCs and the NetApps of the world have been gouging too much for too long. What I'm saying is that "build it yourself" is a bit of an illusion except at the very smallest of scales and most modest of expectations. "Build it with others" is a better answer. Everyone contributes, everyone gets to benefit. If you really want to help speed those dinosaurs toward their extinction, there are any number of open-source projects that are already engaged in doing just that and could benefit from your help.

Am I crazy for thinking that "Motion Picture Ass" had nothing to do with saving headline space?

"If I have more resources than required"

Might as well stop there. That never happens for long. Where there's capability to spare, new workloads will be added until that's no longer the case. It happens with CPU, it happens with memory, and it happens with storage. Always has and always will. The real question is how to maximize the value of the IOPS you're providing when you're providing as many as you can. That means letting higher-value IOPS (e.g. for higher-value apps or tenants) take priority over lower-value IOPS, and that's QoS.

Besides the fact that what they're doing is no different than what GlusterFS (which I work on) and Ceph have done for years, they start off with two lies.

(1) Their FAQ claims that GlusterFS uses a centralized server, which is not true.

(2) They claim to be open-source, but when you follow the link a big fat "coming soon" is all you'll get.

Outfits like this come along every damn month. And they disappear every month too, when they find out that gaining and retaining users is harder than getting a few mentions in the trade press. There's no reason so far to suspect this one will rise above that vile crowd.

Storage has always been a hard place to make a living

Especially for startups. It's one of the first places that enterprises look to cut costs, and one of the last places they're willing to experiment. And it has become a crowded space. The folks at Coho are great, but I could say the same about a dozen other startups of the same vintage. They can't all succeed. In a way, this is a side effect of lowering the barrier to entry. Now that scale-out software on top of commodity hardware (even if it has a fancy faceplate) is more competitive with specialized hardware, it seems like everybody and their brother has a storage startup with a new take on where the "real" storage problems are and how to solve them. Some of those ideas are truly new, and truly great. Some aren't. The problem is that it's hard to tell which is which, so when the lifeboat's too crowded and companies start getting thrown overboard it's not always the ones who should have been. Sadly, technical merit and business value don't usually count as much as cozy relationships with investors, analysts, journalists, and (just once in a while) "whale" customers.

Data ONFIRE? Heh. Good one.

Why do these articles only ever seem to compare against *proprietary* solutions? Another basis of comparison for semi-open-source RozoFS would be truly-open-source Gluster (on which I work) or truly-open-source Ceph, both of which already have erasure coding too. Based on experience with that, I'd say *it doesn't matter* which erasure-coding algorithm involves more addition or multiplication because those calculations are only a minor factor in overall performance. The amount of data that must be transferred, either during normal I/O or during repair, matters far more. The coordination overhead matters even more than that. If you have two clients trying to write overlapping blocks, and they don't coordinate properly, then half of the servers get erasure-coded pieces of one write and half get erasure-coded pieces of the other. This isn't even "last writer wins"; anyone who tries to read that data subsequently gets *garbage* back. The #1 determinant of performance in such systems is how they avoid this issue for every kind of operation (including both data and metadata with all of the atomicity/durability guarantees that must be met to keep users from screaming).

If the Rozo folks want to brag about their erasure-coding efficiency, let's see some actual performance data. While we're at it, let's talk about the scale at which things have really been tested. Anybody can claim hundreds of nodes and multiple exabytes but AFAIK no project in this space has ever successfully run at that scale on the first try. They *always* run into new failure modes and performance anomalies that never appeared at smaller scale and that often require substantial new subsystems to address. Then they find out that customers at this scale are going to want tons of other features as well. Some of these are still only on Rozo's roadmap, after having been shipped years ago by competitors. Others, especially related to multi-tenancy, are still missing entirely.

I think what Rozo is doing is very cool, and I wish them all the success in the world, but let's not lose sight of the fact that there's a *long* row to hoe before even the best ideas turn into a competitive storage solution. They sound a lot like the Ceph folks did *five years ago*, but Ceph (with far more resources at hand) is just now making the transition from bleeding-edge to enterprise-ready. It's not because they lack talent, I can assure you of that. It's just that these problems are *hard*, and solving them takes a lot longer than Evenou and Courtoy seem to think. I'd love to hear from the RozoFS developers about when *they* think RozoFS will be competitive with what's already out there.

Let's not overgeneralize. *This time* he didn't name names. On the other hand, he still did make some pretty strong inferences about "whoever" wrote the code, and "whoever" isn't hard to discover. That's well beyond just criticizing the code.

On the other other hand, I've been on too many projects that *didn't* lay down the law this firmly. Developers are a sneaky lot, and they tend to have their own agendas. They'll keep sneaking in code that they know is crap, if it lets them mark more of their personal tasks complete. If nobody is watching, or nobody responds strongly enough to put the fear of God into them, the result is a codebase that slowly rots into irrelevance. I do think Linus and (even more so) certain other Linux kernel developers behave in some pretty toxic ways sometimes, but as we try to improve that situation we still need to remember that bringing the hammer down once in a while is strictly necessary to maintain any kind of quality. It's all in how it's done, not whether it should be done at all.

A welcome development

This particular case involves siblings (as of last week), but I suspect we'll be seeing a lot more of this kind of thing even among non-siblings - yes, even among current rivals - in the next few years. Among other things, it means folks like Isilon will be forced to compete on the basis of software quality instead of relying on custom-tuned hardware to give them an edge in performance comparisons. Bring it.

(Disclaimer: I'm a Gluster developer)