back to article AMD: Why we had to evacuate 276TB from Oracle DB to Hadoop

AMD has migrated terabytes of information from an Oracle Database installation to an Apache Hadoop stack, claiming Oracle's pricey software was suffering from scaling issues. The chip maker's chief information officer, Jake Dominguez, revealed further details of the transfer in a chat with The Reg. "Within the common Oracle …

COMMENTS

This topic is closed for new posts.
  1. Nate Amsden

    sounds like

    they didn't know what they were doing to begin with. Or perhaps weren't able to do the right thing to begin with and had an organically growing system that nobody seriously managed and they just let it grow until it hit a major breaking point.

    Not that I'd use Hadoop to replace Oracle, rather go with something like Vertica instead especially for a small data set like 276TB.

    1. intrigid

      Re: sounds like

      "especially for a small data set like 276TB."

      lol

      1. Anonymous Coward
        Anonymous Coward

        @intrigid

        276 TB is small in the database world.

        1. AceRimmer

          Re: @intrigid

          "276 TB is small in the database world."

          I'd wager that more than 99.9% of databases in existence are smaller than 276 TB

          1. intrigid

            Re: @intrigid

            99.9999% even.

          2. JEDIDIAH
            Linux

            Re: @intrigid

            > "276 TB is small in the database world."

            >

            > I'd wager that more than 99.9% of databases in existence are smaller than 276 TB

            That just means there's been a proliferation of a lot of small databases as the technology has become cheaper to use. The big jobs continue to get bigger all the time and people aren't terribly impressed when you start throwing around non-trivial numbers.

          3. Anonymous Coward
            Anonymous Coward

            Re: @intrigid

            Perhaps I was exaggerating (or should have said "276TB isn't overly large") but to me how many databases are smaller is less relevant than how unusual a database larger than that size is. Having consulted on a project a couple years ago where one database utilized several fully loaded VMAX arrays, perhaps I'm jaded.

            Probably 99.9% of all programs are shorter than 10K lines, but that doesn't make a 10K line program large.

      2. PorkFriedTech

        Re: sounds like

        When most rack configs are sized at 1PB, 276TB is relatively small.

      3. Anonymous Coward
        Anonymous Coward

        Re: sounds like

        When most rack configurations for big data are sized for around 500TB+, 276TB is relatively small.

    2. Kevin McMurtrie Silver badge

      Re: sounds like

      ...they're doing data mining, which is dumb to put in a relational database. It's not that the Oracle DB failed, it's that Oracle failed to come up with practical distributed computing systems for non-relational data. I know Oracle created some distributed systems in the past but I don't think anyone made it to the end of the installation instructions.

    3. Anonymous Coward
      Anonymous Coward

      Re: sounds like

      Oracle usually shits a brick well before that mark. Depends in your query rate and how much horribly written PL/SQL you're throwing at your doddering old mainframes. I've lost count of the number of places that have come to us wanting to replace their Oracle boxes, thinking they'd pick up something like Teradata or Vertica, only to end up settling for Hadoop.

      Between the completely hassle free configuration and administration, the resilience and fault tolerance, the as-good-as-free hardware and licensing and the massive toolkit you get, it's pretty much a no brainer. Especially now the major distributions come with their MPP SQL engines (Impala, Stinger etc.). Throw in Parquet and Snappy and you literally have a carbon copy of your garden variety Teradata architecture, but open source and really, really sodding cheap.

      Those in traditional data warehousing should be worried.

    4. JEDIDIAH
      Linux

      Re: sounds like

      I also found the remark about the 100K result set limit quite odd.

  2. intrigid

    "According to AMD, the Hadoop software was able to reduce query times by 300 per cent"

    So let's say you have a report that takes 6 hours to run... When your boss says "and I need it yesterday!", you can say "You've got it, literally!"

    1. diodesign (Written by Reg staff) Silver badge

      Re: intrigid

      Yes, OK. AMD claimed it and we weren't awake enough at the time to think it through, but now we've tweaked that sentence.

      Please, don't forget to email corrections@thereg is you spot something odd - we see those emails, but we can't read every comment.

      C.

      1. intrigid

        Re: intrigid

        What would be the fun in that? If your buddy's fly is hanging open during a funeral, you don't quietly mention it to him, you point and laugh! Ok, maybe you have a point.

        1. Captain Save-a-ho
          Pint

          Re: intrigid

          Actually, that depends entirely on whether your buddy has pledged to buy the pints at the after-funeral trip to the pub.

      2. Anonymous Coward
        Anonymous Coward

        Re: intrigid

        > Please, don't forget to email corrections@thereg is you spot something odd

        Couldn't you set up an online form instead of email? I would occasionally send the odd correction but I'd rather not divulge my "useful" email addresses and I really cannot be arsed to log into a throwaway one for this.

    2. h4rm0ny

      >>So let's say you have a report that takes 6 hours to run... When your boss says "and I need it yesterday!", you can say "You've got it, literally!"

      Only if they get you the request before 6:00pm that day.

      And typically that's when the sadists let you know there's something urgently needed right away, ime. :( >:|

  3. Malcolm 5

    Humm "We made the pivot to Hadoop [and] it not only increased our reliability but our response time," I thought you normally tried to reduce your response times?

  4. Steve Knox
    WTF?

    "row limit for query results [of] 100,000 rows on the Oracle system" - !?

    Can someone who works with Oracle's software confirm, deny, or qualify this?

    I cannot believe a modern database system of any kind would have such a tiny row limit...

    1. LHGFLICOD

      Re: "row limit for query results [of] 100,000 rows on the Oracle system" - !?

      never seen this limit, (going back as far as version 7.3.4) so not sure what they are talking about, there is a limit of 1000 columns in a table although that would indicate your doing something really stupid.

      Possibly they have performance problems with data > 100,000 rows being returned but there is no hard limit.

    2. Anonymous Coward
      Anonymous Coward

      Re: "row limit for query results [of] 100,000 rows on the Oracle system" - !?

      I work for Oracle as a developer. Every day I launch lots of queries that return tens of millions of rows. Never been a problem.

  5. Stevie

    Bah!

    I can believe it.

    I can believe almost anything after watching Oracle "specialists" fail to get Oracle RAC running cleanly on a Sun/Solaris stack in a reasonable time, or to be able to say "yes, we know exactly why <very popular document manager application> hosted by our Weblogic Application Server running in Solaris 10 on a dedicated 4450 is acting as though it is running in treacle on wet tar, and here is how to fix it".

    I was approached by an Oracle afterservice drone a couple of years back who asked "How do you like Weblogic on that 4450?" and I had the great pleasure of telling him we liked it so much we were now running the application in webSHPERE on an LPAR in a crowded P7 - and had seen an order of magnitude pick up in response time as a result".

    The fun when doing that is to keep a straight face.

  6. asdf
    Mushroom

    wow

    Its a serious question if Oracle is going to outlive Ellison. Even if it does Oracle may end up being Wang a few decades later.

    1. Roo
      Pint

      Re: wow

      "Even if it does Oracle may end up being Wang a few decades later."

      It looks as though Oracle have been firmly wanged in the short and curlies by AMD + Open Source of all people. If a sufficiently detailed white paper is published I suspect this will develop into a Tsunami of woe for Larry.

  7. Mage Silver badge

    Not surprised

    It's a normal development cycle that incumbents get bloated, expensive and lose touch with what customers want.

    1. Anonymous Coward
      Anonymous Coward

      Re: Not surprised

      But what is surprising is just how long it's taking Oracle to spiral down the plughole. I expected it to happen a decade ago.

      1. Anonymous Coward
        Anonymous Coward

        Re: Not surprised

        I concur. I've always thought of Oracle as Apple without the cool people factor.

  8. Crazy Operations Guy

    Good software, stupid name

    From my experience as a consultant I think there would be a lot more installs of Hadoop if the name didn't sound so goofy. I can't count the number of times I've gone in front of management and had them laugh off the software because of its name.

    1. Anonymous Coward
      Anonymous Coward

      Re: Good software, stupid name

      If that's the reason they won't use it, they deserve to go bankrupt. I've heard the same over Beastie. Apparently so did FreeBSD. The effing suits... Yawn...

      1. Hans 1
        Joke

        Re: Good software, stupid name

        Heard of Apache Pig ?

      2. Jamie Jones Silver badge

        Re: Good software, stupid name

        Yep, I eventually got FreeBSD used in a previous place of employment - ultimately, it's ability spoke for itself - however, in the initial stages, 'suits' weren't impressed with the word 'free' in the title.

        And I''m sure we've all heard of religious American nutjobs who are against it because it 'glorifies satan' etc.

    2. Anonymous Coward
      Anonymous Coward

      Re: Good software, stupid name

      Hadoop is in at least 95 of the FTSE100. It doesn't have a branding problem.

    3. Anonymous Coward
      Anonymous Coward

      Re: Good software, stupid name

      That would be Management from the 1980s. There are a lot of silly names out there that dominate the industry now. Example: Google. Puppet. Red Hat. Git. Linux. Any versions of software with "X" in them (makes it "edgy"!). Android. I mean, comeon, the computer industry is rife with quirky names. In fact, it's pretty much a tradition.

      1. Jamie Jones Silver badge

        Re: Good software, stupid name

        True, I am going back to the early 90's - when open source and free software weren't generally known, and there was still a big 'Nobody gets fired for buying Microsoft' ethos..

        (Yes, I know the phrase originally mentioned IBM, but it was used for Microsoft in the 90's)

  9. Anonymous Coward
    Anonymous Coward

    Didn't

    Oracle stop using AMD processors?

    1. Anonymous Coward
      Anonymous Coward

      Re: Didn't

      Yes, we have some of their AMD-based heads for the 7410 storage system and it is no longer fully supported less than 5 years after it was introduced.

      We are not going back to Oracle in the future, but migrating away as fast as we can.

      1. Anonymous Coward
        Anonymous Coward

        Re: Didn't

        LMAO - It was Sun Microsystems who stopped using AMD, not AMD not providing anymore..

  10. itscoldhere

    Poor Larry -- looks like he'll be building his next America's Cup techno-boat from fiberglass instead of nano-tube-glass!

This topic is closed for new posts.

Other stories you might like