Sign in / up

back to article Big Data? Yeah, nice buzzword. Give us the nuts and bolts this time

Big Data has crossed the chasm from hype to everyday reality remarkably quickly. Its adoption has been accelerated by hungry data warehousers using big data techniques to get better answers from their data mining activities. ™ Big Data In Focus Attending the TM Big Data InFocus 2014 conference in Amsterdam, it became …

COMMENTS

House rules Send corrections
This topic is closed for new posts.
  1. Michael H.F. Wilkinson Silver badge

    Interesting piece

    Sounds like an interesting event. I have been a bit tired of the "big data" and "data mining will solve all your problems" hype. A more realistic discussion on aims, possibilities, difficulties, and limitations, that is very good news.

    We have been doing some fairly big data work (in astronomical data sets and remote sensing), but until now did not use the term "big data". If people start using the term sensibly, I might start using it.

    0 0
    1. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: Interesting piece

      pretty much the same as us. We have about 2.5PB mainly remote sensing but also a sizeable chunk of large model runs. Main issues we have is backing up\archiving and making sure our network is up to the job of moving it all around from client to processing nodes

      0 0
      1. Michael H.F. Wilkinson Silver badge
        Reply Icon

        Re: Interesting piece

        We also run into problems of processing the data rapidly once they are in the processing nodes. Our big data typically consists of fewer HUGE chunks (gigapixel and even terapixel images), compared to many applications where the number of chunks is huge, but each chunk is fairly modest. In all cases there are big issues in feeding the data efficiently to the processing nodes, but in our case there is an additional problem parallellizing the actual processing, once it gets there. Fun problem, really.

        0 0
        1. Anonymous Coward
          Reply Icon
          Anonymous Coward

          Re: Interesting piece

          what do you do the processing on? We've got an HPC for modelling and use a 100'ish node grid engine for remote sensing

          0 0
          1. Michael H.F. Wilkinson Silver badge
            Reply Icon

            Re: Interesting piece

            At the moment we are using a small group of 64-core Opteron compute servers just to develop new algorithms for big images, we also have a 3280 code cluster we use for testing distributed memory algorithms. This is our research side, production computing will be done elsewhere.

            0 0
  2. Anonymous Coward
    Anonymous Coward

    Old as data warehousing (at least)

    Apart from archiving, why on earth do people think organizations have long spent large sums of money collecting and storing lots of data? Obviously, to extract meaningful information from it as far as possible.

    That was the rationale for data warehouses, when that buzzword was introduced about 20 years ago. It's 40 years since Stafford Beer began working on his real-time executive dashboard plan for the Allende government in Chile. Of course, in the newly fashionable sense of "at least as much data as you can afford to pay for and preferably more", Big Data is a transparent sales ploy by Oracle and other database and iron (shouldn't that be "plastic" nowadays?) vendors.

    0 0
  3. Pahhh

    buzzwords

    From the analytic point of view, Big Data is no different to Data Warehousing which every fortune 100 company employs, hence the relative success of niche companies like Teradata for 20+ years.

    I find the term Big Data as ambiguous as Cloud and equally irritating. Still it allows the press to print shite about how its going to change all our lives in the future (even though its been going on forever) and gives the PR men, like the Pied Piper, an instrument to play to draw in the money men.

    1 0
This topic is closed for new posts.

Other stories you might like