Tuesday, July 10, 2012

So now Hadoop's days are numbered?

Earlier this week we all read GigaOM's article with this title:
"Why the days are numbered for Hadoop as we know it"
I know GigaOM like to provoke scandals sometimes, we all remember some other unforgettable piece, but there is something behind it...

Hadoop today (after SOA not so long ago) is one of the worst case of an abused buzzword ever known to men. It's everything, everywhere, can cure illnesses and do "big-data" at the same time! Wow! Actually Hadoop is a software framework that supports data-intensive distributed applications, derived from Google's MapReduce and Google File System (GFS) papers.

My take from the article is this: Hadoop is a foundation, low-level platform. I used the word "platform" just because of a lack of a better word. Wait there is a great word that captures it all! 


This word is Assembler


When computers begun 70 years ago or so, Assembly is the mother of all programming languages, Assembler made it work in real world computers, silicone and copper. In the world of Big Data, map-reduce, massive distribution and parallelism is the mother of all living things (Assembly). And Hadoop enables it to actually run in the real world (Assembler)... 


Like Assembler, Hadoop core is far from being really usable.  Doing something real, good, working, repeatable with it requires skills that only a few people can really master (Like good Assembler programmers, back in 1960's).




While I consider myself lucky to have the chance to actually punch cards with brilliant(?) Assembler code, many of today's brightest minds in Silicone Valleys around the world never wrote one opcode. They're all using PHP, Ruby, Java and node.js, which are great "wrappers" around good old Assembly to bring programming, innovation, disruptiveness - to the masses, make the whole world a better place. It's how it should be.


Hadoop will die only if data and big data dies. Nonsense. Data is by far the most important asset organizations have. Facebook as well as Bank Of America will be worth a fraction of their value in minutes if they loose the same fraction of their data. Both won't be able to compete if they can't be intelligent and analyze their data that multiplies every (low number) days/weeks/months. The data makes a business intelligent and Hadoop helps exactly there. 


Hadoop is the Assembler of all analytical big data processing, ETL and queries. The potential around it and its ecosystem is literally unlimited, tons of innovation and disruptiveness are poured by startups and communities all over, like Splunk, HBase, Cloudera, Hive, Hadapt, and many many more. And we're just in the "FORTRAN" phase...

63 comments:

  1. Your punch card is not showing assembler code but FORTRAN and before you ask, yes it was hard working on computer while the dinosaurs were roaming the land.

    ReplyDelete
    Replies
    1. You got me!! I would have attached a pic of me holding my authentic punchcards, but back then I just couldn't find my cell-phone with a camera... :)

      Thanks for the interest!

      Delete
  2. Love the analogy, Doron. When I started in this business, the moldy old guys were bragging about how they used to string wires between boards to make things work. To them, assembler was a mamby-pamby tool for wimps who couldn't handle real computing!

    Since I was an economics major and feeling a bit inferior about my technical background, I decided to read the S/370 Principles of Operation in detail. Best thing I ever did. A low-level understanding of how the machine itself actually works is indispensable in this business. It certainly served me well at Intel, where the machine's internal architecture isn't nearly as straightforward as that of the S/370!

    But it isn't about bragging rights and how much techno-detail one can master, is it? The fundamental barrier we all face, in life and technology, is TIME. It's incompressible, and it's inexorable.

    If we can find ways to productively use up vast amounts of excess raw computational power to save time, it's worth it. That's what PHP and all the rest are all about.

    An observation: all of the computational power in the world amounts to a hill of beans when it comes to bandwidth and latency (at the limit of compressibility). Since we now know that neutrinos actually do follow the laws of general relativity, we're back to the limitations imposed by the speed of light on that score.

    In the end, time still wins.

    ReplyDelete
    Replies
    1. Mitch - inspiring! Thank you.

      I totally agree and I would add one additional aspect: mass.

      To PHP on a MacBook is much simpler and graspable than to FORTRAN on punchcards (or on wires)! It requires less training so many more can actually be part of "computing" and its innovation, from JCL and batch calculations, all the way to PC, iPad, Amazon, Facebook, Angry Brids!!
      If we take the above to the extreme, we can bring Einstein once again with his famous quote: "Two Things Are Infinite: the Universe and Human Stupidity"... :)

      Thanks for the interest.

      Delete
    2. Believe it or not, I'd never read that quote before. Fantastic!

      Delete
    3. Very informative article post.Really looking forward to read more. Will read on…
      sap online training

      Delete
  3. Hadoop trainings course is designed for developers who want to better understand how to create Apache Hadoop solutions.

    ReplyDelete
  4. You mention that Hadoop runs all the data center analytical processing. I am guessing that it is a software within itself. What about all the other companies that are now coming out with new 2013 enterprise data center software? Where will they come into play in the grand scheme?

    ReplyDelete
  5. hello, i want to say thankyou for your impeasive post.your Hadoop Technology post is very good and knowlable.

    ReplyDelete
  6. it's a nice article, very helpful for us and thanks for sharing. we are providing Hadoop online training


    ReplyDelete
  7. This is nice article and thanks share with us.we providing Hadoop online training

    ReplyDelete
  8. The information which you provided is very much useful for Hadoop Online Training Learners thanks for sharing valuable information

    ReplyDelete
  9. The Information was very much useful for Hadoop Online Training Learners Thank You for Sharing Valuable Information it is very useful for us.

    ReplyDelete
  10. The Information you provided is very much useful for Hadoop Learners. This Information was very Intresting, We also provide
    Hadoop Online training in India

    ReplyDelete
  11. Hi
    I read this post two times.
    I like it so much, please try to keep posting.
    Let me introduce other material that may be good for our community.

    Hadoop training

    ReplyDelete
  12. The information which you have provided is very good. It is very useful who is looking for Hadoop Online Training.

    ReplyDelete
  13. The information which you have provided is very good. It is very useful who is looking for Hadoop Online Training.

    ReplyDelete
  14. This is the information that I was looking for and let me tell you one thing that is it is very useful for who is looking for Hadoop Online Training.

    ReplyDelete
  15. Thanks for providing the best information it's very useful for HADOOP learners.we also provide the best HADOOP Online Training.

    ReplyDelete
  16. Nice article very happy to see this Hadoop Online Training Article.. I came to know 123trainings at hyderabad is also providing excellent hadoop online training.. keep Posting this article.. I also have hadoop online training demo video.

    ReplyDelete
  17. I was really impresed by reading this article about
    Hadoop online training
    , It will be useful for Hadoop online training learners

    ReplyDelete
  18. Thanks for your support, i am very interested in learning Hadoop.. Right now i am learning Hadoop in Attain Technologies at Hyderabad, They will provide the Best Hadoop Online Training..

    ReplyDelete
  19. Thanks for sharing valuable information this is very useful for
    Hadoop Online Training Learners.. I am appreciating you for gathering such a nice information.

    ReplyDelete
  20. Thanks for sahring for this valuble information and it is useful for hadoop learners.Hadoop online trainings also provides best hadoop online training.

    ReplyDelete
  21. This information which you provided is very much useful for us.It was very interesting and useful for hadoop online training learners.We also providing
    hadoop online training .

    ReplyDelete
  22. Thanks for your support,I am very interested in learning HADOOP Right now i am learning HADOOP in 123 trainings. They will provide the Best
    HADOOP Online Training at hyderabad.


    ReplyDelete
  23. hi you have gathered a valuable information on Hadoop...., i am looking for content like this and i am much impressed with the information and nice course content, thanks a lot for the Information regarding Hadoop Online Training.

    ReplyDelete
  24. This comment has been removed by the author.

    ReplyDelete
  25. You have given the most useful Information.. It was useful for us.. I heard that 123trainings at hyderabad provides the best Hadoop Online Training

    ReplyDelete
  26. Thanks for gathering information regarding HADOOP,I have gathered 123trainings is the best HADOOP ONLINE TRAINING provider from hyderabad.

    ReplyDelete
  27. It was nice article it was very useful for me as well as useful for HADOOP online training learners thanks for providing this valuable information.

    ReplyDelete
  28. Great helpful information. Thanks for providing wonderful hadoop information.123trainings provides hadoop online training we can see free demo class
    hadoop online training classes in hyderabad.

    ReplyDelete
  29. it is a good piece of knowledge and it is used for hadoop learners.123trainings provides besthadoop online training to see free demo classHadoo online training demo class in Ameerpet

    ReplyDelete
  30. it is a good piece of knowledge and it is used for hadoop learners.Hadoop online trainings provides besthadoop online training

    ReplyDelete

  31. It's nice information and it is very helpful for us.123trainings provides best online training Hadoop.to see demo Hadoop online training demo class in hyderabad

    ReplyDelete
  32. Itis good and it is very helpful for us.Hadoop online trainings provides best online training Hadoop.

    ReplyDelete
  33. I Appreciate it and It is good and very helpful for us.123trainings provides best online Hadoop training .To see free demo online Hadoop training demo class in Hyderabad

    ReplyDelete
  34. Thanks a lot for the wonderful information and it is useful for us.123trainings provides best Hadoop online training.tosee free demo classHadoop online training class in india

    ReplyDelete
  35. The Information was very much useful for Hadoop Online Training Learners Thank You for Sharing Valuable Information.
    Hadoop online training by Hadoop Online Trainings. Hadoop Online Trainings is identified with quality and reliability.we are the best online training institute with excellent trainers with good experience.Our online Hadoop training programs are interactive, practical, easily understood but at the same time intensive and comprehensive.We have upcoming regular and weekend batches.

    Web : http://hadooponlinetrainings.com/

    ReplyDelete
  36. Thanks for this valuble information and itis useful for us .Biginfosys also provides the best online Hadoop training classes.

    ReplyDelete
  37. This is nice article and thanks share with us.we providing Hadoop Online Training

    ReplyDelete
  38. The information which you have provided is very good. It is very useful who is looking for Hadoop Online Training.

    ReplyDelete
  39. The Information you provided is very much useful for Hadoop Learners. This Information was very Intersting, We also provide Hadoop Online training in India.
    Skypeid: rsonlinehyd
    Please contact us India:+91 9052699906,USA :+1 909-666-5386
    Email:contact@rstrainings.com

    ReplyDelete
  40. Thanks for Sharing this valuble information and itis useful for me and Hadoop learners.We also provides the best Online Hadoop Training classes.

    ReplyDelete
  41. Thanks for sharing this valuble information and it is useful for me and also Hadoop learners.we also provideHadoop Bigdata Online Training Classes In India

    ReplyDelete

  42. Thanks for sharing valuable information and it is useful for hadooponlinetrainings provides the best Hadoop Online Training classes.

    ReplyDelete
  43. For More Information on Hadoop Can also refer to our Hadoop Blog

    ReplyDelete
  44. PHP MYSql Ajax Classes, Courses, LIVE Projects, Bhubaneswar, Orissa
    MySQL has complemented it being the world’s most popular open source database. Together, these two technologies provide you with an excellent platform for building database-driven web applications.
    Website: http://www.waterflame.in/
    Email id: waterflame001@gmail.com

    ReplyDelete
  45. Thanks for sharing useful information with us, Smartmind also provides best Hadoop Online Training

    ReplyDelete
  46. Pretty good post. I just came across your site and wanted to say that I’ve really enjoyed reading your posts. Miracle soft solutions is one of the leading institute for Hadoop online training

    ReplyDelete
    Replies
    1. Pretty good post. I just came across your site and wanted to say that I’ve really enjoyed reading your posts. Miracle soft solutions is one of the leading institute for hadoop online training

      Delete
  47. This company provide the industrial training in the database...
    Interested candidate can join this company..
    6 Month Training

    ReplyDelete
  48. Thank you provide valuable informations and iam seacrching same informations,and saved my time SAS Online Training

    ReplyDelete
  49. Nice information.
    Our institute Is the Leading online institute in Hyderabad, India to provide INFORMATICA online trainings with the real time experts and certified professionals. alot. Informtica ONLINE TRAINING.

    ReplyDelete
  50. Thank you provide valuable informations and iam seacrching same informations,and saved my time SAS Online Training

    ReplyDelete
  51. Nice blog and thanks for sharing your information. Learn communals provide the high-quality online courses training to the students like android, iphone, SAP, Tableau, dotnet, java etc.
    Online Training Institutes In Hyderabad

    ReplyDelete
  52. hi you have gathered a valuable information on Hadoop...., i am looking for content like this and i am much impressed with the information and nice course content, thanks a lot for the Information regarding Hadoop Online Training.

    http://www.primeonlinetraining.com/hadoop-online-training/

    ReplyDelete
  53. The hadoop article you provided is very much useful for Hadoop Learners. This Information was very Intersting, We also provide Hadoop Online training in hyderabad.
    Skypeid: onlinetrainingsexpert
    Please contact us India:+91 9441233734
    Email:onlinestrainingexpert@gmail.com

    ReplyDelete
  54. Nice blog and thanks for sharing your information. Learn communals provide the high-quality online courses training to the students like android, HADOOP, java, SAP, Tableau, Hibernate, Struts, Spring, Salesforce etc.
    HADOOP Online Training Ameerpet

    ReplyDelete
  55. Very nice post i like it very much keep posting and this will to hadoop learners.Hadoop Online Training

    ReplyDelete
  56. Very excellent article this posts useful for that my mind.Thanks a lot.
    Hadoop Training in Chennai

    ReplyDelete