Large-scale data engineering has gone through a remarkable transformation over the past decade. We have seen wide adoption of Big Data frameworks from Apache Hadoop, and Apache Spark to Apache Flink. …


Data engineering is becoming an increasingly important part of scientific discoveries with the adoption of deep learning and machine learning. Data engineering deals with a variety of data formats, storage, data extraction, transformation, and data movements. …


Simple solutions that are capable of solving a large number of problems ignite the creativity side of the engineers. Hadoop is a great system, developed around a single operation called map-reduce. It is simple enough to be understood by many people and generic enough to solve many problems. For a…


Data processing is becoming part of everyday life for any organization looking into the future. Big data needs clusters of computers to store them, manage them and analyze them due to both space and computational requirements. Hadoop (map-reduce) is the first system to manage large data sets and process them…

Supun Kamburugamuve

High performance data analytics

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store