Akka Streams

In this blog I will walk you through the basics of Akka Streams, along with that we will also dive through the journey of how it helped us in creating a stable Data Ingestion pipeline. I have redacted company and project information from this blog for security reasons. At the end of the blog I have shared github link for Akka Streams Tech Primer, I would recommend to go through the blog before jumping on to the primer for better understanding.

Akka Streams is a toolkit for event based processing of streaming data using bounded buffer space. It is an…

Until recently most of the data warehouses and data lakes were batch oriented where data was captured in file systems or/and databases and processed in batches. However, now the current technology advancement it is a competitive disadvantage to rely completely on batch-mode. Hence big-data systems are continuously evolving to be more stream oriented where data is processed as it arrives to get the competitive edge over peers.

Today, there are a number of Open source Streaming frameworks available in the market and almost all of them have been developed in the last few years. As everything is evolving quickly, it’s…

Anup Rawka

Staff Data Engineer at Samsung Research

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store