In the year 2022, world is making great progress with Technology . We are always interested in new upcoming technology whether it's an app , gadget or an electric car. We use technology in Banks, Hospitals, accounting and many other environment to make everyone's life easier.

The more we are interested in it, the more we use it. The more Data is created. This is the Big Data. In today’s world, the essence of the data is more than the essence of crude oil or gases.With the help of data, we can predict each step which will be beneficial to the world if used respectfully.

A teacher keeps watching and analyzing her students everyday and teaches them particularly based on their grasping power. Same is applicable to companies.

Since we live in the era where we have large size data available, its hard to handle. We need to process it well enough before using it.

This is when the Stream and Batch process comes into Picture here.after every work completed, it slides to another piece of array.

Stream Processing

In stream processing, you have a continuous data coming from the user. We have to quickly analyze / Filter / enhance it according to the conditions. Examples: Stocks,Fraud detection, etc.

Batch Processing

In Batch processing, you already have the data or group of data. There is no user interaction.You have to analyze / Filter / enhance after getting the data. Examples: ATM transactions, customer services, etc

We use Batch data processing for complex computation and analysis and for simple computation and analysis, we use streaming.

In stream algorithms, we study space complexity to save records seen,unique records, etc . We save it in the storage and further work with it. This way we reduce the time and enrich our data solutions like data warehouses, data bases, clouds, etc. We use the estimation algorithm to save the data in storage.

Reduce the time complexity in batch processing using Stream line data enrichment method.

You can still reduce time complexity of batch processing before.

Here, we sliding windows. here, lets say you have an array starting from 1-20.You will divide it in particular manner and use it. first, you will take 1-5 and use it, change it, whatever required. after it you will go for 5-10. then 10-15 and will keep on happening till the end. Think the array as a data. Data is in pieces.

This is used extensively for Stocks.

Memphis website allows us to apply various schemes on different data which makes it easy to analyze it.

Through this we can show it in UI and trace the flow. Here, user needs to user needs to analyze the things what they want to show. we just need to think about use case, requirements, components and how to move the data between them.

Thank you for reading the blog.