I thought of building this architecture by learning it is still in progress.But the individual concpets are already built in miniprojects.

image.png

Similar architecture which explains

image.png

Medallion Architecture

image.png

These are the technologies used by Zomato for data engineering pipeline for both real time and batch processing.

image.png

This is apache open source technologies used by Zomato.

Similarly ,now lets talk about Cloud Managed Service.

Eg: Amazon ,Azure, GCP.

Now if you want to use amzon as your infrastructure you can use these technologies.

Similarly you can explore more.