Build a web scrapper and save some raw data in somewhere like S3. Then run some job on top of that data to get aggregated measures and save them somewhere. I bu

Build a web scrapper and save some raw data in somewhere like S3. Then run some job on top of that data to get aggregated measures and save them somewhere. I built a project[1] like this and learned a lot in the process. I used airflow to run the scrapping tasks, save the data in S3, use AWS Athena to run queries and load data into Redshift. I did all of this just to learn more about Airflow and some AWS tools. [1] https://skills.technology/data-engineer https://skills.technology/data-engineer