Data Pipelines with Snowflake and Streamlit

Course Overview
In this course, I construct a data engineering pipeline that gathers multiple sources of information: Kaggle datasets and Google Trends data fetched through SerpAPI. The pipeline aggregates and combines data on Netflix actors along with trends about them on Google within weeks after a new show is released.
You will use Kaggle as a data source for datasets regarding Netflix shows and actors, and Google Trends through SerpAPI to fetch live search data for the actors. All data will be stored and processed within the Snowflake database, using its cloud-native architecture to maximize scalability and performance.
Technical Stack Overview
Snowflake Database
Central repository for storing and querying data.
Streamlit within Snowflake
A web application framework where the data can be rendered directly within Snowflake.
AWS S3
Storage location where some intermediate datasets are stored or fetched.
Snowflake Python Procedures
Code used to automate parts of the data manipulation and processing pipeline.
Snowflake External Access & Storage Integrations
Used to manage secure access to external services and storage.
Outcome
By the end of this course, you will have a fully functional data pipeline that processes and merges streaming data, cloud storage, and APIs to enable trend analysis, visualized in an interactive Streamlit app within Snowflake.