Mastering Data Wrangling with PySpark in Databricks
Learn Key Data Processing Skills and Machine Learning with PySpark in Databricks
Updated on Jun, 2025
Language - English
Lectures -61
Resources -3
Quizzes -5
Duration -6.5 hours
Lifetime Access
Lifetime Access
30-days Money-Back Guarantee
Get your team access to 10000+ top Tutorials Point courses anytime, anywhere.
Course Description
Among the areas of big data analytics, our comprehensive course 'Mastering Data Processing with Spark in Dat bricks' stands at the top level of hands-on skills and knowledge to help you navigate through complexity, both within Spark and Data bricks-these top leaders in efficient data processing and analysis leading into insights from large datasets.
As the technology unfolds day by day, the Big Data becomes increasingly accessible, and therefore the Big Tech Companies look out for those professionals who can process and make sense of all that huge amount of data. Learning Data bricks will upskill you to become that professional the companies are looking for!
Learn by doing: get hands-on experience in how best to use Spark and Data bricks for efficient processing, analysis, and mining of huge datasets. Learn techniques of basic data processing, transformation, query optimization, and machine learning.
With the present times of data-driven decision-making, it is the perfect time to learn Spark in Data bricks. You will be well-equipped upon taking this course so that you can take your data analytics capabilities to a completely new level, making you the most sought-after professional in this data-centric world.
Take the first step toward optimization by joining us.
Add PySpark to Your Resume by the end of this course!
Enroll now and boost your data analytics skills and supercharge your career in the data-driven world!
Goals
Understanding of the basic concepts related to Spark and Data bricks, along with the different implications they hold in the big data analytics world.
Setting up and Configuring your Data bricks Environment: Account Creation; Cluster Management
Working with Data Structures in Spark: Data Frames, Datasets; Creating and Working with Structured Data
Mastering Basic Data Manipulation Techniques in Spark: Selection, Filtering, Transformation, Aggregation and Handling Missing Data.
Learn structured queries with Spark SQL, when to use it compared to Data Frame operations, and the proper uses of both.
Understand how to do the basics of ETL with Spark: how to read and write your data, clean the data, and partition.
Overview of the Spark library and the type of problems that the machine learning will be trying to solve.
Define feature engineering, model selection, evaluation, and hyperparameter tuning to end up with a robust Spark machine learning model.
Learn about data caching and broadcast variables, which are used for performance tuning in Spark as well as query optimization.
Learn strategies to scale your workloads of Spark, including how best to process a large dataset.
Prerequisites
It is expected that the student has a basic knowledge of Python, such as data objects, loops and functions.
Curriculum
Check out the detailed breakdown of what’s inside the course
Introduction
2 Lectures
-
Course Overview
02:38
02:38
-
Notebooks
Getting Started With PySpark and Databricks
4 Lectures
Basics of PySpark
10 Lectures
Data Wrangling With PySpark
26 Lectures
Query Optimization
3 Lectures
Databricks SQL
3 Lectures
Machine Learning with PySpark
9 Lectures
Conclusion
4 Lectures
Instructor Details
Gustavo R Santos
Data ScientistData is changing the way we do business. In my 13+ years working in IT companies, I became experienced and skilled in processing large sets of data to bring up business insights. Transforming, cleaning and analyzing data is the best way to be ahead of the competition. So I have developed my ability to use data to tell stories that support business decisions. That experience led me to writing the book "Data Wrangling with R", with Packt Publishing. I am also experienced improving processes with Excel automation to speed up task completion and decrease errors.
Course Certificate
Use your certificate to make a career change or to advance in your current career.
Our students work
with the Best
Related Video Courses
View MoreAnnual Membership
Become a valued member of Tutorials Point and enjoy unlimited access to our vast library of top-rated Video Courses
Subscribe now
Online Certifications
Master prominent technologies at full length and become a valued certified professional.
Explore Now