Intro
This training is intended for big data practitioners who want to further their understanding of Dataflow in order to advance their data processing applications. Beginning with foundations, this training explains how Apache Beam and Dataflow work together to meet your data processing needs without the risk of vendor lock-in. The section on developing pipelines covers how you convert your business logic into data processing applications that can run on Dataflow. This training culminates with a focus on operations, which reviews the most important lessons for operating a data application on Dataflow, including monitoring, troubleshooting, testing, and reliability.
This training is perfect for;
Data Engineer, Data Analysts and Data Scientists aspiring to develop data engineering skills.
What You'll Learn
- Plan and implement a well-architected logging and monitoring infrastructure
- Define Service Level Indicators (SLIs) and Service Level Objectives (SLOs)
- Create effective monitoring dashboards and alerts
- Monitor, troubleshoot, and improve Google Cloud infrastructure
- Analyze and export Google Cloud audit logs
- Find production code defects, identify bottlenecks, and improve performance
- Optimize monitoring costs
Program
- Module 1: Introduction
- Module 2: Beam Portability
- Module 3: Separating Compute and Storage with Dataflow
- Module 4: IAM, Quotas, and Permissions
- Module 5: Security
- Module 6: Beam Concepts Review
- Module 7: Windows, Watermarks, Triggers
- Module 8: Sources and Sinks
- Module 9: Schemas
- Module 10: State and Timers
- Module 11: Best Practices
- Module 12: Dataflow SQL and DataFrames
- Module 13: Beam Notebooks
- Module 14: Monitoring
- Module 15: Logging and Error Reporting
- Module 16: Troubleshooting and Debug
- Module 17: Performance
- Module 18: Testing and CI/CD
- Module 19: Reliability
- Module 20: Flex Templates
- Module 21: Summary
Prerequisites
To get the most out of this course, participants should have completed the following courses:
- “Building Batch Data Pipelines”
- “Building Resilient Streaming Analytics Systems”
Certification
After this training course, the next step within the learning journey is the Professional Data Engineer Exam.
Learning Journey
- Big Data & Machine Learning Fundamentals
- Data Engineering on Google Cloud
- Serverless Data Processing with Dataflow
- Professional Data Engineer Exam.
The Right Format For Your Preferred Learning Style
At Binx we offer four distinct training modalities:
- In-Classroom Training
- Online, Instructor-Led Training
- Hybrid and Blended Learning
- Self-Paced Training
Learn more about our training modalities
