Build a Secure Data Lake in AWS using AWS Lake Formation

Overview

In this course, we will be creating a data lake using AWS Lake Formation and bring data warehouse capabilites to the data lake to form the lakehouse architecture using Amazon Redshift. Using Lake Formation, we also collect and catalog data from different data sources, move the data into our S3 data lake, and then clean and classify them.

Syllabus

• Introduction to Data Lakes and Lakehouse Architecture
Overview of Data Lakes
Understanding Lakehouse architecture
Key concepts in AWS Lake Formation and Amazon Redshift
• Setting up the AWS Environment
Introduction to AWS services needed
Configuring AWS Identity and Access Management (IAM)
Setting up AWS Simple Storage Service (S3) buckets
• Introduction to AWS Lake Formation
Overview of AWS Lake Formation features
Understanding Lake Formation permissions
• Creating a Data Lake in AWS Lake Formation
Registering S3 locations
Creating databases and tables in the data catalog
Using Blueprints for data import
• Data Ingestion and Cataloging
Collecting data from various sources
Automating data movement into S3 using Lake Formation
Cataloging data with AWS Glue
• Data Cleaning and Classification
Data cleaning best practices
Using transforms and classifiers in Lake Formation
Ensuring data quality and consistency
• Building Lakehouse with Amazon Redshift
Setting up Redshift clusters
Integrating Redshift with Lake Formation
Querying data across the data lake using Redshift Spectrum
• Security and Access Management
Implementing granular data access controls
Configuring data encryption and privacy settings
Auditing and monitoring data activity
• Advanced Topics in Lake Formation
Automating workflows with AWS Glue and Step Functions
Implementing machine learning models in the data lake
• Case Study and Hands-on Project
Real-world example of building a secure data lake
Group project: Design and implement a mini-lakehouse architecture
• Review and Next Steps
Best practices for maintaining a data lake
Upcoming AWS features and updates in data lake technology
Resources for further learning

Taught by

Yomi Owoyemi

Subjects

Business