Data 101: Data Engineering 💾

UC Berkeley, Fall 2024

Ed Lecture Recordings Gradescope Additional Extensions

Professor Lisa Yan

She/Her

yanlisa@berkeley.edu

Professor Michael Ball

He/Him

ball@berkeley.edu

Announcements

Week 3 Announcement

Sep 10

Homework 1 is released! Due next week.

See Week 3 Ed announcement.

Schedule

Week 01

Thu 8/29
Lecture 1 Introduction, Data Engineering Lifecycle
Pre-Semester Form
Discussion 1 SQL Review
Solution, code
Friday 8/30
Project 0 SQL Review
Due Wed 9/4 Thu 9/5, 5pm
notes

Week 02

Tue 9/3
Lecture 2 SQL Review
Wed 9/4
Project 0 Due Date Extended by 24 hours (Ed post)
Thu 9/5
Lecture 3 Relational Model & Algebra
Discussion 2 Relational Algebra
Solution
Project 0 Due, 5pm
Fri 9/6
Homework 1 Homework 1
Due Wed 9/18, 5pm

Week 03

Tue 9/10
Lecture 4 CTEs, Views, Subqueries, Foreign Keys
Thu 9/12
Lecture 5 DML/DDL, Keys, and Constraints
Discussion 3 Views, Windows, DDL
Fri 9/13
Project 1 SQL
Due Wed 9/25, 5pm

Week 04

Tue 9/17
Lecture 6 Foreign Keys, Index Selection, Bits and Memory Model
Wed 9/18
Homework 1 Due, 5pm
Thu 9/19
Lecture 7 Index selection, Query processing, Optimization I
Discussion 4 Constraints, Keys
Fri 9/20
Homework 2 Homework 2
Due Wed 10/2, 5pm

Week 05

Tue 9/24
Lecture 8 Index selection, Query processing, Optimization II
Wed 9/25
Project 1 Due, 5pm
Thu 9/26
Lecture 9 Guest Lecture TBA
Discussion 5 Query Optimization
Fri 9/27
Project 2 Query Performance
Due Wed 10/9, 5pm

Week 06

Tue 10/1
Lecture 10 Data Models: Relations, Tensors, Dataframes
Wed 10/2
Homework 2 Due, 5pm
Thu 10/3
Lecture 11 Data Preparation I: Structural
Discussion 6 Data Prep I

Week 07

Tue 10/8
Lecture 12 Data Preparation II: Numerical, Granularity, Window Functions
Wed 10/9
Project 2 Due, 5pm
Thu 10/10
Lecture 13 Data Preparation III: Outliers
Discussion 7 Data Prep II
Fri 10/11
Homework 3 Homework 3
Due 10/23, 5pm

Week 08

Tue 10/15
Lecture 14 Data Preparation IV: Imputation, Entity Resolution
Wed 10/16
Midterm Midterm Exam (7-9pm)
Thu 10/17
Lecture 15 Normalization + ER
Discussion (no discussion)
Fri 10/18
Project 3 Data Transformation
Due Wed 10/30, 5pm

Week 09

Tue 10/22
Lecture 16 Semistructured Data: NoSQL, JSON, XML
Wed 10/23
Homework 3 Due, 5pm
Thu 10/24
Lecture 17 MongoDB I
Discussion 9 Normalization, MongoDB I
Fri 10/25
Homework 4 Homework 4
Due Wed 11/6, 5pm

Week 10

Tue 10/29
Lecture 18 MongoDB II
Wed 10/30
Project 3 Due, 5pm
Thu 10/31
Lecture 19 Guest Lecture TBA
Discussion 10 MongoDB II
Fri 11/1
Project 4 Mongo
Due Wed 11/13, 5pm

Week 11

Tue 11/5
Election Day Lecture AMA, Q&A
Wed 11/6
Homework 4 Due, 5pm
Thu 11/7
Lecture 20 Transactions and TCL
Discussion 11 Transactions
Fri 11/8
Homework 5 Homework 5
Due Wed 11/20, 5pm
Final Project Optional Final Project
Checkpoint Wed 11/27, 5pm
Due Fri 12/6, 5pm

Week 12

Tue 11/12
Lecture 21 Data Pipelines
Wed 11/13
Project 4 Due, 5pm
Thu 11/14
Lecture 22 Graph data, MapReduce
Discussion 12 Pipelines, MapReduce

Week 13

Tue 11/19
Lecture 23 BI, OLAP, Visualizations
Wed 11/20
Homework 5 Due, 5pm
Thu 11/21
Lecture 24 Parallel and Distributed Computing
Discussion 13 OLAP, Parallel/Distributed Computing
Fri 11/22
Homework 6 Homework 6
Due Wed 12/4, 5pm

Week 14

Mon 11/25
Final Project Checkpoint Due, 5pm
Tue 11/26
Lecture 25 Sampling, Spreadsheets
Thu 11/28
Thanksgiving Day No class

Week 15

Tue 12/3
Lecture 26 Guest Lecture TBA
Wed 12/4
Homework 6 Due, 5pm
Thu 12/5
Lecture 27 Security/Privacy, Closing Thoughts
Discussion 14 Wrap-Up
Fri 12/6
Final Project Due, 5pm

RRR Week

Tue 12/10
Final Review

Finals Week

Tues 12/17
Final Final Exam (3-6pm)