About OH Lectures
About
OH
Lectures
04650-A: Mathematical Foundations of Machine Learning
Fall 2024

In-Person Venue (Monday & Wednesday) :CMU-Africa F203 10:00 - 11:30 AM

Recitations (Fridays) :CMU-Africa F205 13:00 - 14:00

Piazza

Course Description

This course offers a comprehensive mathematical foundation for machine learning, covering essential topics from linear algebra, calculus, probability theory, and optimization to advanced concepts, including information theory and statistical inferencs. The course aims to equip students with the necessary mathematical tools to understand, analyze, and implement various machine learning algorithms and models at a deeper level.

Prerequisites

  1. We will be using Numpy and Pandas in this class, so you will need to be able to program in python3.
  2. You will need familiarity with basic calculus (differentiation, chain rule), linear algebra, and basic probability.

Your Supporters

Instructor:

TAs:

Assignment Policy

Homework not turned before the due will lose 20%. Each additional day loses another 20%. The lowest quiz grade will be dropped. While students in class are encouraged to discuss the assigned problems, each student should write and submit their own solution and should not copy the solutions of other students. This policy will be strictly enforced in this class. If you have any question about whether some activity would constitute cheating, please feel free to ask. You will be granted a maximum of 3 slack days, slack days are self granted extensions which you can use a maximum of one day per assignment.

Attendance:

Every student is expected to attend every class for the whole class duration. Attendance will be checked randomly using different means, including a pop quiz that also assesses the reading assignment. Unexplained absence will result in deduction of point in the attendance score. The attendance score will also include various ways, such as answering your colleague's questions on Piazza, or being active during class. If you need to miss a lecture because of a medical emergency or something equally significant, please inform the instructor before the start of the class.

Grading Algorithm:

35% Homework, 20% Quizzes, 5% Reading & Attendance, 20% Mid-term Exam, 20% Final Exam

Letter Grading Algorithm:

Grade Letter Grade
94 - 100 A
90 - 93.99 A-
87 - 89.99 B+
84 - 86.99 B
80 - 83.99 B-
77 - 79.99 C+
74 - 76.99 C
70 - 73.99 C-
67 - 69.99 D+
64 - 67.99 D
61 - 63.99 D-
0 - 60.99 F

Schedule of Lectures

Lecture Date Topics Slides Additional Materials Quiz Assignment
1 Monday,
Aug 26
  • Introduction and Course Logistics
  • Learning Objectives
  • Grading
  • Deadlines
  1. Youtube
  2. Numpy Fundamentals
Quiz 1 Assignment 1 Released
2 Wednesday,
Aug 28
  • Systems of Linear Equations and Matrices
  1. Tutorial Notes, pdf
  2. mml-book chapter 2.1 and 2.2
  3. Geometry of Linear Equations - Gilbert Strang
  4. Tutorial Notebook
3 Monday,
Sep 02
  • Solving Systems of Linear Equations
  1. Tutorial Notes, pdf
  2. mml-book chapter 2.3
  3. Gilbert Strang lectures 2, and 3
  4. Youtube: Gaussian Elimination
  5. Youtube: Row Enchelon
Quiz 2
4 Wednesday,
Sep 04
5 Monday,
Sep 09
Quiz 3 Assignment 1 Due
Assignment 2 Released
6 Wednesday,
Sep 11
  • Linear Independence, Basis, and Rank
  1. Tutorial Notes, pdf
  2. mml-book chapter 2.4, 2.5, and 2.6
7 Monday,
Sep 16
  • Linear Mappings and Affine Spaces
Quiz 4
8 Wednesday,
Sep 18
  • Orthogonality
9 Monday,
Sep 23
  • Determinant and Trace
Quiz 5 Assignment 2 Due
10 Wednesday,
Sep 25
  • Eigenvalues and Eigenvectors
11 Monday,
Sep 30
  • Eigendecomposition and Diagonalization
Quiz 6
12 Wednesday,
Oct 02
  • Singular Value Decomposition part 1
13 Monday,
Oct 07
  • SVD part 2
  • PCA
No Quiz Assignment 3 Released
14 Wednesday,
Oct 09
  • Midterm Exam
- Monday,
Oct 14
  • No Class - Fall Break
No Quiz
- Wednesday,
Oct 16
  • No Class - Fall Break
15 Monday,
Oct 21
  • Differentiation of Univariate Functions
  • Partial Differentiation and Gradients
No Quiz
16 Wednesday,
Oct 23
  • Gradients of Matrices
17 Monday,
Oct 28
  • Backpropagation and Automatic Differentiation
Quiz 7
Assignment 3 due
Assignment 4 Released
18 Wednesday,
Oct 30
  • Optimization Using Gradient Descent
19 Monday,
Nov 04
  • Introduction to Probability and Statistics
Quiz 8
20 Wednesday,
Nov 06
  • Bayes’ Theorem
21 Monday,
Nov 11
  • Independence, and Law of Total Probability
Quiz 9 Assignment 4 Due
Assignment 5 Released
22 Wednesday,
Nov 13
  • Probability Distribution
23 Monday,
Nov 18
  • Data, Models, and Learning
Quiz 10
24 Wednesday,
Nov 20
  • Parameter Estimation
25 Monday,
Nov 25
  • Entropy and Mutual Information
Quiz 11 Assignment 5 Due
26 Monday,
Dec 02
  • KL Divergence and Cross-Entropy
No Quiz
27 Wednesday,
Dec 04
  • Review

Office Hours

Day Time TAs Zoom Link
Monday 9:00 AM - 10:00 AM Ayebilla Avoka Join Zoom
12:00 PM - 1:00 PM Marie Cynthia Abijuru Kamikazi Join Zoom
Tuesday 5:30 PM - 6:30 PM Brian Kipkirui Join Zoom
7:00 PM - 8:00 PM Marie Cynthia Abijuru Kamikazi Join Zoom
Thursday 1:00 PM - 2:00 PM Ayebilla Avoka Join Zoom
5:30 PM - 6:30 PM Brian Kipkirui Join Zoom
7:00 PM - 8:00 PM Lawrence Francis Join Zoom
Friday 7:00 PM - 8:00 PM Lawrence Francis Join Zoom