EE698R: Advanced Topics in Machine Learning (Spring 2021)

Instructor

Vipul Arora

TAs

Name	Email
Rashmi Yadav	rashmiy@iitk.ac.in
Mohit Vohra	mvohra@iitk.ac.in
Vikas Kanaujia	kvikas@iitk.ac.in
Sumit Kumar	krsumit@iitk.ac.in
Kondamudi Jagadeesh Babu	kbabu@iitk.ac.in
Adhiraj Banerjee	adhiraj@iitk.ac.in
Rahul Kodag	rkodag@iitk.ac.in
Akash Apare	aaapare@iitk.ac.in

Course Objectives:

This course aims at introducing the students to advanced topics in machine learning (ML). The course will begin with lessons on programming that is needed to enable one to efficiently implement ML algorithms. The lectures will focus on mathematical principles, and there will be coding based assignments for implementation.

Pre-requisites:

• Basic course on machine learning (EE698V or equivalent)
• Digital signal processing (EE301A or equivalent)
• Basics of Programming (ESc101 or equivalent)

The course will need a strong background in linear algebra and probability theory.

Topics:

• Data structures and Algorithms
• Deep Neural Networks
• Generative Modeling
  • Random sampling (Monte Carlo methods)
  • Variational Auto Encoders
  • Generative Adversarial Networks
  • Normalizing Flows
• Time Series Modeling
  • Dynamic Programming
  • Hidden Markov Models
  • Recurrent Neural Networks and LSTMs
    • Connectionist Temporal Classification
• Other topics of interest

Lecture Plan

Week of 2021	Topics
3	Introduction
4	Data structures and Algorithms
5	Data structures and Algorithms
6	Neural Networks
7	Time Series Modeling
8	Time Series Modeling
9	Mid-sem Exam
10	Vacation
11	Attention Models, Few shot learning, domain adaptation, explainable ML
12	Sampling - Monte Carlo Methods
13	Sampling - Monte Carlo Methods
14	Variational AutoEncoder
15	Generative Adversarial Network
16	Normalizing Flows
17	Projects
18	Projects
19	End-sem Exam

Grading Scheme

1. Continuous Assessment – 20%
   Assignment - 6.7%, Quiz - 13.3%
2. Mid-semester Exam – 33.3%
   Written exam
3. Project – 46.7%
   
4. Bonus: Demo 5%, Paper 5%

Details of project:

• The project presentations will include an individual viva too to assess each individual member’s contribution.
• Project presentations will be held from 13 May onwards. We will release slots which you can fill.
• Students with difficulties, such as COVID complications, can contact the instructor for an extension in deadline (submission by 17 May, presentation on 18 May). Subject to instructor’s approval.
• if you submit a project demo, you will get extra 5%. Demo means showing your performance (variant goal) on real data in a live run. It should be impressive. E.g.,
  • Melody extraction: take real songs
  • Speaker diarization: record yourself and/or take a real recording from a meeting (TV or movie)
  • Audio event detection: record yourself and/or take a real recording (TV or movie)
• Presentations assessment breakup:

Details	Marks (out of 35)
Problem definition: straightforward + variant	5 marks
No. of methods used (using available libraries)	1 mark/method
No. of methods used (self implemented)	5 marks/method
Report (up to 4 pages + references) Template: https://nips.cc/Conferences/2020/PaperInformation/StyleFiles	10 marks
Codes (readable + reproducible)	10 marks

Plagiarism Penalty:

As heavy as possible. Zero-tolerance policy.

References:

This course will take excerpts from some standard books on machine learning and signal processing. But it will largely be based on articles and research papers in ML and SP conferences (e.g., NeurIPS, ICML, Interspeech, ICASSP, etc.) and journals (e.g., IEEE TASLP, JMLR, IEEE PAMI, etc.).

Books:

• “Pattern Recognition and Machine Learning”, C.M. Bishop, 2nd Edition, Springer, 2011. https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf
• “Bayesian reasoning and machine learning”, D. Barber, Cambridge University Press, 2012. http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/090310.pdf
• “Deep Learning”, I. Goodfellow, Y, Bengio, A. Courville, MIT Press, 2016. https://www.deeplearningbook.org/