Warning: this page is for archival purposes only and is not kept updated.

CS 414, Fall 2010

There are NO required books for the course. More details on the syllabus.


Fall 2010 Syllabus

Project 0: Due 10/4 at 6am
Background reading on Moodle (no response required)

Project 1: Due 11/15 at 6am
"Checkpoints" due on 10/25 (6am) and 11/1 (6am)

Note: Many slides in class are based on Raymond J. Mooney's machine learning class at UT-Austin and Rich Sutton's reinforcement learning class at U-Alberta.


Reading Due
8/30 Introduction to class    
9/1 Syllabus, Introduction The Discipline of Machine Learning (6 pagse)
Sections 1 - 1.3 of Sutton and Barto (7 pages)
9/3 Decision Tree Learning    
9/6 More Decision Tree Learning
Least Mean Squares
9/8 Perceptrons and Artifical Neural Networks    
9/10 Artifical Neural Networks    
9/13 Bayes Theorem    
9/15 Naive Bayes    
9/17 Bayes Nets    
9/20 Bayes Nets + Intro to COLT    
9/22 Version Spaces and PAC Reading Response on Naive Bayes due (on Moodle)  
9/24 (In)finite Hypothesis Spaces Reading Response on Neural Nets due (on Moodle)  
9/27 Sutton + Barto, Chapter 2: Gambling!    
9/29 S&B, Chapter 3: The RL framework    
10/1 Bellman Equations    
10/4 Policy Evaluation, Policy Iteration    
10/6 Value Iteration Reading Response: S&B sections 2.1, 2.2, 2.4, 2.5, 2.7, 2.8, 2.11. Due by 6am.  
10/8 S&B, Chapter 5: Monte Carlo    
No class -- Fall Break
10/13 More Monte Carlo: Policy Evaluations    
10/15 Offline MC and TD S&B Chapter 5. Due 6am Monday 10/18.  
10/18 Sarsa    
10/20 Q-learning    
10/22 Discussion of project 0, project 1, and Keepaway   Project 1, Step 0.
Install rl-competition code.
Hack the getAction() function in agents/marioAgentJava/src/edu/rutgers/rl3/comp/ExMarioAgent.java to always go right and send me the code of this function by 6am Friday.
10/25 Eligibility Traces S&B Chapter 6. Due 6am Wednesday 10/18.
Instead of a summary, you can answer the following two questions: "How would you explain the difference between Dynamic Programming, Monte Carlo, and Temporal Difference Learning? When would you use one of these methods instead of another?"
Project 1, checkpoint #1 due
10/27 Eligibility Traces -- on the board    
10/29 More Eligibility Response Due Monday, 6am: Read 7.0, 7.1, 7.2, 7.3, 7.5, 7.8, and 7.9  
11/1 Function Approximation   Project 1, checkpoint #2 due
11/3 Function Approximation 2, The Remix   Read your section for Friday
11/5 Finishing off Function Approximation and Discussion of paper    
11/8 Planning and Learning    
11/10 Planning and Learning, rest of chapter Skim S&B chapter 8, read section 8.4 (no response due -- project on Monday!)  
11/13 RMax and RL-DT    
11/15 Shaping and TAMER    
11/17 More Shaping, X2 6am, Monday the 22nd: Send Matt a proposal for Project 3.
Suggestions: One of the topics mentioned in class (see 11/17 slides), using some of the UCI data and decision trees, or one of the RL topics in Mario, Tetris, or Mountain Car. Or anything you think of!
Also, send Matt suggestions for what you'd find most interesting to discuss in the final 2 weeks of class.
11/19 Guest Lecture: GAs    
11/22 Ensemble Methods    
No class -- Thanksgiving Break
No class -- Thanksgiving Break
11/29 Instance-Based Methods    
12/1 No lecture -- Matt was sick    
12/3 Hierarchical RL    
12/6 Transfer Learning    
12/8 Discussion of Intrinsic Rewards    
12/10 Discussion of Helicopter Flight