CS60075: Natural Language Processing

Announcements

The course is aimed to have around 100 students. CS students will be preferred. Given we have to reserve seats for scholars, very little room is left for taking more students.
We follow a strict attendance policy which will be recorded and added to grade weightage.
See the prerequisite section to check whether you will be able to cope up with coding assignments in the course.

Content

What is this course about?

Natural language processing (NLP) or computational linguistics is one of the most important technologies of the information age. Applications of NLP are everywhere because people communicate almost everything in language: web search, advertising, emails, customer service, language translation, virtual agents, medical reports, etc. In recent years, Deep Learning approaches have obtained very high performance across many different NLP tasks, using single end-to-end neural models that do not require traditional, task-specific feature engineering. In this course, students will gain a thorough introduction to cutting-edge research in Deep Learning for NLP. Through lectures, assignments and a final project, students will learn the necessary skills to design, implement, and understand their own neural network models, using the PyTorch framework.

Prerequisites

Proficiency in Python
All class assignments will be in Python (using NumPy and PyTorch). If you have a lot of programming experience but in a different language, you will probably be fine.
College Calculus, Linear Algebra (e.g. MA10001, MA10002)
You should be comfortable taking (multivariable) derivatives and understanding matrix/vector notation and operations.
Basic Probability and Statistics
You should know basics of probabilities, gaussian distributions, mean, standard deviation, etc.
Foundations of Machine Learning
We will be formulating cost functions, taking derivatives and performing optimization with gradient descent. Knowing basic ML concepts will be very helpful.

References

https://www.cs.utexas.edu/~mooney/cs388/
https://web.stanford.edu/class/cs224n/index.html#schedule
Daniel Jurafsky and James Martin. Speech and Language Processing. https://web.stanford.edu/~jurafsky/slp3/
Christopher Manning and Hinrich Schütze. Foundations of Statistical Natural Language Processing. https://nlp.stanford.edu/fsnlp/
Goldberg. A Primer on Neural Network Models for Natural Language Processing. https://arxiv.org/abs/1510.00726
Research papers and online materials to be pointed out in class, especially for the recent topics

Coursework

Disclaimer: Coursework is tentative and subject to change!

The final grade for the course will be determined by the following components:

Midsem: 30%
Final Sem: 30%
Assignment 1: 10%
Assignment 2: 15%
Quizzes + Attendances: 15%

Plagiarism policy

Plagiarism in any form - copying from other students or from online resources - will be severely penalized. Every assignment should be done individually or in designated groups. Also you should not use / copy any code that is available online.
While you can discuss the concepts and assignments with other students/groups, you should NOT share your code/answers for any assignment with any other student (or with students outside your designated group), until the grading of the assignment is completed. It is your responsibility to ensure that your codes/answers are not available to others.
We will use standard plagiarism detection software to check the similarity of submitted assignments. If we find submissions that are too similar (beyond what can be expected by chance, or due to discussion), all such submissions will be severely penalized. We will NOT attempt to differentiate between who gave the codes and who copied; all involved students will be penalized equally. The minimum penalty for plagiarism in an assignment is a zero on that assignment. There can be more severe penalties for repeat offences, including grade reduction and de-registration from the course.