Natural language (NL) refers to the language spoken/written by humans. NL is the primary mode of communication for humans. With the growth of the world wide web, data in the form of text has grown exponentially. It calls for the development of algorithms and techniques for processing natural language for the automation and development of intelligent machines. This course will primarily focus on understanding and developing linguistic techniques, statistical learning algorithms, and models for processing language. We will have a statistical approach towards natural language processing, wherein we will learn how one could develop natural language understanding models from statistical regularities in large corpora of natural language texts while leveraging linguistics theories.
Introduction to Machine Learning (CS771) or equivalent course, Proficiency in Linear Algebra, Probability and Statistics, Proficiency in Python Programming
Probabilistic Machine Learning (CS772), Topics in Probabilistic Modeling and Inference (CS775), Deep Learning for Computer Vision (CS776)
Course Instructor: Dr. Ashutosh Modi
Samik Some (Email: firstname.lastname@example.org
Shubham Kumar Nigam (Email: email@example.com
Tushar Shandhilya (Email: firstname.lastname@example.org
Karishma Satchidanand Laud (Email: email@example.com
Gargi Singh (Email: firstname.lastname@example.org
Chayan Dhaddha (Email: email@example.com
Ashwani Bhat (Email: firstname.lastname@example.org
Aishwarya Gupta (Email: email@example.com
Ajitha Shree (Email: firstname.lastname@example.org
Prakhyat Shankesi (Email: email@example.com
In case you want to communicate with the instructor, please do not send any direct emails to the instructor (these will most likely end in spam), use this course email for the communication: firstname.lastname@example.org
Weekly Meeting Session:
Tuesday, Friday 2PM to 3:15 PM
Lectures, assignments and quizzes will be uploaded/conducted on HelloIITK
Virtual classes will be held on MS Teams
. A separate team/channel (CS779-Spring2021: Statistical Natural Language Processing)
has been set up for the course. All course announcements will be made on the Teams channel, HelloIITK and Telegram.
In case you are not there on MS Teams please create an account via IITK subscription. To get IITK sub-scription please fill this form: https://web.iitk.ac.in/ccnew/Office365/Office_365_subscription_at_IITK.htm
. If you are outside IITK, you would need to log into IITK network via VPN to fill the form. Once you have the account, please contact TAs to add you to the Teams channel.
Please check the Resources
Course will have quizzes, assignments, lecture summaries, research project, open competition (NLP Challenge), and exams. Given that course is going to be online, all exams will be conducted online. The tentative weightage for different components are as follows. Please note that this is tentative (due to COVID uncertainties and factors beyond Instructor's control) and weightage might change.
Lecture summaries: 10%
Research Project: 20%
NLP Challenge: 10%
Mid-Sem Exam: 20%
End-Sem Exam: 20%
||Introduction and Logistics
Tentative list of topics we will be covering in this course:
- Introduction to Natural Language (NL): why is it hard to process NL, linguistics fundamentals, etc.
- Language Models: n-grams, smoothing, class-based, brown clustering
- Sequence Labeling: HMM, MaxEnt, CRFs, related applications of these models e.g. Part of Speech tagging, etc.
- Parsing: CFG, Lexicalized CFG, PCFGs, Dependency parsing
- Applications: Named Entity Recognition, Coreference Resolution, text classification, toolkits e.g., Spacy, etc.
- Distributional Semantics: distributional hypothesis, vector space models, etc.
- Distributed Representations: Neural Networks (NN), Backpropogation, Softmax, Hierarchical Softmax
- Word Vectors: Feedforward NN, Word2Vec, GloVE, Contextualization (ELMo etc.), Subword information (FastText, etc.)
- Deep Models: RNNs, LSTMs, Attention, CNNs, applications in language, etc.
- Sequence to Sequence models: machine translation and other applications
- Transformers: BERT, transfer learning and applications
There are no specific references, this course gleans information from a variety of sources likebooks, research papers, other courses, etc. Relevant references would be suggested in the lectures. Some of the frequent references are as follows:
- Speech and Language Processing, Daniel Jurafsky, James H.Martin
- Foundations of Statistical Natural Language Processing, CH Manning, H Schtze
- Introduction to Natural Language Processing, Jacob Eisenstein
- Neural Network Methods for NLP, Yoav Goldberg, Morgan Claypool (If you are in IITK network you can download at: Morgan Claypool Subscription for IITK )
- Linguistic Fundamentals for Natural Language Processing, Emily Bender, Morgan Claypool (If you are in IITK network you can download at: Morgan Claypool Subscription for IITK )
- Natural Language Understanding, James Allen