YSDA Natural Language Processing course [](https://mybinder.org/v2/gh/yandexdataschool/nlp_course/master)

Syllabus

- __week02__ __Text classification__
- Lecture: Text classification. Classical approaches for text representation: BOW, TF-IDF. Neural approaches: embeddings, convolutions, RNNs
- Seminar: Salary prediction with convolutional neural networks; explaining network predictions.

- __week03__ __Language Models__
- Lecture: Language models: N-gram and neural approaches; visualizing trained models
- Seminar: Generating ArXiv papers with language models

- __week04__ __Seq2seq/Attention__
- Lecture: Seq2seq: encoder-decoder framework. Attention: Bahdanau model. Self-attention, Transformer. Pointer networks. Attention for analysis.
- Seminar: Machine translation of hotel and hostel descriptions

- __week05__ __Structured Learning__
- Lecture: Structured Learning: structured perceptron, structured prediction, dynamic oracles, RL basics.
- Seminar: POS tagging

- __week06__ __Expectation-Maximization__
- Lecture: Expectation-Maximization and Word Alignment Models
- Seminar: Implementing expectation maximizaiton

- __week07__ __Machine translation__
- Lecture: Machine Translation: a review of the key ideas from PBMT, the application specific ideas that have developed in NMT over the past 3 years and some of the open problems in this area.
- Seminar: presentations by students

- __week08__ __Transfer learning and Multi-task learning__
- Lecture: What and why does a network learn: "model" is never just "model"! Transfer learning in NLP. Multi-task learning in NLP. How to understand, what kind of information the model representations contain.
- Seminar: Improving named entity recognition by learning jointly with other tasks

- __week09__ __Domain Adaptation__
- Lecture: General theory. Instance weighting. Proxy-labels methods. Feature matching methods. Distillation-like methods.
- Seminar: Adapting general machine translation model to a specific domain.
- __week10__ __Dialogue Systems__
- Lecture: Task-oriented vs general conversation systems. Overview of a framework for task-oriented systems. General conversation: retrieval and generative approaches. Generative models for general conversation. Retrieval-based models for general conversation.
- Seminar: Simple retrieval-based question answering

- __week11__ __Adversarial learning & Latent Variables for NLP__
- Lecture: generative models recap, generative adversarial networks, variational autoencoders and why should you care about them.
- Seminar: semi-supervised dictionary learning with adversarial networks

- __week12__ __Text Summarization__
- Lecture: Text summarization methods. Extractive vs abstractive. A piece of extractive text summarization. Abstractive text summarization.

Contributors & course staff