Natural Language Processing

Machine Learning NLP

Overview

Language models, text processing, and computational linguistics.

Disclaimer: Most of this was stolen from the HuggingFace documentation. These distilled notes were a learning exercise for myself.

Fundamentals

• Types of Tasks

Datasets

• GLUE
• NewsQA
• SQuAD
• Wikitext

Models

• BERT
• ELMo

Papers

• Xu et al. ACL 2020. Curriculum Learning for Natural Language Understanding
• Campos. Arxiv 2021. Curriculum Learning for Language Modeling
• Sanh et al. EMC @ NeurIPS 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Preprocessing

• TF-IDF