Deep Learning for NLP – Part 5

Part 5: Efficient Transformer models

Created by: Manish Gupta
Principal Applied Researcher

What Will I Learn?

• Deep Learning for Natural Language Processing
• Efficient Transformer Models: Star Transformers, Sparse Transformers, Reformer, Longformer, Linformer, Synthesizer
• Efficient Transformer Models: ETC (Extended Transformer Construction), Big bird, Linear Transformer,Performer,Sparse Sinkhorn Transformer,Routing transformers
• Efficient Transformer benchmark: Long Range Arena
• Comparison of various efficient Transformer methods
• DL for NLP

Requirements

• Basics of machine learning
• Basic understanding of Transformer based models and word embeddings

Target audience

• Beginners in deep learning
• Python developers interested in data science concepts
• Masters or PhD students who wish to learn deep learning concepts quickly
• Folks wanting to ship their products across regions and languages (internationalization of their learning/predictive/generative models)