Section 7: Deep Learning for Text

Deep Learning for Text

Lecture slides in HTML format

Lecture slides in HTML format

Code

• Text vectorization, embedding, RMarkdown
• Reusing Word2vec embeddings, RMarkdown
• Attention math demo, RMarkdown
• Transformed architecture, RMarkdown

References

• Text analysis pipeline - text2vec tutorials
• NLP Illustrated, Part 3: Word2Vec - word2vec illustrated guide
• Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. “Attention is all you need.” Advances in neural information processing systems 30 (2017). https://arxiv.org/abs/1706.03762
• LLMs-from-scratch - Implement a ChatGPT-like LLM in PyTorch from scratch, step by step. Youtube playlist