Speech and Language Processing and Text Analytics with Python || Last Minute revision notes
š UNIT 1: Introduction & Basic Text Processing
š¹ 1. What is NLP?
Natural Language Processing (NLP) is a field of AI that enables machines to understand, interpret, and generate human language.
✔ Applications
- Machine Translation (Google Translate)
- Chatbots (ChatGPT)
- Sentiment Analysis
- Speech Recognition
š¹ 2. Problem of Ambiguity
Language is inherently ambiguous.
Types:
- Lexical ambiguity: “bank” (river bank vs financial bank)
- Syntactic ambiguity: “I saw a man with a telescope”
- Semantic ambiguity
š This is a core challenge in NLP.
š¹ 3. Regular Expressions (Regex)
Used for pattern matching in text.
Example:
- Email extraction:
[a-zA-Z0-9]+@[a-z]+\.com - Phone numbers, hashtags, dates
š Widely used in preprocessing tasks.
š¹ 4. Text Normalization
Converting text into a standard form
Steps:
- Lowercasing
- Removing punctuation
- Expanding contractions (don’t → do not)
š¹ 5. Tokenization
Splitting text into smaller units.
Types:
- Sentence tokenization
- Word tokenization
š Example:
"I love NLP" → ["I", "love", "NLP"]
š¹ 7. Stop-word Removal
Removing common words:
- the, is, and, of
š Helps reduce noise in data
š UNIT 2: Formal Language Modeling
š¹ 1. Byte Pair Encoding (BPE)
A subword tokenization technique.
Idea:
- Frequently occurring character pairs are merged
š Helps handle:
- Unknown words
- Rare words
š¹ 2. Minimum Edit Distance
Measures how different two strings are.
✨ Formula:
Operations:
- Insertion
- Deletion
- Substitution
š¹ 3. N-grams
Types:
- Unigram (1 word)
- Bigram (2 words)
- Trigram (3 words)
š Example:
“I love NLP”
- Unigram: I, love, NLP
- Bigram: I love, love NLP
š¹ 4. Markov Assumption
Future depends only on limited past.
š Bigram assumption:
š¹ 5. Maximum Likelihood Estimation (MLE)
š¹ 6. Smoothing
Fixes zero probability problem.
Types:
- Laplace (Add-1)
- Add-k
š¹ 7. Perplexity
Measures how good a language model is.
š Lower perplexity = better model
š UNIT 3: Vector Semantics & Embeddings
š¹ 1. Bag of Words (BoW)
Represents text as word frequency.
š Ignores order
š¹ 2. TF-IDF
š Highlights important words
š¹ 3. Word Embeddings
Convert words → vectors
Why?
Machines understand numbers, not words.
š¹ 4. Word2Vec
Neural model to learn embeddings
Types:
- CBOW (predict word from context)
- Skip-gram (predict context from word)
š¹ 5. Cosine Similarity
š Measures similarity between vectors
š UNIT 4: Sequence Labeling (POS & NER)
š¹ 1. Part-of-Speech (POS) Tagging
Assign grammatical tags:
- Noun, Verb, Adjective
š Example:
"I love NLP"
- I (PRON), love (VERB), NLP (NOUN)
š¹ 2. Named Entity Recognition (NER)
Identifies real-world entities:
Types:
- Person
- Location
- Organization
- Date
š Example:
"Elon Musk founded Tesla"
- Elon Musk → PERSON
- Tesla → ORG
š UNIT 5: Deep Learning for NLP
š¹ 1. Feedforward Neural Networks
Basic neural network structure.
Components:
- Input layer
- Hidden layers
- Output layer
š¹ 2. XOR Problem
Shows need for non-linear models
š¹ 3. NLP Classification
Used for:
- Spam detection
- Sentiment analysis
š¹ 4. Recurrent Neural Networks (RNN)
š Designed for sequential data
Key idea:
- Uses previous output as input
š¹ 5. LSTM (Long Short-Term Memory)
Improves RNN by handling long-term dependencies.
š¹ 6. Applications
✔ Sentiment Analysis
Classify text:
- Positive / Negative
✔ Text Classification
- News categorization
- Email filtering
š UNIT 6: Large Language Models & Transformers
š¹ 1. Limitations of Static Embeddings
Word2Vec gives same vector for word:
- “bank” (river & finance same)
š¹ 2. Contextual Embeddings
Word meaning depends on context.
š¹ 3. Transformers
š Most important modern NLP architecture
Key concept: Attention
Model focuses on important words.
š¹ 4. Transformer Architecture
- Encoder
- Decoder
- Self-attention
š¹ 5. Popular Models
✔ BERT
- Bidirectional understanding
✔ GPT
- Text generation
✔ RoBERTa
- Improved BERT
š¹ 6. Prompting
Giving instructions to LLMs.
š Example:
"Summarize this text"
š¹ 7. Ethical Issues
- Bias
- Hallucination
- Privacy concerns





Comments
Post a Comment