Speech and Language Processing and Text Analytics with Python || Last Minute revision notes

 

šŸ“˜ UNIT 1: Introduction & Basic Text Processing

šŸ”¹ 1. What is NLP?

Natural Language Processing (NLP) is a field of AI that enables machines to understand, interpret, and generate human language.

✔ Applications

  • Machine Translation (Google Translate)
  • Chatbots (ChatGPT)
  • Sentiment Analysis
  • Speech Recognition

šŸ”¹ 2. Problem of Ambiguity

Language is inherently ambiguous.

Types:

  • Lexical ambiguity: “bank” (river bank vs financial bank)
  • Syntactic ambiguity: “I saw a man with a telescope”
  • Semantic ambiguity

šŸ‘‰ This is a core challenge in NLP.


šŸ”¹ 3. Regular Expressions (Regex)

Used for pattern matching in text.

Example:

  • Email extraction: [a-zA-Z0-9]+@[a-z]+\.com
  • Phone numbers, hashtags, dates

šŸ‘‰ Widely used in preprocessing tasks.


šŸ”¹ 4. Text Normalization

Converting text into a standard form

Steps:

  • Lowercasing
  • Removing punctuation
  • Expanding contractions (don’t → do not)

šŸ”¹ 5. Tokenization

Splitting text into smaller units.

Types:

  • Sentence tokenization
  • Word tokenization

šŸ‘‰ Example:
"I love NLP" → ["I", "love", "NLP"]












šŸ”¹ 7. Stop-word Removal

Removing common words:

  • the, is, and, of

šŸ‘‰ Helps reduce noise in data


šŸ“˜ UNIT 2: Formal Language Modeling

šŸ”¹ 1. Byte Pair Encoding (BPE)

A subword tokenization technique.

Idea:

  • Frequently occurring character pairs are merged

šŸ‘‰ Helps handle:

  • Unknown words
  • Rare words

šŸ”¹ 2. Minimum Edit Distance

Measures how different two strings are.

✨ Formula:


Operations:

  • Insertion
  • Deletion
  • Substitution

šŸ”¹ 3. N-grams

Types:

  • Unigram (1 word)
  • Bigram (2 words)
  • Trigram (3 words)

šŸ‘‰ Example:
“I love NLP”

  • Unigram: I, love, NLP
  • Bigram: I love, love NLP

šŸ”¹ 4. Markov Assumption

Future depends only on limited past.

šŸ‘‰ Bigram assumption:



šŸ”¹ 5. Maximum Likelihood Estimation (MLE)



šŸ”¹ 6. Smoothing

Fixes zero probability problem.

Types:

  • Laplace (Add-1)
  • Add-k

šŸ”¹ 7. Perplexity

Measures how good a language model is.


šŸ‘‰ Lower perplexity = better model


šŸ“˜ UNIT 3: Vector Semantics & Embeddings

šŸ”¹ 1. Bag of Words (BoW)

Represents text as word frequency.

šŸ‘‰ Ignores order


šŸ”¹ 2. TF-IDF


šŸ‘‰ Highlights important words


šŸ”¹ 3. Word Embeddings

Convert words → vectors

Why?

Machines understand numbers, not words.


šŸ”¹ 4. Word2Vec

Neural model to learn embeddings

Types:

  • CBOW (predict word from context)
  • Skip-gram (predict context from word)

šŸ”¹ 5. Cosine Similarity


šŸ‘‰ Measures similarity between vectors


šŸ“˜ UNIT 4: Sequence Labeling (POS & NER)

šŸ”¹ 1. Part-of-Speech (POS) Tagging

Assign grammatical tags:

  • Noun, Verb, Adjective

šŸ‘‰ Example:
"I love NLP"

  • I (PRON), love (VERB), NLP (NOUN)

šŸ”¹ 2. Named Entity Recognition (NER)

Identifies real-world entities:

Types:

  • Person
  • Location
  • Organization
  • Date

šŸ‘‰ Example:
"Elon Musk founded Tesla"

  • Elon Musk → PERSON
  • Tesla → ORG

šŸ“˜ UNIT 5: Deep Learning for NLP

šŸ”¹ 1. Feedforward Neural Networks

Basic neural network structure.

Components:

  • Input layer
  • Hidden layers
  • Output layer

šŸ”¹ 2. XOR Problem

Shows need for non-linear models


šŸ”¹ 3. NLP Classification

Used for:

  • Spam detection
  • Sentiment analysis

šŸ”¹ 4. Recurrent Neural Networks (RNN)

šŸ‘‰ Designed for sequential data

Key idea:

  • Uses previous output as input

šŸ”¹ 5. LSTM (Long Short-Term Memory)

Improves RNN by handling long-term dependencies.


šŸ”¹ 6. Applications

✔ Sentiment Analysis

Classify text:

  • Positive / Negative

✔ Text Classification

  • News categorization
  • Email filtering

šŸ“˜ UNIT 6: Large Language Models & Transformers

šŸ”¹ 1. Limitations of Static Embeddings

Word2Vec gives same vector for word:

  • “bank” (river & finance same)

šŸ”¹ 2. Contextual Embeddings

Word meaning depends on context.


šŸ”¹ 3. Transformers

šŸ‘‰ Most important modern NLP architecture

Key concept: Attention

Model focuses on important words.


šŸ”¹ 4. Transformer Architecture

  • Encoder
  • Decoder
  • Self-attention

šŸ”¹ 5. Popular Models

✔ BERT

  • Bidirectional understanding

✔ GPT

  • Text generation

✔ RoBERTa

  • Improved BERT

šŸ”¹ 6. Prompting

Giving instructions to LLMs.

šŸ‘‰ Example:
"Summarize this text"


šŸ”¹ 7. Ethical Issues

  • Bias
  • Hallucination
  • Privacy concerns

Comments

Popular posts from this blog

Why should a magnesium ribbon be cleaned before burning in air?

PHYSICAL WORLD

Current Electricity Notes Unit 3 Class 12 || Unit 3 || Physics || Class 12 || Chapter 3 Notes Class 12 ||

CHEMICAL BONDING AND MOLECULAR STRUCTURE