AI Foundations

Text Classification and Sentiment Analysis

Text classification assigns predefined categories to a piece of text, such as topic, spam, or intent labels. Sentiment analysis is its canonical case, predicting the polarity (positive, negative, or neutral) of an opinion expressed in text.

What text classification is

Text classification is the supervised learning task of assigning one or more predefined categories to a span of text. The category set is fixed in advance, which distinguishes classification from open-ended generation. Common applications include topic labeling (sorting articles into sports, finance, or politics), spam detection (legitimate versus unwanted email), language identification, and intent detection (mapping a user utterance to one of a fixed set of actions in a dialog system).

Sentiment analysis is the best-known instance of text classification. It predicts the polarity of the opinion expressed in a text, most often as positive, negative, or neutral. Because labeled product reviews, movie reviews, and social media posts are abundant, sentiment became an early proving ground for new classification methods, and it remains a standard demonstration task for language models.

A classifier is defined by its label space and its task framing. The same text can be classified along several independent axes at once, for example topic and sentiment together. What stays constant across these settings is the core pattern: map a variable-length string to a probability distribution over a finite, predefined set of classes, then pick the most likely class or classes.

Classification assigns predefined categories; the label set is chosen before training.
Topic labeling, spam detection, language ID, and intent detection are common uses.
Sentiment analysis predicts opinion polarity and is the canonical example.
The unifying pattern is mapping text to a distribution over a fixed label space.

Task framings: binary, multi-class, multi-label

Binary classification has exactly two mutually exclusive classes, such as spam versus not spam, or positive versus negative sentiment. The model outputs a single probability and a threshold (often 0.5) decides the label. Binary sentiment on movie reviews is the most cited example.

Multi-class classification has more than two mutually exclusive classes, where each input belongs to exactly one. Five-star rating prediction (1 to 5 stars) and fine-grained sentiment (very negative to very positive) are multi-class problems. The model produces a probability per class with a softmax, and the highest scoring class wins.

Multi-label classification allows each input to carry several labels at once. A news article can be tagged both economy and technology; a support ticket can have multiple intents. Here the classes are not mutually exclusive, so the model uses an independent sigmoid per label rather than a single softmax, and each label is thresholded separately.

Binary: two exclusive classes decided by a single thresholded probability.
Multi-class: more than two exclusive classes selected via softmax argmax.
Multi-label: several non-exclusive labels, each from an independent sigmoid.
The framing dictates the output layer: one sigmoid, a softmax, or many sigmoids.

From bag-of-words to transformers

Classical pipelines represented a document as a sparse bag-of-words or TF-IDF vector and fed it to a linear classifier. TF-IDF (term frequency times inverse document frequency) weights each word by how often it appears in a document and how rare it is across the corpus, downweighting common words like "the" and emphasizing discriminative terms. Multinomial Naive Bayes, logistic regression, and support vector machines (SVMs) were the standard classifiers on top of these features, and they remain strong, cheap baselines.

The neural era replaced sparse counts with dense word embeddings that place semantically similar words near each other in vector space. Convolutional neural networks (CNNs) over embedded word sequences captured local n-gram patterns, while recurrent networks such as LSTMs modeled longer-range word order. These models improved accuracy on tasks where word order and context matter, at the cost of more compute and data than the linear baselines.

The current standard is to fine-tune a pretrained transformer. BERT (introduced in 2018) and its successors are pretrained on large corpora, then adapted to a specific classification task with a small labeled set and a classification head on top of the model's pooled representation. Since the rise of large language models, zero-shot and few-shot classification has also become practical: an instruction-following model labels text directly from a prompt, with no task-specific training, though a fine-tuned smaller model is often cheaper and more accurate when ample labels exist.

tfidf(t, d, D) = tf(t, d) · idf(t, D),   idf(t, D) = log( N / n_t )

TF-IDF weight of term t in document d. N is the number of documents in corpus D and n_t is the number of documents containing t, so rare-but-frequent terms score highest.

python

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline

texts = ["the film was great", "a boring waste of time"]
labels = [1, 0]  # 1 = positive, 0 = negative

clf = make_pipeline(
    TfidfVectorizer(ngram_range=(1, 2)),
    LogisticRegression(max_iter=1000),
)
clf.fit(texts, labels)
print(clf.predict(["an absolute delight"]))  # -> [1]

A TF-IDF plus logistic regression sentiment baseline in scikit-learn, the cheapest strong starting point before reaching for transformers.

TF-IDF plus Naive Bayes, logistic regression, or SVM are durable linear baselines.
Word embeddings with CNNs or LSTMs added context and word-order sensitivity.
Fine-tuned transformers like BERT became the accuracy standard from 2018 on.
LLMs enable zero-shot and few-shot classification with no task-specific training.

Sentiment analysis specifics and hard cases

Beyond document-level polarity, aspect-based sentiment analysis (ABSA) extracts sentiment toward specific aspects of an entity. A review reading "great battery life but a terrible camera" is positive about battery and negative about camera, so a single document label would lose information. ABSA pairs each opinion target with its own polarity, which is more useful for product and service analytics.

Sentiment is hard because language is indirect. Negation flips polarity ("not good" is negative even though "good" is positive), and the negation cue can sit far from the word it modifies. Sarcasm and irony invert literal meaning ("oh great, another delay"), often with no surface signal, so they remain among the hardest cases even for large models. Intensifiers, comparatives, and mixed opinions within one sentence add further difficulty.

Domain shift is a practical failure mode. A model trained on movie reviews can degrade on tweets or clinical notes because vocabulary, register, and label conventions differ. The word "unpredictable" is praise for a thriller plot but a complaint about a car. Handling domain shift typically requires in-domain labeled data, domain-adaptive pretraining, or careful prompt design when using an LLM.

Aspect-based sentiment ties each opinion to a specific target, not the whole text.
Negation can flip polarity and may sit far from the word it modifies.
Sarcasm and irony invert literal meaning and stay hard even for large models.
Domain shift degrades accuracy when train and test vocabularies diverge.

Evaluating a classifier

Accuracy, the fraction of correct predictions, is intuitive but misleading under class imbalance. If 99 percent of email is legitimate, a model that labels everything legitimate scores 99 percent accuracy while catching no spam. Precision (of the items predicted positive, how many were correct) and recall (of the actual positives, how many were found) expose this gap, and their harmonic mean, the F1 score, balances the two in a single number.

With more than two classes, per-class F1 scores are aggregated. Macro averaging takes the unweighted mean of per-class F1, giving every class equal weight regardless of size, so rare classes count fully. Micro averaging pools all true positives, false positives, and false negatives across classes before computing one global F1, which weights classes by their frequency. Macro-F1 is preferred when minority-class performance matters; micro-F1 (equal to accuracy in single-label settings) reflects overall throughput.

Reporting choices should follow the cost structure of the application. Spam filtering tolerates missing some spam but punishes flagging real mail, favoring precision; disease screening favors recall. Confusion matrices, precision-recall curves, and per-class breakdowns reveal where a single aggregate number hides errors on the classes that matter most.

F1 = 2 · (precision · recall) / (precision + recall),   macro-F1 = (1/C) · Σ_{i=1}^{C} F1_i

F1 is the harmonic mean of precision and recall. Macro-F1 averages per-class F1 over all C classes with equal weight, so small classes are not drowned out.

Accuracy is unreliable under class imbalance and can hide a useless model.
F1 is the harmonic mean of precision and recall, balancing both error types.
Macro-F1 weights every class equally; micro-F1 weights by class frequency.
Choose the metric to match the application's cost of false positives versus negatives.

Benchmark datasets

The Large Movie Review Dataset, known as IMDb, contains 50,000 polar movie reviews split evenly into 25,000 training and 25,000 test examples, plus additional unlabeled reviews. Introduced by Maas and colleagues at ACL 2011, it is a standard binary sentiment benchmark and has been used to evaluate models from ELMo to BERT and beyond.

The Stanford Sentiment Treebank (SST), introduced by Socher and colleagues in 2013, annotates sentiment over the full parse trees of about 11,855 sentences from movie reviews, enabling fine-grained, phrase-level sentiment. SST-2 is the binary (positive versus negative) sentence-level version of this data and is one of the most reported sentiment tasks.

SST-2 is included as a task in GLUE (the General Language Understanding Evaluation benchmark), a multi-task suite introduced in 2018 that aggregates several language understanding tasks under one leaderboard. Its presence in GLUE made SST-2 a routine reporting target whenever a new model claims general language understanding ability.

python

from transformers import pipeline

# "sentiment-analysis" is an alias for the text-classification task
classifier = pipeline("sentiment-analysis")
result = classifier("This restaurant is awesome")
print(result)
# [{'label': 'POSITIVE', 'score': 0.9998743534088135}]

The Hugging Face transformers sentiment pipeline. Its default model is a DistilBERT fine-tuned on SST-2, so two lines give a transformer-grade classifier.

IMDb: 50,000 polar reviews, a 25k/25k train/test split, a binary sentiment standard.
SST annotates sentiment over parse trees of about 11,855 movie-review sentences.
SST-2 is the binary sentence-level version of the Stanford Sentiment Treebank.
GLUE bundles SST-2 with other tasks, making it a routine model-comparison target.

Key takeaways

Text classification assigns predefined categories to text; sentiment analysis predicting opinion polarity is its canonical case.
Task framing sets the output: binary and multi-class use one or a softmax of mutually exclusive labels, multi-label uses independent sigmoids.
Methods progressed from TF-IDF with Naive Bayes, logistic regression, or SVM, to embeddings with CNNs and LSTMs, to fine-tuned transformers, and now zero-shot and few-shot LLMs.
Sentiment is hard because of negation, sarcasm, aspect-level mixing, and domain shift between training and deployment text.
Under class imbalance, prefer precision, recall, and F1 over accuracy, and distinguish macro from micro averaging by which classes matter.
IMDb and SST-2 (the latter part of GLUE) are the standard sentiment benchmarks for comparing models.

Frequently asked questions

Text classification is the general task of assigning predefined categories to text, covering topic labeling, spam detection, and intent detection. Sentiment analysis is one specific kind of text classification whose categories are opinion polarities such as positive, negative, or neutral. Every sentiment analyzer is a text classifier, but not every text classifier does sentiment.

Start with a TF-IDF plus logistic regression or SVM baseline because it is fast, cheap, and often surprisingly strong on topic and polarity tasks. Move to a fine-tuned transformer like BERT when the baseline plateaus and you have enough labeled data and compute to justify it. If you have very few labels, a zero-shot or few-shot LLM prompt can beat both without any training.

Accuracy counts the fraction of correct predictions, which is misleading when classes are imbalanced. A spam filter on email that is 99 percent legitimate can reach 99 percent accuracy by predicting legitimate every time while catching no spam. Precision, recall, and F1 reveal performance on the minority class that accuracy hides.

Aspect-based sentiment analysis (ABSA) assigns sentiment to specific aspects of an entity rather than to the whole text. For a review saying the battery is great but the camera is terrible, ABSA records positive for battery and negative for camera. This is more informative than a single document-level label when a text expresses mixed opinions.

Macro-F1 computes F1 for each class and takes the unweighted average, so every class counts equally regardless of size. Micro-F1 pools the true positives, false positives, and false negatives across all classes before computing one global F1, which weights classes by frequency. Use macro-F1 when rare-class performance matters and micro-F1 when overall volume matters; in single-label settings micro-F1 equals accuracy.

Negation flips polarity, and the negation cue can be far from the word it modifies, so simple bag-of-words features miss it. Sarcasm and irony invert literal meaning while often leaving no surface signal, making them ambiguous even for large models. These cases require real contextual understanding and remain among the hardest in sentiment analysis.

Put the idea into practice

MemX is an AI memory agent built on these ideas: store anything, skip the folders, and find it again by asking in plain English.

Try MemX Free