Text classification means giving a piece of text a label from a fixed set of categories: spam or not spam, a positive or negative review, the right folder for an email. There are three broad ways to do it. You can write the rules yourself. You can train a classic machine learning model, such as naive Bayes, logistic regression or a support vector machine, on examples people have already labeled. Or you can start from a pre-trained language model: fine-tune one such as BERT on your labels, or prompt a large language model with none.
Below are the 10 methods and when to use each, but first a warning that applies to all of them. In scikit-learn’s own tutorial on sorting newsgroup posts, one of the strongest clues for the atheism category was the word “caltech”. Email addresses from earlier messages had leaked into the text, and the scores were over-optimistic until that metadata was stripped out. Whatever you pick, check what it has actually learned.
Need labeled text to train on? Every method here except rules and zero-shot prompting learns from labeled examples, and all of them need labeled test data before you can trust them. Zilo, based in Bangalore, provides text annotation for exactly this, alongside image and voice annotation and audio, video and multilingual transcription, and says its team of more than 1,600 trained annotation and speech recognition experts has annotated over 10 million data points.
Text classification algorithms at a glance
| # | Method | Needs labeled data? | Best for |
|---|---|---|---|
| 1 | Rule-based classification | No | A few stable categories with telltale keywords |
| 2 | Naive Bayes | Yes, but copes with little | Fast baselines, small datasets, short texts |
| 3 | Logistic regression | Yes | A baseline with probabilities you can act on |
| 4 | Support vector machines | Yes | Longer documents |
| 5 | k-nearest neighbors | Yes | Showing the similar documents behind each label |
| 6 | Random forest and XGBoost | Yes | Text mixed with structured data |
| 7 | fastText | Yes | Huge datasets and label sets, on CPUs |
| 8 | CNNs and LSTMs | Yes, lots | Many labeled samples of short text |
| 9 | Fine-tuned transformers | Yes, or a handful per class with SetFit | The highest accuracy |
| 10 | Zero-shot models and LLM prompts | No | Starting with no labels, or labels that change |
| + | Labeled training data from Zilo AI | Supplies the labels | Training and testing every method above except rules and zero-shot |
1. Rule-based text classification
With no labeled data, you can still classify text: write the rules yourself. The Stanford textbook Introduction to Information Retrieval gives an example: if a document mentions wheat or grain, and not whole or bread, file it under grain. Carefully tuned rules can be very accurate, but the authors estimate about two days of work per category, plus upkeep as the language drifts. They also suggest layering rules over a trained linear classifier, because writing a rule beats retuning weights when one document must be fixed today.
Best for: a few stable categories with obvious keywords, and as a manual override on a trained model.
2. Naive Bayes
Naive Bayes, like the next four methods, usually works on a bag of words: each document becomes a row of word counts, with word order thrown away and common words like “the” often down-weighted using TF-IDF. The model learns how often each word appears in each category, then picks the category that best explains a new document. It is “naive” because it treats each word as independent of the others once the category is known. That is rarely true of language, and it works well anyway, famously for spam filtering.
It trains extremely fast and needs little data. In scikit-learn’s newsgroup benchmark, it gave the best balance of accuracy and speed of the eight classic models tested. Pick the variant carefully: the section on probabilistic models below explains how.
Best for: a first baseline, small labeled sets and short texts.
3. Logistic regression
Despite the name, logistic regression is a classifier. It learns a weight for every word, adds up the weights in a document and turns the total into a probability. The weights are readable, so you can list the words that push a document toward each category. That is how scikit-learn’s tutorial caught the “caltech” leak, using a similar linear model.
A comparative review of text classification research, recently revised, notes findings that logistic regression still outperforms some much newer techniques.
Best for: an explainable baseline, especially when you need probabilities you can act on.
4. Support vector machines (SVM)
An SVM finds the boundary between two categories that leaves the widest gap to the nearest examples on either side. That suits text, where every word in the vocabulary is a dimension, and SVMs stay effective even with more dimensions than training samples. On a bag of words, a linear SVM is a standard baseline.
Wang and Manning found that naive Bayes did better on short sentiment snippets and SVMs on longer documents, and that an SVM fed naive Bayes word ratios (NBSVM) performed well across tasks and datasets. The catch: an SVM gives a score, not a probability, and converting it takes an extra round of cross-validation that is expensive on large datasets.
Best for: longer documents, such as full-length reviews.
5. k-nearest neighbors (kNN)
kNN barely trains. It stores every labeled document and labels a new one by a majority vote of the most similar ones. You can explain each decision by showing the neighbors, but prediction is slow, because every new document is compared with the whole training set. In scikit-learn’s benchmark its accuracy was relatively low, which the tutorial blames on the curse of dimensionality in text’s huge feature space.
Best for: cases where you want to show the similar past documents behind each label.
6. Random forest and XGBoost
Both methods combine many decision trees, and XGBoost, a gradient-boosted tree system, is widely used to get top results in machine learning challenges. On text alone, though, random forest struggled in scikit-learn’s benchmark: it was slow to train, expensive to predict and comparatively inaccurate. The tutorial explains that with 10,000 or more features, most problems can be split by a straight boundary, so linear models often fit better.
Consider gradient boosting when text is only part of the picture. XGBoost was built with a sparsity-aware algorithm and accepts sparse matrices directly, so you can feed it TF-IDF features next to ordinary columns such as order value or account age.
Best for: predictions that combine text with structured data.
7. fastText
fastText, published by Facebook AI Research in 2016, looks up a vector for each word and short word sequence (n-gram) in a document, averages them and passes the result to a linear classifier. Its paper reported accuracy often on par with deep learning classifiers, with training many orders of magnitude faster: more than a billion words in under ten minutes on a standard multicore CPU, and half a million sentences sorted among 312,000 classes in under a minute. One caution for 2026: the fastText GitHub repository was archived in March 2024, so the code is now read-only.
Best for: very large datasets or label sets when you want to stay on CPUs.
8. CNNs and LSTMs
These neural networks turn each word into a dense vector and read the sequence, so word order counts. A convolutional network (CNN) applies filters to windows of a few words to pick out telling phrases: Yoon Kim showed in 2014 that simple CNNs built on pre-trained word vectors improved on the best published results on 4 of 7 tasks, including sentiment and question classification. An LSTM reads one token at a time, using input, forget and output gates to control what it remembers.
To judge whether they beat a bag of words, Google’s text classification guide, based on about 450,000 experiments across 12 datasets, suggests dividing your number of samples by the median number of words per sample. Below 1,500, use word n-grams with a simple multi-layer perceptron. Above it, treat the text as sequences and use a separable CNN.
Best for: large labeled datasets of short texts, by that ratio.
9. Fine-tuned transformers (BERT and newer)
BERT, from Google AI Language in 2018, was pre-trained to predict hidden words from the context on both sides, reading BooksCorpus (800 million words) and English Wikipedia (2.5 billion words). To make it a classifier, you add one output layer and fine-tune it on your labels; every fine-tuning result in the paper can be reproduced in a few hours on a GPU. The review above finds that BERT-style models still lead supervised text classification, and Hugging Face’s guide walks you through fine-tuning DistilBERT on movie reviews.
Three variants fix the usual problems:
- Long documents: the original BERT reads at most 512 tokens. ModernBERT, introduced in December 2024, reads 8,192.
- Speed: DistilBERT is 40% smaller and 60% faster than BERT while keeping 97% of its language understanding, its authors report.
- Few labels: SetFit fine-tunes a small sentence-embedding model instead. Its authors report that with 8 labeled examples per class on a customer review dataset, it was competitive with RoBERTa Large trained on all 3,000.
Best for: the highest accuracy when you have labeled data and a GPU.
10. Zero-shot classification and LLM prompts
With no labels and no obvious keywords, you can classify by describing the categories. Hugging Face’s zero-shot pipeline uses a natural language inference model: it turns each label into a statement such as “This example is sports.” and scores how strongly your text supports it, so you can change the labels at run time without retraining. A large language model goes further: describe the task, add a few examples if you like, and it answers with no training at all, the setup the GPT-3 paper tested.
The catch is accuracy. A 2024 study found that smaller BERT-style models fine-tuned on task data beat zero-shot GPT-3.5, GPT-4 and Claude Opus on every classification task it tried. Start with prompts, and plan to fine-tune once you have labels.
Best for: new projects with no labels yet, and label sets that change often.
The best text classification model for your data
- You want a trained model but have no labels yet: get a labeled set first, for example from Zilo AI’s text annotation team, then choose from the options below.
- No labeled data: rules if your categories have clear keywords, otherwise a zero-shot model or an LLM prompt.
- A small labeled set: naive Bayes, or SetFit if you have only a handful of examples per class.
- A reasonable labeled set: logistic regression or a linear SVM first, then a fine-tuned BERT-style model if the gain justifies a GPU.
- Huge datasets: fastText. With huge amounts of data, the Stanford textbook notes, the choice of classifier matters less, so pick the one that scales.
- Long documents: a linear SVM, or ModernBERT for its 8,192-token window.
- Text plus structured columns: XGBoost.
Whatever you choose, beat this baseline first. Given a list of texts and their labels, it builds TF-IDF features from single words and word pairs, trains logistic regression and scores it with 5-fold cross-validation:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
model = make_pipeline(
TfidfVectorizer(ngram_range=(1, 2), sublinear_tf=True),
LogisticRegression(max_iter=1000),
)
scores = cross_val_score(model, texts, labels, cv=5, scoring="f1_macro")
print(scores.mean())Macro F1 weights every category equally, so a rare category the model keeps missing drags the score down instead of hiding behind high overall accuracy. If the score disappoints, check the labels before you change the model: our guide to data quality for machine learning shows how.
How to choose the right probabilistic model for a text classification task
If you need a probability rather than just a label, say to let only confident predictions through automatically, the two classic probabilistic models are naive Bayes and logistic regression. Three questions settle it.
- How much labeled data do you have? Ng and Jordan showed that naive Bayes can approach its best performance with far fewer examples, while logistic regression ends with lower error. Little data favors naive Bayes; more favors logistic regression.
- What do your documents look like? Following scikit-learn’s guidance, use multinomial naive Bayes for word counts and try Bernoulli naive Bayes for short texts. If some categories are far rarer than others, use complement naive Bayes, which regularly beats the multinomial version on text classification.
- Will you act on the scores? Naive Bayes is a decent classifier but a poor judge of its own confidence, and SVMs give no probabilities without an extra step. Logistic regression is more likely to be calibrated out of the box, so a score of 0.8 should mean right about 80% of the time.
Frequently asked questions
What is text classification?
Assigning a piece of text to one of a fixed set of categories, such as spam or not spam, usually by learning from examples people have labeled. Labeling individual words inside the text, such as the names of people and places, is a different task called named entity recognition.
Which model is best for text classification?
With enough labeled data, usually a fine-tuned BERT-style model, though logistic regression sometimes beats newer techniques, so test it first. With no labels, start with a zero-shot model or an LLM prompt.
Are LLMs better than BERT for text classification?
Not when you have labeled data: a 2024 study found fine-tuned BERT-style models beat zero-shot GPT-4 and Claude Opus on every task it tested. LLMs win on flexibility, since you can change the categories by editing a prompt.
What is rule-based text classification?
Sorting text with conditions you write yourself, such as keyword lists, so no training data is needed. Tuned rules can be very accurate, but each category takes days to build and needs upkeep.