AI & Machine Learning Topic
Natural Language Processing: Build a Small Text Classifier
Natural language processing, or NLP, works with human language. An app may classify a message, find relevant documents, extract a date or generate a reply. These are different tasks, so start with one clear goal.
How text becomes numbers
A token is a unit used to process text. It can be a word, part of a word or a character. Tokenisation rules depend on the system and language. Splitting on spaces is not a universal language solution.
A count-based representation records word counts. TF-IDF changes their weights using how common terms are across documents. An embedding is a numeric representation learned or constructed to capture useful relationships. Nearby vectors are related according to that representation, not guaranteed to mean the same thing in every context.
Start with a small classification task
We will label invented lesson feedback as positive or negative. Install scikit-learn. Save this as nlp_demo.py:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
# Invented training examples; too small to establish real accuracy.
texts = ["clear helpful lesson","very useful guide","easy good example",
"excellent clear explanation","confusing broken example",
"very poor lesson","unclear difficult guide","bad confusing explanation"]
labels = [1,1,1,1,0,0,0,0]
model = make_pipeline(TfidfVectorizer(ngram_range=(1,2)),LogisticRegression())
model.fit(texts,labels)
for text in ["helpful clear guide","confusing lesson","not a good lesson"]:
print(text,"->",int(model.predict([text])[0]))
Run python nlp_demo.py. The labels are one for positive and zero for negative. The program prints predictions for new phrases. The dataset is deliberately tiny; it demonstrates the pipeline but cannot establish real accuracy.
Read the limitations
The model may handle a familiar word and still misunderstand “not a good lesson”. Negation, irony, spelling, dialect and mixed languages can change meaning. A confident-looking result is not proof of understanding.
The vectoriser and model are in one pipeline. For a real experiment, split labelled data first and fit the entire pipeline only on training folds. Keep a final test set. Do not learn a vocabulary from the full dataset before testing.
Should you remove words?
Lowercasing, removing punctuation or removing common words can help some tasks and harm others. “Not” may be crucial for sentiment. Names and punctuation may matter for extraction. Choose preparation from the task and compare results rather than copying a universal cleaning recipe.
What transformers add
A transformer-based model can produce representations that depend on surrounding text. The word “bank” can have a different representation near “river” than near “account”. That flexibility can improve many tasks, but it needs the right model, data and evaluation.
A pretrained model may need a large download and suitable memory. Check the model card, language coverage, licence and intended uses. Start with a simple baseline so you can measure whether the extra complexity helps.
Evaluate with real mistakes
Look at precision and recall for each class. Test short and long messages, spelling errors and relevant language groups. Check privacy before storing text. Separate repeated messages or the same author's related texts where needed to avoid an overly easy test.
For extracting a date, check whether the value is correct and valid. For summaries, check factual support and missing information. A general text similarity score does not answer every quality question.
Practice
Add invented examples with negation. Keep some separate for evaluation instead of teaching the model every phrase you later test. Explain why fitting one more example is not proof of a general improvement. Continue with RAG to learn how text search can support an answer.
Official references
Reviewed on 25 October 2026. Use documentation for your installed versions.
Comments (0)
No comments yet. Be the first to comment!