Build your first machine learning project in Python

Machine learning is a way of writing software where, instead of spelling out the rules, you show the program labelled examples and let it learn the pattern. The fastest way to understand it is to build one small project from start to finish, and Python is the usual language for that because its libraries cover every step.

This guide builds a spam filter for text messages. It is a classic first project for good reasons: the data is public and free, the problem is easy to picture, and it teaches the steps every real machine learning project follows. You will get the data, look at it, train two models, evaluate them honestly, use the better one on new messages, and see what changes when a project like this goes into production.

What you need before you start

You need Python 3.10 or later, a terminal and about an hour. Create a project folder with its own virtual environment so the libraries don’t clash with anything else on your machine:

mkdir spam-filter && cd spam-filter
python3 -m venv .venv
source .venv/bin/activate        # on Windows: .venv\Scripts\activate
pip install pandas scikit-learn joblib

If you like to experiment line by line, install Jupyter too (pip install notebook) and paste each script into a notebook cell. Everything below also runs as plain scripts.

The Python libraries for machine learning, and what each one does

Python’s machine learning tools are a set of libraries that each do one job well:

• NumPy holds numbers in fast arrays. Almost every other library is built on it, so you will use it even when you don’t import it.
• pandas loads, cleans and reshapes tables of data: reading a CSV file, dropping duplicates, counting values.
• scikit-learn is the standard library for classic machine learning: classification, regression and clustering models, plus the tools to split data, build features and measure results. Every model has the same fit and predict methods, which makes it easy to learn.
• Matplotlib and seaborn draw charts, so you can see your data and your model’s mistakes instead of reading tables of numbers.
• PyTorch and TensorFlow are deep learning frameworks for neural networks: images, audio, long text and very large datasets. You don’t need them for a first project.
• Jupyter gives you notebooks, where code, output and notes sit together. It is the usual place to explore data.

This project uses pandas and scikit-learn, plus joblib (a scikit-learn dependency, listed above to be explicit) to save the trained model.

Step 1: pick a problem a model can learn

A good first problem has three things: a clear yes-or-no (or pick-one) answer, a few thousand examples with that answer already attached, and a way to tell whether the model is right. Spam detection has all three. Each message is either spam or not, people have already labelled thousands of them, and you can check the model’s guesses against those labels.

Problems that make poor first projects are ones with no labelled data (“predict which customers will love our next product”), ones where the right answer is a matter of opinion, and ones where a simple rule would do the job just as well. If a single keyword catches almost all the cases, write the rule and skip the model.

Step 2: get the data

The SMS Spam Collection is a public dataset of 5,574 English text messages, each labelled spam or ham (not spam). It was published by Tiago Almeida, José María Gómez Hidalgo and Akebo Yamakami in 2011 and is hosted by the UCI Machine Learning Repository. This script downloads it and saves the data file:

# get_data.py: download the SMS Spam Collection (UCI Machine Learning Repository)
import io
import urllib.request
import zipfile
from pathlib import Path

URL = "https://archive.ics.uci.edu/static/public/228/sms+spam+collection.zip"

Path("data").mkdir(exist_ok=True)
with urllib.request.urlopen(URL, timeout=60) as response:
    archive = zipfile.ZipFile(io.BytesIO(response.read()))
archive.extract("SMSSpamCollection", "data")
print("Saved data/SMSSpamCollection")

Run it with python get_data.py. If the download fails, open the dataset’s page on the UCI repository in a browser, download the zip by hand and put the SMSSpamCollection file in a data folder.

Step 3: look at the data before you train anything

Looking at the data first is the step beginners skip and professionals never do. It tells you how the data is shaped, how balanced the labels are and what needs cleaning:

# explore.py: load the messages and look at them before training anything
import csv

import pandas as pd

df = pd.read_csv(
    "data/SMSSpamCollection",
    sep="\t",
    header=None,
    names=["label", "text"],
    quoting=csv.QUOTE_NONE,  # messages contain stray quote marks
)
print(df.shape)
print(df["label"].value_counts())
print(df.sample(5, random_state=1))

dupes = df.duplicated().sum()
print(f"{dupes} exact duplicate rows")

Three findings matter:

• There are 5,574 messages, and 747 of them are spam. That is about 13%. When one label is much rarer than the other, plain accuracy becomes a misleading score, as Step 5 shows.
• There are 403 exact duplicate rows. The same message appears more than once. If a copy lands in the training data and another in the test data, the model is tested on something it has already seen, and the score looks better than it really is. Remove them before splitting.
• The text is messy. Abbreviations, slang, odd spacing and quote marks are normal for text messages. The quoting=csv.QUOTE_NONE setting stops pandas from treating a stray quote mark as the start of a quoted field.

Step 4: train and compare two models

A model can’t read text, so the first job is to turn each message into numbers. TF-IDF (term frequency, inverse document frequency) does this by giving each word a weight: high when the word is frequent in this message but rare across all messages, so words like “the” count for little and words like “prize” count for a lot.

This script cleans the data, holds back a test set, sets a baseline, compares two models with cross-validation, then trains the better one and saves it:

# train.py: clean the data, train a spam classifier, evaluate it and save it
import csv

import joblib
import pandas as pd
from sklearn.dummy import DummyClassifier
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import make_pipeline

# 1. Load and clean
df = pd.read_csv(
    "data/SMSSpamCollection", sep="\t", header=None,
    names=["label", "text"], quoting=csv.QUOTE_NONE,
)
df = df.dropna().drop_duplicates(subset="text")  # a repeated message must not sit in both train and test
X = df["text"]
y = (df["label"] == "spam").astype(int)  # 1 = spam, 0 = not spam

# 2. Hold back a test set the model never sees while training
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42,
)

# 3. A baseline: always guess "not spam"
baseline = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print(f"Baseline accuracy: {baseline.score(X_test, y_test):.3f}")

# 4. Two candidate models, each a pipeline: text -> TF-IDF features -> classifier
candidates = {
    "naive_bayes": make_pipeline(TfidfVectorizer(lowercase=True), MultinomialNB()),
    "logistic_regression": make_pipeline(
        TfidfVectorizer(lowercase=True, ngram_range=(1, 2), min_df=2),
        LogisticRegression(max_iter=1000, class_weight="balanced"),
    ),
}
for name, model in candidates.items():
    scores = cross_val_score(model, X_train, y_train, cv=5, scoring="f1")
    print(f"{name}: cross-validated F1 {scores.mean():.3f} (+/- {scores.std():.3f})")

# 5. Train the chosen model on all the training data, then test it once
model = candidates["logistic_regression"].fit(X_train, y_train)
predictions = model.predict(X_test)
print(confusion_matrix(y_test, predictions))
print(classification_report(y_test, predictions, target_names=["not spam", "spam"]))

# 6. Save it for the prediction script
joblib.dump(model, "spam_model.joblib")
print("Saved spam_model.joblib")

What each part does, and why:

• train_test_split with stratify=y keeps 20% of the messages aside, with the same share of spam as the whole set. The model never sees them during training, so they stand in for the messages it will meet in real use.
• A pipeline (make_pipeline) bundles the text-to-numbers step and the model together. The TF-IDF weights are learned from the training data only, and the saved model can take raw text later without any extra steps.
• Cross-validation (cross_val_score with cv=5) trains and scores each candidate five times on different slices of the training data. That gives a fairer comparison than a single split, and it never touches the test set.
• Two candidates. Naive Bayes is fast and a common starting point for text. Logistic regression with word pairs (ngram_range=(1, 2)) and class_weight="balanced", which makes mistakes on the rare spam class count for more, usually does better on this data. Try others: the point is to compare on the same footing, not to guess.

Run it with python train.py. On scikit-learn 1.6, logistic regression scored a cross-validated F1 of about 0.93 against about 0.75 for Naive Bayes with default settings, so the script trains logistic regression. Your numbers may differ slightly with other library versions.

Step 5: evaluate it honestly

The baseline in the script always answers “not spam”. It scores about 87% accuracy, because about 87% of the messages aren’t spam, and it catches no spam at all. That is why accuracy alone is the wrong score for this project. Use these instead:

• Precision for spam: of the messages the model flagged as spam, how many really were. Low precision means real messages end up in the spam folder.
• Recall for spam: of all the real spam, how much the model caught. Low recall means spam gets through.
• F1: a single score that balances precision and recall.
• The confusion matrix: the four counts behind those scores. Real messages kept, real messages wrongly flagged, spam missed and spam caught.

On the held-back test set, the trained model reached about 0.92 precision and 0.92 recall for spam: it wrongly flagged 11 of 904 real messages and missed 11 of 131 spam messages. Whether that is good enough depends on the cost of each mistake. For a spam filter, a lost real message is usually worse than a missed spam, so you might raise the threshold for calling something spam and accept more misses.

Test on the held-back set once, at the end. If you keep adjusting the model until the test score improves, the test set stops being a fair stand-in for new data, and the score you report will flatter the model.

Step 6: use the model on new messages

A trained model is only useful when something can call it. This script loads the saved model and scores messages you pass on the command line:

# predict.py: load the saved model and score new messages
import sys

import joblib

# joblib files can run code when loaded: only load model files you created yourself
model = joblib.load("spam_model.joblib")
messages = sys.argv[1:] or [
    "Are we still on for lunch tomorrow?",
    "WINNER! You have been selected for a free prize. Call now to claim.",
]
for text, p in zip(messages, model.predict_proba(messages)[:, 1]):
    label = "spam" if p >= 0.5 else "not spam"
    print(f"{label:>8}  {p:.2f}  {text}")

Run python predict.py to score the two examples, or python predict.py "your message here" to try your own. The number is the model’s estimated probability that the message is spam.

Try a modern scam, such as a fake parcel-delivery message asking you to confirm your address at a link. The model is far less sure about it than about the prize message, because scams like that were rare when the dataset was collected over a decade ago. That is your first lesson in data drift: a model only knows the world its training data showed it, and that world changes.

Designing a learning system in machine learning

Designing a learning system means deciding four things before writing code: the task, the measure of success, the experience the system learns from, and how that experience is represented. Tom Mitchell’s textbook Machine Learning (1997) frames it this way: a program learns from experience E with respect to task T and performance measure P if its performance at T, as measured by P, improves with E.

For this project:

• Task (T): label a text message as spam or not spam.
• Performance measure (P): precision and recall for spam on messages the model hasn’t seen, not accuracy.
• Experience (E): 5,574 messages labelled by people, cleaned of duplicates.
• Representation: TF-IDF weights for words and word pairs, fed to a linear model.

Writing these four lines down before you start is the most useful habit you can take from a first project. Most failed projects get one of them wrong: the task is vague, the measure doesn’t match what the business cares about, or the labelled experience doesn’t exist.

When is a large language model the better tool?

A large language model (LLM) is often the better tool when you have few or no labelled examples, when the categories change often, or when you need the system to explain its answer or write text. You describe the task in a prompt instead of training a model, so you can have a working version in an afternoon.

A small trained model like this one is usually the better tool when:

• You have labelled data and the task is stable.
• You need volume. It scores thousands of messages a second on an ordinary server, at almost no cost per prediction.
• Speed and predictability matter. It answers in milliseconds and gives the same answer for the same input.
• The data must stay on your machines. Nothing is sent to an outside API.

Many production systems use both: a cheap trained model handles the clear cases, and an LLM or a person handles the uncertain ones. If you want to try the LLM side next, our guide to build a chatbot in Python is a good second project.

What changes in a production project

A notebook that scores well is the smaller part of the work. A production machine learning project adds:

• A data pipeline. Data arrives continuously from real systems, gets cleaned the same way every time, and is versioned, so you can rebuild any past model exactly.
• Labelling. Someone has to supply correct answers for new examples, with clear guidelines, so the model can be retrained and checked.
• An evaluation set that reflects real use. A fixed set of hard, recent and important examples, checked before every release.
• Human review where mistakes are costly. Uncertain predictions go to a person, and their decisions become new training data.
• Monitoring. Tracking the share of each prediction and the model’s confidence over time shows drift, like the parcel scam above, before users notice it.
• A way to roll back to the previous model if a new one does worse.

A real example of the data-collection and extraction side is Crime and Punishment, which Tailwebs built and maintains for the Vidhi Centre for Legal Policy. It is a database of 7,305 crimes across 370 central laws, where a scraping engine plus AI extracts punishment data and tags law sections.

Projects where collection, extraction and the application around them have to work together are what AI development covers. For models and LLM features inside a product, see AI and LLM development. Python is a common choice for the back ends and data pipelines around them; see Python development.

Where to go next

Once the spam filter works, try these in order:

1. Plot the mistakes. Print the messages the model got wrong and look for patterns. This teaches more than any new algorithm.
2. Tune the threshold. Change the 0.5 cut-off in predict.py and watch precision and recall move in opposite directions.
3. Try a new dataset with the same workflow, such as classifying support tickets or product reviews. Kaggle hosts many public datasets.
4. Learn the theory alongside. The scikit-learn user guide and Google’s Machine Learning Crash Course are both free and practical.
5. Move to deep learning with PyTorch when you reach images, audio or long text.

The short version

• A spam classifier is a good first machine learning project in Python: public data, a clear answer and an honest way to check it.
• Look at the data first. Remove duplicates and notice when one label is rare.
• Hold back a test set, set a baseline and compare models with cross-validation.
• Judge a classifier by precision and recall, not accuracy alone.
• Production adds pipelines, labelling, evaluation sets, human review and monitoring.