Spam or Ham: An Email Classifier

A classifier labels each email as spam or legitimate. The training set holds 4,171 emails, 1,200 spam and 2,971 legitimate, with 1,000 more for testing.

scikit-learnSVMTF-IDFFeature Engineering

0.984

Kaggle private accuracy

4,171

training emails

5,072

engineered features

+15.9

points over pre-built features

Problem

Understanding the challenge and user needs

Labeling every email legitimate already scores 71 percent, so accuracy alone flatters a weak model. Word counts alone miss part of the signal, so the features have to describe how an email reads as well as what it says.

Approach

Strategy and methods used to address the problem

Feature engineering combined 5,000 TF-IDF terms with 72 hand-built signals for a total of 5,072 features. Cross-validation tuned the kernel and regularization strength of a support vector machine.

Waterfall chart decomposing the accuracy gain into three steps — An estimated breakdown of the gain: scaling the custom features accounts for most of it, with refinements and tuning adding the rest.

Final Solution

The final design and outcome

An RBF-kernel SVM scored 98.4 percent on Kaggle's private leaderboard, against 82.5 percent for the same search on the 32 pre-built features the competition supplied.

Bar chart comparing accuracy of the pre-built feature pipeline and the raw data pipeline — The same model family and search scores 0.825 on the pre-built features and 0.984 on features engineered from the raw emails.

Grouped bar chart of local validation, public, and private accuracy per submission — Local validation tracked the private leaderboard closely on every submission.

Four submissions, three scores each

Submission	Local validation	Kaggle public	Kaggle private
Raw emails, baseline	0.721	0.690	0.712
Pre-built features, baseline	0.810	0.816	0.822
Raw emails, tuned	0.974	0.986	0.984
Pre-built features, tuned	0.823	0.810	0.825

The pre-built set holds 32 anonymous features; the raw pipeline builds 5,072.

Impact & Validation

Results, learnings, and outcomes

The sixteen-point gap came from the features, not the algorithm, which makes the case for understanding the data before tuning the model.

Bar chart of the gap between public and private scores per submission — The gap between public and private scores stays within sampling noise for all but one linear-only search.