SMOTE and Oversampling Variants

You've just diagnosed the problem: your fraud detection model is useless because the data is 90:1 imbalanced. Accuracy says 0.90, but recall is 0.0 — the model never caught a single fraud. The fix seems obvious: resample the minority class until the dataset is balanced. So you copy your 6 fraud samples until they match the majority. Now the dataset is balanced, training loss drops nicely, and validation F1 hits 0.93. You feel good until you see the test F1: 0.50. The model memorized the same six points. It has no idea what a real fraud looks like in the wild.

There must be a better way — a way to create genuinely new minority samples instead of just recycling the same ones. That better way is SMOTE (Synthetic Minority Oversampling Technique), which generates new points along line segments between existing minority samples instead of duplicating them. This post walks through SMOTE, then four variants that address its weaknesses.

This post assumes you have read the previous one on imbalance diagnosis. The fraud detection context continues.

What SMOTE Does

SMOTE creates synthetic minority samples by interpolating between existing minority points — it places new points on the line segments connecting them. This is NOT the same as random oversampling (which copies the same points) because the synthetic points are genuinely new — they fall between real examples, not on top of them. The key insight: if your minority class has only 6 real points, SMOTE can generate dozens of plausible new ones by drawing lines between them and sampling anywhere along those lines.

The Anchor

To make every calculation traceable, use a 10-sample working subset in 2D — 6 minority (fraud) samples and 4 majority (legit) samples — just enough to hand-trace each oversampling algorithm and see when variants actually change the result. The features are scaled versions of amount and hour from the original fraud dataset.

import numpy as np

X_minority = np.array([
    [0.10, 0.90],   # x0
    [0.20, 0.80],   # x1
    [0.15, 0.85],   # x2
    [0.25, 0.75],   # x3
    [0.10, 0.70],   # x4
    [0.20, 0.95],   # x5
])  # fraud (class 1)

X_majority = np.array([
    [0.60, 0.30],   # m0
    [0.70, 0.40],   # m1
    [0.80, 0.20],   # m2
    [0.90, 0.50],   # m3
])  # legit (class 0)

X = np.vstack([X_minority, X_majority])
y = np.array([1, 1, 1, 1, 1, 1, 0, 0, 0, 0])

The minority cluster is in the upper-left, the majority in the lower-right. They are well separated — that becomes important for the Borderline-SMOTE and ADASYN sections later.

The Plan — Six Variants of Oversampling

We'll walk through six techniques on the same 10-sample fraud anchor. First we see why random oversampling fails (memorization), then trace SMOTE's interpolation math step by step, then examine four variants that address specific weaknesses.

Why Random Oversampling Fails

The simplest fix: duplicate the 6 minority samples until the dataset is balanced. The 6 samples become 24, with each point appearing 4 times.

from imblearn.over_sampling import RandomOverSampler

random_state=42 makes the random duplication pattern reproducible. Without it, you'd get different rows duplicated each run.
ros = RandomOverSampler(random_state=42)
X_resampled, y_resampled = ros.fit_resample(X, y)
print("Class counts:", np.bincount(y_resampled))

Class counts: [12 12]

The model now sees 12 of each class and trains evenly. The catch: 6 distinct minority points are now 12 copies. The decision boundary it learns wraps tightly around those 6 exact coordinates. Anything between them — anything new — is a wild guess.

The overfit signature shows up in the gap between training and validation:

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold
from sklearn.metrics import f1_score

n_splits=3 with 12 resampled samples gives 4 samples per fold — small but enough to illustrate the overfit pattern.
skf = StratifiedKFold(n_splits=3, shuffle=True, random_state=42)
train_f1, val_f1 = [], []
for tr, va in skf.split(X_resampled, y_resampled):
    m = LogisticRegression(random_state=42).fit(X_resampled[tr], y_resampled[tr])
    p_tr = m.predict(X_resampled[tr])
    p_va = m.predict(X_resampled[va])
    train_f1.append(f1_score(y_resampled[tr], p_tr))
    val_f1.append(f1_score(y_resampled[va], zero_division=0))
print(f"Train F1: {np.mean(train_f1):.3f}, Val F1: {np.mean(val_f1):.3f}")

Train F1: 0.929, Val F1: 0.500

Train F1 of 0.93 against validation F1 of 0.50 is the overfit signature. The model has memorized the duplicated minority points and learned to wrap a boundary around them, but it cannot generalize to the new majority samples in the validation fold.

✓ Step 1 complete. Random oversampling duplicated the 6 minority points to match the majority, producing a train F1 of 0.929 and validation F1 of 0.500 — the classic overfit signature of a model that memorized rather than learned.

Visually, the duplicated copies sit on top of the originals — six distinct positions with overlaid markers:

The faded markers behind each circle are the duplicate copies — the model only ever sees 6 unique positions.

How SMOTE interpolates between minority samples

Instead of duplicating, SMOTE interpolates. For each minority sample, SMOTE picks one of its k nearest minority neighbors and places a new point on the line between them.

Algorithm — four steps:

1. For each minority sample xᵢ, find its k=5 nearest minority neighbors (Euclidean distance).
2. Randomly select one of those k neighbors; call it xₙ.
3. The interpolation formula is straightforward — we take the starting point, add λ times the vector pointing from start to neighbor: x_syn = xᵢ + λ · (xₙ − xᵢ).
4. Add x_syn to the training set.

Each xᵢ can spawn multiple synthetic points, each on a different line segment with a different λ.

Worked Example for x₀ = [0.10, 0.90]

Let's compute the distance from x₀ to each of the other 5 minority samples. We'll use Euclidean distance in the 2D feature space:

Compute the Euclidean distance from x₀ to every other minority point:

The k=5 nearest minority neighbors are all five other points. Pick xₙ = x₁ = [0.20, 0.80] and λ = 0.6:

x_syn = [0.10 + 0.6 · (0.20 − 0.10), 0.90 + 0.6 · (0.80 − 0.90)] = [0.10 + 0.6 · 0.10, 0.90 + 0.6 · (−0.10)] = [0.10 + 0.06, 0.90 − 0.06] = [0.16, 0.84]

Three synthetic points with different λ values:

Each x_syn lies on the line segment between xᵢ and xₙ, at fraction λ along the way from xᵢ to xₙ.

Visually, the line segments connect each minority sample to a chosen neighbor, and the synthetic points sit on those lines:

The three green triangles are the synthetic points S1, S2, S3 — each on a dashed orange line between two minority circles.

Apply SMOTE to the anchor:

from imblearn.over_sampling import SMOTE

k_neighbors=5 is sklearn's default — with 6 minority samples, k=5 uses all other minority points as potential neighbors. random_state=42 controls which neighbor is selected and which λ value is drawn, ensuring every run produces the same synthetic points.
smote = SMOTE(k_neighbors=5, random_state=42)
X_resampled, y_resampled = smote.fit_resample(X, y)
print(f"Original shape: {X.shape}, Resampled shape: {X_resampled.shape}")
print(f"Original class counts: {np.bincount(y)}")
print(f"Resampled class counts: {np.bincount(y_resampled)}")

Original shape: (10, 2), Resampled shape: (12, 2)
Original class counts: [4 6]
Resampled class counts: [6 6]

Two new minority samples were generated, bringing 6 minorities up to match 4 majorities.

A Limitation SMOTE Introduces

SMOTE interpolates blindly between minority samples. If two minority points are on opposite sides of a majority cluster, the line between them crosses majority territory and the synthetic point lands inside the majority region. The model then sees a "minority" point in the wrong place.

In this anchor, the minority cluster is tight and well separated from the majority, so this problem is mild. In real datasets where minority and majority overlap, a simple SMOTE can amplify noise. The next two variants — Borderline-SMOTE and ADASYN — were designed to handle this.

✓ Step 2 complete. SMOTE generated 2 synthetic minority samples by interpolating along line segments between existing minority points. S1=[0.16, 0.84], S2=[0.215, 0.785], S3=[0.12, 0.91] — each on a line between two real minority samples.

Borderline-SMOTE: focusing on the decision boundary

The intuition: minority samples deep inside the minority region already get learned correctly. Synthesizing more points around them is wasted. Only the minority samples near the decision boundary — the borderline cases — need reinforcement.

The classification rule. For each minority sample xᵢ, look at the k=5 nearest neighbors from the entire dataset (both classes). Count how many of those 5 are majority:

• Safe: 0–2 majority neighbors → well inside the minority region
• Borderline / Danger: 3–4 majority neighbors → near the decision boundary
• Noise: 5 of 5 majority neighbors → surrounded by majority, likely mislabeled

Only borderline samples get SMOTE applied.

Classify Every Minority Sample on the Anchor

Compute the distance from each minority point to each of the 4 majority points:

Every minority point is at least 0.570 away from its nearest majority. By contrast, the nearest minority-to-minority distances (computed in the SMOTE section) are 0.07–0.21. The k=5 nearest neighbors of every minority sample are all other minority samples — zero majority in the neighborhood of any minority point.

The honest finding: on this anchor, Borderline-SMOTE does nothing. Every minority sample is safely tucked inside the minority cluster, with no majority neighbors in its k=5 nearest. This is a feature of the data, not a bug in the algorithm — the anchor's clusters are well separated.

This is exactly the case where Borderline-SMOTE has nothing to add. On a real fraud dataset where minority and majority overlap, samples near the boundary would be classified borderline and get reinforcement. On a clean separation like this, every minority is "safe" and the algorithm correctly does not over-generate.

from imblearn.over_sampling import BorderlineSMOTE

kind='borderline-1' means we classify minority samples based on the number of majority neighbors in k=5 and only synthesize new points for borderline ones. kind='borderline-2' also considers the distance to majority neighbors when synthesizing — borderline-1 is the more common choice.
bsmote = BorderlineSMOTE(kind="borderline-1", k_neighbors=5, random_state=42)
X_resampled, y_resampled = bsmote.fit_resample(X, y)
print(f"Resampled class counts: {np.bincount(y_resampled)}")

Resampled class counts: [4 6]

The resampled shape is identical to the original — the algorithm saw no borderline samples to amplify.

✓ Step 3 complete. Borderline-SMOTE classified all 6 minority samples as "safe" (zero majority neighbors in k=5), so it produced no synthetic points. The honest finding: on well-separated clusters, this variant is unnecessary.

ADASYN: weighting synthetics by local difficulty

ADASYN's twist: generate more synthetic samples near minority samples that are harder to learn. The "hardness" weight rᵢ is the fraction of majority neighbors:

• rᵢ = (# majority in k=5) / 5
• r̂ᵢ = rᵢ / Σrⱼ — normalize so the weights sum to 1
• nᵢ = round(r̂ᵢ · G) — number of synthetic samples to generate for xᵢ

Where G is the total number of synthetic samples to add.

Compute rᵢ for the 6 minority samples using the k=5 nearest from the entire dataset:

Sum: Σrᵢ = 0. The normalization step r̂ᵢ = rᵢ / 0 is undefined. With no difficulty signal in the data, ADASYN has no basis for adaptive allocation.

from imblearn.over_sampling import ADASYN

n_neighbors=5 defines the neighborhood for computing the difficulty weight rᵢ.
adasyn = ADASYN(n_neighbors=5, random_state=42)
X_resampled, y_resampled = adasyn.fit_resample(X, y)
print(f"Resampled class counts: {np.bincount(y_resampled)}")

Resampled class counts: [4 6]

The implementation falls back to a uniform distribution when all rᵢ are zero, so it does not crash. But the "adaptive" part of ADASYN — the part that concentrates synthetics near hard samples — is inert on this anchor.

This finding generalizes: if the minority and majority clusters are well separated, every oversampling variant degrades to plain SMOTE (or does nothing). The variants earn their keep only when minority and majority overlap and there is a non-trivial boundary.

✓ Step 4 complete. ADASYN computed rᵢ = 0 for all 6 minority samples — no difficulty signal to adapt to. The adaptive allocation was inert on this well-separated anchor.

SMOTE-Tomek: removing boundary ambiguity

SMOTE-Tomek is a two-phase hybrid. Phase 1: oversample with SMOTE. Phase 2: remove Tomek links.

A Tomek link is a pair of samples (a, b) from different classes where a is the nearest neighbor of b and b is the nearest neighbor of a. These pairs sit on the decision boundary, and the majority sample in the pair is usually the one to remove (it is the "least informative" majority).

Check whether any cross-class mutual nearest neighbor pairs exist in the anchor. The minority point closest to the majority cluster is x₃=[0.25,0.75] (distance 0.570 to m₀). The majority point closest to the minority cluster is m₀=[0.6,0.3] (same distance). For a Tomek link, we need each to be the other's nearest neighbor:

• x₃'s nearest neighbor is x₁=[0.20,0.80] at distance 0.0707 (minority), not m₀ at 0.570.
• m₀'s nearest neighbor is m₁=[0.7,0.4] at distance 0.141 (majority), not x₃ at 0.570.

No cross-class pair is a mutual nearest neighbor. The minority-to-minority distances (0.07–0.21) and majority-to-majority distances (0.14–0.36) are all far smaller than the closest cross-class gap of 0.570.

The SMOTE step generates 2 new minority samples, then the Tomek step looks for new Tomek links. With the well-separated anchor, no new Tomek links form — the SMOTE-generated points are far from the majority cluster.

from imblearn.combine import SMOTETomek

st = SMOTETomek(random_state=42)
X_resampled, y_resampled = st.fit_resample(X, y)
print(f"Resampled class counts: {np.bincount(y_resampled)}")

Resampled class counts: [4 6]

The output is identical to plain SMOTE on this anchor — the Tomek step had nothing to clean. On a noisier dataset, Tomek would remove the most ambiguous majority samples and tighten the boundary.

✓ Step 5 complete. No Tomek links were found — the minimum cross-class distance (0.570) was far larger than the nearest within-class distances (0.07–0.21). The cleaning step had nothing to do.

SMOTE-ENN: removing noise clusters

ENN (Edited Nearest Neighbors) is more aggressive than Tomek. The rule: remove any sample (majority or minority) whose k=3 nearest neighbors disagree with its class.

Apply the rule to the four samples closest to the decision boundary (two from each class):

Every sample's three nearest neighbors are same-class. The k=3 vote is unanimous for all 10 points — the well-separated clusters guarantee that the decision boundary region is empty and ENN has nothing to clean.

from imblearn.combine import SMOTEENN

senn = SMOTEENN(random_state=42)
X_resampled, y_resampled = senn.fit_resample(X, y)
print(f"Resampled class counts: {np.bincount(y_resampled)}")

Resampled class counts: [4 6]

Again, no change on a well-separated anchor. ENN's strength is removing clusters of noisy boundary samples, but the anchor has none.

✓ Step 6 complete. Every sample's k=3 nearest neighbors were same-class — unanimous votes across all 10 points. No noisy boundary samples to remove.

How the variants compare on the anchor

When It Works and When It Doesn't

SMOTE-family methods work best when the minority class is moderately sized (50+ samples), classes overlap enough that borderline detection methods have signal, and the minority class is roughly contiguous. They fail when:

• Minority samples are too few (<20) — synthetic points are interpolations of too little real data and don't add information
• The minority class has isolated sub-clusters — SMOTE interpolates between them into majority territory, creating noise
• The data is already well-separated — Borderline-SMOTE, ADASYN, and the hybrid cleaners have nothing to contribute beyond plain SMOTE

Hyperparameter Sensitivity

The key hyperparameter for SMOTE-family methods is k_neighbors — the number of neighbors used to draw a synthetic point from.

for k in [2, 3, 5, 7, 10]:
    s = SMOTE(k_neighbors=k, random_state=42)
    Xr, yr = s.fit_resample(X, y)
    print(f"k={k}: shape={Xr.shape}, n_synthetic_added={Xr.shape[0] - X.shape[0]}")

k=2: shape=(8, 2), n_synthetic_added=2
k=3: shape=(8, 2), n_synthetic_added=2
k=5: shape=(8, 2), n_synthetic_added=2
k=7: shape=(7, 2), n_synthetic_added=1
k=10: shape=(4, 2), n_synthetic_added=0

At small k (2, 3, 5), SMOTE generates 2 synthetic samples — the maximum the algorithm can produce with 6 minorities. As k grows, the neighbor set pulls in majority points (k=10 includes all 4 majority points in the "nearest" set since there are only 9 other points), which violates the assumption that neighbors are minority-class. The implementation refuses to generate points and produces an imbalanced output. In practice, k=5 is the standard default; smaller values generate tightly clustered synthetics, larger values may silently produce none.

Related Concepts

Backward: the previous post in this series, Handling Imbalanced Datasets, established why imbalanced classes break accuracy and how the four primary metrics (precision, recall, F1, balanced accuracy) expose the failure. SMOTE is a response to that failure mode — it changes the training data, not the model or the metric.

Forward: the next post in this series, Borderline-SMOTE, ADASYN, SMOTE-Tomek, and SMOTE-ENN in the 02-imbalaned-dataset/ section, treat these variants as their own anchor and walk through each algorithm step by step on a 2D fraud anchor. Beyond SMOTE-family methods, generative approaches (CTGAN, conditional VAEs) learn a full distribution of the minority class and sample from it — useful when minority samples are too few to interpolate reliably.

Honest Limitations

• Below ~50 minority samples, SMOTE cannot reliably help. Synthetic points are interpolations of too few real points. The "new" samples do not add information; they restate the existing distribution more densely. Use domain knowledge or active learning instead.
• SMOTE assumes the minority class is contiguous. If the minority actually has two separate sub-clusters, SMOTE synthesizes points between the sub-clusters — in the gap, which may be the majority region. The result is more noise, not less.
• Variants assume overlap exists. Borderline-SMOTE, ADASYN, and SMOTE-ENN all rely on a non-trivial boundary between classes. On a well-separated dataset like this anchor, they degrade to plain SMOTE or do nothing. Do not assume the more elaborate method is always better — measure on your data.

Test Your Understanding

1. Conceptual — SMOTE generates synthetic points along the line between a minority sample and one of its k nearest minority neighbors. What happens to the synthetic point if the two selected minority samples lie on opposite sides of a majority cluster? Why is this more likely with large k values than with small k values?
2. Applied — For x₀ = [0.10, 0.90] and chosen neighbor xₙ = [0.20, 0.80] with λ = 0.3, compute the synthetic point by hand. Where on the line segment does it land (fraction of the way from x₀ to xₙ)?
3. Applied — Compute the rᵢ value for a hypothetical minority point that has 4 majority points and 1 minority point in its k=5 nearest from the entire dataset. If Σrᵢ across all 6 anchor minority samples is 0.4, what is r̂ᵢ for this hypothetical point?
4. Edge case — Two minority samples are positioned on opposite sides of a majority cluster, distance 2.0 apart with the majority cluster in the middle. What does SMOTE place on the line between them? Why is this a problem?
5. Edge case — On a perfectly separated dataset (minority and majority 5+ distance apart), what does Borderline-SMOTE produce compared to plain SMOTE? What does ADASYN do when Σrᵢ = 0?