Distance of a Point from a Plane

A bank receives a loan application: income $45k, debt ratio 0.35. Should they approve it? The decision depends on where this applicant sits relative to the bank's decision boundary — a line separating "likely to repay" from "likely to default." The single number that captures both the decision and the confidence is the signed distance from the applicant's data point to that line.

The house price anchor from earlier posts can't demonstrate this — predicting a continuous price doesn't have a "sides of a line" concept. For signed distance, we need a classification scenario, so we'll introduce a minimal sub-dataset for this post only.

Anchor setup: Predict loan default from income ($x_1$, in $k) and debt ratio ($x_2$). A candidate decision line is $0.5x_1-3x_2-20=0$, so $a=0.5$, $b=-3$, $c=-20$. The denominator is $\sqrt{0.5^2+3^2}=\sqrt{0.25+9}=\sqrt{9.25}\approx 3.04$.

This is not the same as regression error (vertical distance from point to fitted line). Here the distance is perpendicular, and its sign — not just magnitude — carries the prediction.

Computing distances for three anchor points:

Point $(45,0.35)$ — income $45k, debt ratio 0.35:

$d=\frac{0.5\times 45+(-3)\times 0.35+(-20)}{3.04}=\frac{22.5-1.05-20}{3.04}=\frac{1.45}{3.04}\approx 0.477$

Point $(31,0.62)$ — income $31k, debt ratio 0.62:

$d=\frac{0.5\times 31+(-3)\times 0.62+(-20)}{3.04}=\frac{15.5-1.86-20}{3.04}=\frac{-6.36}{3.04}\approx -2.09$

Point $(95,0.12)$ — income $95k, debt ratio 0.12:

$d=\frac{0.5\times 95+(-3)\times 0.12+(-20)}{3.04}=\frac{47.5-0.36-20}{3.04}=\frac{27.14}{3.04}\approx 8.93$

Signed Distance — Which Side of the Line?

The sign of $a{x}_0+b{x}_0+c$ directly identifies which side of the decision boundary a point is on:

• Positive ($+1.45$ for point (45, 0.35)): on the side where $0.5x_1-3x_2-20>0$. Classifier predicts class +1.
• Negative ($-6.36$ for point (31, 0.62)): on the opposite side. Classifier predicts class −1.

This signed quantity is exactly what SVMs use. The sign is the prediction; the magnitude is the confidence. A point 8.93 units from the decision boundary is much more confidently classified than one 0.48 units away.

Normalizing the Line Equation

Dividing $a$, $b$, $c$ by $\sqrt{a^2+b^2}$ creates a normalized form where the denominator equals 1, simplifying the distance formula to just the numerator:

$a^{\prime }=\frac{0.5}{3.04}=0.164,b^{\prime }=\frac{-3}{3.04}=-0.987,c^{\prime }=\frac{-20}{3.04}=-6.58$

Check for point $(45,0.35)$:

$|0.164\times 45+(-0.987)\times 0.35-6.58|=|7.38-0.35-6.58|=|0.45|\approx 0.48✓$

The small difference from 0.477 is rounding. Normalizing is why SVM theory often assumes $\Vert \mathbf{w}\Vert =1$ — it makes the distance formula clean.

Extension to 3D: Distance from a Point to a Plane

In 3D, the plane equation is $ax+by+cz+d=0$. The distance formula extends naturally:

$\text{dist}=\frac{|a{x}_0+b{y}_0+c{z}_0+d|}{\sqrt{a^2+b^2+c^2}}$

For the plane $x+2y-3z+6=0$ and point $P=(1,2,3)$:

$\text{Numerator:}|1\times 1+2\times 2+(-3)\times 3+6|=|1+4-9+6|=|2|=2$

$\text{Denominator:}\sqrt{1^2+2^2+3^2}=\sqrt{14}\approx 3.742$

$\text{dist}=\frac{2}{3.742}\approx 0.535$

The geometry is identical to the 2D case: the denominator is the length of the normal vector to the plane, and dividing by it projects the point's displacement onto the unit normal.

Extension to p Dimensions: Distance from a Point to a Hyperplane

In $p$ dimensions the hyperplane is $\mathbf{w}\cdot \mathbf{x}+w_0=0$, and the distance from a point ${\mathbf{x}}_0$ to it is:

$d=\frac{|\mathbf{w}\cdot {\mathbf{x}}_0+w_0|}{\Vert \mathbf{w}\Vert }$

This is the margin formula in SVMs. The margin between two classes is defined as $\frac{2}{\Vert \mathbf{w}\Vert }$ — twice the minimum distance from the closest training point to the decision hyperplane. Maximizing the margin means minimizing $\Vert \mathbf{w}\Vert$, which is the SVM optimization objective.

Distance Formula Reference

Related Concepts and Honest Limitations

The signed distance formula is the mathematical backbone of SVMs (margin maximization), the log-odds in logistic regression (which is $\mathbf{w}\cdot \mathbf{x}$ before the sigmoid), and the geometric interpretation of regularization (constraining $\Vert \mathbf{w}\Vert$ effectively widens the margin). In the house price anchor from earlier posts, the same distance formula appears as the residual — but without the sign, because regression cares about magnitude only.

A common mistake is treating the signed distance as a reliable probability. A point 8.93 units from the boundary is confidently classified, but that doesn't mean it's 8.93 times more likely to belong to the positive class — the distance is in raw feature space, not probability space. The sigmoid in logistic regression exists precisely to convert one into the other.

Test Your Understanding

1. For the decision line $0.5x_1-3x_2-20=0$, compute the distance for the point $(67,0.28)$ using the formula. Which class does the sign predict?

2. If you multiply both sides of the line equation by $-1$ (so $-0.5x_1+3x_2+20=0$), does the distance change? Does the signed distance change?

3. In SVM, the margin is $2/\Vert \mathbf{w}\Vert$. If $\mathbf{w}=[0.5,-3]$ as in our anchor, what is the margin? To double the margin, what would you need to do to $\mathbf{w}$?

4. The normalization step changes $a=0.5$ to $a^{\prime }=0.164$. If you use the normalized coefficients in the distance formula, why does the denominator disappear?

5. A data point lies exactly on the decision hyperplane. What is its signed distance? What prediction does an SVM make for it, and why is this a problem in practice?