It's always good to have a stock of non-examples.

§ Line with hole: Algebraic proof

The set of points $V \equiv \{ (t, t) : t \neq 42, t \in \mathbb R \} \subseteq mathbb R^2$ is not a variety. To prove this, assume it is a variety defined by equations $I(V)$. Let $f(x, y) \in I(V) \subseteq \mathbb R[x, y]$. Since $f$ vanishes on $V$, we must have $f(a, a) = 0$ for all $a \neq 42$ (since $(a, a) \in V$ for all $a \neq 42$). So create a new function $g(a) \equiv (a, a)$. Now $f \circ g: \mathbb R \rightarrow \mathbb R = f(g(a)) = f(a, a) = 0$. This polynomial (it is a composition of polynomial, and is thus a polynomial) has infinitely many zeroes, and is thus identically zero. So, $f(g(a)) = 0$, So $f(a, a) = 0$ for all $a$. In particular, $f(42, 42) = 0$ for all equations that define $V$, hence $(42, 42) \in I(V)$. But this does not give us the variety $V$. Hence $V$ is not a variety.

§ Line with hole: Analytic proof

The set of points $V \equiv \{ (t, t) : t \neq 42, t \in \mathbb R \} \subseteq mathbb R^2$ is not a variety. To prove this, assume it is a variety defined by equations $I(V)$. Let $f(x, y) \in I(V) \subseteq \mathbb R[x, y]$. Since $f$ vanishes on $V$, we must have $f(a, a) = 0$ for all $a \neq 42$ (since $(a, a) \in V$ for all $a \neq 42$). Since $f$ is continuous, $f$ preserves limits. Thus, $\lim_{x \to 42} f(x, x) = f(\lim_{x \to 42} (x, x))$. The left hand side is zero, hence the right hand size must be zero. Thus, $f(42, 42) = 0$. But this can't be, because $(42, 42) \not \in V$.

§ $\mathbb Z$

The set $\mathbb Z$ is not an algebraic variety. Suppose it is, and is the zero set of a collection of polynomials $\{ f_i \}$. Then for some $f_i$, they must vanish on at least all of $\mathbb Z$, and maybe more. This means that $f_i(z) = 0$ for all $z \in \mathbb Z$. But a degree $n$ polynomial can have at most $n$ roots, unless it is the zero polynomial. Since $f_i$ does not have a finite number of roots, $f_i = 0$. Thus, all the polynomials are identically zero, and so their zero set is not $\mathbb Z$; it is all of $\mathbb R$.

§ The general story

In general, we are using a combinatorial fact that a $n$ degree polynomial has at most $n$ roots. In some cases, we could have used analytic facts about continuity of polynomials, but it suffices to simply use combiantorial data which I find interesting.