§ The implicit and inverse function theorem

I keep forgetting the precise conditions of these two theorems. So here I'm writing it down as a reference for myself.

§ Implicit function: Relation to function

  • Example 1: x2+y2=1x^2 + y^2 = 1 to y=1x2y = \sqrt{1 - x^2}.
If we have a function y=g(p,q)y = g(p, q), we can write this as yg(p,q)=0y - g(p, q) = 0. This can be taken as an implicit function h(y;p,q)=yg(p,q)h(y; p, q) = y - g(p, q). We then want to recover the explicit version of y=g(p,q)y = g'(p, q) such that h(g(p,q);p,q)=0h(g'(p, q); p, q) = 0. That is, we recover the original explicit formulation of y=g(p,q)y = g'(p, q) in a way that satisfies hh.

§ The 1D linear equation case

In the simplest possible case, assume the relationship between yy and pp is a linear one, given implicitly. So we have h(y;p)=αy+βp+γ=0h(y; p) = \alpha y + \beta p + \gamma = 0. Solving for h(y,p)=0h(y,p) = 0, one arrives at: y=1/α(βp+γ)y = -1/\alpha (\beta p + \gamma).
  • Note that the solution exists iff α0\alpha \neq 0.
  • Also note that the the existence of the solution is equivalent to asking that h/y=α0\partial h / \partial y = \alpha \neq 0.

§ The circle case

In the circle case, we have h(y;p)=p2+y21h(y; p) = p^2 + y^2 - 1. We can write y=±p21y = \pm \sqrt{p^2 - 1}. These are two solutions, not one, and hence a relation, not a function.
  • We can however build two functions by taking two parts. y+=+p21y+ = +\sqrt{p^2 - 1}; y=p21y- = -\sqrt{p^2 - 1}.
  • In this case, we have h/y=2y\partial h / \partial y = 2y, which changes sign for the two solutions. If y>0y^\star > 0, then (h/y)(y=0)(\partial h / \partial y)(y^\star = 0). Similarly for the negative case.

§ Assuming that a solution for h(y,p)h(y, p) exists

Let us say we wish to solve h(y;p)=y3+p23yp7=0h(y; p) = y^3 + p^2 - 3 yp - 7 = 0. Let's assume that we have a solution y=sol(p)y = sol(p) around the point (y=3,p=4)(y=3, p=4). Then we must have: sol(p)3+p23sol(p)p7=0sol(p)^3 + p^2 - 3 sol(p) p - 7 = 0. Differentiating by pp, we get: 3sol(p)2sol(p)+2p3sol(p)p3sol(p)=03 sol(p)^2 sol'(p) + 2p - 3 sol'(p) p - 3 sol(p) = 0. This gives us the condition on the derivative:
3sol(p)2sol(p)+2p3sol(p)p3sol(p)=0sol(p)[3sol(p)23p]=3sol(p)2psol(p)=[3sol(p)2p]/[3(sol(p)23p)] \begin{aligned} &3 sol(p)^2 sol'(p) + 2p - 3 sol'(p) p - 3 sol(p) = 0 \\ &sol'(p)\left[ 3 sol(p)^2 - 3p \right] = 3 sol(p) - 2p \\ &sol'(p) = [3 sol(p) - 2p] / \left[ 3(sol(p)^2 - 3p) \right] \\ \end{aligned}
The above solution exists if 3(sol(p)23p0)3(sol(p)^2 - 3p \neq 0). This quantity is again h/y\partial h / \partial y.

§ Application to economics

  • We have two inputs which are purchaed as x1x_1 units of input 1, x2x_2 units of input 22.
  • The price of the first input is w1BTC/unitw_1 BTC/unit. That of the second input is w2BTC/unitw_2 BTC/unit.
  • We produce an output which is sold at price wBTC/unitw BTC/unit.
  • For a given (x1,x2)(x_1, x_2) units of input, we can produce x1ax2bx_1^a x_2^b units of output where a+b<1a + b < 1. The Coob-douglas function .
  • The profit is going to be profit(x1x2,w1,w2,w)=w(x1ax2b)w1x1w2x2profit(x_1 x_2, w_1, w_2, w) = w(x_1^a x_2^b) - w_1 x_1 - w_2 x_2.
  • We want to select x1,x2x_1, x_2 to maximize profits.
  • Assume we are at break-even: profit(x1,x2,w1,w2,w)=0profit(x_1, x_2, w_1, w_2, w) = 0.
  • The implicit function theorem allows us to understand how any variable changes with respect to any other variable. It tells us that locally, for example, that the number of units of the first input we buy ( x1x_1) is a function of the price w1w_1. Moreover, we can show that it's a decreasing function of the price.

§ Inverse function: Function to Bijection

  • Given a differentiable function ff, at a point pp, we will have a continuous inverse f1(p)f^{-1}(p) if the derivative f(p)f'(p) is locally invertible.
  • The intuition is that we can approximate the original function with a linear function. y=f(p+δ)=f(p)+f(p)δy = f(p + \delta) = f(p) + f'(p) \delta. Now since f(p)f'(p) is locally invertible, we can solve for δ\delta. y=f(p)+f(p)δy = f(p) + f'(p) \deltaimplies that δ=1/f(p)[yf(p+δ)]\delta = 1/f'(p) [y - f(p + \delta) ]. This gives us the pre-image (p+delta)y(p + delta) \mapsto y.
  • The fact that 1/f(p)1/f'(p) is non-zero is the key property. This generalizes in multiple dimensions to saying that f(p)f'(p) is invertible.
One perspective we can adopt is that of Newton's method. Recall that Newton's method allows us to find xx^* for a fixed yy^* such that y=f(x)y^* = f(x^*). It follows the exact same process!
  • We start with some x[1]x[1].
  • We then find the tangent f(x[1])f'(x[1]).
  • We draw the tangent at the point x[1]x[1] as (y[2]y[1])=f(x[1])(x[2]x[1])(y[2] - y[1]) = f'(x[1])(x[2] - x[1]).
  • To find the yy^* we set y[2]=yy[2] = y^*.
  • This gives us x[2]=x[1](yy[1])/f(x[1])x[2] = x[1] - (y^* - y[1])/f'(x[1]).
  • Immediately generalizing, we get x[n+1]=x[n](yy[n])/f(x[n])x[n+1] = x[n] - (y^* - y[n]) / f'(x[n]).

§ Idea of proof of implicit function theorem (from first principles)

  • Let F(x,y)0F(x, y) \neq 0, point pp be the point where we wish to implicitize.
  • To apply implicit fn theorem, take ypF(x,y)0\partial_y|_pF(x, y) \neq 0. Say WLOG that ypF(x,y)>0\partial_y|_p F(x, y) > 0, since FF is assumed to be continuously differentiable.
  • Since yF\partial_y F is continuous and positive at pp, it's positive in a nbhd of pp by continuity.
  • Consider F(px,y)F(p_x, y) as a single variable function of yy. Its derivative with respect to yy is positive; it's an increasing function in terms of yy.
  • Since F(x0,y0)F(x_0, y_0) and is an increasing function of y0y_0, we must have two yy values y,y+y_-, y_+ such that F(px,y+)F(p_x, y_+) is positive, and F(px,y)F(p_x, y_-)is negative.
  • Since FF is zero and continuous, we have that for all xx near x0x_0, that F(x0,y+)>0    F(x,y+)>0F(x_0, y_+) > 0 \implies F(x, y_+) > 0 and F(x0,y)<0    F(x,y)<0F(x_0, y_-) < 0 \implies F(x, y_-) < 0. We have released x0x_0 into a wild xx in the neighbourhood!
  • Now pick some xx_* near xx. Since we have that F(x,y+)>0F(x_*, y_+) > 0 and F(x,y)<0F(x_*, y_-) < 0, there exists a unique yy_* (by MEAN VALUE THEOREM) that F(x,y)=0F(x_*, y_*) = 0
  • Since the yy_* is unique, we found a function: xfyx_* \xmapsto{f} y_*.
  • We are not done. We need to prove the formula for f(x)f'(x) where ff is the implicit mapping.
  • F(x,f(x))=0F(x, f(x)) = 0 by defn of f(x)f(x). Apply chain rule!
F(x,f(x))=0dF=0Fxxx+Fydfdx=0Fx1+Fydfdx=0dfdx=FyFx1 \begin{aligned} &F(x, f(x)) = 0 \\ &dF = 0 \\ &\frac{\partial F}{\partial x} \cdot \frac{\partial x}{\partial x} + \frac{\partial F}{\partial y} \frac{df}{dx} = 0 \\ &\frac{\partial F}{\partial x} \cdot 1 + \frac{\partial F}{\partial y} \frac{df}{dx} = 0 \\ &\frac{df}{dx} = \frac{- \frac{\partial F}{\partial y}}{\frac{\partial F}{\partial x} \cdot 1} \\ \end{aligned}

§ Idea of proof of inverse function theorem (from implicit function theorem)

  • Given the implicit function, say we want to locally invert f(x)f(x).
  • Pick the implicit function F(x,y)=f(y)xF(x, y) = f(y) - x. If we consider the level set F(x,y)=0F(x, y) = 0, the implicit function theorem grants us a g(x)g(x) such that F(x,y=g(x))=0F(x, y=g(x)) = 0. That is, we get f(g(x))x=0f(g(x)) - x = 0, or f(g(x))=xf(g(x)) = x.
  • To show that it's also a right inverse, consider F(f(k),k)=f(k)f(k)=0F(f(k), k) = f(k) - f(k) = 0. Since y=g(x)y = g(x), we have that k=g(f(k))k = g(f(k)).
  • Hence, gg and ff are both left and right inverses, and hence bijections.

§ Idea of proof of implicit function theorem (from inverse function theorem)

  • We are given a function F(x,y)F(x, y). We wish to find a formula y=g(x)y = g(x) such that F(x,g(x))=0F(x, g(x)) = 0.
  • The idea is to consider a function f(x,y)=(x,F(x,y))f(x, y) = (x, F(x, y)).
  • Since this is invertible, we get a local inverse function gg such that g(f(x,y))=(x,y)g(f(x, y)) = (x, y). That is, g(x,F(x,y))=(x,y)g(x, F(x,y)) = (x, y).
  • Now, for a given xx, set y:=snd(g(x,0))y := snd(g(x, 0)). This gives us a (x,y)(x, y) such that g(x,F(x,y)=0)=(x,y)g(x, F(x,y)=0) = (x, y). That is, we get a yy such that F(x,y)=0F(x, y) = 0 and this a bijection from xx to yy since gg is a bijection.

§ Idea of proof of inverse function theorem (from newton iterates)

We know that F(x+δx)=F(x)+Jδx+ηF(x + \delta x) = F(x) + J \delta x + \eta where JJ is the jacobian, η\eta is error term. Upto first order, this works. We take iterates of this process to get the full inverse.

§ References