“People have so many important things to communicate throughout their lives. They have so many things to talk about over a good beer.“
An ordinary differential equation (ODE) is an equation that relates a function to its derivative(s). Two example might be:
dy(x)dx=x,d2y(x)dx2=x+sin(x)Each differential equation can be viewed as an operator d:F→F that maps elements of a set to elements of the same set. For instance, consider the set of continuous, differentiable functions over the interval [a,b], denoted C1(a,b). For y(x)∈C1(a,b), one differential operator could be:
d(y(x))=d2y(x)dx2−x−sin(x)Then d(y(x))=0 expresses the second example above.
We say that an operator l:F→F is linear if it satisfies two properties. Letting y1(x),y2(x)∈C1(a,b) and λ∈R, the properties are
Element addition: l(y1(x)+y2(x))=l(y1)+l(y2)
Scalar multiplication: l(λy(x))=λl(y(x))
3Blue1Brown has an amazing series on what linear operators mean geometrically, so for that insight, I highly recommend watching his material.
An operator is a differential operator if it takes some derivative of its argument. Note that a derivative dx=ddx is a linear operator:
dx(λ1y1(x)+λ2y2(x))=λ1dxy1(x)+λ2dxy2(x)For brevity, we’ll often omit the argument to the function and just write y. A complete solution to any linear equation l(y(x))=h(x) has two parts: the null solution yn, which solves the homogeneous equation l(yn)=0, and the particular solution yp, which solves the inhomogeneous equation l(yp)=h(x). The sum of the two solutions provides the complete solution due to linearity:
l(yp+yn)=l(yp)+l(yn)=h(x)+0=h(x)A generic linear, first-order ODE can be written as:
l(y(x))=a(x)dxy(x)+b(x)y(x)=h(x)For linear, first-order ODEs, exponentials are key. Why? Because all null solutions aim to solve the homogeneous equation:
l(y)=a(x)dxy(x)+b(x)y(x)=0If a(x)=a and b(x)=b are constants, we immediately see that we are trying to find a function that is proportional to its own derivative - the exponential function!
dxy(x)=−bay(x)→y(x)=y(x0)e−b(x−x0)/aIf that isn’t persuasive, we can be a bit more rigorous, dividing by y and integrating both sides:
dxy+bay(x)=0∫dxyydy+ba∫dy=logy(x)−logy(x0)+b(x−x0)/a=y(x)=y(x0)e−b(x−x0)/aFor concreteness, suppose x is time and y(x) is your bank balance. Over time, interest will accrue, depositing more in your account as a function of the amount you put in initially y(x0), the ratio b/a and the elapsed time x−x0. Now, suppose we deposit an additional dollar at a particular time x1. That added dollar will also continue growing exponentially, but the elapsed time since its deposit will be x−x1, not x−x0. Thus, exponentials remain key for the inhomogenous equation.
As we saw above, if l(y)=adxy(x)+by(x)=0, the solution is straightforward:
y(x)=y(x0)e−bx/aFor non-constant coefficients a(x),b(x), we can use the same approach as before:
l(y(x))=a(x)dxy+b(x)y=0dxyy+b(x)a(x)=y(x)=y(x0)e∫xx0b(u)a(u)duGilbert Strang dubs the exponential term the growth factor G(x0,x) because it describes how much a quantity will grow/decay from x0 to x.
G(x0,x)=exp(∫xx0b(u)a(u)du)This growth factor will reappear in the solution to the particular equation.
To find the solution y(x) to the linear, first order inhomogeneous equation
l(y)=a(x)dxy(x)+b(x)y(x)=h(x)we’ll use a function f(x) called an integrating factor. Before starting, we’ll first divide by a(x) to clean things up.
dxy+b(x)a(x)y=haThe motivation for an integrating factor is that if a function f(x) exists such that 1f(fy)′=y′+bay, then we could replace the inconvenient y′+bay with a derivative (fy)′ that can be more easily integrated:
y′+bay=1f(fy)′=hafy−f(x0)y(x0)=∫tt=x0h(t)f(t)a(t)dty(x)=f(x0)f(x)y(x0)+1f(x)∫xx0h(u)f(u)a(u)duThe question is now how to find f(x)? Set 1f(fy)′=y′+bay and solve for the integrating factor f(x):
1f(fy)′=y′+bay1f(f′y+yf′)=y′+bayf′f=balogf(x)−logf(x0)=∫xt=x0b(t)a(t)dtf(x)=f(x0)exp(∫xx0b(u)a(u)du)Note that the integrating factor has the same growth factor as in the homogeneous case! So what is our final complete solution? We have
y(x)=f(x0)f(x)y(x0)+1f(x)∫xx0h(v)f(v)a(v)dv=exp(∫xx0b(u)a(u)du)y(x0)+exp(−∫xx0b(u)a(u)du)∫xx0h(v)a(v)exp(−∫vx0b(u)a(u)du)dvAlthough we have a general formula, certain input functions h(x) are more important than others.
Constant h(x)=h:
Step Function h(x)=cH(x−x∗):
Delta Function h(x)=δ(x−x∗):
Exponential Input h(x)=ect:
Resonating Expoential Input h(x)=e−ax:
It has the complete solution:
y(x)=exp{(∫u=xu=x0b(u)a(u))du}[y(x0)+exp{(∫u=xu=x0b(u)a(u))du]where f(x)=exp(∫u=xu=x0b(u)a(u))du)
We start by trying to find the null solution, which corresponds to identifying what the kernel of the operator is. Recall that for any operator l from one set to another, the kernel of the operator is the set of inputs that map to 0 i.e. ker(l)={v∈V:l(v)=0}. One property worth noting is that for a first-order linear operator l:V→V with l(y)=a(x)y′(x)+b(x)y(x), the kernel of l is one dimensional. To see this, we see that the solution $y_n$ to the homogeneous equation is:
l(y)=0⇔ay′n+byn=0y′nyn=bayn=cexp(−∫xx0b(t)a(t)dt)ker(l)=span(exp(−∫xx0b(t)a(t)dt))This tells us that any solution y can be written as a particular solution yp plus some constant c times the homogeneous aka null solution yn.
l(y−yp)=l(y)−l(yp)=h(x)−h(x)=0y−yp∈ker(l)y−yp=cyny=yp+cynAnother way to solve the differential equation is to Taylor Series expand both sides of the system and match coefficients.
For the linear, first-order ODE, we could solve the equation by first finding the null/homogeneous solution and then finding the particular/inhomogeneous solution by treating the constant as a variable. This approach will also work for a linear, second-order ODE. Suppose we know
l(y)=a(x)y″(x)+b(x)y′(x)+c(x)y=h(x) and ker(l)={f1(x),f2(x)}The null/homogeneous solution (with constants $c_1, c_2 \in \mathbb{R}$) is
yn(x)=c1f1(x)+c2f2(x)We suppose that the particular/inhomogeneous solution might have the same form but with variable coefficients:
yp(x)=c1(x)f1(x)+c2(x)f2(x)Dropping $x$ for brevity and differentiating, we see that:
y′p=c′1f1+c1f′1+c′2f2+c2f′2and that
y″p=c″1f1+2c′1f′1+c1f″1+2c′2f′2+c″2f2+c2f″2Plugging into the inhomogeneous equation, we see a mess of terms that we can simplify a bit, taking advantage of the fact that some terms live in the kernel of $l$:
\begin{align}
h(x) &= l(y_p) = a y_p’’ + b y_p’ + c y
&= a (c_1 ‘’ f_1 + 2 c_1’ f_1’ + c_1 f_1’’ + 2 c_2’ f_2 ‘ + c_2’’ f_2 + c_2 f_2’’) +
b (c_1’ f_1 + c_1 f_1’ + c_2’ f_2 + c_2 f_2’) + c(c_1(x) f_1(x) + c_2(x) f_2(x))
&= a (c_1’’ f_1 + 2 c_1’ f_1’ + 2 c_2’ f_2’ + c_2’’ f_2) +
b (c_1’ f_1 + c_2’ f_2) +
c_1 (a f_1’’ + b f_1’ + c f_1) + c_2 (a f_2’’ + b f_2’ + c f_2)
&= a (c_1’’ f_1 + 2 c_1’ f_1’ + 2 c_2’ f_2’ + c_2’’ f_2) +
b (c_1’ f_1 + c_2’ f_2) +
c_1 l(f_1) + c_2 l(f_2)
&= a (c_1’’ f_1 + 2 c_1’ f_1’ + 2 c_2’ f_2’ + c_2’’ f_2) +
b (c_1’ f_1 + c_2’ f_2) +
0 + 0
\end{align}
Suppose someone gives you a hint and suggests that $c_1’ f_1 + c_2’ f_2 = 0$, which also implies that its derivative $c_1’’ f_1 + c_1’ f_1’ + c_2’’ f_2 + c_2’ f_2’ = 0$. This simplifies our equation tremendously:
h=a(c′1f′1+c′2f′2)The question now is whether we can find two functions, c1,c2, that satisfy both equations:
c′1f1+c′2f2=0c′1f′1+c′2f′2=h(x)a(x)We have two equations and two unknowns, meaning we can find a solution!
[f1(x)f2(x)f′1(x)f′2(x)][c′1(x)c′2(x)]=[0h(x)a(x)]We invert the matrix:
[c′1(x)c′2(x)]=1f1f′2−f′1f2[f′2(x)−f2(x)−f′1(x)f1(x)][0h(x)a(x)]A commonly used term for the prefactor is the Wronskian, which we denote W(x)\defeqf1(x)f′2(x)−f′1(x)f2(x). We then solve our the varying coefficients:
c1(x)=−∫t=xt=x0f2(t)h(t)a(t)W(t)dt and c2(x)=∫t=xt=x0f1(t)h(t)a(t)W(t)dtThus our final solution to the inhomogeneous equation is:
yp=c1f1+c2f2=−f1(x)∫t=xt=x0f2(t)h(t)a(t)W(t)dt+f2(x)∫t=xt=x0f1(t)h(t)a(t)W(t)dt