Processing math: 100%

Rylan Schaeffer

Logo
Resume
Research
Learning
Blog
Teaching
Jokes
Kernel Papers
Failures


“People have so many important things to communicate throughout their lives. They have so many things to talk about over a good beer.“

Ordinary Differential Equations

An ordinary differential equation (ODE) is an equation that relates a function to its derivative(s). Two example might be:

dy(x)dx=x,d2y(x)dx2=x+sin(x)

Each differential equation can be viewed as an operator d:FF that maps elements of a set to elements of the same set. For instance, consider the set of continuous, differentiable functions over the interval [a,b], denoted C1(a,b). For y(x)C1(a,b), one differential operator could be:

d(y(x))=d2y(x)dx2xsin(x)

Then d(y(x))=0 expresses the second example above.

Linear ODEs


We say that an operator l:FF is linear if it satisfies two properties. Letting y1(x),y2(x)C1(a,b) and λR, the properties are

  1. Element addition: l(y1(x)+y2(x))=l(y1)+l(y2)

  2. Scalar multiplication: l(λy(x))=λl(y(x))

3Blue1Brown has an amazing series on what linear operators mean geometrically, so for that insight, I highly recommend watching his material.

An operator is a differential operator if it takes some derivative of its argument. Note that a derivative dx=ddx is a linear operator:

dx(λ1y1(x)+λ2y2(x))=λ1dxy1(x)+λ2dxy2(x)

For brevity, we’ll often omit the argument to the function and just write y. A complete solution to any linear equation l(y(x))=h(x) has two parts: the null solution yn, which solves the homogeneous equation l(yn)=0, and the particular solution yp, which solves the inhomogeneous equation l(yp)=h(x). The sum of the two solutions provides the complete solution due to linearity:

l(yp+yn)=l(yp)+l(yn)=h(x)+0=h(x)

Linear, First Order ODEs


A generic linear, first-order ODE can be written as:

l(y(x))=a(x)dxy(x)+b(x)y(x)=h(x)

Intuition

For linear, first-order ODEs, exponentials are key. Why? Because all null solutions aim to solve the homogeneous equation:

l(y)=a(x)dxy(x)+b(x)y(x)=0

If a(x)=a and b(x)=b are constants, we immediately see that we are trying to find a function that is proportional to its own derivative - the exponential function!

dxy(x)=bay(x)y(x)=y(x0)eb(xx0)/a

If that isn’t persuasive, we can be a bit more rigorous, dividing by y and integrating both sides:

dxy+bay(x)=0dxyydy+bady=logy(x)logy(x0)+b(xx0)/a=y(x)=y(x0)eb(xx0)/a

For concreteness, suppose x is time and y(x) is your bank balance. Over time, interest will accrue, depositing more in your account as a function of the amount you put in initially y(x0), the ratio b/a and the elapsed time xx0. Now, suppose we deposit an additional dollar at a particular time x1. That added dollar will also continue growing exponentially, but the elapsed time since its deposit will be xx1, not xx0. Thus, exponentials remain key for the inhomogenous equation.

Null (Homogeneous) Solution

As we saw above, if l(y)=adxy(x)+by(x)=0, the solution is straightforward:

y(x)=y(x0)ebx/a

For non-constant coefficients a(x),b(x), we can use the same approach as before:

l(y(x))=a(x)dxy+b(x)y=0dxyy+b(x)a(x)=y(x)=y(x0)exx0b(u)a(u)du

Gilbert Strang dubs the exponential term the growth factor G(x0,x) because it describes how much a quantity will grow/decay from x0 to x.

G(x0,x)=exp(xx0b(u)a(u)du)

This growth factor will reappear in the solution to the particular equation.

Particular (Inhomogeneous) Solution via Integrating Factors

To find the solution y(x) to the linear, first order inhomogeneous equation

l(y)=a(x)dxy(x)+b(x)y(x)=h(x)

we’ll use a function f(x) called an integrating factor. Before starting, we’ll first divide by a(x) to clean things up.

dxy+b(x)a(x)y=ha

The motivation for an integrating factor is that if a function f(x) exists such that 1f(fy)=y+bay, then we could replace the inconvenient y+bay with a derivative (fy) that can be more easily integrated:

y+bay=1f(fy)=hafyf(x0)y(x0)=tt=x0h(t)f(t)a(t)dty(x)=f(x0)f(x)y(x0)+1f(x)xx0h(u)f(u)a(u)du

The question is now how to find f(x)? Set 1f(fy)=y+bay and solve for the integrating factor f(x):

1f(fy)=y+bay1f(fy+yf)=y+bayff=balogf(x)logf(x0)=xt=x0b(t)a(t)dtf(x)=f(x0)exp(xx0b(u)a(u)du)

Note that the integrating factor has the same growth factor as in the homogeneous case! So what is our final complete solution? We have

y(x)=f(x0)f(x)y(x0)+1f(x)xx0h(v)f(v)a(v)dv=exp(xx0b(u)a(u)du)y(x0)+exp(xx0b(u)a(u)du)xx0h(v)a(v)exp(vx0b(u)a(u)du)dv

Key Inhomogeneous Solutions

Although we have a general formula, certain input functions h(x) are more important than others.

  1. Constant h(x)=h:

  2. Step Function h(x)=cH(xx):

  3. Delta Function h(x)=δ(xx):

  4. Exponential Input h(x)=ect:

  5. Resonating Expoential Input h(x)=eax:

It has the complete solution:

y(x)=exp{(u=xu=x0b(u)a(u))du}[y(x0)+exp{(u=xu=x0b(u)a(u))du]

where f(x)=exp(u=xu=x0b(u)a(u))du)

Dimension of Linear, 1st-Order Kernel

We start by trying to find the null solution, which corresponds to identifying what the kernel of the operator is. Recall that for any operator l from one set to another, the kernel of the operator is the set of inputs that map to 0 i.e. ker(l)={vV:l(v)=0}. One property worth noting is that for a first-order linear operator l:VV with l(y)=a(x)y(x)+b(x)y(x), the kernel of l is one dimensional. To see this, we see that the solution $y_n$ to the homogeneous equation is:

l(y)=0ayn+byn=0ynyn=bayn=cexp(xx0b(t)a(t)dt)ker(l)=span(exp(xx0b(t)a(t)dt))

This tells us that any solution y can be written as a particular solution yp plus some constant c times the homogeneous aka null solution yn.

l(yyp)=l(y)l(yp)=h(x)h(x)=0yypker(l)yyp=cyny=yp+cyn

Solving Linear, 1st-Order: Variation of Parameters

Solving Linear, 1st-Order: Power Series Expansion

Another way to solve the differential equation is to Taylor Series expand both sides of the system and match coefficients.

Linear, 2nd-Order ODEs


Solving Linear, 2nd-Order: Variation of Parameters

For the linear, first-order ODE, we could solve the equation by first finding the null/homogeneous solution and then finding the particular/inhomogeneous solution by treating the constant as a variable. This approach will also work for a linear, second-order ODE. Suppose we know

l(y)=a(x)y(x)+b(x)y(x)+c(x)y=h(x) and ker(l)={f1(x),f2(x)}

The null/homogeneous solution (with constants $c_1, c_2 \in \mathbb{R}$) is

yn(x)=c1f1(x)+c2f2(x)

We suppose that the particular/inhomogeneous solution might have the same form but with variable coefficients:

yp(x)=c1(x)f1(x)+c2(x)f2(x)

Dropping $x$ for brevity and differentiating, we see that:

yp=c1f1+c1f1+c2f2+c2f2

and that

yp=c1f1+2c1f1+c1f1+2c2f2+c2f2+c2f2

Plugging into the inhomogeneous equation, we see a mess of terms that we can simplify a bit, taking advantage of the fact that some terms live in the kernel of $l$:

\begin{align} h(x) &= l(y_p) = a y_p’’ + b y_p’ + c y
&= a (c_1 ‘’ f_1 + 2 c_1’ f_1’ + c_1 f_1’’ + 2 c_2’ f_2 ‘ + c_2’’ f_2 + c_2 f_2’’) + b (c_1’ f_1 + c_1 f_1’ + c_2’ f_2 + c_2 f_2’) + c(c_1(x) f_1(x) + c_2(x) f_2(x))
&= a (c_1’’ f_1 + 2 c_1’ f_1’ + 2 c_2’ f_2’ + c_2’’ f_2) + b (c_1’ f_1 + c_2’ f_2) + c_1 (a f_1’’ + b f_1’ + c f_1) + c_2 (a f_2’’ + b f_2’ + c f_2)
&= a (c_1’’ f_1 + 2 c_1’ f_1’ + 2 c_2’ f_2’ + c_2’’ f_2) + b (c_1’ f_1 + c_2’ f_2) + c_1 l(f_1) + c_2 l(f_2)
&= a (c_1’’ f_1 + 2 c_1’ f_1’ + 2 c_2’ f_2’ + c_2’’ f_2) + b (c_1’ f_1 + c_2’ f_2) + 0 + 0
\end{align
}

Suppose someone gives you a hint and suggests that $c_1’ f_1 + c_2’ f_2 = 0$, which also implies that its derivative $c_1’’ f_1 + c_1’ f_1’ + c_2’’ f_2 + c_2’ f_2’ = 0$. This simplifies our equation tremendously:

h=a(c1f1+c2f2)

The question now is whether we can find two functions, c1,c2, that satisfy both equations:

c1f1+c2f2=0c1f1+c2f2=h(x)a(x)

We have two equations and two unknowns, meaning we can find a solution!

[f1(x)f2(x)f1(x)f2(x)][c1(x)c2(x)]=[0h(x)a(x)]

We invert the matrix:

[c1(x)c2(x)]=1f1f2f1f2[f2(x)f2(x)f1(x)f1(x)][0h(x)a(x)]

A commonly used term for the prefactor is the Wronskian, which we denote W(x)\defeqf1(x)f2(x)f1(x)f2(x). We then solve our the varying coefficients:

c1(x)=t=xt=x0f2(t)h(t)a(t)W(t)dt and c2(x)=t=xt=x0f1(t)h(t)a(t)W(t)dt

Thus our final solution to the inhomogeneous equation is:

yp=c1f1+c2f2=f1(x)t=xt=x0f2(t)h(t)a(t)W(t)dt+f2(x)t=xt=x0f1(t)h(t)a(t)W(t)dt