On a fixed interval \(I\) on the real line, we have a point \(a\in I\). The \(N\)th-order Taylor polynomial for \(f : I\to\mathbb{R}\) at \(a\) is
\[P_N(x) = \sum\limits_{n=0}^N\frac{f^{(n)}(a)}{n!}(x-a)^n\]
where the constant \(\frac{f^{(n)}(a)}{n!}\) of the polynomal is chosen such that the \(N\) derivatives of the polynomial \(P_N(x)\) of degree \(N\) and the original function \(f(x)\) are equal at point \(a\):
\[\begin{array}{rl} P_N(a) &= f(a)\\ P_N'(a) &= f'(a)\\ P_N''(a) &= f''(a)\\ &= \vdots\\ P_N^{(N)}(a) &= f^{(N)}(a)\\ \end{array}\]
For example, to approximate \(e^x\) with \(P(x) = c_0 + c_1x + c_2x^2 + \dots\), we match derivatives at \(x = 0\):
- \(P(0) = f(0) \implies c_0 = 1\),
- \(P'(0) = f'(0) \implies c_1 = 1\),
- \(P''(0) = f''(0) \implies 2c_2 = 1 \implies c_2 = \frac{1}{2}\),
- \(P'''(0) = f'''(0) \implies ...\)
Taylor series of a real or complex-valued function \(f(x)\) that is infinitely differentiable at a real or complex number \(a\) is the power series
\[ P_\infty(x)=\sum_{n=0}^\infty \frac{f^{(n)}(a)}{n!}(x-a)^n \]
where \(f^{(n)}(a)\) denotes the \(n\)th derivative of \(f(x)\) evaluated at point \(a\). The derivative of order zero of \(f(x)\) is defined to be \(f(x)\) itself.
The Taylor series for any polynomial is the polynomial itself.
Taylor Approximation and Truncation Error
In practical use, we do not sum infinitely many terms. We stop after \(N\) terms and use the finite approximation
\[ P_N(x)=\sum\limits_{n=0}^{N}\frac{f^{(n)}(a)}{n!}(x-a)^n. \]
A standard remainder form (Lagrange form) states that for some \(\xi\) between \(a\) and \(x\),
\[ R_{N+1}(x)=f(x)-P_N(x)=\frac{f^{(N+1)}(\xi)}{(N+1)!}(x-a)^{N+1}. \]
Consequently, if \(\lvert f^{(N+1)}(t)\rvert\le M\) on the interval between \(a\) and \(x\), then
\[ \lvert f(x)-P_N(x)\rvert \le \frac{M}{(N+1)!}\lvert x-a\rvert^{N+1}. \]
This is why local approximations such as \(\tan(x)\approx x\) are useful for small \(\lvert x\rvert\): the first non-linear correction starts at order \(x^3\).
As a concrete example, take \(f(x)=\sin(x)\) at \(a=0\). Every derivative of \(\sin(x)\) is \(\pm\sin(x)\) or \(\pm\cos(x)\), so \(M=1\) always works. The first-order approximation \(P_1(x)=x\) then comes with the guarantee
\[ \lvert \sin(x)-x\rvert \le \frac{x^2}{2}. \]
For \(x=0.2\) this bounds the error by \(0.02\), so \(\sin(0.2)\in[0.18,0.22]\) — correct, since the true value is \(\sin(0.2)=0.198669\dots\). Now compare this to \(f(x)=\cos(x)\): the second-order approximation \(P_2(x)=1-\frac{x^2}{2}\) only guarantees \(\lvert\cos(x)-P_2(x)\rvert\le\frac{x^3}{6}\), i.e. an error of at most \(0.00133\) at \(x=0.2\). But \(\cos'''(x)=\sin(x)\) vanishes at \(a=0\), so the third-order polynomial \(P_3(x)\) equals \(P_2(x)\) exactly — the very same polynomial therefore also satisfies the sharper fourth-order bound \(\lvert\cos(x)-P_2(x)\rvert\le\frac{x^4}{24}\), only \(0.0000667\) at \(x=0.2\). Whenever an odd-order derivative happens to vanish at the expansion point, the next-higher bound comes for free.
Maclaurin series
When \(a=0\), the Taylor series is also called a Maclaurin series
\[ \sum_{n=0}^\infty \frac{f^{(n)}(0)}{n!} x^n \]
Geometric Series
We know the geometric series \(\sum\limits_{i=0}^\infty x^i\) converges to \(\frac{1}{1-x}\) for \(|x|<1\). Reversing this convergence with Maclaurin series is possible by determining the \(n\)th derivative
\[ \frac{\partial ^n}{\partial x^n} \frac{1}{1-x} = \frac{n!}{(1 - x)^{n + 1}} \]
From which we can find the original geometric series as a Maclaurin series:
\[\frac{1}{1-x} =\sum _{n=0}^{\infty }x^n = 1+x+x^2+x^3+\dots \forall x\in(-1,1)\]
Exponential function
\[ e^{x}=\exp(x)=\sum _{n=0}^{\infty }{\frac {x^{n}}{n!}}=1+x+{\frac {x^{2}}{2!}}+{\frac {x^{3}}{3!}}+\dotsb \]
This way \(e\) can be defined:
\[ e = \exp(1)=\sum _{n=0}^{\infty}{\frac {1}{n!}}=1+{\frac {1}{1!}}+{\frac {1}{2!}}+{\frac {1}{3!}}+\dotsb\]
Because every term of the series is positive for \(x>0\), keeping only the \((n+1)\)th term gives the lower bound \(\exp(x)>\dfrac{x^{n+1}}{(n+1)!}\), hence
\[ \frac{\exp(x)}{x^n} > \frac{x}{(n+1)!} \xrightarrow{x\to\infty} \infty \]
for every fixed \(n\). In other words, no matter how large a polynomial degree \(n\) we pick, the exponential function eventually outgrows \(x^n\).
Natural logarithms
\[\ln(1+x) = \sum \limits_{n=1}^{\infty }(-1)^{n+1}\frac{x^n}{n} = x-\frac{x^2}{2}+\frac{x^3}{3}-\frac{x^4}{4}+\dots\forall x\in(-1,1]\]
\[\ln(1-x) = \sum \limits_{n=1}^{\infty }(-1)^{n}\frac{x^n}{n} = -x+\frac{x^2}{2}-\frac{x^3}{3}+\frac{x^4}{4}-\dots\forall x\in(-1,1]\]
For the boundary on \(x=1\) we see that \(\ln(1+1) = \sum \limits_{n=1}^{\infty }(-1)^{n+1}\frac{1}{n}\) and therefore
\[\ln(2) = 1-\frac{1}{2}+\frac{1}{3}-\frac{1}{4}+\dots\]
\[\log\left(\frac{1+x}{1-x}\right) = \sum\limits_{n=1}^\infty \frac{2x^{2n-1}}{2n-1}\]
Cosine and Sine function
\[ \cos(x)=\sum _{n=0}^{\infty }(-1)^{n}{\frac {x^{2n}}{(2n)!}}={\frac {x^{0}}{0!}}-{\frac {x^{2}}{2!}}+{\frac {x^{4}}{4!}}-\frac{x^6}{6!}+ \dotsb \]
\[ \sin(x)=\sum _{n=0}^{\infty }(-1)^{n}{\frac {x^{2n+1}}{(2n+1)!}}={\frac {x^1}{1!}}-{\frac {x^{3}}{3!}}+{\frac {x^{5}}{5!}}- \frac{x^7}{7!}+\dotsb \]
It is interesting to see that the polynomial of sine has all the odd exponents divided by odd factorials and the cosine has all even exponents divided by even factorials, so they are kind of complementary.
Tangent function
\[ \tan(x) = x + \frac{1}{3}x^3 + \frac{2}{15}x^5 + \frac{17}{315}x^7 + \dotsb \]
Arcus Tangent function
\[\tan^{-1}(x) = \sum _{n=0}^{\infty }(-1)^{n}\frac{x^{2n+1}}{2n+1} = \frac{x^1}{1} -\frac{x^3}{3}+\frac{x^5}{5} -\frac{x^7}{7}+\dots\forall x\in[-1,1]\]
Arcus Cotangent function
\[\begin{array}{rl} \cot^{-1}(x) &= \frac{\pi}{2} - \tan^{-1}(x)\\ &= \frac{\pi}{2} - \sum _{n=0}^{\infty }(-1)^{n}\frac{x^{2n+1}}{2n+1} \end{array}\]
Hyperbolic functions
\[{\displaystyle {\begin{aligned}\cosh(z)&=1+{\frac {z^{2}}{2!}}+{\frac {z^{4}}{4!}}+{\frac {z^{6}}{6!}}+\dots =\sum _{n=0}^{\infty }{\frac {z^{2n}}{(2n)!}}\\\sinh(z)&=z+{\frac {z^{3}}{3!}}+{\frac {z^{5}}{5!}}+{\frac {z^{7}}{7!}}+\dots =\sum _{n=0}^{\infty }{\frac {z^{2n+1}}{(2n+1)!}}\end{aligned}}}\]
Binomial Series
For a real exponent \(\alpha\), consider \(f(x):=(1+x)^\alpha\). Its \(n\)th derivative at \(0\) is
\[ f^{(n)}(0) = \alpha(\alpha-1)(\alpha-2)\cdots(\alpha-n+1), \]
which motivates the generalized binomial coefficient
\[ \binom{\alpha}{n} := \frac{\alpha(\alpha-1)\cdots(\alpha-n+1)}{n!}, \qquad \binom{\alpha}{0} := 1. \]
The Maclaurin series of \(f(x)\) is then the binomial series
\[ (1+x)^\alpha = \sum_{n=0}^\infty \binom{\alpha}{n} x^n. \]
Two cases are worth telling apart:
- If \(\alpha\) is a non-negative integer, then \(\alpha-n+1=0\) as soon as \(n=\alpha+1\), so \(\binom{\alpha}{n}=0\) for every \(n>\alpha\). The series collapses to the finite sum of the ordinary binomial theorem \((1+x)^\alpha=\sum_{n=0}^\alpha\binom{\alpha}{n}x^n\), which holds for every \(x\in\mathbb{R}\) since it is just a polynomial identity.
- Otherwise the series never terminates, and its radius of convergence is \(1\): the ratio test gives \[ \left\lvert\frac{\binom{\alpha}{n+1}x^{n+1}}{\binom{\alpha}{n}x^n}\right\rvert = \left\lvert\frac{\alpha-n}{n+1}\right\rvert\lvert x\rvert \xrightarrow{n\to\infty} \lvert x\rvert, \] so \((1+x)^\alpha=\sum_{n=0}^\infty\binom{\alpha}{n}x^n\) for every \(\lvert x\rvert<1\).
The case \(\alpha=\frac12\) gives a Taylor series for square roots, and it is a convenient way to see an expansion around a point other than \(0\). Writing \(\sqrt{25+t}=5\sqrt{1+t/25}\) and expanding \((1+x)^{1/2}\) at \(x=t/25\) gives
\[ \sqrt{25+t} = 5\left(1+\frac12\cdot\frac{t}{25}-\frac18\cdot\frac{t^2}{25^2}+\dots\right). \]
For \(t=5\), i.e. approximating \(\sqrt{30}\), already the first three terms give \(5\left(1+0.1-0.005\right)=5.475\), close to the true value \(\sqrt{30}=5.477225\dots\). Choosing an expansion point near the value we care about — here \(25\), whose square root is known exactly — rather than always expanding at \(0\), keeps \(t/25\) small and makes the series converge quickly.
The case \(\alpha=-\frac12\) is what turns the relativistic mass \(m=m_0/\sqrt{1-v^2/c^2}\) into a series in powers of \(v^2/c^2\); keeping just the linear term recovers the classical kinetic energy \(\frac12 m_0v^2\) as the leading-order correction to \(m_0\) for \(v\ll c\).
Derivation of complex numbers
When we add the Maclaurin series of \(\cos(x)\) and \(\sin(x)\), we get
\[\cos(x) + \sin(x) = {\frac {x^{0}}{0!}}+{\frac {x}{1!}}-{\frac {x^{2}}{2!}}-{\frac {x^{3}}{3!}}+{\frac {x^{4}}{4!}}+{\frac {x^{5}}{5!}}-\frac{x^6}{6!}- \frac{x^7}{7!}+\dotsb\]
which is pretty similar to the Maclaurin series of \(e^x\). The only difference are the signs. In order to properly add the series, we need something that is negative when squared. The problem is that \(i^2\geq 0\forall i\in\mathbb{R}\). The solution is the clever trick of defining \(i^2=-1\), which gives the famous equation
\[\begin{array}{rl} e^{ix} &= \frac{(ix)^0}{0!} + \frac{(ix)^1}{1!} + \frac{(ix)^2}{2!} + \frac{(ix)^3}{3!} + \frac{(ix)^4}{4!} + \frac{(ix)^5}{5!} + \frac{(ix)^6}{6!} + \dotsb\\ &= \left({\frac {x^{0}}{0!}}-{\frac {x^{2}}{2!}}+{\frac {x^{4}}{4!}}-\frac{x^6}{6!}+ \dotsb\right)+ i\left({\frac {x}{1!}}-{\frac {x^{3}}{3!}}+{\frac {x^{5}}{5!}}- \frac{x^7}{7!}+\dotsb\right)\\ &= \cos(x) + i\sin(x) \end{array}\]
Plugging in \(x:= \pi\) results in what some call the most beautiful equation in mathematics, Euler’s identity:
\[e^{i\pi} + 1 = 0\]
But what is \(i\) anyway? Instead of seeing numbers on a line, this opens a whole plane of numbers, the Complex Numbers.