01. Differentiability in ℝ

Norm on ${\mathbb{R}}^n$

For $x=(x_1,x_2,\cdots ,x_n)\in {\mathbb{R}}^n$

$$|x|=\sqrt{\sum _{i=1}^n{x}_i^2}$$

Link to original

Limits in Higher Dimensions

For $f:\Omega \subseteq {\mathbb{R}}^n\to {\mathbb{R}}^m$ then

$f(\underline{x})\to \underline{b}$ as $\underline{x}\to \underline{a}$ if

$$\forall ϵ>0,\exists \delta >0\text{ such that }\forall x\in \Omega ,0<|\underline{x}-\underline{a}|<\delta ⟹|f(x)-b|<ϵ$$

Then we can write

$$\underset{\underline{x}\to \underline{a}}{\lim }f(\underline{x})=\underline{b}$$

Underlining is meant to represent vectors

Link to original

1.1 The Total Derivative

Total Derivative

Function $\Omega \subseteq {\mathbb{R}}^n\to {\mathbb{R}}^m$ is differentiable at $\underline{a}\in \Omega$ if

There exists Linear Map $L:{\mathbb{R}}^n\to {\mathbb{R}}^m$ such that

$$\underset{\underline{h}\to \underline{0}}{\lim }\frac{f(\underline{a}+\underline{h})-f(\underline{a})-\underline{L}h}{|\underline{h}|}=\underline{0}$$

$L$ is known as the (total) derivative of $f$ at $\underline{a}$ denoted as $df(\underline{a})$

Link to original

Differentiable Function in $\Omega$

If $f$ is differentiable at every point $\underline{a}\in \Omega$

Link to original

Recall that the matrix for linear map $L$ with respect to the usual bases of ${\mathbb{R}}^n$ and ${\mathbb{R}}^m$ is the Jacobian Matrix

Link to original

Proof - Usual Definition of Derivative not enough for $n>1$

Assume that $f$ is differentiable at $\underline{a}$ then
Define

$$R(\underline{h}):=\frac{f(\underline{a}+\underline{h})-f(\underline{a})-L\underline{h}}{|\underline{h}|}\in {\mathbb{R}}^m$$

Then

$$f(\underline{a}+\underline{h})-f(\underline{a})=\underline{L}h+R(\underline{h})|\underline{h}|$$

So for $\underline{h}\ne \underline{0}$

$$\frac{f(\underline{a}+\underline{h})-f(\underline{a})}{|\underline{h}|}=L\left(\frac{\underline{h}}{|\underline{h}|}\right)+R(\underline{h})$$

Where now $R(\underline{h})\to \underline{0}$ as $\underline{h}\to \underline{0}$
Letting $\underline{h}\to \underline{0}$ and as $L$ is independent of $\underline{h}$, then for $\underline{h}\ne \underline{0}$

$$\underset{\underline{h}\to \underline{0}}{\lim }\frac{f(\underline{a}+\underline{h})-f(\underline{a})}{|\underline{h}|}=L\underset{\underline{h}\to \underline{0}}{\lim }\left(\frac{\underline{h}}{|\underline{h}|}\right)$$

Assuming that both limits exists (if one does so does the other)

Case for $n=1$ For $n=1$ then $\underline{h}=h$ so $\frac{\underline{h}}{|\underline{h}|}=1\text{ or }-1$ Regardless of the sign, the limit exists so we get

$$\underset{h\to 0}{\lim }\frac{f(a+h)-f(a)}{h}=L$$

Now $L$ is the usual derivative but just interpreted as a linear map on $\mathbb{R}$

However for $n=2$ then $\frac{\underline{h}}{|\underline{h}|}$ is a unit length vector in direction of $\underline{h}\ne 0$
As there is complete freedom in $\underline{h}\to \underline{0}$, then any such sequence for $\underline{h}$ works

Hence the usual definition of the derivative is inadequate for $n>1$

Link to original

---

1.2 Directional Derivatives

Directional Derivatives

Let $\underline{n}={\mathbb{R}}^n$ be a unit vector then

$f:\Omega \subseteq {\mathbb{R}}^n\to {\mathbb{R}}^m$ is differentiable at $\underline{a}\in \Omega$ in direction $\underline{n}$ if

$$\underset{\lambda \to 0}{\lim }\frac{f(\underline{a}+\lambda \underline{n})-f(\underline{a})}{\lambda }\text{ exists}$$

Proof - Relation between total derivatives and directional derivatives

Assume that $f$ is (totally) differentiable at $\underline{a}$
Restrict $\underline{h}=\lambda \underline{n}$ for $\lambda \in \mathbb{R}$ so

$$\underset{\underline{h}\to \underline{0}}{\lim }\left(\frac{\underline{h}}{|\underline{h}|}\right)=\underline{n}\text{ does now exist}$$

Hence

$$\underset{\lambda \to 0}{\lim }\frac{f(\underline{a}+\lambda \underline{n})-f(\underline{a})}{\lambda }=L\underline{n}$$

So the derivative in direction $\underline{n}$ exists and given by applying matrix $L$ to unit vector $\underline{n}$

Link to original

---

1.3 Continuous Partial Derivatives give Differentiability

Partial Derivatives $n=2$ and $m=1$ such that we have

For simplicity consider

$$f:{\mathbb{R}}^2\to \mathbb{R}$$

Then for the directional derivatives consider the Basis Vectors $(1,0)$ and $(0,1)$

Suppose the directional derivatives exist then they are denoted (respectively)

$$f_x(a,b)\text{ and }{f}_y(a,b)$$

Assuming that $f:{\mathbb{R}}^2\to \mathbb{R}$ is differentiable at $\underline{a}=(a,b)$ then we have

$$df(\underline{a}):(1,0)\mapsto f_x(a,b)df(\underline{a}):(0,a)\mapsto f_y(a,b)$$

Writing $df(\underline{a})$ as a matrix with respect to the usual basis we get

$$df(\underline{a})=(f_x(a,b),f_y(a,b))$$

Link to original

01 - Differentiability Criterion

Differentiability Criterion

Suppose $f:{\mathbb{R}}^2\to \mathbb{R}$ has continuous partial derivatives then

$f$ is differentiable in ${\mathbb{R}}^2$ with derivative at $\underline{a}=(a,b)$ by

$$df(\underline{a})=(f_x(a,b),f_y(a,b))$$

Proof

Let $(a,b)\in \Omega$ and $(h,k)\in {\mathbb{R}}^2$
Define

$$L:=(f_x(a,b),f_y(a,b))$$

Need to show that

$$\underset{(h,k)\to \underline{0}}{\lim }\frac{f(a+h,b+k)-f(a,b)-L(h,k)}{|(h,k)|}=0$$

Using the telescoping sum

$$\begin{array}{rl}& f(a+h,b+k)-f(a,b)\\ & =(f(a+h,b+k)-f(a+h,b))+(f(a+h,b)-f(a,b))\end{array}$$

By Mean Value Theorem in one variable, there exists ${\theta }_h,{\theta }_k\in (0,1)$ s.t.

$$\begin{array}{rl}f(a+h,b+k)-f(a+h,b)& =f_y(a+h,b+{\theta }_{k}k)k\\ f(a+h,b)-f(a,b)& =f_x(a+{\omega }_{h}h,b)h\end{array}$$

(This works as the partial derivatives exist so we can use MVT)

Hence

$$f(a+h,b+k)-f(a,b)=f_y(a+h,b+{\theta }_{k}k)k+f_x(a+{\theta }_{h}h,b)h$$

Treating $(h,k)$ as a column vector for matrix $L$ then

$$L(h,k)=f_x(a,b)h+f_y(a,b)k$$

So

$$\begin{array}{rl}& f(a+h,b+k)-f(a,b)-L(h,k)\\ & =(f_x(a+{\theta }_{h}h,b)-f_x(a,b))h+(f_y(a+h,b+{\theta }_{k}k)-f_y(a,b))k\\ & =(f_x(a+{\theta }_{h}h,b)-f_x(a,b),f_y(a+h,b+{\theta }_{k}k)-f_y(a,b))\cdot (h,k)\end{array}$$

As it is a dot product of two vector, then by Cauchy Schwarz Inequality
The absolute value is at most

$$\sqrt{(f_x(a+{\theta }_{h}h,b)-f_x(a,b){)}^2+(f_y(a+h,b+{\theta }_{k}k)-f_y(a,b){)}^2}\cdot \sqrt{h^2+k^2}$$

For the limit to equal $0$ we need

$$\sqrt{(f_x(a+{\theta }_{h}h,b)-f_x(a,b){)}^2+(f_y(a+h,b+{\theta }_{k}k)-f_y(a,b){)}^2}\to 0$$

as $(h,k)\to \underline{0}$

Therefore

$$f_x(a+{\theta }_{h}h,b)\to f_x(a,b)\text{ and }{f}_y(a+h,b+{\theta }_{k}k)\to f_y(a,b)$$

Hence it tends to $0$ thus the limit equals $0$

Proof - Uniqueness of Total Derivatives from Partial Derivatives

For notational convenience and simplicity suppose

$$n=2\text{ and }m=1$$

Let $\Omega ={\mathbb{R}}^2$ and function $f:{\mathbb{R}}^2\to \mathbb{R}$
Suppose $\underline{a}=(a,b)$ and $\underline{h}=(h,k)$, and $f(a,b)\text{ represent }f((a,b))$ for notation

Consider the two directions $(0,1)$ and $(1,0)$ as the basis for ${\mathbb{R}}^2$

Let the two directional derivatives be $f_x(a,b)$ and $f_y(a,b)$ (partial derivatives)
Note that they can also be written as $\frac{\partial f}{\partial x}(a,b)$ and $\frac{\partial f}{\partial y}(a,b)$ but it takes longer :)

Assume that $f:{\mathbb{R}}^2\to \mathbb{R}$ is differentiable at point $\underline{a}=(a,b)$
With linear map $L=df(\underline{a}):{\mathbb{R}}^2\to \mathbb{R}$ being the derivative, then

$$df(\underline{a}):(1,0)\to f_x(a,b),df(\underline{a}):(0,1)\to f_y(a,b)$$

Hence $df(\underline{a})$ is unique, as partial derivatives are uniquely defined

Link to original

---

1.4 Example, counterexample, and some geometric intuition

NA

---

1.5 Continuity and the Chain Rule

Continuous Function

Function $f:\Omega \subseteq {\mathbb{R}}^n\to {\mathbb{R}}^m$ is continuous at $\underline{a}\in \Omega$ if

$$\forall ϵ>0,\exists \delta >0\text{ such that }\forall \underline{x}\in \Omega \text{ with }|\underline{x}-\underline{a}|<\delta ⟹|f(\underline{x})-f(\underline{a})|<ϵ$$

Which also means that

$$f(\underline{x})-f(\underline{a})\to \underline{0}\text{ as }\underline{x}\to \underline{a}$$

Link to original

02 - Differentiability Implies Continuity

Differentiability Implies Continuity

Assume $f:\Omega \subseteq {\mathbb{R}}^n>.{\mathbb{R}}^m$ is differentiable at $\underline{a}\subseteq \Omega$ then

$$f\text{ is continuous at }\underline{a}$$

Proof $L:{\mathbb{R}}^n\to {\mathbb{R}}^m$ then

Let the derivative be

$$f(\underline{a}+\underline{h})-f(\underline{a})=\underline{L}h+R(\underline{h})|\underline{h}|$$

where $R(\underline{h})\to \underline{0}$ as $\underline{h}\to \underline{0}$

Taking $\underline{h}\to 0$ on the right hand side gives $\underline{0}$ hence

$$f(\underline{a}+\underline{h})-f(\underline{a})\to 0\text{ as }\underline{h}\to \underline{0}$$

Link to original

03 - Chain Rule

Chain Rule

Let $f:{\mathbb{R}}^n\to {\mathbb{R}}^m$, $g:{\mathbb{R}}^m\to {\mathbb{R}}^p$, $\underline{a}\in {\mathbb{R}}^m$

Assume that $f$ is differentiable at $\underline{a}$ and $g$ is differentiable at $f(\underline{a})$

Let $h:=g\circ f$ then

$$h\text{ is differentiable at }\underline{a}\text{ with }dh(\underline{a})=dg(f(\underline{a}))df(\underline{a})$$

Omitted + Non-Examinable

Link to original

---

1.6 Some topology in ${\mathbb{R}}^{}$

1.6.1 Path-connected and connected sets

Open Ball

Let $r>0$ and $x\in {\mathbb{R}}^2$ then

$$B(x,r):=\{y\in {\mathbb{R}}^2:|x-y|<r\}$$

Link to original

Closed Ball

Let $r>0$ and $x\in {\mathbb{R}}^2$ then

$$\overline{B}(x,r):=\{y\in {\mathbb{R}}^2:|x-y|\le r\}$$

Link to original

Open Set

Let set $U\subseteq {\mathbb{R}}^2$ be open if

$$\forall x\in U:\exists r>0\text{ such that }B(x,r)\subseteq U$$

Link to original

Closed Set

Let set $U\subseteq {\mathbb{R}}^2$ be closed if

$$\text{complement }{\mathbb{R}}^2\setminus U\text{ is open}$$

Link to original

Open Neighbourhood

Given $a\in {\mathbb{R}}^2$ then

Any open set $U\subseteq {\mathbb{R}}^2$ containing $a$ is an open neighbourhood of $a$

Link to original

Topology in ${\mathbb{R}}^2$

The collection of all open sets in ${\mathbb{R}}^2$

Link to original

Path

Let $U\subseteq {\mathbb{R}}^2$ and $a_0,a_1\in U$

A path in $U$ between $a_0$ and $a_1$ is a continuous map $\gamma$ defined by

$$\gamma :[0,1]\to U\text{ with }\gamma (0)=a_1\text{ and }\gamma (1)=a_1$$

Link to original

Path Connected

For every $a_0,a_1\in U$ there exists a path between $a_0$ and $a_1$

Link to original

Homotopic

Let $U\subseteq {\mathbb{R}}^2$ and $a_0,a_1\in U$
Let ${\gamma }_0,{\gamma }_1$ be paths in $U$ connecting $a_0,a_1$

Then paths ${\gamma }_0,{\gamma }_1$ are homotopic if there exists $\Gamma$ defined by

$$\Gamma :[0,1]\times [0,1]\to U$$

with

$$\begin{array}{rlrlrlrl}& \Gamma (s,0)=a_0& & \text{ and }& & \Gamma (s,1)=a_1& & \text{ for all }s\in [0,1]\\ & \Gamma (0,t)={\gamma }_0(t)& & \text{ and }& & \Gamma (1,t)={\gamma }_1(t)& & \text{ for all }t\in [0,1]\end{array}$$

Can be useful to treat $\Gamma$ as a way of interpolation between the two paths

Link to original

Simply-Connected

Let $U\subseteq {\mathbb{R}}^2$ then

$U$ is simply connected if

1. path-connected
2. For $a_0,a_1\in U$ then any two paths in $U$ between $a_0$ and $a_1$ is homotopic

Link to original

04 - Path Connectedness of Images

Path Connectedness of Images

Let $U\subseteq {\mathbb{R}}^2$ be path-connected and $f:U\to {\mathbb{R}}^2$ be continuous then

$$f(U)\text{ is path-connected}$$

Proof

Let $v_0,v_1\in f(U)$ then

$$v_0=f(u_0)\text{ and }{v}_1=f(u_1)\text{ for some }{u}_0,u_1\in U$$

As $U$ is path connected then there is a path $\gamma$ joining $u_0$ and $v_0$

Then $f\circ \gamma$ gives the required path between $v_0$ and $v_1$

To show continuity use this to show the composition of two continuous maps is also continuous

Link to original

Connected

Let $U\subseteq {\mathbb{R}}^2$ then

$U$ is connected if for any open sets $A,B\in {\mathbb{R}}^2$ with

$$U\subseteq A\cup B\text{ and }(A\cap B)\cap U=\varnothing$$

Then

$$U\subseteq A\text{ or }U\subseteq B$$

Link to original

05 - Connectedness of Images

Connectedness of Images

Let $U\subseteq {\mathbb{R}}^2$ be connected and $f:U\to {\mathbb{R}}^2$ be continuous then

$$f(U)\text{ is continious}$$

Link to original

Path Connected Sets are always Connected

Link to original

Open Connected Set implies Path Connected (in ${\mathbb{R}}^2$)

Link to original

---

1.6.2 Compactness

Compactness

Let $U\subseteq {\mathbb{R}}^2$ then

$U$ is connected if
Given any collection $\{V_i:i\in I\}$ of open sets with $U\subseteq {\bigcup }_{i\in I}{V}_i$
There exists a finite subset $J\subseteq I$ such that $U\subseteq {\bigcup }_{j\in J}{V}_j$

Link to original

Bounded

$U\subseteq {\mathbb{R}}^2$ is bounded if

$$\exists x\in {\mathbb{R}}^2\text{ and }r>0\text{ such that}U\subseteq B(x,r)$$

Link to original

06 - Images of a Continuous Functions is Bounded

Images of a Continuous Functions is Bounded

Let $f:U\to {\mathbb{R}}^2\to \mathbb{R}$ be a continuous function
where $U$ is non-empty and compact then

$$f(U)\text{ is bounded and achieves that boud}$$

That is $\exists b\in \mathbb{R}$ and $a\in U$ such that

$$|f(x)|\le b\text{ for all }x\in U\text{ and }|f(a)|=b$$

Proof - Uses Lecture Topic 10 $U$ is compact then it is sequentially compact

Since

As $|f|$ is continuous (composition of two continuous functions)
Then $|f(U)|$ is non-empty and bounded so it has a supremum
Since $|f(U)|$ is closed then the supremum is in $|f(U)|$
Hence there exists $b\in \mathbb{R}$ and $a\in U$ such that $|f(x)|\le b$ for all $x\in U$ and $|f(a)|=b$

Link to original

---

1.6.3 Notions of distance in ${\mathbb{R}}^2$

Distances in ${\mathbb{R}}^2$

For $\underline{x}=(x_1,x_2)$ and $\underline{y}=(y_1,y_2)$
Let

$$d_2(\underline{x},\underline{y}):=|\underline{x}-\underline{y}|=\sqrt{(x_1-y_1{)}^2+(x_2-y_2{)}^2}$$

Link to original

Alternate Definition of Distance

$$d_{\infty }(\underline{x},\underline{y}):=\max (|x_1-y_1|,|x_2-y_2|)$$

Hence this gives the open ball of radius $r>0$ around point $\underline{a}\in {\mathbb{R}}^2$ to be

$$B_{\infty }(\underline{x},r)=\{\underline{y}\in {\mathbb{R}}^2:d_{\infty }(\underline{x},\underline{y})<r\}$$

Link to original

---

1.7 The Inverse Function Theorem

07 - Inverse Function Theorem in ℝ²

Inverse Function Theorem in ${\mathbb{R}}^2$

Let $\Omega \subseteq {\mathbb{R}}^2$ be open, $f\in C^1(\Omega ,{\mathbb{R}}^2)$ and $\underline{a}\in \Omega$
Suppose that $df(\underline{a}):{\mathbb{R}}^2\to {\mathbb{R}}^2$ is invertible then

There exists an open neighbourhood $U$ of $\underline{a}$ such that

$$f(U)\text{ is open }\text{ and }f:U\to f(U)\text{ is a bijection }$$

Where $f$ has an inverse $f^{-1}\in C^1(f(U),U)$
With $d{f}^{-1}(f(\underline{a}))=df(\underline{a}{)}^{-1}$

Showing if a function is locally invertible $f$ be a function of $(x,y)$ Need to find the Jacobian Matrix $df(x,y)$ We need the determinant of the matrix to be positive (and continuous) in the region So that it is invertible and hence the theorem applies

Let

Link to original