Open ballA concept that we will use in the rest of the document is that of an open ball. An open ball centered at a point (a,b) with radius r is the following set:{(x,y):(x-a)2+(y-b)2<r2}In simple terms, it is the set of all points within the circle centered at (a,b) with radius r. Note that the boundary is not a part of an open ball. Visually:All points in the shaded region belong to the open ball. Note that the boundary is given bya dotted line and is not a part of the open ball.(a,b)r
2. Limits and Continuity2.1. Limits of SequencesConsider an=
1n
. This sequence converges to 0 as n→∞. That is, as n increases, an gets closer and closer to 0. We denote this as an→0. A sequence (an,bn) in R2 converges to (a,b) if an→a and bn→b. For instance (1
n,nsin(1
n))→(0,1). We will use this fact to define limits of functions of two variables.2.2. Limits of functions
DefinitionConsider a function f:D→R where D⊂R2. Let (a,b)∈D. The limit of f at (a,b) exists if there exists a real number L such that:(an,bn)→(a,b)⟹f(an,bn)→Lfor all possible sequences (an,bn) that converge to (a,b).
For simple functions, we can just substitute the value (a,b) into the function to get the limit. For instance:lim(x,y)→(a,b) exsin(x+y)=easin(a+b)This method of substituting the value is not foolproof. There is a useful theorem that can be used to show the non-existence of limits:
TheoremThe limit of a function f at (a,b) exists and is equal to L if for every curve C in the domain D of f passing through (a,b), the limit of f along C is exists and is equal to L.
An example of this theorem in action:lim(x,y)→(0,0)x2-y2
x2+y2The limit along the x-axis is 1 and the limit along the y-axis is -1. Since we have two different limits along the two curves, the limit doesn't exist.2.3. Properties of limits2.3.1. Arithmeticf and g denote two functions on a domain D. If lim(x,y)→(a,b) f(x,y)=F and lim(x,y)→(a,b) g(x,y)=G, then:• lim(x,y)→(a,b) f(x,y)+g(x,y)=F+G• • lim(x,y)→(a,b)(fg)(x,y)=FG• • lim(x,y)→(a,b)f(x,y)
g(x,y)=F
G, provided G≠0 and g is defined in a small interval around (a,b).2.3.2. CompositionLet f be a multivariable function and g be a single variable function such that g∘f is defined. If lim(x,y)→(a,b) f(x,y)=F and limx→F g(x)=L, then:lim(x,y)→(a,b)(g∘f)(x,y)=L2.3.3. Sandwich principleIf lim(x,y)→(a,b) f(x,y)=L and lim(x,y)→(a,b) g(x,y)=L, and f(x)⩽h(x)⩽g(x), we have:lim(x,y)→(a,b) h(x,y)=L2.4. ContinuityA function f is continuous at the point (a,b) in its domain if the limit at the point exists and is equal to the value of the function at that point. That is:lim(x,y)→(a,b) f(x,y)=f(a,b)If a function is continuous at all points in its domain, it is said to be a continuous function.3. Derivatives3.1. Partial DerivativesThe partial derivative of a function f with respect to x and y at the point (a,b) is denoted by fx and fy respectively and is given as:a
fx(a,b)
=limh→0f(a+h,b)-f(a,b)
h
fy(a,b)
=limh→0f(a,b+h)-f(a,b)
h
For example, fx(1,2) for f(x,y)=x2+y2 can be computed as:a
fx(1,2)
=limh→0(1+h)2+22-(12+22)
h
=limh→0(1+h)2-1
h
=limh→0h(2+h)
h
=2
fx(a,b) can be viewed as the instantaneous rate of change of f at (a,b) along the x-axis. A similar idea holds for fy(a,b). The partial derivatives are themselves functions of (x,y). We denote this as fx(x,y) and fy(x,y). To compute the partial derivative with respect to x, we can view f as function of x while y remains constant, and differentiate with respect to x. For example, if f(x,y)=x2+y2, fx(x,y)=2x and fy(x,y)=2y.3.2. GradientThe partial derivatives can be clubbed together into a vector called the gradient:∇f(x,y)=a
fx(x,y)
fy(x,y)
For example, if f(x,y)=x2+y2, ∇f(x,y)=a
2x
2y
.3.3. Directional DerivativesDirectional derivatives extend the notion of partial derivatives to any direction. The directional derivative of f in the direction given by the unit vector u=(u1,u2) at (a,b) is given by fu(a,b):fu(a,b)=limh→0f(a+hu1,b+hu2)-f(a,b)
hfu(x,y) is again a function of x,y. As an example, if f(x,y)=xy, the directional derivative along u=(u1,u2) can be computed as:a
fu(x,y)
=limh→0f(x+hu1,y+hu2)-f(x,y)
h
=limh→0(x+hu1)(y+hu2)-xy
h
=limh→0xy+xhu2+yhu1+h2u1u2-xy
h
=yu1+xu2
TheoremIf the partial derivatives are continuous in an open ball centered at (a,b), the directional derivative at (a,b) is defined for all directions. For any direction given by the unit vector u=(u1,u2), the directional derivative is given by:fu(a,b)=∇f(a,b)TuRecall that uTv is just the dot product of u and v.
To see this theorem in action, for f(x,y)=xy, we have:a
fu(x,y)
=∇f(x,y)Ta
u1
u2
=a
y
x
a
u1
u2
=yu1+xu2
3.4. Steepest Ascent and DescentSince ||u||=1, we see that:a
fu(a,b)
=∇f(a,b)Tu
=||∇f(a,b)||⋅||u||⋅cos𝜃
=||∇f(a,b)||⋅cos𝜃
The maximum and minimum values of fu(a,b) occur when 𝜃=0 and 𝜃=180∘ respectively. If 𝜃=90∘, then the directional derivative is zero. Therefore:• The direction of steepest ascent of the function at (a,b) is ∇f(a,b)
||∇f(a,b)||.• • The direction of steepest descent of the function at (a,b) is -∇f(a,b)
||∇f(a,b)||.• The direction in which the function's value doesn't change at (a,b) is orthogonal to ∇f(a,b).Note that all these are local phenomenon. When we talk about the direction of steepest ascent at a point (a,b), we are only concerned about the function's behavior in the immediate vicinity of (a,b).3.5. Second Order Partial DerivativesThe second order partial derivatives are given below:fxx=(fx)x=∂
∂x(∂f
∂x)=∂2f
∂x2fyy=(fy)y=∂
∂y(∂f
∂y)=∂2f
∂y2fxy=(fx)y=∂
∂y(∂f
∂x)=∂2f
∂y∂xfyx=(fy)x=∂
∂x(∂f
∂y)=∂2f
∂x∂yfxy and fyx are called mixed partial derivatives.
TheoremLet f be a function defined on a domain D with a point (a,b) and an open ball around it contained in D. If the second order partial derivatives are continuous in an open ball around (a,b), then fxy(a,b)=fyx(a,b).
All the second order partial derivatives can be collected in a matrix termed the Hessian matrix:H=a
fxx
fxy
fyx
fyy
If the theorem above holds, H is a symmetric matrix. For a function of three variables, the Hessian will be a square matrix of order three:H=a
fxx
fxy
fxz
fyx
fyy
fyz
fzx
fzy
fzz
QuestionFind all partial derivatives up to order 2 at (0,0).f(x,y)=a
xy(x2-y2)
x2+y2,
(x,y)≠(0,0)
0,
(x,y)=(0,0)
For (x,y)=(0,0):fx(0,0)=limh→0f(h,0)-f(0,0)
h=0fy(0,0)=limh→0f(0,h)-f(0,0)
h=0For (x,y)≠(0,0):a
fx(x,y)
=(x2+y2)[3x2y-y3]-2x2y(x2-y2)
(x2+y2)2
=x4y+4x2y3-y5
(x2+y2)2
Therefore:fx(x,y)=a
y[x4+4x2y2-y4]
(x2+y2)2,
(x,y)≠(0,0)
0,
(x,y)=(0,0)
fxx(0,0)=limh→0fx(h,0)-fx(0,0)
h=0fxy(0,0)=limh→0fx(0,h)-fx(0,0)
h=limh→0-h5
h5=-1Similarly, we compute fy(x,y):fy(x,y)=a
x(x4-4x2y2-y4)
(x2+y2)2,
(x,y)≠(0,0)
0,
(x,y)=(0,0)
This gives us fyy(0,0)=0 and fyx(0,0)=1.
NOTE: IGNORE FOR END-TERM, CAN BE CONFUSINGDuring the live session, there was a mistake in the expression for fy(x,y). This is what was being attempted. One way to quickly compute fy(x,y) is to note the following relationship:a
f(x,y)
=-f(y,x)
⟹fy(x,y)
=-fy(y,x)
fy(y,x) is the partial derivative of f with respect to the first variable. Therefore, this is the same as partial derivative of f(x,y) with respect to x, and then swapping the variables x and y. Therefore, we have fy(y,x)=fx(y,x). This gives us:fy(x,y)=-fx(y,x)
4. TangentsFor all subsections, consider a function f defined on a domain D⊂R2 that has continuous partial derivatives in an open ball centered at the point (a,b)∈D. The existence of continuous partial derivatives implies that the directional derivative exists in all directions and is given by:fu(a,b)=∇f(a,b)Tu4.1. Tangent LinesThe tangent line to a function f at the point (a,b) in the direction of the unit vector u=(u1,u2) is given by the following equivalent forms:Form-1(a,b,f(a,b))+t(u1,u2,fu(a,b))(a,b,f(a,b))+span{(u1,u2,fu(a,b))}Form-2a
x
=a+tu1
y
=b+tu2
z
=f(a,b)+tfu(a,b)
Form-3x-a
u1=y-b
u2=z-f(a,b)
fu(a,b)The tangent line is an affine subspace of R3.
QuestionFind the tangent line to the function f(x,y)=xy at the point (1,3) in the direction (1
2,1
2).
(1,3,f(1,3))+t(1
2,1
2,fu(1,3))fu(1,3)=a
3
1
a
1/2
1/2
=4
2The tangent line at the point (1,3) to f in the direction (1
2,1
2) is given by:(1,3,3)+t(1
2,1
2,4
2)Alternatively:a
x
=1+t
2
y
=3+t
2
z
=3+4t
2
Alternatively:x-1
1/2=y-3
1/2=z-3
4/2
4.2. Tangent HyperplanesThe tangent plane to the function at (a,b) is given by:Form-1z=f(a,b)+fx(a,b)(x-a)+fy(a,b)(y-b)Form-2(a,b,f(a,b))+{(x,y,z):fx(a,b)x+fy(a,b)y-z=0}Form-3z=f(a,b)+∇f(a,b)Ta
x-a
y-b
The vector (fx(a,b),fy(a,b)-1) is orthogonal to the tangent plane.
QuestionFind the tangent plane to the function f(x,y)=9-x2-y2 at the point (1,2).
z-f(1,2)=fx(1,2)(x-1)+fy(1,2)(y-2)a
fx
=(-2x)
29-x2-y2=-x
9-x2-y2=-1
2=-1
2
fy
=(-2y)
29-x2-y2=-y
9-x2-y2=-2
2=-1
z-2=-1
2(x-1)-(y-2)The equation of the tangent plane at (1,2) to f is:x+2y+2z=9
4.3. Best Linear ApproximationThe best linear approximation to the function at (a,b) is given by Lf(x,y)|(a,b):Lf(x,y)|(a,b)=f(a,b)+∇f(a,b)Ta
x-a
y-b
Expanding it, we get:Lf(x,y)|(a,b)=f(a,b)+fx(a,b)(x-a)+fy(a,b)(y-b)
QuestionFind the best linear approximation to f at (0,0) where, f(x,y)=exsiny+xex.
Lf(x,y)|(0,0)=f(a,b)+∇f(a,b)Ta
x-a
y-b
f(0,0)=0∇f(x,y)=a
ex(siny+x+1)
excosy
=a
1
1
Lf(x,y)|(0,0)=x+y
5. DifferentiabilityConsider a function f defined on a domain D⊂R2. Let fx and fy be defined at (a,b)∈D. Then, f is differentiable at (a,b)∈D if:lim(h1,h2)→(0,0)f(a+h1,b+h2)-f(a,b)-fx(a,b)h1-fy(a,b)h2
h21+h22=0This is equivalent to:• The tangent plane to f exists at (a,b).• The best linear approximation to f exists at (a,b).Intuitively, this means that very close to the point (a,b)f looks like a plane (behaves like a linear function).
TheoremIf the partial derivatives exist in an open ball at (a,b) and are continuous, then f is differentiable at (a,b).
6. OptimizationConsider a function defined over D⊂R2 and a point (a,b)∈D with an open ball centered at (a,b) that is contained in D.6.1. Maxima and Minima• A point (a,b) is called a local minimum of f if f(x,y)⩾f(a,b) for all (x,y) in an open ball centered at (a,b).• A point (a,b) is called a local maximum of f if f(x,y)⩽f(a,b) for all (x,y) in an open ball centered at (a,b).• A point (a,b) is called a absolute minimum of f if f(x,y)⩾f(a,b) for all (x,y) in the domain of f.• A point (a,b) is called a absolute maximum of f if f(x,y)⩽f(a,b) for all (x,y) in the domain of f.A point that is a maximum or a minimum is called an extremum.6.2. Critical PointsConsider a function f with domain D⊂R2. A point (a,b) is a critical point of f if one of two things happen:• The gradient of f is not defined at (a,b).• The gradient of f is zero.For example, consider f(x,y)=x2+6xy+4y2+2x-4y.a
fx(x,y)
=2x+6y+2=0
fy(x,y)
=8y+6x-4=0
⟹(x,y)=(2,-1)(2,-1) is a critical point of f.
TheoremIf a point (a,b) is a local extremum, then ∇f(a,b)=0.
To see why this is true, consider what happens if ∇f(a,b)≠0. The function's value would increase along the direction ∇f(a,b) and decrease along -∇f(a,b) suggesting that (a,b) is not an extremum. This is a loose argument. For a more concrete argument, consider the slice of the function along y=b and note that for the corresponding single variable function in terms of x, x=a is an extremum, which implies that fx(a,b)=0.Every local extremum is a critical point. All critical points are not local extrema. A point which is a critical point but not a local extremum is called a saddle point. An example is the function f(x,y)=xy. This function has a critical point at (0,0), but this is neither a local maximum nor a local minimum. The function has a minimum along y=x and a maximum along y=-x.6.3. Continuous functions on Closed, Bounded DomainA set D⊂R2 is • closed if it contains all its boundary points• bounded if it is contained within a ball around 0 with finite radius.An example of a closed, bounded domain is a square that contains its boundary:{(x,y):0⩽x⩽2,0⩽y⩽2}An example of a domain that is bounded but not closed is a square that doesn't contain its boundary:{(x,y):0<x<2,0<y<2}Visually:Open, BoundedClosed, Bounded
TheoremConsider a continuous function f on a domain D that is closed and bounded. Then f has a global maximum and global minimum on D.
The maximum and minimum could be one of these points:• A point in the interior of D.• A point on the boundary of D.
QuestionFind the global maximum and minimum of the function f(x,y)=x2+y2 on the closed triangular plate bounded by the lines x=0,y=0,y=2x+2 in the second quadrant.
Step-1: Critical points∇f(x,y)=(2x,2y)(0,0) is the only critical point and is one of the candidates for global maximum/minimum.Step-2: Graph the regionStep-3: Check the boundariesBoundary-1: x=0h(y)=f(0,y)=y2Check for (0,0) and (0,2).Boundary-2: y=0h(x)=f(x,0)=x2Check for (0,0) and (-1,0).Boundary-3: y=2x+2h(x)=f(x,2x+2)=x2+(2x+2)2=5x2+8x+4Note that h is defined in the interval [-1,0]. In this interval, the global maximum of h can be a critical point or one of the boundary points.h′(x)=0⟹10x+8=0⟹x=-4
5Check for (-4/5,2/5), (-1,0) and (0,2).
(a,b)
f(a,b)
(0,0)
0
(0,2)
4
(-1,0)
1
(-4/5,2/5)
4
5
Therefore, the global maximum of f is 4 at (0,2) and the global minimum is 0 at (0,0).
AsideD is the triangular region mentioned above:f:D→Rf(x,y)=x2+y2What is fx(0,0)?fx(0,0)=limh→0-f(0+h,0)-f(0,0)
h=limh→0-h2-0
h=0fy(0,0)=limh→0+f(0,0+h)-f(0,0)
h=limh→0+h2-0
h=0
6.4. Hessian TestThe Hessian test can be used to determine the nature of the critical point. The test may fail to be conclusive in certain situations.6.4.1. Two variablesConsider a function f defined on D⊂R2 which has a critical point at (a,b)∈D with continuous first and second order partial derivatives in an open ball centered at (a,b). Let H be the Hessian at (a,b) and let fxx,fyy,fxy,fyx denote the second order partial derivatives at (a,b):• If |H|>0 and fxx>0, (a,b) is a local minimum.• If |H|>0 and fxx<0, (a,b) is a local maximum.• If |H|<0, (a,b) is a saddle point.• If |H|=0, the test is inconclusive.Here |H| denotes the determinant of H.6.4.2. Three variablesConsider a function f defined on D⊂R3 which has a critical point at (a,b,c)∈D with continuous first and second order partial derivatives in an open ball centered at (a,b,c). Let H be the Hessian at (a,b,c) and let fxx,fyy,fzz,fxy,fyz,fzx denote the second order partial derivatives at (a,b,c):• If |H|>0, fxx>0 and fxxfyy-f2xy>0, then (a,b,c) is a local minimum.• If |H|<0, fxx<0 and fxxfyy-f2xy>0, then (a,b,c) is a local maximum.• If |H|≠0 and the above two cases do not occur, then (a,b,c) is a saddle point.• If |H|=0, the test is inconclusive.Here |H| denotes the determinant of H. Visually, this test involves the determinants of three matrices:a
fxx
fxy
fxz
fyx
fyy
fyz
fzx
fzy
fzz
,a
fxx
fxy
fxz
fyx
fyy
fyz
fzx
fzy
fzz
,a
fxx
fxy
fxz
fyx
fyy
fyz
fzx
fzy
fzz
fxx
fxxfyy-f2xy
|H|
Outcome
+
+
+
Local minimum
-
+
-
Local maximum
For additional reading, lookup Sylvester's theorem in Wikipedia.
QuestionFind all the points of local extrema for the function f(x,y)=x3-3x+y3-3y2.
a
fx(x,y)
=3x2-3=0
fy(x,y)
=3y2-6y=0
⟹x=±1,y=0,2This function has 4 critical points, namely:(1,0),(1,2),(-1,0),(-1,2)a