Differentiation of Functions of Several Variables

Introduction

A function of several variables is a function that depends on more than one input variable. For a function f(x, x, ..., x), the input is an n-dimensional vector of real numbers, and the output is a real number.

Differentiation of such functions extends the concept of derivatives from single-variable calculus. In multivariable calculus, we examine how a function changes as each of its input variables changes independently.

The simplest case is a function of two variables, z = f(x, y), which we can visualize as a surface in three-dimensional space. The derivative now becomes more complex as there are multiple directions in which the function can change.

Partial Derivatives

The partial derivative of a function of several variables is its derivative with respect to one of those variables, while the other variables are held constant.

For a function f(x, y), the partial derivative with respect to x is denoted by f/x or f(x, y), and with respect to y by f/y or f(x, y):

Directional Derivatives

While partial derivatives measure the rate of change along the coordinate axes, directional derivatives measure the rate of change of a function in any specified direction.

The directional derivative of f at point (x, y) in the direction of a unit vector u = (a, b) is denoted by Df(x, y) and is defined as:

If f is a differentiable function, then the directional derivative can be computed using the gradient:

Gradient Vector

The gradient is a vector operator that operates on a scalar-valued function to produce a vector field. For a function f(x, x, ..., x), the gradient is denoted by f and is given by:

In two dimensions, the gradient of f(x, y) is:

The gradient points in the direction of the greatest rate of increase of the function with respect to its input variables, and its magnitude gives the rate of increase.

Chain Rule for Multivariable Functions

The chain rule extends to functions of several variables when these variables themselves depend on other variables.

For a function f(x(t), y(t)), where x and y depend on a single variable t, the derivative with respect to t is:

In more general cases, for a function f(x(t, t, ..., t), x(t, t, ..., t), ..., x(t, t, ..., t)), the partial derivative with respect to t is:

Second-order Partial Derivatives

Just as with single-variable functions, we can take second derivatives of functions of several variables. For a function f(x, y), we have four possible second-order partial derivatives:

Under reasonable conditions (if the mixed partials are continuous), Clairaut's theorem states that the mixed partial derivatives are equal:

The Hessian Matrix

The Hessian matrix is a square matrix of second-order partial derivatives of a scalar-valued function. For a function f(x, x, ..., x), the Hessian H is:

For a function f(x, y), the Hessian is the 22 matrix:

The Hessian matrix is important in optimization problems, particularly for determining the nature of critical points:

• If the Hessian is positive definite (all eigenvalues positive), the critical point is a local minimum.
• If the Hessian is negative definite (all eigenvalues negative), the critical point is a local maximum.
• If the Hessian has both positive and negative eigenvalues, the critical point is a saddle point.

Applications of Multivariable Differentiation

Differentiation of functions of several variables has numerous applications across mathematics, science, and engineering:

Optimization

Finding the values of variables that maximize or minimize a multivariable function is a fundamental problem with applications in economics, physics, and engineering. Critical points where the gradient is zero help identify potential extrema, and the Hessian matrix helps classify them.

Physics

In physics, many quantities depend on multiple spatial coordinates. For instance, electric and magnetic fields vary with position, and the gradient of the potential gives the force field. The divergence, curl, and Laplacian operators are all based on partial derivatives.

Machine Learning

In machine learning, functions with many parameters are optimized using gradient descent algorithms. The gradient (and sometimes the Hessian) guides the algorithm toward better parameter values.

Economics

In economic theory, utility functions often depend on multiple goods, and production functions depend on various inputs. Partial derivatives help understand marginal utility and marginal productivity concepts.

Engineering

In engineering, partial differential equations (PDEs) model phenomena like heat transfer, fluid flow, and electromagnetic fields. These equations involve derivatives of functions with respect to multiple variables.