9.1 Calculus of variations

Calculus of variations is a powerful mathematical tool for optimizing functionals. It finds the function that extremizes a given functional, subject to constraints. This field has wide-ranging applications in physics, engineering, and economics.

The fundamental problem is finding a function that minimizes or maximizes a functional. Key concepts include functionals, function spaces, extrema, variations, and the Euler-Lagrange equation. These tools allow us to solve complex optimization problems in many fields.

Fundamental concepts of calculus of variations

• Calculus of variations is a field of mathematical analysis that deals with optimizing functionals, which are mappings from a set of functions to the real numbers
• It has wide-ranging applications in physics, engineering, and economics, where it is used to find the function that extremizes a given functional subject to certain constraints
• The fundamental problem in calculus of variations is to find a function $u(x)$ that minimizes or maximizes a given functional $J[u]$, often subject to boundary conditions or other constraints

Functionals and function spaces

• A functional $J[u]$ is a mapping that assigns a real number to each function $u$ in a certain function space
• Function spaces are sets of functions that share certain properties, such as continuity, differentiability, or integrability
• Examples of functionals include the length of a curve $L[u] = \int_a^b \sqrt{1 + u'(x)^2} dx$ and the energy of a system $E[u] = \int_a^b (u'(x)^2 - u(x)^2) dx$

Extrema of functionals

• An extremum of a functional is a function $u^*$ that minimizes or maximizes the functional among all functions in the function space
• A necessary condition for $u^*$ to be an extremum is that the first variation of the functional vanishes at $u^*$, i.e., $\delta J[u^*; v] = 0$ for all admissible variations $v$
• A sufficient condition for $u^*$ to be a minimum is that the second variation of the functional is non-negative at $u^*$, i.e., $\delta^2 J[u^*; v, v] \geq 0$ for all admissible variations $v$

Weak and strong extrema

• A function $u^*$ is a weak extremum of a functional if it satisfies the necessary condition $\delta J[u^*; v] = 0$ for all admissible variations $v$
• A function $u^*$ is a strong extremum of a functional if it satisfies both the necessary and sufficient conditions for an extremum
• Weak extrema are not necessarily global extrema, while strong extrema are always global extrema in their function space

Variations and variational derivatives

• A variation of a function $u$ is a small perturbation $v$ added to $u$, i.e., $u_\epsilon = u + \epsilon v$, where $\epsilon$ is a small parameter
• The first variation of a functional $J[u]$ at $u$ in the direction $v$ is defined as $\delta J[u; v] = \lim_{\epsilon \to 0} \frac{J[u + \epsilon v] - J[u]}{\epsilon}$
• The variational derivative of a functional $J[u]$ is a function $\frac{\delta J}{\delta u}$ such that $\delta J[u; v] = \int_a^b \frac{\delta J}{\delta u}(x) v(x) dx$ for all admissible variations $v$

Euler-Lagrange equation

• The Euler-Lagrange equation is a necessary condition for a function to be an extremum of a functional
• It is a second-order differential equation that the extremizing function must satisfy, along with any boundary conditions or constraints
• The Euler-Lagrange equation arises from setting the first variation of the functional equal to zero and applying the fundamental lemma of calculus of variations

Derivation of Euler-Lagrange equation

• Consider a functional of the form $J[u] = \int_a^b L(x, u(x), u'(x)) dx$, where $L$ is the Lagrangian of the system
• The first variation of $J[u]$ at $u$ in the direction $v$ is given by $\delta J[u; v] = \int_a^b \left(\frac{\partial L}{\partial u} v + \frac{\partial L}{\partial u'} v'\right) dx$
• Setting $\delta J[u; v] = 0$ for all admissible variations $v$ and applying integration by parts leads to the Euler-Lagrange equation $\frac{\partial L}{\partial u} - \frac{d}{dx}\frac{\partial L}{\partial u'} = 0$

First variation and its properties

• The first variation $\delta J[u; v]$ measures the rate of change of the functional $J[u]$ in the direction of the variation $v$
• The first variation is a linear functional in $v$, i.e., $\delta J[u; \alpha v_1 + \beta v_2] = \alpha \delta J[u; v_1] + \beta \delta J[u; v_2]$ for all scalars $\alpha, \beta$ and variations $v_1, v_2$
• The first variation satisfies the product rule $\delta (J_1[u] J_2[u]) = J_1[u] \delta J_2[u] + J_2[u] \delta J_1[u]$ and the chain rule $\delta (J[G[u]]) = \frac{\delta J}{\delta G} \delta G[u]$

Second variation and sufficient conditions

• The second variation $\delta^2 J[u; v, w]$ measures the second-order change of the functional $J[u]$ in the directions of the variations $v$ and $w$
• The second variation is a bilinear functional in $v$ and $w$, i.e., $\delta^2 J[u; \alpha v_1 + \beta v_2, w] = \alpha \delta^2 J[u; v_1, w] + \beta \delta^2 J[u; v_2, w]$ for all scalars $\alpha, \beta$ and variations $v_1, v_2, w$
• A sufficient condition for a function $u^*$ to minimize the functional $J[u]$ is that $\delta J[u^*; v] = 0$ for all admissible variations $v$ and $\delta^2 J[u^*; v, v] \geq 0$ for all non-zero admissible variations $v$

Generalizations of Euler-Lagrange equation

• The Euler-Lagrange equation can be generalized to functionals involving higher-order derivatives, such as $J[u] = \int_a^b L(x, u(x), u'(x), u''(x)) dx$
• In this case, the Euler-Lagrange equation becomes $\frac{\partial L}{\partial u} - \frac{d}{dx}\frac{\partial L}{\partial u'} + \frac{d^2}{dx^2}\frac{\partial L}{\partial u''} = 0$
• The Euler-Lagrange equation can also be generalized to functionals involving multiple functions, such as $J[u_1, \ldots, u_n] = \int_a^b L(x, u_1(x), \ldots, u_n(x), u_1'(x), \ldots, u_n'(x)) dx$, leading to a system of coupled Euler-Lagrange equations

Variational problems with constraints

• Many variational problems involve finding the extrema of a functional subject to certain constraints on the admissible functions
• Constraints can arise from physical or geometrical considerations, such as fixed boundary conditions, prescribed arc length, or conservation laws
• The presence of constraints modifies the Euler-Lagrange equation and introduces additional terms, such as Lagrange multipliers or transversality conditions

Isoperimetric problems and Lagrange multipliers

• An isoperimetric problem is a variational problem where the functional is extremized subject to a constraint of the form $G[u] = \int_a^b g(x, u(x), u'(x)) dx = c$, where $c$ is a constant
• The isoperimetric constraint can be incorporated into the functional using a Lagrange multiplier $\lambda$, leading to the modified functional $J_\lambda[u] = J[u] + \lambda G[u]$
• The Euler-Lagrange equation for the modified functional is $\frac{\partial L}{\partial u} - \frac{d}{dx}\frac{\partial L}{\partial u'} + \lambda \left(\frac{\partial g}{\partial u} - \frac{d}{dx}\frac{\partial g}{\partial u'}\right) = 0$, along with the isoperimetric constraint $G[u] = c$

Holonomic and non-holonomic constraints

• Holonomic constraints are constraints that can be expressed as equations involving the functions and their derivatives, such as $u(a) = u_a$, $u(b) = u_b$, or $G[u] = c$
• Non-holonomic constraints are constraints that cannot be expressed as equations, such as inequality constraints $u(x) \geq 0$ or integral constraints $\int_a^b u(x) dx = c$
• Holonomic constraints can be incorporated into the functional using Lagrange multipliers, while non-holonomic constraints require more advanced techniques, such as the Karush-Kuhn-Tucker conditions or the Pontryagin maximum principle

Transversality conditions and natural boundary conditions

• Transversality conditions are additional conditions that the extremizing function must satisfy at the boundary points where the function or its derivatives are not prescribed
• Transversality conditions arise from the boundary terms in the first variation of the functional and ensure that the extremizing function is "perpendicular" to the boundary in a suitable sense
• Natural boundary conditions are boundary conditions that are not prescribed a priori but follow from the transversality conditions, such as $\frac{\partial L}{\partial u'}(a) = 0$ or $\frac{\partial L}{\partial u'}(b) = 0$

Hamilton's principle and variational mechanics

• Hamilton's principle is a variational principle in classical mechanics that states that the motion of a system between two fixed points in configuration space is such that the action integral $S[q] = \int_{t_1}^{t_2} L(q(t), \dot{q}(t), t) dt$ is stationary, where $L$ is the Lagrangian of the system
• The Euler-Lagrange equation for Hamilton's principle yields the equations of motion for the system, which are second-order differential equations in the generalized coordinates $q(t)$
• Hamilton's principle provides a unified framework for deriving the equations of motion in various branches of mechanics, such as particle dynamics, rigid body dynamics, and continuum mechanics

Direct methods in calculus of variations

• Direct methods are techniques for proving the existence of minimizers of functionals without explicitly solving the Euler-Lagrange equation
• Direct methods rely on functional analytic tools, such as compactness, lower semicontinuity, and coercivity, to establish the existence of minimizers in suitable function spaces
• Direct methods are particularly useful for functionals that lack smoothness or convexity properties, or for problems with constraints or singular behavior

Existence of minimizers and lower semicontinuity

• A functional $J[u]$ is said to have a minimizer in a function space $X$ if there exists a function $u^* \in X$ such that $J[u^*] \leq J[u]$ for all $u \in X$
• A sufficient condition for the existence of a minimizer is the sequential lower semicontinuity of the functional, which means that $J[u] \leq \liminf_{n \to \infty} J[u_n]$ for any sequence $u_n$ converging to $u$ in $X$
• Lower semicontinuity is often established using convexity or compactness arguments, such as the Banach-Alaoglu theorem or the Rellich-Kondrachov compactness theorem

Tonelli's theorem and its applications

• Tonelli's theorem provides sufficient conditions for the existence of a minimizer of a functional of the form $J[u] = \int_a^b L(x, u(x), u'(x)) dx$ in the Sobolev space $W^{1,p}(a,b)$
• The conditions are: (1) $L(x, u, p)$ is convex in $p$ for almost all $x$ and all $u$; (2) $L(x, u, p) \geq \alpha(x) + \beta |p|^p$ for some $\alpha \in L^1(a,b)$, $\beta > 0$, and $p > 1$; (3) $L(x, u, p)$ is measurable in $x$ for all $(u, p)$ and continuous in $(u, p)$ for almost all $x$
• Tonelli's theorem has applications in various areas, such as nonlinear elasticity, image processing, and optimal control theory

Regularity of minimizers and Hilbert's invariant integral

• Regularity theory studies the smoothness properties of minimizers of functionals, such as their continuity, differentiability, or higher regularity
• A key tool in regularity theory is the Euler-Lagrange equation, which provides a necessary condition for minimizers in the form of a differential equation
• Hilbert's invariant integral is a functional of the form $J[u] = \int_a^b L(x, u(x), u'(x)) dx$ that is invariant under a certain group of transformations, such as translations or rotations
• The invariance of the functional leads to conservation laws for the minimizers, which can be used to establish their regularity or symmetry properties

Lavrentiev phenomenon and relaxation methods

• The Lavrentiev phenomenon refers to the situation where the infimum of a functional over a certain function space is strictly less than the infimum over a dense subspace
• The Lavrentiev phenomenon occurs when the functional lacks lower semicontinuity or coercivity, and it indicates a gap between the original problem and its relaxed version
• Relaxation methods are techniques for extending the functional to a larger space where the infimum is attained, often by convexifying the functional or by adding lower semicontinuous terms
• Relaxation methods provide a way to approximate the original problem by a sequence of more tractable problems, and to establish the existence of generalized minimizers

Applications of calculus of variations

• Calculus of variations has numerous applications in geometry, physics, engineering, and other fields where optimization problems arise
• Many physical principles, such as the principle of least action, the principle of minimum potential energy, or the maximum entropy principle, can be formulated as variational problems
• The solutions to variational problems often have important geometric or physical interpretations, such as geodesics, minimal surfaces, or equilibrium configurations

Geodesics and shortest paths in Riemannian geometry

• A geodesic on a Riemannian manifold is a curve that locally minimizes the length functional $L[\gamma] = \int_a^b \sqrt{g_{\gamma(t)}(\dot{\gamma}(t), \dot{\gamma}(t))} dt$, where $g$ is the Riemannian metric and $\gamma(t)$ is a parametrized curve
• Geodesics are the generalization of straight lines in Euclidean space to curved spaces, and they play a fundamental role in Riemannian geometry
• The Euler-Lagrange equation for the length functional yields the geodesic equation $\ddot{\gamma}^k + \Gamma^k_{ij} \dot{\gamma}^i \dot{\gamma}^j = 0$, where $\Gamma^k_{ij}$ are the Christoffel symbols of the metric
• Geodesics have applications in general relativity, where they describe the motion of free-falling particles in curved spacetime, and in computer graphics, where they are used for shortest path planning and mesh parameterization

Minimal surfaces and soap films

• A minimal surface is a surface that locally minimizes the area functional $A[S] = \int_S dA$, where $dA$ is the area element of the surface
• Minimal surfaces are characterized by having zero mean curvature, which means that their principal curvatures are equal and opposite at each point
• Soap films are physical realizations of minimal surfaces, as they minimize the surface tension energy subject to the constraint of enclosing a fixed volume
• The Euler-Lagrange equation for the area functional yields the minimal surface equation $\Delta_S X = 0$, where $\Delta_S$ is the Laplace-Beltrami operator on the surface and $X$ is the position vector of the surface
• Minimal surfaces have applications in architecture, where they are used for designing efficient and aesthetically pleasing structures, and in materials science, where they arise in the study of crystal growth and self-assembly

Elastica and rod theory

• An elastica is a curve that minimizes the bending energy functional $$E[\gamma] = \int_a^b \kappa(t)^2 ds