8.2 Maximal monotone operators and resolvent operators

Maximal monotone operators are key players in variational analysis. They're set-valued mappings with special properties that make them super useful in optimization. These operators help us solve tricky problems by providing a framework for analysis.

Resolvent operators are closely tied to maximal monotone operators. They're like the inverse of a maximal monotone operator plus the identity. Resolvents are single-valued and well-behaved, making them handy tools for solving equations and optimization problems.

Maximal Monotone Operators

Definition and Characterizations

Top images from around the web for Definition and Characterizations

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

1 of 2

Top images from around the web for Definition and Characterizations

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

• 

  Hilbert space - Wikipedia View original

  Is this image relevant?

1 of 2

• A monotone operator is a set-valued mapping $T : H \to 2^H$ from a Hilbert space $H$ to its power set $2^H$ that satisfies the monotonicity condition: $\langle x - y, u - v \rangle \geq 0$ for all $x, y \in H$ and $u \in T(x), v \in T(y)$
• A monotone operator $T$ is maximal if its graph is not properly contained in the graph of any other monotone operator
  • If $(x, u) \in H \times H$ satisfies $\langle x - y, u - v \rangle \geq 0$ for all $(y, v) \in \text{gph}(T)$, then $(x, u) \in \text{gph}(T)$
• The graph of a maximal monotone operator is closed and convex in the product space $H \times H$
• A monotone operator $T$ is maximal if and only if the range of $T + \lambda I$ is the entire space $H$ for all $\lambda > 0$, where $I$ is the identity operator
• The subdifferential of a proper, lower semicontinuous, convex function is a maximal monotone operator (subdifferential of a convex function)

Properties and Examples

• The sum of two maximal monotone operators is also maximal monotone
• The composition of a maximal monotone operator with a linear, bounded, and self-adjoint operator is maximal monotone
• Examples of maximal monotone operators include:
  • The subdifferential of the $l_1$-norm: $\partial \|\cdot\|_1$
  • The normal cone operator of a closed convex set: $N_C(x) = \{u \in H : \langle u, y - x \rangle \leq 0, \forall y \in C\}$
  • The linear operator $Ax = -\Delta x$ with domain $H^2(\Omega) \cap H^1_0(\Omega)$ in $L^2(\Omega)$, where $\Delta$ is the Laplacian operator and $\Omega$ is a bounded domain

Resolvent Operator for Maximal Monotone Operators

Definition and Existence

• The resolvent operator $J_\lambda^T$ of a maximal monotone operator $T$ with parameter $\lambda > 0$ is defined as $J_\lambda^T = (I + \lambda T)^{-1}$, where $I$ is the identity operator
• The resolvent operator is single-valued and well-defined for all $\lambda > 0$ due to the maximal monotonicity of $T$
• The existence and uniqueness of the resolvent operator can be proved using the Minty surjectivity theorem
  • For a maximal monotone operator $T$ and $\lambda > 0$, the range of $I + \lambda T$ is the entire space $H$
  • The proof involves showing that for any $z \in H$, there exists a unique $x \in H$ such that $z \in x + \lambda T(x)$, which is equivalent to $x = J_\lambda^T(z)$

Uniqueness and Explicit Formulas

• The resolvent operator is unique for a given maximal monotone operator $T$ and parameter $\lambda > 0$
• For the subdifferential of the $l_1$-norm, the resolvent operator (proximal operator) has an explicit formula: $J_\lambda^{\partial \|\cdot\|_1}(x) = \text{sign}(x) \max(|x| - \lambda, 0)$, where $\text{sign}(x)$ is the sign function and $|\cdot|$ is the absolute value
• For the normal cone operator of a closed convex set $C$, the resolvent operator (projection operator) is given by $J_\lambda^{N_C}(x) = \text{proj}_C(x) = \argmin_{y \in C} \|y - x\|$, which is the projection of $x$ onto the set $C$

Properties of Resolvent Operators

Nonexpansiveness and Firm Nonexpansiveness

• The resolvent operator $J_\lambda^T$ is nonexpansive: $\|J_\lambda^T(x) - J_\lambda^T(y)\| \leq \|x - y\|$ for all $x, y \in H$ and $\lambda > 0$
• The resolvent operator $J_\lambda^T$ is firmly nonexpansive: $\|J_\lambda^T(x) - J_\lambda^T(y)\|^2 \leq \langle x - y, J_\lambda^T(x) - J_\lambda^T(y) \rangle$ for all $x, y \in H$ and $\lambda > 0$
  • Firm nonexpansiveness implies nonexpansiveness, but the converse is not true in general
• The resolvent operator is a contraction mapping when $\lambda < 1$: $\|J_\lambda^T(x) - J_\lambda^T(y)\| \leq (1 - \lambda)\|x - y\|$ for all $x, y \in H$ and $0 < \lambda < 1$
• The composition of resolvent operators with different parameters, $J_\mu^T \circ J_\lambda^T$, is also nonexpansive for $\lambda, \mu > 0$

Resolvent Identity and Yosida Approximation

• The resolvent identity holds for resolvent operators with different parameters: $J_\lambda^T = J_\mu^T((1 - \frac{\mu}{\lambda})I + \frac{\mu}{\lambda}J_\lambda^T)$ for all $\lambda, \mu > 0$
• The Yosida approximation of a maximal monotone operator $T$ is defined as $T_\lambda = \frac{1}{\lambda}(I - J_\lambda^T)$ for $\lambda > 0$
  • $T_\lambda$ is a single-valued, monotone, and Lipschitz continuous operator with Lipschitz constant $\frac{1}{\lambda}$
  • As $\lambda \to 0$, $T_\lambda(x)$ converges to the minimal norm element of $T(x)$ for all $x \in \text{dom}(T)$

Maximal Monotone Operators vs Subdifferentials

Relationship between Maximal Monotone Operators and Subdifferentials

• The subdifferential $\partial f$ of a proper, lower semicontinuous, convex function $f : H \to (-\infty, +\infty]$ is a maximal monotone operator
• For a proper, lower semicontinuous, convex function $f$, the resolvent operator of its subdifferential $\partial f$ is the proximal operator $\text{prox}_\lambda^f$, defined as $\text{prox}_\lambda^f(x) = \argmin_y \{f(y) + \frac{1}{2\lambda}\|y - x\|^2\}$
• The proximal operator is related to the resolvent operator by $\text{prox}_\lambda^f = J_\lambda^{\partial f} = (I + \lambda \partial f)^{-1}$

Moreau Envelope and Moreau-Yosida Regularization

• The Moreau envelope of a proper, lower semicontinuous, convex function $f$ is the Moreau-Yosida regularization $f_\lambda(x) = \min_y \{f(y) + \frac{1}{2\lambda}\|y - x\|^2\}$, which is a continuously differentiable approximation of $f$
• The gradient of the Moreau envelope $\nabla f_\lambda$ is related to the proximal operator by $\nabla f_\lambda(x) = \frac{1}{\lambda}(x - \text{prox}_\lambda^f(x))$
• The Moreau envelope $f_\lambda$ converges pointwise to $f$ as $\lambda \to 0$, and the gradient $\nabla f_\lambda$ converges pointwise to the minimal norm element of $\partial f$ as $\lambda \to 0$
• The proximal operator and the Moreau envelope provide a way to smooth and regularize non-smooth convex functions, which is useful in optimization and variational analysis (Moreau-Yosida regularization of the $l_1$-norm)