4.3 Nyquist stability criterion

Definition of Nyquist stability criterion

The Nyquist stability criterion lets you determine whether a closed-loop system is stable by examining the open-loop frequency response. Instead of finding closed-loop poles directly (which can be difficult or impossible for systems with delays), you plot the open-loop transfer function in the complex plane and count how many times the plot wraps around a single critical point: $-1 + j0$.

This makes it one of the most practical stability tools in control engineering. It works with experimental frequency response data, handles time delays naturally, and gives you quantitative stability margins all from one plot.

Nyquist plot basics

A Nyquist plot traces out the complex value of the open-loop transfer function $G(j\omega)H(j\omega)$ as frequency $\omega$ varies. The horizontal axis is the real part, and the vertical axis is the imaginary part. Each point on the curve corresponds to a specific frequency.

Unlike a Bode plot (which splits magnitude and phase into two separate graphs), the Nyquist plot captures both in a single curve. This makes it possible to visually assess encirclements of the critical point.

Open-loop transfer function in Nyquist plots

The open-loop transfer function $G(s)H(s)$ is the product of the forward-path transfer function $G(s)$ and the feedback-path transfer function $H(s)$. To build the Nyquist plot, you evaluate this product along the imaginary axis by substituting $s = j\omega$:

$G(j\omega)H(j\omega) \quad \text{for } \omega \in [0, \infty)$

This gives you the positive-frequency portion of the plot. The negative-frequency portion ($\omega < 0$) is the mirror image reflected across the real axis, since for real-coefficient transfer functions $G(-j\omega)H(-j\omega)$ is the complex conjugate of $G(j\omega)H(j\omega)$.

Mapping contours in Nyquist plots

The full Nyquist plot comes from mapping a closed contour in the $s$-plane through the function $G(s)H(s)$. That contour is the Nyquist D-contour, which encloses the entire right-half plane:

1. Travel up the imaginary axis from $s = -j\infty$ to $s = +j\infty$.
2. Close the contour with a semicircular arc of infinite radius sweeping clockwise through the right-half plane.

If the open-loop transfer function has poles on the imaginary axis (e.g., an integrator at $s = 0$), you indent the D-contour around those poles with infinitesimally small semicircles into the right-half plane. This avoids evaluating $G(s)H(s)$ at its own poles while still enclosing the entire RHP.

The image of this entire contour through $G(s)H(s)$ is the complete Nyquist plot.

Encirclements and stability

The core idea: the number of times the Nyquist plot encircles the critical point $-1 + j0$ tells you about the closed-loop poles in the right-half plane. This connection comes from the Cauchy argument principle in complex analysis, which relates encirclements of the origin by a mapped contour to the zeros and poles of the mapping function enclosed by the original contour.

Number of encirclements and poles

The closed-loop characteristic equation is $1 + G(s)H(s) = 0$, so closed-loop poles are the zeros of $1 + G(s)H(s)$. Applying the argument principle to $1 + G(s)H(s)$ mapped around the D-contour gives:

$N = Z - P$

where:

• $N$ = number of clockwise encirclements of $-1 + j0$ by the Nyquist plot
• $Z$ = number of closed-loop poles in the RHP (zeros of $1 + G(s)H(s)$ in the RHP)
• $P$ = number of open-loop poles in the RHP (poles of $G(s)H(s)$ in the RHP)

You want $Z = 0$ for a stable closed-loop system. Rearranging: $Z = N + P$.

Clockwise vs counterclockwise encirclements

To count encirclements, draw a line (a "test ray") from $-1 + j0$ outward in any direction and count how many times the Nyquist curve crosses it:

• Each crossing where the curve goes one way counts as $+1$.
• Each crossing the opposite way counts as $-1$.
• The net count gives $N$.

With the clockwise-positive convention ($N = Z - P$): clockwise encirclements add to $Z$, and counterclockwise encirclements subtract from it.

Stability assessment using Nyquist criterion

For a stable closed-loop system, you need $Z = 0$ (no closed-loop poles in the RHP). Using $Z = N + P$:

• If the open-loop system is stable ($P = 0$): the Nyquist plot must make zero net encirclements of $-1 + j0$. Any encirclement means instability.
• If the open-loop system is unstable ($P > 0$): the Nyquist plot must make exactly $P$ counterclockwise encirclements of $-1 + j0$. This is the only way to get $Z = N + P = 0$ (since counterclockwise encirclements give $N = -P$).

Stable vs unstable systems

A closed-loop system is stable when all closed-loop poles lie in the left-half plane ($Z = 0$). The output remains bounded for any bounded input.

A closed-loop system is unstable when at least one closed-loop pole lies in the RHP ($Z > 0$). Even a bounded input can produce an output that grows without bound.

The Nyquist criterion connects these conditions to a single visual test: count encirclements, know $P$, and compute $Z$.

Marginally stable systems and Nyquist criterion

A marginally stable system has closed-loop poles exactly on the imaginary axis, with none in the RHP. On the Nyquist plot, this shows up as the curve passing directly through the critical point $-1 + j0$.

When this happens, the encirclement count is undefined (the curve hits the test point rather than going around it). You'll need additional analysis, such as examining the direction the curve approaches $-1 + j0$ or using small-gain perturbation arguments, to characterize the system's behavior.

Gain and phase margins

Gain margin and phase margin quantify how close the Nyquist plot comes to the critical point. A system can be stable but barely so, and these margins tell you how much room you have before instability.

Definition of gain and phase margins

Gain margin (GM): Find the frequency $\omega_{pc}$ where the phase of $G(j\omega)H(j\omega)$ equals $-180°$ (the phase crossover frequency). At that frequency, the Nyquist plot crosses the negative real axis. The gain margin is:

$GM = \frac{1}{|G(j\omega_{pc})H(j\omega_{pc})|}$

In decibels: $GM_{dB} = -20\log_{10}|G(j\omega_{pc})H(j\omega_{pc})|$. This tells you how much you could multiply the loop gain before the plot reaches $-1 + j0$.

Phase margin (PM): Find the frequency $\omega_{gc}$ where the magnitude $|G(j\omega)H(j\omega)| = 1$ (the gain crossover frequency). The phase margin is:

$PM = 180° + \angle G(j\omega_{gc})H(j\omega_{gc})$

This tells you how much additional phase lag the system can tolerate before the plot rotates onto $-1 + j0$.

Relationship between margins and stability

• Positive GM and positive PM → stable closed-loop system (for minimum-phase, open-loop stable systems).
• Larger margins → more robust to modeling errors, parameter variations, and unmodeled dynamics.
• Negative GM or negative PM → the Nyquist plot already encircles $-1 + j0$, indicating instability.

Typical design targets are a gain margin of at least 6 dB and a phase margin of at least 30°–60°, though the exact requirements depend on the application.

Nyquist criterion for systems with time delays

A pure time delay of $T$ seconds contributes a transfer function $e^{-sT}$, which on the imaginary axis becomes $e^{-j\omega T}$. This has magnitude 1 at all frequencies but adds phase $-\omega T$ (in radians) that grows linearly with frequency.

Effect of time delays on Nyquist plots

Because the phase keeps decreasing as $\omega \to \infty$, the Nyquist plot spirals around the origin rather than converging to a single point. The curve wraps around infinitely many times, and each wrap potentially crosses near $-1 + j0$.

This spiraling behavior is what makes time-delay systems tricky: even a system that's stable without delay can become unstable if the delay is large enough, because the extra phase lag pushes the curve around the critical point.

Stability analysis for time-delayed systems

The standard Nyquist criterion (count encirclements of $-1 + j0$) still applies directly to time-delay systems. This is actually one of the major advantages of the Nyquist approach: unlike root-locus or Routh-Hurwitz, it handles the transcendental term $e^{-sT}$ without approximation.

You apply the criterion the same way:

1. Plot $G(j\omega)H(j\omega)$ including the $e^{-j\omega T}$ factor.
2. Count net encirclements of $-1 + j0$.
3. Use $Z = N + P$ to determine closed-loop stability.

The practical challenge is that the spiraling makes it harder to count encirclements accurately, especially at higher frequencies. Software tools are almost always used for these plots.

Advantages and limitations of Nyquist criterion

Comparison with other stability methods

The Nyquist criterion's ability to work directly with experimental frequency response data (measured magnitude and phase at various frequencies) makes it especially valuable for systems where an accurate transfer function model isn't available.

Challenges in applying Nyquist criterion

• Accurately sketching the plot by hand for high-order systems requires careful calculation at multiple frequencies.
• Counting encirclements can be ambiguous when the plot passes very close to $-1 + j0$ or makes complicated loops.
• The D-contour indentation for open-loop poles on the imaginary axis (like integrators) requires careful handling to get the correct plot.
• For non-minimum-phase or conditionally stable systems, the relationship between margins and stability is less straightforward than for simple systems.