5.1 Properties of Continuous Probability Density Functions

Continuous probability distributions are essential in statistics, allowing us to model real-world phenomena with infinite possible outcomes. They use probability density functions (PDFs) to represent the likelihood of values, with the area under the curve indicating probabilities for specific ranges.

Understanding continuous distributions involves mastering concepts like cumulative distribution functions (CDFs), which give probabilities for values below a certain point. These tools help analyze complex data and make predictions in fields ranging from finance to physics, forming the backbone of advanced statistical analysis.

Continuous Probability Distributions

Area under curve for probability

• In continuous probability distributions, the probability of a continuous random variable falling within a specific range is represented by the area under the probability density function (PDF) curve within that range
  • The total area under the PDF curve always equals 1, representing the total probability of all possible outcomes ($X$ taking any value from $-\infty$ to $\infty$)
  • The probability of a random variable taking on a specific value is 0, as there are infinitely many possible values in a continuous distribution (probability of $X = 3.14159...$ is 0)
• To find the probability of a random variable falling within a range $[a, b]$, calculate the definite integral of the PDF $f(x)$ from $a$ to $b$:
  • $P(a \leq X \leq b) = \int_a^b f(x) dx$
  • Example: For a standard normal distribution, $P(-1 \leq X \leq 1) = \int_{-1}^1 \frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}} dx \approx 0.6827$
• The area under the curve can be calculated using integration techniques, such as the fundamental theorem of calculus or numerical methods (trapezoidal rule, Simpson's rule)

Intervals in cumulative distribution functions

• The cumulative distribution function (CDF) of a continuous random variable $X$, denoted as $F(x)$, gives the probability that $X$ is less than or equal to a specific value $x$
  • $F(x) = P(X \leq x) = \int_{-\infty}^x f(t) dt$, where $f(t)$ is the probability density function
• To calculate the probability of a random variable falling within an interval $[a, b]$ using the CDF:
  • $P(a \leq X \leq b) = F(b) - F(a)$
  • Example: For a standard normal distribution, $P(-1 \leq X \leq 1) = F(1) - F(-1) \approx 0.8413 - 0.1587 = 0.6826$
• For the probability of a random variable being greater than a value $a$:
  • $P(X > a) = 1 - F(a)$
  • Example: For an exponential distribution with rate parameter $\lambda = 0.5$, $P(X > 2) = 1 - F(2) = 1 - (1 - e^{-0.5 \cdot 2}) \approx 0.3679$
• CDF properties:
  1. Non-decreasing: $F(a) \leq F(b)$ for $a < b$
  2. Limits: $\lim_{x \to -\infty} F(x) = 0$ and $\lim_{x \to \infty} F(x) = 1$
  3. Right-continuous: $\lim_{x \to a^+} F(x) = F(a)$ for all $a$
  4. Monotonicity: The CDF is a monotonically increasing function

Density vs cumulative distribution functions

• Probability Density Function (PDF):
  • Denoted as $f(x)$
  • Represents the relative likelihood of a random variable taking on a specific value
  • The area under the PDF curve between two points represents the probability of the random variable falling within that range
  • Properties:
    • Non-negative: $f(x) \geq 0$ for all $x$
    • Total area under the curve is 1: $\int_{-\infty}^{\infty} f(x) dx = 1$
    • Does not directly give probabilities for specific values or ranges (area under a single point is 0)
    • The support of the PDF is the set of all possible values the random variable can take
• Cumulative Distribution Function (CDF):
  • Denoted as $F(x)$
  • Represents the probability that a random variable is less than or equal to a specific value
  • Can be obtained by integrating the PDF: $F(x) = \int_{-\infty}^x f(t) dt$
  • Directly gives probabilities for specific values or ranges ($P(X \leq x) = F(x)$)
  • Properties:
    1. Non-decreasing: $F(a) \leq F(b)$ for $a < b$
    2. Limits: $\lim_{x \to -\infty} F(x) = 0$ and $\lim_{x \to \infty} F(x) = 1$
    3. Right-continuous: $\lim_{x \to a^+} F(x) = F(a)$ for all $a$
• The PDF and CDF are related by differentiation and integration:
  • $F(x) = \int_{-\infty}^x f(t) dt$
  • $f(x) = \frac{d}{dx} F(x)$ (for continuous CDFs)
  • Example: For a standard normal distribution, the PDF is $f(x) = \frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}$ and the CDF is $F(x) = \frac{1}{2}\left[1 + \text{erf}\left(\frac{x}{\sqrt{2}}\right)\right]$

Probability Space and Transformations

• Probability space: A mathematical construct that models a real-world process consisting of events that occur randomly
  • It includes the sample space, event space, and probability measure
• Transformation of variables: A method used to derive the probability distribution of a function of a random variable
  • This technique is useful when dealing with functions of random variables or when changing coordinate systems