Marginal likelihood

Marginal likelihood is the probability of your data under a model after you average over all possible parameter values. In Intro to Probability, it shows up in Bayesian reasoning when comparing models or updating beliefs.

What is Marginal likelihood?

Marginal likelihood is the chance of seeing your observed data after you take the model’s parameters out of the picture by averaging over them. In Intro to Probability, that usually means you start with a Bayesian model, treat the parameters as uncertain, and ask: if this model were true, how likely is the data overall?

That “averaging” step is the whole point. Instead of plugging in one parameter value, you combine the likelihood with the prior distribution and sum or integrate across every possible parameter value. For continuous parameters, this is an integral. For discrete parameters, it is a sum. The result is a single number that scores the model’s overall fit to the data.

A good way to think about it is that marginal likelihood measures how well a model explains the data before you fine-tune the parameters. A model that only fits the data for one tiny, lucky parameter value may not score very high, because most parameter values under that model do a poor job. A model that gives decent support across many parameter values can earn a larger marginal likelihood.

This is why marginal likelihood is different from the ordinary likelihood you may see in basic probability problems. Likelihood usually treats the parameter as fixed and asks how compatible the data are with that parameter value. Marginal likelihood treats the parameter as uncertain and builds that uncertainty into the calculation.

In Bayesian notation, marginal likelihood is often the denominator in Bayes’ theorem. It normalizes the posterior so the probabilities across competing hypotheses add up correctly. In practice, that denominator can be hard to compute, especially when a model has several parameters, so Intro to Probability courses often emphasize the meaning of the quantity more than the full technical calculation.

A small example makes the idea clearer. Suppose two coin-flip models could explain a run of heads and tails. One model assumes the coin is probably fair, with a prior centered near 0.5. Another allows almost any bias. Marginal likelihood compares how well each model explains the observed flips after accounting for all the parameter values each model allows, not just the best-fitting one.

Why Marginal likelihood matters in Intro to Probability

Marginal likelihood matters because it is what lets Bayesian probability compare whole models, not just individual parameter values. In Intro to Probability, that shows up when you want to choose between competing explanations for the same data, like whether a coin is fair, whether a spam filter should treat an email as suspicious, or whether one probability model fits a data set better than another.

It also connects directly to Bayes’ theorem. If you have a prior over models and a likelihood for the data under each model, the marginal likelihood is the normalizing piece that turns those raw scores into posterior model probabilities. Without it, you can describe evidence, but you cannot cleanly update your belief across different hypotheses.

The concept also explains a subtle Bayesian tradeoff: models with more flexibility are not automatically better. A flexible model can fit many data sets, but marginal likelihood rewards models that fit well while staying efficient, not just models that can contort themselves to match one sample. That idea shows up a lot in classroom comparisons of simpler versus more complex probability models.

It is also a useful bridge to later Bayesian topics like posterior predictive checks and Bayesian networks, because both rely on reasoning about a model as a whole, not just a single parameter estimate.

How Marginal likelihood connects across the course

Bayes' theorem

Marginal likelihood sits inside Bayes’ theorem as the denominator that turns a raw likelihood into a posterior probability. When you compare competing models, you are using Bayes’ theorem on the model level, and the marginal likelihood is what makes that update work correctly.

Prior distribution

The prior distribution is part of the marginal likelihood calculation because you average the likelihood across parameter values using the prior as the weighting rule. A different prior can change the marginal likelihood a lot, which is why prior choice matters in Bayesian model comparison.

Posterior distribution

The posterior distribution tells you what the parameter values look like after you see the data. Marginal likelihood is not the posterior itself, but it helps produce it by normalizing Bayes’ theorem. One is about updated parameter beliefs, the other is about how strongly the data support the model overall.

posterior predictive checks

Posterior predictive checks ask whether a model generates data that look like what you actually observed. Marginal likelihood comes earlier in the workflow, helping you judge the model as a whole before or alongside checking how well its posterior predictions match the data.

Is Marginal likelihood on the Intro to Probability exam?

A quiz or problem-set question will usually ask you to recognize marginal likelihood as the Bayesian probability of the data under a model after integrating over parameters. You may be asked to compare two models and explain which one the data support more strongly, or to identify the marginal likelihood term inside Bayes’ theorem. If a problem gives a prior and a likelihood, watch for the step where you average over all parameter values instead of choosing one best estimate. That is the move that separates marginal likelihood from ordinary likelihood.

Marginal likelihood vs likelihood

Likelihood measures how well one parameter value explains the data. Marginal likelihood averages that likelihood over all possible parameter values, using the prior as weights. So likelihood is about a parameter choice, while marginal likelihood is about the whole model.

Key things to remember about Marginal likelihood

• Marginal likelihood is the probability of the observed data under a model after you average over all possible parameter values.

• In Bayesian probability, it is the piece that lets you compare models, not just estimate a single parameter.

• You compute it by combining the likelihood with the prior and summing or integrating across the parameter space.

• A model with more flexibility does not automatically get a better marginal likelihood, because the calculation rewards overall support, not just one perfect fit.

• If you see Bayes’ theorem in a problem, the marginal likelihood is often the normalizing denominator that turns raw evidence into a posterior.

Frequently asked questions about Marginal likelihood

What is marginal likelihood in Intro to Probability?

Marginal likelihood is the probability of the observed data after you average over all possible values of the model parameters. In Intro to Probability, it comes up in Bayesian problems where you compare models or update beliefs. It is not the same as plugging in one parameter value and checking the likelihood.

How is marginal likelihood different from likelihood?

Likelihood evaluates how well a specific parameter value explains the data. Marginal likelihood averages that likelihood across all parameter values, usually using a prior distribution. That makes marginal likelihood a model-level quantity, not just a parameter-level one.

Why do I need marginal likelihood in Bayes' theorem?

You need it because Bayes’ theorem uses it as the denominator that normalizes the posterior. Without it, the probabilities would not add up correctly. In model comparison, it also tells you how strongly the data support one model versus another.

Is marginal likelihood hard to calculate?

It can be, especially when a model has many parameters or a complicated prior. In class, you may focus more on the setup and interpretation than on the full computation. For harder models, people often use numerical methods or approximations.