🎳Intro to Econometrics Unit 3 Review

Functional form refers to the mathematical specification of how your dependent variable relates to your independent variables in a regression model. Getting this right matters a lot: if you specify the wrong functional form, your coefficient estimates can be biased and inconsistent, your standard errors can be wrong, and any policy conclusions you draw from the model may be misleading.

Choosing the right form means balancing simplicity against realism. A linear model is easy to interpret but might miss important curvature in the data. A nonlinear model can capture more complex patterns but brings its own risks, including overfitting and harder interpretation.

Linear vs nonlinear models

Advantages of linear models

Linear models assume a constant marginal effect of each independent variable on the dependent variable. If you increase $X$ by one unit, $Y$ always changes by the same amount $\beta$ , regardless of where you start.

This simplicity is a real strength:

They're computationally straightforward with well-established statistical properties (OLS is BLUE under the Gauss-Markov assumptions)
Coefficient interpretation is immediate: $\beta_1$ is the change in $Y$ for a one-unit change in $X_1$ , holding other variables constant
For many economic relationships, especially over a limited range of the data, a linear approximation works well enough

Limitations of linear models

They can't capture curvature. If the true relationship between income and spending is concave (diminishing marginal effect), a straight line will systematically over-predict in some ranges and under-predict in others.
The constant marginal effect assumption is often unrealistic. The effect of an extra year of education on wages, for example, likely differs between year 8 and year 16.
Linear models aren't appropriate for bounded dependent variables. If your outcome is a probability (between 0 and 1), a linear model can generate predictions outside that range.

Types of nonlinear models

Polynomial models add squared, cubic, or higher-order terms (e.g., $Y = \beta_0 + \beta_1 X + \beta_2 X^2$ ) to capture curvature
Logarithmic models transform variables using natural logs to capture proportional or multiplicative relationships (log-log, log-lin, lin-log)
Threshold/spline models allow different slopes across different ranges of an independent variable, connected at specified "knot" points
Logistic and probit models handle binary dependent variables by modeling the probability of an event as a nonlinear function of the regressors

Consequences of misspecification

Bias in coefficient estimates

When you use the wrong functional form, your coefficient estimates won't converge to their true values even as your sample grows. This is a consistency problem, not just a small-sample issue.

Bias can come from omitting a relevant nonlinear term (like leaving out $X^2$ when the true relationship is quadratic), using logs when the relationship is linear, or ignoring interactions between variables. The result is incorrect conclusions about both the size and direction of effects.

Inefficiency of estimates

Even if bias isn't severe, misspecification can make your estimates noisier than they need to be. The variances of your coefficients will be larger than the minimum achievable, leading to wider confidence intervals and reduced statistical power. You'll have a harder time detecting relationships that actually exist.

Invalid inference and hypothesis testing

This is where things get especially dangerous. If the functional form is wrong, your estimated standard errors may not reflect the true sampling variability of your coefficients. That means your t-statistics, p-values, and confidence intervals can all be wrong. You might conclude a variable is statistically significant when it isn't, or miss a genuinely important relationship.

Detecting functional form misspecification

Residual plots and patterns

Your first diagnostic tool is simple: plot the residuals against fitted values and against each independent variable. If the functional form is correct, residuals should look like random scatter with no systematic pattern.

Watch for these warning signs:

Curvature in the residual plot suggests you need nonlinear terms
Fanning out (residuals getting larger as fitted values increase) suggests heteroscedasticity, which can also signal a wrong functional form
Systematic clusters of positive or negative residuals across ranges of $X$

Advantages of linear models, Characteristics of Linear Functions | College Algebra

Ramsey RESET test

The Ramsey RESET test is a formal test for functional form misspecification. Here's how it works:

Estimate your original model and obtain the fitted values $\hat{Y}$
Add powers of the fitted values (typically $\hat{Y}^2$ and $\hat{Y}^3$ ) as additional regressors
Run an F-test for the joint significance of these added terms
If you reject the null (the added terms are jointly significant), your original functional form is likely misspecified

The RESET test is general-purpose: it doesn't tell you what the correct form is, only that yours is probably wrong.

Lagrange Multiplier (LM) tests

LM tests are useful for testing specific types of misspecification. Unlike the RESET test, you design an LM test around a particular alternative hypothesis, such as "there should be a quadratic term" or "there's omitted heteroscedasticity."

The test compares the fit of your restricted (original) model to a more general model that includes the suspected source of misspecification. If the LM statistic is large enough to reject the null, the more general specification is preferred.

Addressing functional form issues

Transformations of variables

Variable transformations can linearize a nonlinear relationship so you can still use OLS. The most common transformations and what they do:

Log-log model ( $\ln Y = \beta_0 + \beta_1 \ln X$ ): coefficients are elasticities. A 1% increase in $X$ is associated with a $\beta_1$ % change in $Y$ .
Log-lin model ( $\ln Y = \beta_0 + \beta_1 X$ ): $\beta_1 \times 100$ gives the approximate percentage change in $Y$ for a one-unit change in $X$ .
Lin-log model ( $Y = \beta_0 + \beta_1 \ln X$ ): $\beta_1 / 100$ gives the unit change in $Y$ for a 1% change in $X$ .
Box-Cox transformations offer a flexible family that nests several of these as special cases.

The choice should be guided by economic theory and the nature of your variables, not just what gives the best $R^2$ .

Polynomial and interaction terms

Adding a squared term like $X^2$ lets you model a relationship where the marginal effect of $X$ changes as $X$ increases. For example, $Y = \beta_0 + \beta_1 X + \beta_2 X^2$ captures a U-shaped or inverted U-shaped relationship depending on the sign of $\beta_2$ .

Interaction terms (e.g., $X_1 \times X_2$ ) model situations where the effect of one variable depends on the level of another. For instance, the return to education might differ by gender, captured by including $Education \times Female$ .

Be cautious: adding many polynomial and interaction terms increases model complexity and the risk of overfitting, especially with smaller samples.

Piecewise linear (spline) functions

Spline functions let you fit different slopes across different ranges of an independent variable. This is useful when you expect a structural break or threshold effect, like a policy that kicks in above a certain income level.

To construct a spline:

Choose "knot" points that divide the range of $X$ into segments
Estimate separate linear slopes for each segment
Impose continuity constraints so the fitted line doesn't jump at the knots

The result is a flexible, piecewise-linear approximation that can capture kinks and changes in slope without requiring a specific nonlinear form.

Model selection and validation

Goodness-of-fit measures

$R^2$ measures the proportion of variance in $Y$ explained by the model. Higher is better, but $R^2$ never decreases when you add variables, so it can reward overfitting.
Adjusted $R^2$ ( $\bar{R}^2$ ) penalizes for additional regressors. It can decrease if an added variable doesn't improve the fit enough to justify the lost degree of freedom.
RMSE (root mean squared error) measures the typical size of prediction errors in the units of $Y$ .

These measures are useful for comparing models with the same dependent variable. You can't compare $R^2$ across models where one uses $Y$ and another uses $\ln Y$ as the dependent variable.

Cross-validation techniques

Cross-validation assesses how well your model predicts data it wasn't trained on, which guards against overfitting:

Split your data into $k$ roughly equal subsets ("folds")
For each fold, estimate the model on the remaining $k-1$ folds and predict the held-out fold
Average the prediction error across all $k$ iterations

The functional form with the lowest average prediction error is preferred. Common choices are 5-fold or 10-fold cross-validation.

Akaike and Bayesian information criteria

AIC and BIC both balance fit against complexity, but they do it differently:

AIC penalizes each additional parameter by 2. It tends to favor slightly more complex models.
BIC penalizes each additional parameter by $\ln(n)$ , where $n$ is the sample size. For any sample larger than about 8 observations, BIC penalizes complexity more heavily than AIC.

In both cases, lower values are better. Choose the functional form that minimizes AIC or BIC. When AIC and BIC disagree, BIC's preference for parsimony is often a safer bet in econometrics.

Advantages of linear models, Build linear models | College Algebra

Interpreting nonlinear models

Marginal effects and elasticities

In a linear model, the marginal effect of $X$ on $Y$ is just $\beta$ , a constant. In nonlinear models, the marginal effect depends on where you evaluate it.

For a quadratic model $Y = \beta_0 + \beta_1 X + \beta_2 X^2$ , the marginal effect of $X$ is $\beta_1 + 2\beta_2 X$ . This changes with every value of $X$ .

Common practice is to report marginal effects evaluated at the sample means of the independent variables, or to report average marginal effects (the average of marginal effects computed at each observation).

Elasticities measure the percentage change in $Y$ for a 1% change in $X$ . In a log-log model, the coefficient is the elasticity. In other specifications, you compute it as:

$\text{Elasticity} = \frac{\partial Y}{\partial X} \cdot \frac{X}{Y}$

Graphical representations

Plotting predicted values of $Y$ against $X$ (holding other variables at their means) is one of the best ways to communicate nonlinear results. You can also plot marginal effects against $X$ to show how the relationship changes across the range of the data. These visuals are especially helpful for conveying results to audiences who aren't comfortable interpreting regression coefficients directly.

Challenges in interpretation

Coefficients in nonlinear models don't have the clean "one-unit change" interpretation that linear coefficients do. You almost always need to compute and report marginal effects.
With interaction terms or polynomials, the effect of one variable depends on the values of others, so you need to specify those values when reporting results.
Predictions from nonlinear models can become unreliable outside the range of the observed data. Extrapolation is risky with any model, but especially so with polynomials, which can behave erratically beyond the sample range.

Common nonlinear functional forms

Logarithmic and exponential models

Log-log models ( $\ln Y = \beta_0 + \beta_1 \ln X$ ) are popular because coefficients directly represent elasticities. A Cobb-Douglas production function estimated in log form is a classic example: $\ln Q = \beta_0 + \beta_1 \ln L + \beta_2 \ln K$ .

Semi-log models come in two flavors. In a log-lin model ( $\ln Y = \beta_0 + \beta_1 X$ ), a one-unit increase in $X$ is associated with an approximate $\beta_1 \times 100$ % change in $Y$ . The Mincer wage equation, where log wages are regressed on years of education, is a well-known example.

Exponential models are used when the dependent variable exhibits exponential growth or decay, such as compound interest or population growth.

Quadratic and higher-order polynomials

A quadratic specification $Y = \beta_0 + \beta_1 X + \beta_2 X^2$ is the most common polynomial form. If $\beta_2 < 0$ , the relationship is an inverted U (think of the Environmental Kuznets Curve, where pollution first rises then falls with income). If $\beta_2 > 0$ , it's U-shaped.

The turning point occurs at $X^* = -\beta_1 / (2\beta_2)$ , which you can calculate directly from the estimated coefficients.

Higher-order polynomials (cubic, quartic) can capture more complex shapes but are rarely used beyond cubic in practice. They're sensitive to outliers and can produce wild predictions outside the sample range.

Logistic and probit models

When your dependent variable is binary (0 or 1), linear probability models can predict probabilities outside [0, 1]. Logistic and probit models solve this by using nonlinear link functions that keep predicted probabilities bounded.

Logit uses the logistic CDF: $P(Y=1|X) = \frac{e^{X\beta}}{1 + e^{X\beta}}$
Probit uses the standard normal CDF: $P(Y=1|X) = \Phi(X\beta)$

The raw coefficients represent changes in log-odds (logit) or z-scores (probit), which aren't directly intuitive. You'll typically report marginal effects instead: the change in the predicted probability for a one-unit change in $X$ , evaluated at specific values or averaged across the sample.

Functional form in specific contexts

Production and cost functions

Production functions model how inputs (labor $L$ , capital $K$ ) translate into output $Q$ . The functional form you choose has direct implications for properties like returns to scale and the elasticity of substitution between inputs.

Cobb-Douglas: $Q = A L^{\alpha} K^{\beta}$ . Estimated in log form. Returns to scale are $\alpha + \beta$ . The elasticity of substitution is always 1.
CES (Constant Elasticity of Substitution): more flexible than Cobb-Douglas, allowing the elasticity of substitution to differ from 1, but harder to estimate.
Translog: a second-order Taylor approximation in logs that nests Cobb-Douglas as a special case. Very flexible but requires many parameters.

Cost functions can be derived from production functions via duality theory and often take similar forms (linear, quadratic, translog).

Demand and supply equations

Demand equations relate quantity demanded to price, income, and prices of related goods. The functional form determines the implied elasticities:

A linear demand model implies elasticities that vary along the demand curve
A log-log demand model implies constant elasticities, which is convenient but may not always be realistic
The AIDS (Almost Ideal Demand System) model is a flexible form used for estimating systems of demand equations across multiple goods

The choice of form matters for welfare analysis. Estimated consumer surplus, deadweight loss, and the effects of tax changes all depend on the assumed functional form.

Growth and convergence models

Growth models relate GDP growth to factors like capital accumulation, human capital, technology, and population growth. The Solow-Swan model assumes a Cobb-Douglas production function with constant returns to scale, leading to the prediction that poorer countries grow faster (conditional convergence).

Beta-convergence models regress growth rates on initial income levels. A negative coefficient on initial income supports convergence.
Sigma-convergence examines whether the cross-country dispersion of income narrows over time.

The functional form affects the estimated speed of convergence and the implied steady-state income levels. Using a log specification for GDP per capita is standard in this literature, since growth is inherently a proportional concept.

🎳Intro to Econometrics Unit 3 Review