We want the PDF of \(Y = g(X)\) for a continuous RV \(X\) with known PDF \(f_X\).
Step 1. Find the CDF of \(Y\): \[F_Y(y) = P(Y \le y) = P(g(X) \le y) = \int_{\{x \mid g(x) \le y\}} f_X(x)dx\] by translating the event \(\{g(X)\le y\}\) into an event about \(X\) and using \(f_X\) to compute its probability.
Why not go directly PMF-style? In the discrete case, if \(g\) is invertible we can write \(p_Y(y) = p_X(g^{-1}(y))\) directly. We need to go through the Fundamental Law of Calculus/differentiate the CDF route because we can only say that the mass of events are equal, not the density!
Plugging the Normal PDF into the linear-case formula gives \[f_Y(y) = \frac{1}{|a|\sigma\sqrt{2\pi}} \exp\left(-\frac{1}{2}\left(\frac{\frac{y-b}{a}-\mu}{\sigma}\right)^2\right)\] which is exactly the \(\mathcal{N}(a\mu + b,\; a^2\sigma^2)\) density.
We saw this previously, using the PDF of a monotonic function of a RV theorem!
Exercise
Exercise
\(X\) is Exponential(\(\lambda\)), and \(Y = aX + b\) with \(a > 0\).
What is the PDF of \(Y\)?
Is \(Y\) exponential?
Exercise – Solution
For \(a>0\): \(x = \frac{y-b}{a} \ge 0 \iff y \ge b\), so \[f_Y(y) = \frac{1}{a} \lambda\, e^{-\lambda (y-b)/a}, \qquad y \ge b\]\[F_Y(y) = 1 - e^{-\lambda (y-b)/a}, \qquad y \ge b\]
Is \(Y\) exponential? Only if \(b = 0\) – then \(Y \sim \text{Exponential}(\lambda/a)\) exactly. If \(b \ne 0\), \(Y\) has the exponential shape but shifted to start at \(b\) instead of \(0\): it’s memoryless from \(b\) onward, not from \(0\), so it is not itself an Exponential random variable.
(If \(a<0\) instead, the same formula applies but the support flips to \(y \le b\).)
Code
import numpy as npimport matplotlib.pyplot as pltrng = np.random.default_rng(2)lam, a =0.5, 2.0fig, ax = plt.subplots(figsize=(8, 4.5))for b, color in [(0.0, "C0"), (3.0, "C3")]: X = rng.exponential(1/ lam, 200_000) Y = a * X + b y = np.linspace(b, Y.max(), 400) fy = (lam / a) * np.exp(-lam * (y - b) / a) ax.hist(Y, bins=60, density=True, alpha=0.35, color=color) label =r"$b=0$: exactly Exponential($\lambda/a$)"if b ==0elser"$b=3$: shifted, not Exponential" ax.plot(y, fy, color=color, lw=2, label=label)ax.set_title(r"$Y = aX+b$ for $X\sim\mathrm{Exponential}(\lambda)$")ax.legend()plt.tight_layout()plt.show()
Recall: Monotonic Case
The linear case generalizes: if \(g\) is strictly monotonic and differentiable, with inverse \(h = g^{-1}\), \[f_Y(y) = f_X(h(y))\, |h'(y)|\]
Specializing to \(g(x) = ax+b\): \(h(y) = \frac{y-b}{a}\), \(h'(y) = \frac{1}{a}\), recovering exactly the linear-case formula above.
Review of Proof
The formula follows from the same two-step process, using that monotonicity lets us rewrite the event \(\{g(X) \le y\}\) as an event about \(X\) alone: \[\{g(X) \le y\} = \{X \le h(y)\}\] if \(g\) is increasing, or \(\{X \ge h(y)\}\) if \(g\) is decreasing.
Functions of Two Random Variables
Same two-step process applies: For \(Z = g(X,Y)\), jointly continuous \(X,Y\):
The only new wrinkle is that “the event” is now a region of the plane rather than an interval of the line.
Example: \(X, Y \sim \text{Uniform}(0,1)\) independent, \(Z = Y/X\)
\(\{Z \le z\} = \{Y \le zX\}\): the region of the unit square below the line \(y = zx\). Its area looks different depending on whether the line exits through the top of the square or the right edge.
Code
import numpy as npimport matplotlib.pyplot as pltfig, axes = plt.subplots(1, 2, figsize=(11, 5))for ax, z, title in [(axes[0], 0.5, "z < 1: line exits the right edge"), (axes[1], 2.0, "z >= 1: line exits the top edge")]: xx = np.linspace(0, 1, 200) yy = np.clip(z * xx, 0, 1) ax.fill_between(xx, 0, yy, color="C0", alpha=0.4) ax.plot(xx, np.clip(z * xx, 0, 1.3), color="C0", lw=2, label=f"y = {z}x") ax.set_xlim(0, 1) ax.set_ylim(0, 1) ax.set_aspect("equal") ax.set_title(f"z = {z}\n{title}") ax.set_xlabel("x") ax.set_ylabel("y") ax.legend(loc="upper left")plt.tight_layout()plt.show()
Integrating the shaded area in each case: \[F_Z(z) = \begin{cases} \dfrac{z}{2} & 0 \le z < 1 \\[4pt] 1 - \dfrac{1}{2z} & z \ge 1 \end{cases}
\qquad\Longrightarrow\qquad
f_Z(z) = \begin{cases} \dfrac12 & 0 < z < 1 \\[4pt] \dfrac{1}{2z^2} & z \ge 1 \end{cases}\]
For independent discrete RVs \(X, Y\) and \(Z = X+Y\): \[
\begin{aligned}
p_Z(z) &= P(X + Y = z)
\\&= \sum_{\{(x,y) \mid x+y = z\}}P(X=x, Y=y)
\\&= \sum_x P(X=x, Y = z-x)
\\&=\sum_x p_X(x)\, p_Y(z-x)
\end{aligned}
\]
We sum over every way \(x\) and \(z-x\) could combine to produce the total \(z\). This is the convolution of \(p_X\) and \(p_Y\).
The Continuous Case
Replacing sums with integrals: for independent continuous \(X,Y\), \[F_Z(z) = P(X+Y\le z) = \int_{-\infty}^{\infty} f_X(x)\, F_Y(z-x)\, dx\] Differentiating in \(z\) gives the continuous convolution formula: \[f_Z(z) = \int_{-\infty}^{\infty} f_X(x)\, f_Y(z-x)\, dx\]
Example: Sum of i.i.d. \(\text{Uniform}(0,1)\) RVs
Suppose \(X, Y \sim \text{Uniform}([0,1])\) are independent. The PDF of \(Z = X + Y\) is \[f_Z(z) = \int_{-\infty}^\infty f_X(x)f_Y(z-x)dx.\] The integrand is nonzero (equal to 1) for \(0 \le x \le 1\) and \(0 \le z-x \le 1\).
Combining these, we see the integrand is nonzero for \(\max\{0,z-1\} \le x \le \min\{1,z\}\). Thus, \[f_Z(z) = \min\{1,z\} - \max\{0,z-1\} \text{ for } 0 \le z \le 2.\]
Example: Sum of Independent Normal RVs is Normal
If \(X\sim\mathcal N(\mu_1,\sigma_1^2)\), \(Y\sim\mathcal N(\mu_2,\sigma_2^2)\) independent, then \[X+Y \sim \mathcal N(\mu_1+\mu_2,\ \sigma_1^2+\sigma_2^2)\]
Sketch
Plugging both Gaussian densities into \(f_Z(z)=\int f_X(x)f_Y(z-x)\,dx\) gives an integrand of the form \(\exp(-(\text{quadratic in } x))\); completing the square in \(x\) separates the integral into a constant (in \(x\)) factor times a Gaussian integral over \(x\) that evaluates to \(1\). What’s left as a function of \(z\) is exactly the \(\mathcal N(\mu_1+\mu_2,\sigma_1^2+\sigma_2^2)\) density.
Graphical Calculation of Convolutions
\(f_Z(z) = \int f_X(x)\, f_Y(z-x)\, dx\) has a picture: flip\(f_Y\) about the vertical axis, slide it by \(z\), multiply pointwise against \(f_X\), and the shaded overlap area is \(f_Z(z)\).
