English:

In the theory of probability, regression refers to the statistical relationship between a dependent random variable $Y$ (also called the response or outcome) and one or more independent random variables $X$ (also called predictors, covariates, or explanatory variables). Regression aims to model how the conditional expectation of $Y$ depends on $X$.

Core Idea of Regression in Probability

The goal is to find the conditional expectation of $Y$ given $X$, denoted as $mathbb{E}[Y | X]$. This is called the regression function. It provides the best estimate of $Y$, in the sense of minimizing the expected squared error, given the value of $X$.

Mathematically:

$$mathbb{E}[Y | X] = g(X),$$

where $g(X)$ is a function determined by the joint probability distribution of $(X, Y)$. This function is the regression function, which captures the relationship between $X$ and $Y$.

Types of Regression

1. Linear Regression:

   • Assumes a linear relationship between $Y$ and $X$: $$mathbb{E}[Y | X] = beta_0 + beta_1 X.$$
   • The parameters $beta_0$ (intercept) and $beta_1$ (slope) are determined by minimizing the expected squared difference between $Y$ and $mathbb{E}[Y | X]$: $$min_{beta_0, beta_1} mathbb{E}left[left(Y - (beta_0 + beta_1 X)right)^2right].$$

2. Nonlinear Regression:

   • The relationship between $Y$ and $X$ is not linear. The regression function $g(X)$ can be a more general, nonlinear function such as $mathbb{E}[Y | X] = alpha + beta sin(X)$.

3. Multiple Regression:

   • When $Y$ depends on multiple independent variables $mathbf{X} = (X_1, X_2, dots, X_k)$, the regression function becomes: $$mathbb{E}[Y | mathbf{X}] = g(X_1, X_2, dots, X_k).$$

4. Logistic Regression:

   • Used for modeling the probability of a binary outcome $Y in {0, 1}$: $$mathbb{P}(Y = 1 | X) = frac{1}{1 + e^{-(beta_0 + beta_1 X)}}.$$

Key Theoretical Properties

1. Best Predictor in Mean-Square Sense:

   • The regression function $mathbb{E}[Y | X]$ minimizes the mean squared error: $$mathbb{E}left[left(Y - g(X)right)^2right].$$
   • This makes $mathbb{E}[Y | X]$ the optimal prediction of $Y$ given $X$.

2. Law of Total Expectation:

   • Regression relates to the law of total expectation: $$mathbb{E}[Y] = mathbb{E}[mathbb{E}[Y | X]].$$
   • This decomposes the expectation of $Y$ into the expectation of its conditional expectation.

3. Decomposition of Variance:

   • The total variance of $Y$ can be decomposed as: $$text{Var}(Y) = mathbb{E}[text{Var}(Y | X)] + text{Var}(mathbb{E}[Y | X]),$$ where:
     • $mathbb{E}[text{Var}(Y | X)]$: Expected variance of $Y$ given $X$.
     • $text{Var}(mathbb{E}[Y | X])$: Variance of the conditional expectation.

Regression vs. Correlation

While correlation measures the strength and direction of a linear relationship between two random variables, regression explicitly models the conditional relationship, focusing on predicting $Y$ given $X$. Correlation is symmetric (no dependent/independent distinction), but regression is asymmetric (predicting one variable from another).

Applications in Probability and Statistics

1. Prediction:

   • Use regression to estimate $Y$ based on observed $X$, such as forecasting sales or weather.

2. Causal Inference:

   • Analyze how changes in $X$ affect $Y$, such as studying the effect of a drug dose on recovery time.

3. Exploratory Data Analysis:

   • Understand relationships between variables by modeling $mathbb{E}[Y | X]$.

4. Uncertainty Quantification:

   • Incorporate probabilistic methods to assess the variability of predictions.

In summary, regression in the theory of probability focuses on modeling the conditional expectation of one random variable $Y$ based on another $X$, providing a foundational tool for prediction, inference, and understanding relationships in data.

Vietnamese: