Probit Model

What Is The Probit Model?

The probit model is a statistical technique used in econometrics and other fields to model the relationship between a binary dependent variable and one or more independent variables. The probit model aims to estimate the probability that the dependent variable takes a particular value (usually 1) based on the values of the independent variables.

This model helps in understanding the relationship between predictors and the likelihood of an event, enabling predictions and making inferences about the probability of an outcome occurring. Its applications are widespread, from economics (for example, predicting the likelihood of default on a loan) to social sciences and medicine, where it can predict the likelihood of a patient having a particular condition based on various diagnostic factors.

• The probit model estimates the probability of a binary outcome, such as success/failure, presence/absence, yes/no, etc., based on predictor variables.
• It employs the cumulative distribution function of the standard normal distribution (probit function) to estimate probabilities, often generating an S-shaped curve when graphed against predictor variables.
• The model assumes linearity between predictors and probabilities, regular distribution of errors, and independence of observations, which should be evaluated and validated for reliable interpretation of results.
• They are widely used in various fields such as economics, medicine, social sciences, and more to predict outcomes like default/non-default, disease presence, customer choices, and others where binary outcome prediction is crucial.

Probit Model Explained

The probit model is a statistical method used to analyze the relationship between one or more independent variables and a binary outcome, providing a framework to predict the probability of a particular event occurring. It originated from the field of statistics and was primarily introduced by economist Chester Bliss in the 1930s. Later, the concept was refined and popularised by economist Robert Probit, leading to its current name.

This model assumes a linear relationship between the predictors and the underlying probability of the binary outcome. It then employs the cumulative distribution function of the standard normal distribution, known as the probit function, to map this relationship. The probit function converts the linear combination of predictors into probabilities, constrained from 0 to 1. Essentially, it estimates the likelihood of a binary event taking place based on the given set of predictor variables.

The probit model finds application in various fields like economics, medicine, social sciences, and more. For instance, in economics, it can predict consumer choices or the likelihood of a person defaulting on a loan. In medical research, it might predict the probability of a patient having a particular disease based on various health indicators.

Assumptions

The probit model relies on several key assumptions:

1. Binary Outcome: The dependent variable must be dichotomous, representing two mutually exclusive and exhaustive outcomes, often coded as 0 and 1.
2. Linearity: The relationship between the independent variables and the probability of the outcome occurring is linear. This linear relationship maps through the probit function.
3. Independence of Errors: The errors in the model should be independent of each other. This is crucial to ensure that one observation's error doesn't influence another, allowing for an accurate estimation of the model's coefficients.
4. No Multicollinearity: The independent variables should not be highly correlated with each other. High multicollinearity can make it difficult to estimate the individual effect of each variable on the outcome.
5. Normality of Errors: The errors (residuals) in the model should follow a standard normal distribution. While the errors themselves aren't observable, the assumption of normality is necessary for the probit function to estimate probabilities accurately.
6. Homoscedasticity: The variance of the errors should be constant across all values of the independent variables. In other words, the spread of errors should be consistent along the range of predictors.

Formula

The probit model formulates the relationship between the probability of a binary outcome (typically denoted as "Y") and one or more independent variables (often denoted as "X") through a cumulative distribution function. The probit model assumes a linear relationship between the independent variables and the probability of the outcome. The formula for the probit model can be written as:

P(Y = 1) = Φ(β0 + β1X1 + β2X2 + ... + βkXk)

Here's a breakdown of the formula:

• P(Y = 1) represents the probability of the binary outcome Y being equal to 1 (success or event occurrence).
• Φ denotes the cumulative distribution function of the standard normal distribution. It transforms the linear combination of the independent variables into a probability, ensuring that the outcome probability is between 0 and 1.
• β0, β1, β2, ..., βk are the coefficients that the model estimates. These coefficients represent the impact or contribution of each independent variable X1, X2, ..., Xk to the probability of the binary outcome.
• X1, X2, ..., Xk are the independent variables that influence the probability of the binary outcome.

In practical applications, the coefficients (β0, β1, β2, ...) use maximum likelihood estimation, and the values of the independent variables (X1, X2, ...) for a specific case plug into the formula to calculate the predicted probability of the event (Y = 1).

Examples

Let us understand it better with the help of examples:

Example #1

Let's suppose an imaginary example where one wants to predict the probability of a customer making a purchase (Y = 1) based on two independent variables: the customer's age (X1) and the number of previous purchases (X2). One can estimate the following coefficients for the probit model:

β0 = -2.0 β1 = 0.04 β2 = 0.5

Now, suppose one has a customer who is 35 years old (X1 = 35) and has made three previous purchases (X2 = 3). If one wants to calculate the probability of this customer making a purchase (Y = 1) using the probit model formula:

P(Y = 1) = Φ(β0 + β1X1 + β2X2)

= Φ(-2.0 + 0.0435 + 0.53)

= Φ(-2.0 + 1.4 + 1.5) = Φ(0.9)

Now, consult a standard regular distribution table or use statistical software to find the cumulative probability associated with 0.9. Let's assume that Φ(0.9) is approximately 0.8159.

So, in this imaginary example, the estimated probability of the customer making a purchase is approximately 0.8159 or 81.59%. This means there's an 81.59% chance that this customer will make a purchase, given their age and previous purchase history, according to the probit model.

Example #2

Goldman Sachs analysts released a report in 2023, shedding light on factors that increase the likelihood of a company becoming a target for activist investors seeking to make fundamental changes. The analysis examined over 2,000 shareholder activism campaigns launched since 2006. The report identified four financial variables that might prompt an activist attack: slower sales growth, lower valuations, weaker net margin, and trailing underperformance. Sales growth was identified as the most significant variable. A probit model was used to analyze characteristics associated with companies targeted by activists, leading to the identification of 116 stocks susceptible to activist campaigns in the Russell 3000 index.

The report also noted that the top three demands of activist investors have been for companies to separate their businesses, review strategic alternatives, and return cash to shareholders. Shareholder activism has been on the rise, with 148 campaigns against 120 U.S. companies in 2022. Analysts expect activism to remain popular in 2023. Notable companies like Walt Disney and Salesforce were targeted in the first quarter of this year, indicating that shareholder activism remains a significant force in corporate governance.

Graph

The graph of a probit model typically represents the relationship between the probability of a binary outcome and one of the independent variables. Since a probit model estimates probabilities, the graph visualizes how changes in an independent variable affect the predicted probability of the binary outcome.

Suppose a scenario where one has a single independent variable (X-axis) and the predicted probability of the binary outcome (Y = 1) (Y-axis). The graph would demonstrate the relationship between this independent variable and the probability of the event occurring.

The graph might show an S-shaped or sigmoid curve due to the nature of the probit function, which is based on the cumulative distribution function of the standard normal distribution. As the independent variable changes, the predicted probability increases or decreases, following the pattern determined by the coefficients and the functional form of the model.

Advantages And Disadvantages

Here is a brief representation of the advantages and disadvantages of the probit model:

Advantages	Disadvantages
1. Provides accurate probability estimates for binary outcomes.	2. Requires large sample sizes to produce reliable estimates, which can be limited in certain cases.
4. The assumption of normal distribution might not always hold in real-world scenarios.	2. Applicable to a wide range of fields, including economics, medicine, and social sciences.
3. Handles well with binary dependent variables and accommodates multiple independent variables.	3. Difficulty in interpretation as coefficients do not directly signify the impact on the outcome.
4. Robustness to outliers in the data.	4. Assumption of normal distribution might not always hold in real-world scenarios.
5. Capable of handling non-linear relationships between predictors and probabilities through transformations.	5. Prone to overfitting when dealing with small or unrepresentative datasets.
6. Enables comparison of the effects of different predictors on the probability of the outcome.	6. Interpretation can be challenging for non-statisticians.

Difference Between Probit Model And Logit Model

Below is a comparison between the probit model and the logit model:

Probit Model	Logit Model
1. Uses the cumulative distribution function of the standard normal distribution (probit function).	1. Applies the logistic function (sigmoid curve) for estimation.
2. Assumes errors follow a standard normal distribution.	2. Assumes errors follow a logistic distribution.
3. Estimates coefficients that do not have a direct probability interpretation.	3. Estimates coefficients that have a direct probability interpretation (odds ratios).
4. Commonly used in disciplines like economics and bio-statistics.	4. Widely used in fields such as medicine and social sciences.
5. Less intuitive interpretation of results due to the probit function.	5. Coefficients directly represent the change in the odds of the event.
6. It approximates the cumulative distribution function of the standard normal distribution.	6. Estimates the log odds of the probability.
7. May provide a slightly better fit if the underlying assumptions are met.	7. More computationally efficient and easier to interpret.

Frequently Asked Questions (FAQs)

Recommended Articles

This article has been a guide to what is Probit Model. Here, we explain its formula, examples, comparison with logit model, assumptions, advantages, and graph. You may also find some useful articles here -

• Leading Indicators
• Ordinal Regression
• Ordinal Logistic Regression