10.3 Hybrid algorithms

Hybrid algorithms are game-changers in causal inference, combining machine learning and statistical methods to estimate causal effects in complex data. They leverage the flexibility of machine learning while maintaining desirable statistical properties like double robustness and efficiency.

Popular hybrid algorithms include targeted maximum likelihood estimation (TMLE), augmented inverse probability weighting (AIPW), and double machine learning (DML). These methods use techniques like cross-fitting and efficient influence functions to provide robust, efficient estimates of causal effects in various study designs.

Hybrid algorithms overview

• Hybrid algorithms combine machine learning and statistical methods to estimate causal effects and deal with complex data structures in causal inference
• Leverage the flexibility and predictive power of machine learning while maintaining desirable statistical properties like double robustness and efficiency
• Commonly used hybrid algorithms include targeted maximum likelihood estimation (TMLE), augmented inverse probability weighting (AIPW), and double machine learning (DML)

Targeted maximum likelihood estimation (TMLE)

TMLE procedure

Top images from around the web for TMLE procedure

• 

  Research on Parameter Optimization in Collaborative Filtering Algorithm View original

  Is this image relevant?

• 

  Frontiers | Causal Learning From Predictive Modeling for Observational Data View original

  Is this image relevant?

• 

  Frontiers | Causal Learning From Predictive Modeling for Observational Data View original

  Is this image relevant?

• 

  Research on Parameter Optimization in Collaborative Filtering Algorithm View original

  Is this image relevant?

• 

  Frontiers | Causal Learning From Predictive Modeling for Observational Data View original

  Is this image relevant?

1 of 3

Top images from around the web for TMLE procedure

• 

  Research on Parameter Optimization in Collaborative Filtering Algorithm View original

  Is this image relevant?

• 

  Frontiers | Causal Learning From Predictive Modeling for Observational Data View original

  Is this image relevant?

• 

  Frontiers | Causal Learning From Predictive Modeling for Observational Data View original

  Is this image relevant?

• 

  Research on Parameter Optimization in Collaborative Filtering Algorithm View original

  Is this image relevant?

• 

  Frontiers | Causal Learning From Predictive Modeling for Observational Data View original

  Is this image relevant?

1 of 3

• TMLE is an iterative procedure that updates an initial estimator of the outcome regression and propensity score to achieve a targeted bias-variance trade-off
• Involves constructing a targeted estimator by maximizing a targeted likelihood, which incorporates information about the target parameter
• Requires specifying a loss function (e.g., negative log-likelihood) and a fluctuation submodel for updating the initial estimators
• Iteratively updates the estimators until convergence, ensuring the final estimator solves the efficient influence function equation

TMLE for causal effect estimation

• TMLE can be used to estimate various causal effects, such as the average treatment effect (ATE), average treatment effect on the treated (ATT), and conditional average treatment effect (CATE)
• Requires specifying the target parameter as a function of the potential outcomes (e.g., $ATE = E[Y(1) - Y(0)]$)
• Involves estimating the outcome regression and propensity score using machine learning algorithms (e.g., Super Learner)
• The targeted estimator is obtained by updating the initial estimators using the efficient influence function for the target parameter

TMLE vs traditional methods

• TMLE is doubly robust, meaning it is consistent if either the outcome regression or propensity score is correctly specified
• Achieves optimal asymptotic efficiency when both models are correctly specified
• Allows for flexible estimation of nuisance parameters using machine learning, reducing model misspecification bias
• Provides valid inference and confidence intervals based on the efficient influence function
• Traditional methods, such as inverse probability weighting (IPW) and outcome regression, are sensitive to model misspecification and may have suboptimal efficiency

Augmented inverse probability weighting (AIPW)

AIPW estimator

• AIPW is a doubly robust estimator that combines inverse probability weighting (IPW) and outcome regression
• The AIPW estimator is defined as: $\hat{\psi}_{AIPW} = \frac{1}{n} \sum_{i=1}^n \left(\frac{A_i Y_i}{\hat{e}(X_i)} - \frac{A_i - \hat{e}(X_i)}{\hat{e}(X_i)} \hat{m}(X_i)\right)$ where $\hat{e}(X_i)$ is the estimated propensity score and $\hat{m}(X_i)$ is the estimated outcome regression
• Achieves double robustness by incorporating both the propensity score and outcome regression in the estimator
• Can be used to estimate various causal effects, such as the ATE, ATT, and CATE

AIPW for missing data problems

• AIPW can be applied to missing data problems, such as missing outcomes or covariates
• Involves estimating the propensity score for missingness and the outcome regression using observed data
• The AIPW estimator adjusts for missing data by weighting observed outcomes by the inverse probability of being observed and augmenting with the estimated outcome regression
• Provides consistent estimates under the missing at random (MAR) assumption and correct specification of either the propensity score or outcome regression

AIPW vs IPW and outcome regression

• AIPW is doubly robust, while IPW and outcome regression are singly robust
• AIPW is more efficient than IPW when the propensity score is correctly specified and more efficient than outcome regression when the outcome model is correctly specified
• AIPW can achieve the semiparametric efficiency bound when both models are correctly specified
• AIPW provides valid inference and confidence intervals based on the efficient influence function
• IPW and outcome regression may be sensitive to model misspecification and have suboptimal efficiency

Efficient influence functions (EIF)

EIF definition and properties

• The efficient influence function (EIF) is a key concept in semiparametric theory and plays a central role in the construction of efficient estimators
• EIF is the influence function of the efficient estimator, which achieves the smallest asymptotic variance among all regular asymptotically linear (RAL) estimators
• EIF satisfies the following properties:
  • It is a mean-zero function of the observed data and the target parameter
  • It is the pathwise derivative of the target parameter functional
  • It is the score function of the least favorable submodel for the target parameter
• The variance of the EIF provides a lower bound for the asymptotic variance of any RAL estimator (i.e., the semiparametric efficiency bound)

EIF in TMLE and AIPW

• In TMLE, the targeted estimator is constructed by solving the EIF estimating equation, ensuring that the final estimator is asymptotically efficient
• The EIF for the target parameter (e.g., ATE) is used to define the fluctuation submodel and update the initial estimators in the TMLE procedure
• In AIPW, the estimator is defined as the sample average of the EIF evaluated at the estimated nuisance parameters (propensity score and outcome regression)
• The AIPW estimator is efficient when both the propensity score and outcome regression are correctly specified, as it solves the EIF estimating equation

EIF-based confidence intervals

• The EIF can be used to construct asymptotically valid confidence intervals for the target parameter
• The variance of the EIF estimator provides a consistent estimate of the asymptotic variance of the efficient estimator
• A Wald-type confidence interval can be constructed as: $\hat{\psi} \pm z_{\alpha/2} \sqrt{\frac{1}{n} \sum_{i=1}^n \widehat{EIF}(\hat{\psi}, \hat{\eta})^2}$ where $\hat{\psi}$ is the efficient estimator, $\hat{\eta}$ denotes the estimated nuisance parameters, and $z_{\alpha/2}$ is the $1-\alpha/2$ quantile of the standard normal distribution
• EIF-based confidence intervals have correct asymptotic coverage and are robust to model misspecification, as long as the estimator is consistent and asymptotically normal

Double machine learning (DML)

DML framework

• Double machine learning (DML) is a framework for estimating causal effects and other statistical parameters using machine learning methods while maintaining valid inference
• DML involves estimating nuisance parameters (e.g., propensity score and outcome regression) using machine learning algorithms and constructing a doubly robust estimator based on the efficient influence function (EIF)
• The key steps in the DML framework are:
  1. Split the data into K folds for cross-fitting
  2. For each fold k, estimate the nuisance parameters using the other K-1 folds
  3. Construct the EIF estimator using the estimated nuisance parameters and the left-out fold
  4. Average the EIF estimators across all folds to obtain the final DML estimator
• DML ensures that the bias induced by machine learning estimation of the nuisance parameters does not affect the asymptotic distribution of the final estimator

DML for treatment effect estimation

• DML can be used to estimate various treatment effects, such as the average treatment effect (ATE), average treatment effect on the treated (ATT), and conditional average treatment effect (CATE)
• For the ATE, the EIF estimator in the DML framework is given by: $\hat{\psi}_{DML} = \frac{1}{K} \sum_{k=1}^K \frac{1}{n_k} \sum_{i \in I_k} \left(\frac{A_i (Y_i - \hat{m}^{(-k)}(X_i))}{\hat{e}^{(-k)}(X_i)} + \hat{m}^{(-k)}(X_i) - \hat{\psi}^{(-k)}\right)$ where $I_k$ is the set of indices in fold k, $n_k$ is the size of fold k, $\hat{m}^{(-k)}$ and $\hat{e}^{(-k)}$ are the estimated outcome regression and propensity score using the other K-1 folds, and $\hat{\psi}^{(-k)}$ is the estimated ATE using the other K-1 folds
• DML estimators are doubly robust, efficient, and provide valid inference under mild conditions on the nuisance parameter estimators (e.g., $n^{1/4}$-consistency)

DML vs traditional machine learning

• Traditional machine learning focuses on prediction and often relies on cross-validation for model selection and performance assessment
• DML, on the other hand, is designed for estimating causal effects and other statistical parameters while maintaining valid inference
• DML uses cross-fitting to avoid overfitting and ensure that the bias induced by machine learning estimation does not affect the asymptotic distribution of the final estimator
• DML estimators are doubly robust and efficient, whereas traditional machine learning estimators may be biased and lack efficiency guarantees
• DML provides asymptotically valid confidence intervals and hypothesis tests, which are not directly available in traditional machine learning

Cross-fitting technique

Cross-fitting procedure

• Cross-fitting is a sample-splitting technique used in hybrid algorithms like DML and TMLE to avoid overfitting and ensure valid inference
• The cross-fitting procedure involves the following steps:
  1. Randomly split the data into K folds (e.g., K = 5 or 10)
  2. For each fold k, estimate the nuisance parameters (e.g., propensity score and outcome regression) using the other K-1 folds as training data
  3. Construct the efficient influence function (EIF) estimator for each observation in fold k using the estimated nuisance parameters from step 2
  4. Repeat steps 2-3 for all K folds
  5. Average the EIF estimators across all observations to obtain the final cross-fitted estimator
• Cross-fitting ensures that the nuisance parameters are estimated on a separate dataset from the one used to construct the final estimator, reducing overfitting bias

Cross-fitting in TMLE and DML

• In TMLE, cross-fitting is used to estimate the initial outcome regression and propensity score models
• The targeted update step in TMLE is then performed using the estimated nuisance parameters from the corresponding cross-fitting fold
• The final TMLE estimator is obtained by averaging the targeted estimators across all cross-fitting folds
• In DML, cross-fitting is used to estimate the nuisance parameters and construct the EIF estimator for each fold
• The final DML estimator is obtained by averaging the EIF estimators across all cross-fitting folds
• Cross-fitting in both TMLE and DML ensures that the bias induced by machine learning estimation of the nuisance parameters does not affect the asymptotic distribution of the final estimator

Cross-fitting benefits and trade-offs

• Benefits of cross-fitting:
  • Reduces overfitting bias by estimating nuisance parameters on a separate dataset from the one used to construct the final estimator
  • Ensures valid inference by avoiding the bias induced by machine learning estimation of the nuisance parameters
  • Improves efficiency by allowing the use of more flexible machine learning methods for nuisance parameter estimation
• Trade-offs of cross-fitting:
  • Increases computational complexity, as nuisance parameters need to be estimated K times (once for each fold)
  • May reduce the effective sample size for nuisance parameter estimation, especially when K is large
  • The choice of K involves a bias-variance trade-off: larger K reduces bias but may increase variance due to smaller training sets for nuisance parameter estimation
• In practice, the choice of K depends on the sample size, the complexity of the nuisance parameter models, and the computational resources available

Hybrid algorithms performance

Efficiency and robustness

• Hybrid algorithms like TMLE, AIPW, and DML are designed to achieve optimal asymptotic efficiency while maintaining robustness to model misspecification
• Efficiency refers to the ability of an estimator to achieve the smallest possible asymptotic variance among all regular asymptotically linear (RAL) estimators
• Robustness refers to the ability of an estimator to remain consistent and asymptotically normal even when some of the nuisance parameter models are misspecified
• Hybrid algorithms achieve efficiency and robustness by leveraging the efficient influence function (EIF) and the double robustness property
• The EIF is used to construct the estimators and provide a lower bound for the asymptotic variance, ensuring efficiency
• Double robustness ensures that the estimators remain consistent and asymptotically normal as long as either the propensity score or the outcome regression model is correctly specified

Finite sample properties

• While hybrid algorithms have desirable asymptotic properties, their finite sample performance may depend on several factors:
  • Sample size: Hybrid algorithms may require larger sample sizes to achieve their asymptotic properties, especially when using complex machine learning methods for nuisance parameter estimation
  • Choice of machine learning methods: The performance of hybrid algorithms depends on the ability of the machine learning methods to accurately estimate the nuisance parameters
  • Tuning parameters: Machine learning methods often involve tuning parameters (e.g., regularization strength, depth of trees) that can affect the finite sample performance of hybrid algorithms
  • Degree of model misspecification: The finite sample performance of hybrid algorithms may deteriorate when the degree of model misspecification is high
• In practice, it is important to assess the finite sample performance of hybrid algorithms through simulation studies and sensitivity analyses

Asymptotic properties

• Hybrid algorithms have attractive asymptotic properties under mild conditions on the nuisance parameter estimators:
  • $\sqrt{n}$-consistency: The estimators converge to the true parameter value at a rate of $\sqrt{n}$, where $n$ is the sample size
  • Asymptotic normality: The estimators are asymptotically normally distributed, allowing for the construction of confidence intervals and hypothesis tests
  • Semiparametric efficiency: The estimators achieve the semiparametric efficiency bound, meaning they have the smallest possible asymptotic variance among all RAL estimators
• These asymptotic properties hold under the following conditions:
  • The nuisance parameter estimators are consistent and converge at a rate faster than $n^{-1/4}$
  • The propensity score is bounded away from 0 and 1
  • The outcome regression and propensity score models satisfy certain smoothness and complexity conditions
• The asymptotic properties of hybrid algorithms provide a strong theoretical foundation for their use in causal inference and other statistical applications

Applications of hybrid algorithms

Observational studies

• Hybrid algorithms are particularly useful in observational studies, where confounding and selection bias are common challenges
• In observational studies, the treatment assignment is not randomized and may depend on observed and unobserved confounders
• Hybrid algorithms can be used to estimate causal effects by adjusting for observed confounders through the propensity score and outcome regression models
• The double robustness property of hybrid algorithms provides protection against model misspecification, which is especially important in observational studies where the true data-generating process is unknown
• Examples of observational studies where hybrid algorithms have been applied include:
  • Estimating the effect of a job training program on earnings
  • Assessing the impact of a medical treatment on patient outcomes
  • Evaluating the effectiveness of a policy intervention on social welfare

Randomized trials with non-compliance

• Hybrid algorithms can also be used in randomized trials with non-compliance, where some participants do not adhere to their assigned treatment
• Non-compliance can bias the intention-to-treat (ITT) estimator, which compares outcomes between the treatment and control groups based on their assigned treatment
• Hybrid algorithms can be used to estimate the complier average causal effect (CACE), which is the average treatment effect among the subpopulation of compliers (i.e., those who would adhere to their assigned treatment)
• The CACE can be estimated by using the randomized treatment assignment as an instrumental variable (IV) and applying hybrid algorithms to the IV estimation problem
• Examples of randomized trials with non-compliance where hybrid algorithms have been applied include:
  • Evaluating the effectiveness of a school voucher program on student achievement
  • Assessing the impact of a medication on patient outcomes in the presence of non-adherence
  • Estimating the effect of a behavioral intervention on substance abuse, accounting for participant dropout

Longitudinal studies

• Hybrid algorithms can be extended to handle longitudinal data, where participants are followed over time and repeated measurements of the treatment, confounders, and outcomes are collected
• In longitudinal studies, time-varying confounding and selection bias due to informative censoring are common challenges
• Hybrid algorithms can be adapted to estimate causal effects in the presence of time-varying confounding by using g-computation, inverse probability weighting of marginal structural models, or targeted maximum likelihood estimation (TMLE) for longitudinal data
• These approaches involve estimating the propensity score and outcome