causalml Documentation

Uplift modeling and causal inference with machine learning algorithms

6,028 stars Python Python 23 files · ~54,377 tokens #causal-inference#incubation#machine-learning#uplift-modeling
RAW Doc

Technical Documentation: uber/causalml

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (README.md)





Disclaimer

This project is stable and being incubated for long-term support. It may contain new experimental code, for which APIs are subject to change.

Causal ML: A Python Package for Uplift Modeling and Causal Inference with ML

Causal ML is a Python package that provides a suite of uplift modeling and causal inference methods using machine learning algorithms based on recent
research [1]. It provides a standard interface that allows user to estimate the Conditional Average Treatment Effect (CATE) from experimental or observational data. Essentially, it estimates the causal impact of intervention T on outcome Y for users
with observed features X, without strong assumptions on the model form. Typical use cases include

  • Campaign targeting optimization: An important lever to increase ROI in an advertising campaign is to target the ad to the set of customers who will have a favorable response in a given KPI such as engagement or sales. CATE identifies these customers by estimating the effect of the KPI from ad exposure at the individual level from A/B experiment or historical observational data.

  • Personalized engagement: A company has multiple options to interact with its customers such as different product choices in up-sell or messaging channels for communications. One can use CATE to estimate the heterogeneous treatment effect for each customer and treatment option combination for an optimal personalized recommendation system.

Documentation

Documentation is available at https://causalml.readthedocs.io/. Good places to start:

Installation

Installation instructions are available at:

https://causalml.readthedocs.io/en/stable/installation.html

Quickstart

Quickstarts with code-snippets are available at:

https://causalml.readthedocs.io/en/stable/quickstart.html

Example Notebooks

Example notebooks are available at:

https://causalml.readthedocs.io/en/stable/examples.html

Benchmark Datasets and Leaderboard

CausalML ships loaders for the standard causal inference benchmarks (LaLonde, IHDP, Twins) with SHA256-verified downloads, plus the ground-truth metrics (PEHE, ATE error, policy risk) they enable. See the benchmark datasets page for each dataset's provenance and terms, and the leaderboard notebook, which regenerates every published number end to end.

Contributing

We welcome community contributors to the project. Before you start, please read our code of conduct and check out contributing guidelines first.

Versioning

We document versions and changes in our changelog.

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

References

Documentation

Workshops, Talks, and Publications

Citation

To cite CausalML in publications, you can refer to the following sources:

Whitepaper:
CausalML: Python Package for Causal Machine Learning

Bibtex:

@misc{chen2020causalml,
title={CausalML: Python Package for Causal Machine Learning},
author={Huigang Chen and Totte Harinen and Jeong-Yoon Lee and Mike Yung and Zhenyu Zhao},
year={2020},
eprint={2002.11631},
archivePrefix={arXiv},
primaryClass={cs.CY}
}

Literature

  1. Chen, Huigang, Totte Harinen, Jeong-Yoon Lee, Mike Yung, and Zhenyu Zhao. "Causalml: Python package for causal machine learning." arXiv preprint arXiv:2002.11631 (2020).
  2. Radcliffe, Nicholas J., and Patrick D. Surry. "Real-world uplift modelling with significance-based uplift trees." White Paper TR-2011-1, Stochastic Solutions (2011): 1-33.
  3. Zhao, Yan, Xiao Fang, and David Simchi-Levi. "Uplift modeling with multiple treatments and general response types." Proceedings of the 2017 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, 2017.
  4. Hansotia, Behram, and Brad Rukstales. "Incremental value modeling." Journal of Interactive Marketing 16.3 (2002): 35-46.
  5. Jannik Rößler, Richard Guse, and Detlef Schoder. "The Best of Two Worlds: Using Recent Advances from Uplift Modeling and Heterogeneous Treatment Effects to Optimize Targeting Policies". International Conference on Information Systems (2022)
  6. Su, Xiaogang, et al. "Subgroup analysis via recursive partitioning." Journal of Machine Learning Research 10.2 (2009).
  7. Su, Xiaogang, et al. "Facilitating score and causal inference trees for large observational studies." Journal of Machine Learning Research 13 (2012): 2955.
  8. Athey, Susan, and Guido Imbens. "Recursive partitioning for heterogeneous causal effects." Proceedings of the National Academy of Sciences 113.27 (2016): 7353-7360.
  9. Künzel, Sören R., et al. "Metalearners for estimating heterogeneous treatment effects using machine learning." Proceedings of the national academy of sciences 116.10 (2019): 4156-4165.
  10. Nie, Xinkun, and Stefan Wager. "Quasi-oracle estimation of heterogeneous treatment effects." arXiv preprint arXiv:1712.04912 (2017).
  11. Bang, Heejung, and James M. Robins. "Doubly robust estimation in missing data and causal inference models." Biometrics 61.4 (2005): 962-973.
  12. Van Der Laan, Mark J., and Daniel Rubin. "Targeted maximum likelihood learning." The international journal of biostatistics 2.1 (2006).
  13. Kennedy, Edward H. "Optimal doubly robust estimation of heterogeneous causal effects." arXiv preprint arXiv:2004.14497 (2020).
  14. Louizos, Christos, et al. "Causal effect inference with deep latent-variable models." arXiv preprint arXiv:1705.08821 (2017).
  15. Shi, Claudia, David M. Blei, and Victor Veitch. "Adapting neural networks for the estimation of treatment effects." 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019.
  16. Zhao, Zhenyu, Yumin Zhang, Totte Harinen, and Mike Yung. "Feature Selection Methods for Uplift Modeling." arXiv preprint arXiv:2005.03447 (2020).
  17. Zhao, Zhenyu, and Totte Harinen. "Uplift modeling for multiple treatments with cost optimization." In 2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 422-431. IEEE, 2019.
  • uplift: uplift models in R
  • grf: generalized random forests that include heterogeneous treatment effect estimation in R
  • rlearner: A R package that implements R-Learner
  • DoWhy: Causal inference in Python based on Judea Pearl's do-calculus
  • EconML: A Python package that implements heterogeneous treatment effect estimators from econometrics and machine learning methods

About (`docs/about.rst`)

About CausalML

CausalML is a Python package that provides a suite of uplift modeling and causal inference methods using machine learning algorithms based on recent research.
It provides a standard interface that allows user to estimate the Conditional Average Treatment Effect (CATE) from experimental or observational data.
Essentially, it estimates the causal impact of intervention W on outcome Y for users with observed features X, without strong assumptions on the model form.

GitHub Repo

https://github.com/uber/causalml

Mission

From the CausalML Charter <https://github.com/uber/causalml/blob/master/CHARTER.md>_:

text
CausalML is committed to democratizing causal machine learning through accessible, innovative, and well-documented open-source tools that empower data scientists, researchers, and organizations. At our core, we embrace inclusivity and foster a vibrant community where members exchange ideas, share knowledge, and collaboratively shape a future where CausalML drives advancements across diverse domains.

Contributing and Governance

See :doc:contributing for how to set up a development environment and open a
pull request, and for the project's charter, governance and maintainer list.
Contributors <https://github.com/uber/causalml/graphs/contributors>_ are
listed on GitHub.

Causalml (`docs/causalml.rst`)

============= API Reference

Submodules

causalml.inference.tree module

.. automodule:: causalml.inference.tree
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.inference.meta module

.. automodule:: causalml.inference.meta
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.inference.iv module

.. automodule:: causalml.inference.iv
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.inference.torch module

.. automodule:: causalml.inference.torch
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.inference.tf module

.. automodule:: causalml.inference.tf
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.inference.jax module

.. automodule:: causalml.inference.jax
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.optimize module

.. automodule:: causalml.optimize
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.dataset module

.. automodule:: causalml.dataset
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.match module

.. automodule:: causalml.match
:members:
:undoc-members:
:show-inheritance:

causalml.propensity module

.. automodule:: causalml.propensity
:members:
:undoc-members:
:show-inheritance:

causalml.metrics module

.. automodule:: causalml.metrics
:members:
:imported-members:
:undoc-members:
:show-inheritance:
:exclude-members: mae, r2_score, roc_auc_score

causalml.feature_selection module

.. automodule:: causalml.feature_selection
:members:
:imported-members:
:undoc-members:
:show-inheritance:

causalml.features module

.. automodule:: causalml.features
:members:
:undoc-members:
:show-inheritance:

Module contents

.. automodule:: causalml
:members:
:undoc-members:
:show-inheritance:

Changelog (`docs/changelog.rst`)

.. :changelog:

Changelog

You can find the latest changes in the GitHub releases <https://github.com/uber/causalml/releases>_

0.18.0 (Oct 2026)

  • Deprecates positional treatment and y ahead of the v1.0 switch to
    fit(X, y, treatment, ...) (#854). Positional order is unchanged in this release;
    see the :ref:migration guide <fit-argument-order>.
  • Causal trees and forests estimate leaves honestly by default (#584), and a single
    CausalTreeRegressor can select its size by cross-validation with ccp_alpha="cv".
  • Uplift trees and forests run on the shared Cython tree kernel (#945), with native
    missing-value support.
  • New evaluation tools: benchmark dataset loaders and ground-truth metrics,
    dr_score / plug_in_t_score / rlearner_score, bootstrap confidence intervals
    for auuc_score / qini_score, and Marginal Sensitivity Model bounds for the ATE.
  • Model save() / load() with version metadata, a JAX backend for CEVAE, and
    XGBTClassifier / XGBRClassifier.
  • A rebuilt documentation site, with an end-to-end tutorial, an estimator guide and
    an FAQ.

New Features

text
* **Benchmark dataset loaders and ground-truth metrics (v1.0 M3).** ``fetch_lalonde``,
  ``fetch_ihdp`` and ``fetch_twins`` download a benchmark from its original source on
  first use and cache it under ``~/causalml-data`` (``CAUSALML_DATA`` overrides), with
  SHA256 verification so a file that has moved or been truncated raises instead of
  being parsed as data. ``download_if_missing=False`` stays offline, and
  ``clear_data_dir()`` empties the cache. None of the datasets are redistributed with
  CausalML; :doc:`datasets` records where each comes from and on what terms, including
  the four benchmarks that are documented rather than shipped.

  ``causalml.metrics`` gains the three metrics that need a known truth: ``pehe``
  (Hill's precision in estimating heterogeneous effects, with ``squared=False`` for the
  root), ``ate_error`` for the absolute error in the average effect, and
  ``policy_risk`` for the expected loss of the treat-if-positive policy on randomized
  data. Each docstring names what it requires — the ranking metrics stay the only ones
  computable on ordinary observational data.

  ``docs/examples/benchmark_leaderboard.ipynb`` is the leaderboard itself: running it
  end to end regenerates every published number. It reports three tables rather than
  one, because what is measurable is a property of how each dataset was made.

  By @jeongyoonlee in https://github.com/uber/causalml/pull/1015

* **`CausalTreeRegressor` and `CausalRandomForestRegressor` accept** ``ccp_alpha="cv"``
  **(#584).** ``honesty=True`` supplies held-out leaf estimation; this setting adds the
  two remaining pieces of :cite:`athey2016recursive`. The splitting objective's variance penalty is scaled by
  ``1 + N_structure / N_estimation`` — the paper's factor of 2 at an even split — and tree
  size is selected by ``cv_folds``-fold cross-validation over the cost-complexity path,
  scoring each candidate subtree with that same objective evaluated on the held-out fold.

  Without it the tree grows until ``min_samples_leaf`` / ``min_group_samples`` stop it, and
  the variance penalty only ranks candidate splits instead of choosing tree size, which is
  the job it does in the paper. Held-out CATE RMSE for a single tree on a weak effect
  (16 seeds, ``min_samples_leaf=25``, paired by seed):

  .. list-table::
     :header-rows: 1

     * - Noise
       - ``honesty`` only
       - ``ccp_alpha="cv"``
       - Change
       - Leaves
     * - ``sigma=0.1``
       - 0.1212
       - 0.1192
       - -1.7% (n.s.)
       - 46 → 14
     * - ``sigma=0.5``
       - 0.2128
       - 0.1676
       - **-21.2%**
       - 46 → 13
     * - ``sigma=1.0``
       - 0.3499
       - 0.2287
       - **-34.6%**
       - 46 → 10
     * - ``sigma=2.0``
       - 0.7011
       - 0.2977
       - **-57.5%**
       - 46 → 8

  Better in 16 of 16 seeds at every noise level above 0.1, ``p < 0.0001``; at
  ``sigma=0.1`` there is no overfitting to remove and the two are equivalent. The cost is
  ``cv_folds`` extra fits per tree, roughly 5x fit time. Off by default because it changes
  fitted trees.

  This applies to a single tree. On ``CausalRandomForestRegressor`` the same option
  measured no gain: held-out CATE RMSE went from 0.084 to 0.130 at ``sigma=0.5`` (+55%,
  0 of 10 paired seeds better) and was unchanged at ``sigma=2.0``. Averaging across trees
  already removes the variance the cross-validated pruning targets, so pruning each tree to
  a few leaves adds bias and reduces ensemble diversity, at a larger cost when noise is
  lower. Prefer it on ``CausalTreeRegressor``.

  ``ccp_alpha="cv"`` requires ``honesty=True`` and raises otherwise, since the
  cross-validation scores candidate subtrees with the honest objective.

  By @jeongyoonlee in https://github.com/uber/causalml/pull/1001

* **`CausalTreeRegressor.estimate_ate` and `BaseDRIVLearner.estimate_ate` accept**
  ``pretrain`` **(#517).** ``pretrain=True`` estimates from the already-fitted model
  instead of refitting on ``X``, so the rows passed in can be held out from the fit.
  The default refits on the rows it then estimates from, which leaves the estimate
  carrying whatever the model overfit. Fitting on half a randomized draw and
  estimating on the other half gave a mean absolute ATE error of 0.099 against 0.114
  in-sample, over 20 seeds on ``CausalTreeRegressor``.

  This completes the parameter across the package: the meta-learners have had it
  since #511. ``TMLELearner.estimate_ate`` is the remaining one without it and stays
  that way, since the class has no ``fit`` and so no fitted model to reuse.

  ``BaseDRIVLearner`` requires ``p`` when ``pretrain=True``: the propensity stored by
  ``fit`` is indexed by the rows it was fit on, while the standard error indexes it by
  the rows passed here. ``assignment`` and ``pZ`` are only used to fit and go unused.

  By @jeongyoonlee in https://github.com/uber/causalml/pull/1024

* **`UpliftTreeClassifier` can prune inside** ``fit`` **(#1003).** ``prune_fraction``
  (default ``None``, off) holds out that fraction of the rows stratified on
  (treatment, outcome), grows the tree on the rest, and runs the existing ``prune()``
  on the holdout with ``min_gain`` / ``prune_rule``. Reaching validation-based pruning
  previously meant splitting the data, fitting, and calling ``prune()`` as a second
  step. ``prune()`` is unchanged for callers who manage their own holdout, and
  ``n_nodes_before_pruning_`` records the size pruning started from.

  The pruning rows are removed before the honest split, so neither the split search nor
  the estimation half sees them. Pruning renumbers nodes and promotes internal nodes to
  leaves, so the honest re-estimation and the per-node group counts are redone against
  the pruned tree.

  ``fit`` now rejects a held-out fraction outside ``(0, 1)`` — ``prune_fraction`` on the
  uplift tree and ``estimation_sample_size`` on both the uplift and causal trees —
  raising a ``ValueError`` that names the parameter instead of ``train_test_split``'s
  ``test_size``. ``prune_fraction=0.0`` was read as off.

  On ``make_uplift_classification`` (n=3000, 6 seeds, ``max_depth=None``,
  ``min_samples_leaf=20``), held-out qini went from -1.74 unpruned to -1.44 with
  ``prune_fraction=0.3``, and to 0.83 combined with ``honesty=True``. One simulated
  design, so the magnitudes are indicative.

  By @jeongyoonlee in https://github.com/uber/causalml/pull/1004

Also in this release:

* Add a model serialization API (``save()`` / ``load()`` / ``load_learner``) with version metadata to the meta-learners by @genrichez in https://github.com/uber/causalml/pull/929
* Extend serialization to the causal and uplift trees and forests, ``BaseDRIVLearner`` and ``IVRegressor`` by @genrichez in https://github.com/uber/causalml/pull/932
* Add ``dr_score()`` and ``plug_in_t_score()`` for evaluating CATE models without ground truth by @aman-coder03 in https://github.com/uber/causalml/pull/930
* Add ``rlearner_score()`` (R-loss) for CATE model evaluation, matching EconML's ``RScorer`` by @aman-coder03 in https://github.com/uber/causalml/pull/935
* Add bootstrap confidence intervals to ``auuc_score()`` and ``qini_score()`` by @ipezygj in https://github.com/uber/causalml/pull/994
* Add ``SensitivityMSM``: ATE bounds under the Marginal Sensitivity Model by @aman-coder03 in https://github.com/uber/causalml/pull/925
* Improve ``SensitivityMSM`` robustness and test coverage by @aman-coder03 in https://github.com/uber/causalml/pull/933
* Add ``return_components`` to the R-learner's ``predict()`` and ``fit_predict()`` by @aman-coder03 in https://github.com/uber/causalml/pull/923
* Add ``XGBTClassifier`` and ``XGBRClassifier`` meta-learners (#824) by @jeongyoonlee in https://github.com/uber/causalml/pull/942
* Add a JAX/flax.nnx backend for CEVAE by @xrhd in https://github.com/uber/causalml/pull/957
* Expose matched index pairs as ``NearestNeighborMatch.matched_indexes_`` (#621), reimplementing @jbbqqf's #897 by @jeongyoonlee in https://github.com/uber/causalml/pull/940
* Add node p-value display to ``plot_causal_tree`` (#805) by @genrichez in https://github.com/uber/causalml/pull/976

Tree Unification (#945)

Bug Fixes

text
* **`CausalTreeRegressor.estimate_ate` understated the standard error (#1006).** The
  interval came from ``dhat.std() / n``, which divides by ``n`` rather than ``sqrt(n)``
  and measures how the predicted effect varies across units instead of the sampling
  variability of its mean. At a nominal 95% over 100 randomized-trial draws
  (``n=1000``), the interval contained the true ATE 0 times.

  It is now the standard error of the estimate's influence function, which adds the
  model's residuals on the observed outcomes to the spread of the predicted effects —
  the quantity ``BaseTLearner.estimate_ate`` already computes as a three-term
  variance. Coverage on the same draws is 88%, the remainder being that ``estimate_ate``
  fits and estimates on the same rows (#517). Point estimates are unchanged; only the
  interval moves, and it is wider.

  By @jeongyoonlee in https://github.com/uber/causalml/pull/1007

* **`UpliftTreeClassifier.prune` left `_node_group_counts` stale (#1003).** ``prune()``
  replaced ``tree_`` without rebuilding the per-node group counts, which are indexed by
  node id and read by the uplift-score p-value and the plot's ``group_size``. The stale
  array is longer than the pruned tree, so it returned another node's counts instead of
  raising. ``prune()`` now carries the counts over to the pruned node ids, so a
  standalone call is fixed too, not only the fit-time path. A surviving node is reached
  by the same rows either way, so the values transfer unchanged; the remap replays the
  pruner's traversal and matches a full recomputation exactly.

* **`uplift_tree_plot` failed with pyparsing 3.3.3 (#1052).** Its graph was parsed by
  ``pydotplus``, unmaintained since 2014, whose parser raises ``TypeError`` on the
  HTML-style node labels the function writes once pyparsing 3.3.3 is installed. The
  dependency is now `pydot <https://github.com/pydot/pydot>`__, the maintained library
  ``pydotplus`` was forked from; the returned ``pydot.Dot`` has the same
  ``create_png()`` / ``write_png()`` methods.

  By @jeongyoonlee in https://github.com/uber/causalml/pull/1053

* **macOS without the OpenMP runtime now reports an actionable error (#908).**
  The installation guide now documents the required OpenMP runtime and
  installation options. By @su-jin1425 in https://github.com/uber/causalml/pull/1005

* Fix ``LRSRegressor.estimate_ate`` raising ``KeyError`` on DataFrame input (#1026) by @su-jin1425 in https://github.com/uber/causalml/pull/1031
* Thread ``random_state`` to ``BaseRLearner`` bootstraps (#1029) by @su-jin1425 in https://github.com/uber/causalml/pull/1032
* Accept ``RandomState`` instances as the meta-learners' ``random_state`` (#1035) by @su-jin1425 in https://github.com/uber/causalml/pull/1040
* Supply a default LightGBM outcome learner in the public ``cate_scoring`` functions (#1028) by @su-jin1425 in https://github.com/uber/causalml/pull/1030
* Give ``rlearner_score`` a default outcome learner, and raise ``ValueError`` instead of asserting in the CATE scoring functions (#1034) by @su-jin1425 in https://github.com/uber/causalml/pull/1039
* Seed and stratify ``GradientBoostedPropensityModel``'s early-stopping validation split (#1045) by @ismailvc1111 in https://github.com/uber/causalml/pull/1051
* Use a custom ``propensity_learner`` on the R-learner's first fit by @arpitjain099 in https://github.com/uber/causalml/pull/937
* Raise in ``mape`` when every target is near zero, and define ``smape``'s 0/0 term as 0 (#1047) by @arpitjain099 in https://github.com/uber/causalml/pull/1050
* Stop ``logloss`` from clipping the caller's prediction array in place by @arpitjain099 in https://github.com/uber/causalml/pull/943
* Fix NaN scores in ``FilterSelect``'s KL filter when a bin has no treated converters by @arpitjain099 in https://github.com/uber/causalml/pull/999
* Take the exact KL limits at ``pk = 0`` and ``pk = 1`` instead of clamping by @arpitjain099 in https://github.com/uber/causalml/pull/1000
* Validate a binary outcome in ``filter_LR`` / ``filter_D`` (#349) by @jeongyoonlee in https://github.com/uber/causalml/pull/941
* Fix the ``ATE`` column filled with NaN in ``SensitivitySelectionBias.summary`` by @arpitjain099 in https://github.com/uber/causalml/pull/958
* Fix a ``NearestNeighborMatch`` crash when the control pool runs below ``ratio`` by @arpitjain099 in https://github.com/uber/causalml/pull/934
* Preserve the decorated function's name, docstring and signature in ``timeit`` by @jeongyoonlee in https://github.com/uber/causalml/pull/987

Behavior Changes

  • CausalTreeRegressor and CausalRandomForestRegressor now estimate leaves honestly by
    default (#584).     The new honesty parameter defaults to True: the sample is split
    in two, the tree structure is grown on one half and each leaf's per-group outcome means
    are re-estimated on the other, so a leaf value no longer inherits the selection bias of
    the split search that produced it :cite:athey2016recursive.
    estimation_sample_size (default 0.5) sets the held-out fraction, and the split is
    stratified on treatment. Both are the names UpliftTreeClassifier already uses, and
    the default matches grf's honesty = TRUE and EconML's honest=True.

    This changes fitted models and predictions. An existing script gets a different tree,
    different leaf values and different CATE estimates without any edit. Pass
    honesty=False to keep the previous behavior.

    Measured over 200 replications on data with no treatment effect anywhere, with the tree
    structure held fixed across both arms: in-sample leaves reported a mean absolute
    estimated effect of 0.163 versus 0.095 for honest leaves, a 42% reduction in spurious
    heterogeneity. The trade is variance: each half sees only part of the data, so an honest
    tree is shallower and noisier per leaf, and at small sample sizes an individual estimate
    can be worse. Use honesty=False when halving the sample leaves too little to split
    on.

    On the forest, honesty composes with bootstrap rather than replacing it: the
    bootstrap counts arrive as each tree's sample_weight and weight both the structure
    fit and the leaf re-estimation, so this is not the subsample-without-replacement scheme
    of :cite:athey2019generalized.

    By @jeongyoonlee in https://github.com/uber/causalml/pull/1001

  • UpliftRandomForestClassifier now defaults to n_jobs=None (one worker) instead of
    n_jobs=-1 (#991). Trees are fitted in parallel and each concurrent fit holds its own
    working set, so peak memory grew in proportion to the machine's core count — the same fit
    needed more memory on a larger machine. On a 100k × 440 benchmark, peak memory was 8.8× the
    input array at n_jobs=-1 (10 cores) versus 1.4× at n_jobs=1.

    Fitted models and predictions are unchanged; only peak memory and wall time move. Existing
    code gets slower and much lighter without any edit. Pass n_jobs=-1 explicitly to restore
    the previous behavior. Note that n_estimators does not affect peak memory once n_jobs
    is fixed.

    This matches CausalRandomForestRegressor and scikit-learn's forests, which use the same
    across-trees parallelism and default to serial.

    By @jeongyoonlee in https://github.com/uber/causalml/pull/992

  • causalml.features.load_data now returns numpy.ndarray instead of
    numpy.matrix (#978). This was the only function in the package that returned a
    numpy.matrix. Code that relied on matrix semantics — * as matrix multiplication,
    or an always-2-D result from indexing — needs np.asmatrix() around the call or,
    preferably, the equivalent ndarray operation. numpy.matrix carries a
    PendingDeprecationWarning upstream.

    The same change fixes an AssertionError raised whenever the one-hot encoder produced
    no columns: with an all-numeric feature list, a constant categorical column, or a
    high-cardinality column whose levels all fall below min_obs. All three now return a
    feature matrix instead of raising (#978, #979).

    By @arpitjain099 in https://github.com/uber/causalml/pull/978 and https://github.com/uber/causalml/pull/979

  • The default propensity models produce different scores (#1027).
    ElasticNetPropensityModel and LogisticRegressionPropensityModel now select the
    regularization strength by log-loss over a wider grid: the previous grid held only
    C >= 1 and the default accuracy scoring could not tell candidates apart on
    imbalanced treatment, so on IHDP the model collapsed to the treated share. Also,
    GradientBoostedPropensityModel defaults to max_depth=3 instead of 8, which
    had fit the training data almost perfectly and pushed scores to the clip bounds. With
    early_stop=True its validation split is now seeded and stratified, so a fixed
    random_state gives the same scores on every fit. Any estimate built on a default
    propensity model moves. By @arpitjain099 in https://github.com/uber/causalml/pull/944,
    @su-jin1425 in https://github.com/uber/causalml/pull/1033 and @ismailvc1111 in
    https://github.com/uber/causalml/pull/1051

  • UpliftTreeClassifier and UpliftRandomForestClassifier run on the shared tree
    kernel (#955).     Names, arguments, defaults and the predict output shape are kept.
    fit's X_val / treatment_val / y_val and
    early_stopping_eval_diff_scale are accepted but ignored, with a UserWarning,
    because the kernel has no validation-set early stopping; for validation-based pruning,
    use prune_fraction (above). DecisionTree stays importable behind a DeprecationWarning.
    By @jeongyoonlee in https://github.com/uber/causalml/pull/973

Deprecations

text
* **Positional** ``treatment`` **and** ``y`` **are deprecated (#854).** In v1.0 every
  learner's positional argument order becomes ``fit(X, y, treatment, ...)``, matching
  the scikit-learn convention so a CausalML learner can be a ``Pipeline`` step. Passing
  ``treatment`` or ``y`` by position now emits a ``FutureWarning``; positional order is
  unchanged in this release, so nothing breaks yet.

  The fix is to pass them by keyword — ``learner.fit(X=X, treatment=treatment, y=y)`` —
  which is order-independent and therefore correct both before and after the flip.

  This one cannot be caught at runtime: ``y`` and ``treatment`` are both same-length
  arrays, so a call left in the old positional form will silently train on swapped
  arguments at v1.0 rather than raising ``TypeError``. See the
  :ref:`migration guide <fit-argument-order>` for the per-family table and for the
  signatures that are not a plain swap (``IVRegressor.fit`` and ``BaseDRIVLearner``).

  The warning is live for three minor releases — 0.18.0, 0.19.0 and 0.20.0 — before
  the flip in v1.0, roughly nine months.

  By @jeongyoonlee in https://github.com/uber/causalml/pull/975

Breaking Changes
  • BaseSLearner.estimate_ate always returns (ate, lb, ub) and no longer accepts
    return_ci (#1026). It previously returned the bare ATE unless
    return_ci=True, unlike the T-, X- and R-learners, which always return the triple.
    ate = learner.estimate_ate(...) now binds a tuple, and passing return_ci raises
    TypeError. Unpack the result instead: ate, lb, ub = learner.estimate_ate(...).
    By @su-jin1425 in https://github.com/uber/causalml/pull/1031

Documentation

text
* Switch the docs theme to pydata-sphinx-theme by @jeongyoonlee in https://github.com/uber/causalml/pull/1014
* Fix broken references and drop internal files from ``docs/`` by @jeongyoonlee in https://github.com/uber/causalml/pull/1016
* Correctness pass over the published pages, and a five-section structure by @jeongyoonlee in https://github.com/uber/causalml/pull/1018
* Take the docs build to zero warnings and fail on new ones by @jeongyoonlee in https://github.com/uber/causalml/pull/1020
* Document honest estimation and pruning in the methodology by @jeongyoonlee in https://github.com/uber/causalml/pull/1021
* Group the example notebooks by topic by @jeongyoonlee in https://github.com/uber/causalml/pull/1022
* Order the validation page as an evaluation workflow by @jeongyoonlee in https://github.com/uber/causalml/pull/1023
* Add the IHDP tutorial, choosing-an-estimator guide, FAQ and uncertainty catalog by @jeongyoonlee in https://github.com/uber/causalml/pull/1025
* Document DragonNet and CEVAE in the methodology (#417) by @jeongyoonlee in https://github.com/uber/causalml/pull/998
* Add CatBoost documentation and test coverage for the Explainer by @aman-coder03 in https://github.com/uber/causalml/pull/924
* Add the ``fit()`` argument-order migration guide (#984) by @jeongyoonlee in https://github.com/uber/causalml/pull/990
* Publish the release schedule and the #854 deprecation window by @jeongyoonlee in https://github.com/uber/causalml/pull/993
* Record the ``load_data`` return-type change in the changelog by @jeongyoonlee in https://github.com/uber/causalml/pull/996

Build / CI
~~~~~~~~~~
* Run the optional-backend tests on pull requests by @mohsinm-dev in https://github.com/uber/causalml/pull/977
* Fail the suite on causalml's own arg-order ``FutureWarning`` (#995) by @jeongyoonlee in https://github.com/uber/causalml/pull/997
* Stop the #854 shim warning on causalml's own internal ``fit`` calls by @jeongyoonlee in https://github.com/uber/causalml/pull/988
* Migrate tests and docs to keyword ``fit`` / ``predict`` calls (#982, #983) by @jeongyoonlee in https://github.com/uber/causalml/pull/989
* Filter caught warnings by category in the serialization tests by @jeongyoonlee in https://github.com/uber/causalml/pull/1002

Release Schedule

Quarterly, targeting:

.. list-table::
:header-rows: 1

    • Release
    • Target
    • Argument order
    • 0.18.0
    • October 2026
    • fit(X, treatment, y, ...) — deprecation warning added
    • 0.19.0
    • December 2026
    • unchanged, warning continues
    • 0.20.0
    • March 2027
    • unchanged, warning continues
    • 1.0.0
    • June 2027
    • fit(X, y, treatment, ...) — the flip, shim removed

Dates are targets, not commitments. The ordering is fixed: no signature moves before
v1.0.

0.17.0 (Jul 2026)

  • Adds scikit-learn 1.9 support, resolving an import causalml.dataset failure present on the 0.16.0 wheel (#926).
  • Adds native Polars DataFrame/Series/LazyFrame support across all meta-learners, and a JAX/flax.nnx backend for DragonNet.
  • Makes the meta-learners scikit-learn compliant (BaseEstimator), so clone() and get_params() work.
  • Adds the RATE evaluation metric and post-fit confidence intervals for BaseTLearner, plus numerous bug fixes.

New Features

text
* Add native Polars DataFrame, Series, and LazyFrame support for all meta-learners by @aman-coder03 in https://github.com/uber/causalml/pull/901
* Polish Polars support and add bootstrap CI test coverage by @aman-coder03 in https://github.com/uber/causalml/pull/921
* Add JAX/flax.nnx backend for DragonNet by @xrhd in https://github.com/uber/causalml/pull/918
* docs: document JAX backend for DragonNet by @xrhd in https://github.com/uber/causalml/pull/919
* Make meta-learners scikit-learn compliant via BaseEstimator by @aman-coder03 in https://github.com/uber/causalml/pull/912
* Add Rank-weighted Average Treatment Effect (RATE) metric by @aman-coder03 in https://github.com/uber/causalml/pull/887
* Add ``rate_score()`` with bootstrap confidence intervals and p-values by @aman-coder03 in https://github.com/uber/causalml/pull/890
* Add AIPW docstring warning to ``get_toc()`` and ``rate_score()`` by @jeongyoonlee in https://github.com/uber/causalml/pull/891
* Add post-fit confidence intervals to ``BaseTLearner`` via ``store_bootstraps`` and ``return_ci`` by @aman-coder03 in https://github.com/uber/causalml/pull/886
* Support NaN values in UpliftTree and UpliftRandomForest by @aman-coder03 in https://github.com/uber/causalml/pull/860

scikit-learn 1.9 Support

Bug Fixes

text
* Fix #904: Prevent deepcopy of fitted templates in bootstrap and correct predict validation ordering by @Saurav-Gupta-9741 in https://github.com/uber/causalml/pull/910
* Optimised training, inference and memory for metalearners in multitreatment settings by @Ic3fr0g in https://github.com/uber/causalml/pull/896
* Fix UpliftRandomForest predict shape mismatch with multiple treatments by @jeongyoonlee in https://github.com/uber/causalml/pull/884
* Fix uplift tree p-value NaN from division by zero by @jeongyoonlee in https://github.com/uber/causalml/pull/882
* Fix CausalRandomForestRegressor predicting inf from division by zero by @jeongyoonlee in https://github.com/uber/causalml/pull/883
* Fix SensitivityPlaceboTreatment ignoring actual treatment groups by @jeongyoonlee in https://github.com/uber/causalml/pull/880
* Fix seed parameter TypeError in BaseDRLearner bootstrap CI by @mohsinm-dev in https://github.com/uber/causalml/pull/879
* Fix ValueError on read-only arrays in BaseSLearner.predict() by @mohsinm-dev in https://github.com/uber/causalml/pull/878
* Add input validation to auuc_score for missing model columns by @jeongyoonlee in https://github.com/uber/causalml/pull/881
* Make xgboost optional in synthetic dataset generation by @Si-ra-kri in https://github.com/uber/causalml/pull/872
* Bug Fix: use iloc to index pd.Series by @bekojuniranjan in https://github.com/uber/causalml/pull/877

Build / CI

Breaking Changes

text
* **Meta-learner** ``__init__`` **signatures (#912):** to become scikit-learn ``BaseEstimator`` s, each learner now stores its constructor arguments verbatim and builds models in ``fit()``. Most visibly, ``XGBRRegressor`` no longer accepts arbitrary ``**kwargs`` — pass XGBoost parameters via the explicit ``xgb_kwargs=<dict>`` argument.

0.16.0 (Feb 2026)
-----------------
* **BREAKING CHANGE:** This release upgrades from manylinux2014 to manylinux_2_28 for Linux wheel distribution.
* Pre-built wheels now require glibc 2.28 or later (Ubuntu 20.04+, RHEL 8+, Debian 10+).
* Users on older Linux distributions (Ubuntu 18.04, RHEL 7, etc.) must build from source.
* Removes scipy version pin, enabling compatibility with both scipy 1.16.x and 1.17.x.

Updates
~~~~~~~
* Upgrade to manylinux_2_28 and remove scipy version constraints by @jeongyoonlee in https://github.com/uber/causalml/pull/869
* Upgrade cibuildwheel to v3.3.1 and remove deprecated macos-13 runner by @jeongyoonlee in https://github.com/uber/causalml/pull/867
* Fix Ubuntu packaging failure - scipy manylinux compatibility by @jeongyoonlee in https://github.com/uber/causalml/pull/865
* Fix Ubuntu packaging failure by aligning cibuildwheel config with Python version requirement by @jeongyoonlee in https://github.com/uber/causalml/pull/864

Breaking Changes
~~~~~~~~~~~~~~~~
* **Linux wheel compatibility:** Pre-built wheels require glibc 2.28+ (manylinux_2_28)

  * ✅ **Supported:** Ubuntu 20.04+, RHEL/CentOS 8+, Debian 10+, Fedora 32+
  * ⚠️ **Requires source build:** Ubuntu 18.04, RHEL 7, Ubuntu 16.04, Debian 9

* **Python version:** Minimum Python version is 3.11 (updated from 3.9)

Migration Notes
~~~~~~~~~~~~~~~
If you are on an older Linux distribution:

1. **Check your glibc version:** ``ldd --version``
2. **If glibc < 2.28:** Install from source instead of using pip wheels:

   .. code-block:: bash

       git clone https://github.com/uber/causalml.git
       cd causalml
       pip install -e .

3. **Recommended:** Upgrade to a modern Linux distribution (Ubuntu 20.04+, RHEL 8+)

0.15.1 (Apr 2024)
-----------------
* This release fixes the build failure on macOS and a few bugs in ``UpliftTreeClassifier``.
* We have two new contributors, @lee-junseok and @IanDelbridge. Thanks for your contributions!

Updates
~~~~~~~
* Relax ``pandas`` version requirement by @jeongyoonlee in https://github.com/uber/causalml/pull/743
* Remove undefined variables in ``match.__main__()`` by @jeongyoonlee in https://github.com/uber/causalml/pull/749
* Fix ``distr_plot_single_sim()`` by @jeongyoonlee in https://github.com/uber/causalml/pull/750
* Add ``with_std``, ``with_counts`` to ``create_table_one`` by @lee-junseok in https://github.com/uber/causalml/pull/748
* fix stratified sampling call by @IanDelbridge in https://github.com/uber/causalml/pull/756
* 20240207 honest leaf size by @IanDelbridge in https://github.com/uber/causalml/pull/753
* 757: add ``return_ci=True`` in sensitivity by @lee-junseok in https://github.com/uber/causalml/pull/758
* Update sensitivity tests with more meta-learners by @jeongyoonlee in https://github.com/uber/causalml/pull/759
* manually specify ``multiprocessing`` use ``fork`` in ``setup.py`` by @IanDelbridge in https://github.com/uber/causalml/pull/754

New contributors
~~~~~~~~~~~~~~~~
* @lee-junseok made their first contribution in https://github.com/uber/causalml/pull/748
* @IanDelbridge made their first contribution in https://github.com/uber/causalml/pull/756

0.15.0 (Feb 2024)
-----------------
* In this release, we revamped documentation, cleaned up dependencies, and improved installation - in addition to the long list of bug fixes.
* We have three new contributors, @peterloleungyau, @SuperBo, and @ZiJiaW, who submitted their first PRs to CausalML. @erikcs also contributed to @ras44's PR #729 to add the wrapper for his MAQ implementation to CausalML. Thanks for your contributions!

Updates
~~~~~~~
* Update python-publish.yml by @jeongyoonlee in https://github.com/uber/causalml/pull/673
* Add build.[os, tools.python] to .readthedocs.yml by @jeongyoonlee in https://github.com/uber/causalml/pull/676
* Update notebook example with causal trees interpretation by @alexander-pv in https://github.com/uber/causalml/pull/683
* Remove the numpy and pandas version restriction in pyproject.toml by @jeongyoonlee in https://github.com/uber/causalml/pull/681
* Add governance documents by @jeongyoonlee in https://github.com/uber/causalml/pull/688
* Update GOVERNANCE.md by @ras44 in https://github.com/uber/causalml/pull/691
* Dev/governance docs to snake-case by @ras44 in https://github.com/uber/causalml/pull/693
* Reduce sklearn dependency in causalml by @alexander-pv in https://github.com/uber/causalml/pull/686
* Update MAINTAINERS.md by @jeongyoonlee in https://github.com/uber/causalml/pull/696
* Modified to speed up UpliftTreeClassifier.growDecisionTreeFrom. by @peterloleungyau in https://github.com/uber/causalml/pull/695
* Update README.md by @ras44 in https://github.com/uber/causalml/pull/698
* Add notebook examples to docs by @jeongyoonlee in https://github.com/uber/causalml/pull/697
* resolves change requests in #166 by @ras44 in https://github.com/uber/causalml/pull/701
* Fix the readthedocs build error by @jeongyoonlee in https://github.com/uber/causalml/pull/702
* Replace Stack and PriorityHeap with cpp stack/heap methods in trees by @SuperBo in https://github.com/uber/causalml/pull/700
* Hotfix for #701 by @jeongyoonlee in https://github.com/uber/causalml/pull/705
* Dev/699 win build fix by @ras44 in https://github.com/uber/causalml/pull/710
* expose n_jobs for rlearner by @ZiJiaW in https://github.com/uber/causalml/pull/714
* minimal fix to resolve #707 by @ras44 in https://github.com/uber/causalml/pull/720
* Add Python 3.10, 3.11, 3.12 to the testing by @cclauss in https://github.com/uber/causalml/pull/454
* Remove Python 3.12 from the build tests in python-test.yaml by @jeongyoonlee in https://github.com/uber/causalml/pull/726
* fix plot_std_diffs, add bal_tol, condense to one plot by @ras44 in https://github.com/uber/causalml/pull/723
* Dev/677 documentation by @ras44 in https://github.com/uber/causalml/pull/725
* documentation updates by @ras44 in https://github.com/uber/causalml/pull/728
* resolves #730, docs clean conda install by @ras44 in https://github.com/uber/causalml/pull/731
* minimal wrapper of MAQ #662 by @ras44 in https://github.com/uber/causalml/pull/729
* Temporary fix for causal trees missing values support #733 by @alexander-pv in https://github.com/uber/causalml/pull/734
* resolves #639, credit due to Dong Liu by @ras44 in https://github.com/uber/causalml/pull/722

New contributors
~~~~~~~~~~~~~~~~
* @peterloleungyau made their first contribution in https://github.com/uber/causalml/pull/695
* @SuperBo made their first contribution in https://github.com/uber/causalml/pull/700
* @ZiJiaW made their first contribution in https://github.com/uber/causalml/pull/714


0.14.1 (Aug 2023)
-----------------
* This release mainly addressed installation issues and updated documentation accordingly.
* We have 4 new contributors. @bsaunders27, @xhulianoThe1, @zpppy, and @bsaunders23. Thanks for your contributions!

Updates
~~~~~~~
* Update the python-publish workflow file to fix the package publish Gi… by @jeongyoonlee in https://github.com/uber/causalml/pull/633
* Update Cython dependency by @alexander-pv in https://github.com/uber/causalml/pull/640
* Fix for builds on Mac M1 infrastructure by @bsaunders27 in https://github.com/uber/causalml/pull/641
* code cleanups by @xhulianoThe1 in https://github.com/uber/causalml/pull/634
* support valid error early stopping by @zpppy in https://github.com/uber/causalml/pull/614
* fix: update to ``envs/`` conda build for precompiled M1 installs by @bsaunders27 in https://github.com/uber/causalml/pull/646
* Installation updates to README and .github/workflows by @ras44 in https://github.com/uber/causalml/pull/637
* fix: simulate_randomized_trial by @bsaunders23 in https://github.com/uber/causalml/pull/656
* issue 252 by @vincewu51 in https://github.com/uber/causalml/pull/660
* ras44/651 graph viz, resolves #651 by @ras44 in https://github.com/uber/causalml/pull/661
* linted with black by @ras44 in https://github.com/uber/causalml/pull/663
* Fix issue 650 by @vincewu51 in https://github.com/uber/causalml/pull/659
* Install graphviz in the workflow builds by @jeongyoonlee in https://github.com/uber/causalml/pull/668
* Update docs/installation.rst by @jeongyoonlee in https://github.com/uber/causalml/pull/667
* Schedule monthly PyPI install tests by @jeongyoonlee in https://github.com/uber/causalml/pull/670

New contributors
~~~~~~~~~~~~~~~~
* @bsaunders27 made their first contribution in https://github.com/uber/causalml/pull/641
* @xhulianoThe1 made their first contribution in https://github.com/uber/causalml/pull/634
* @zpppy made their first contribution in https://github.com/uber/causalml/pull/614
* @bsaunders23 made their first contribution in https://github.com/uber/causalml/pull/656


0.14.0 (July 2023)
------------------
- CausalML surpassed `2MM downloads <https://pepy.tech/project/causalml>`_ on PyPI and `4,100 stars <https://github.com/uber/causalml/stargazers>`_ on GitHub. Thanks for choosing CausalML and supporting us on GitHub.
- We have 7 new contributors: @darthtrevino, @ras44, @AbhishekVermaDH, @joel-mcmurry, @AlxClt, @kklein, and @volico. Thanks for your contributions!

Updates
~~~~~~~
- Fix the readthedocs build failure by @jeongyoonlee in https://github.com/uber/causalml/pull/545
- Add ``pyproject.toml`` with basic build dependencies for PEP518 compliance by @darthtrevino in https://github.com/uber/causalml/pull/553
- bump ``numpy`` from 1.20.3 to 1.23.2 in ``environment-py38.yml`` #338 by @ras44 in https://github.com/uber/causalml/pull/550
- CausalTree split criterions fix and fit optimization by @alexander-pv in https://github.com/uber/causalml/pull/557
- fixing math notations for proper rendering by @AbhishekVermaDH in https://github.com/uber/causalml/pull/558
- Update ``methodology.rst`` by @joel-mcmurry in https://github.com/uber/causalml/pull/568
- Causal trees bootstrapping and ``max_leaf_nodes`` fixes with minor update by @alexander-pv in https://github.com/uber/causalml/pull/583
- Fix #596 by @AlxClt in https://github.com/uber/causalml/pull/597
- Add ``**kwargs`` to ``Explainer.plot_shap_values()`` by @jeongyoonlee in https://github.com/uber/causalml/pull/603
- Make the Adam optimization optional and learning rate/epochs configurable in DragonNet by @jeongyoonlee in https://github.com/uber/causalml/pull/604
- Fix bug in variance calculation in drivlearner. by @huigangchen in https://github.com/uber/causalml/pull/606
- Bug Fix in Dragonnet: Adam parameter name lr depreciation by @huigangchen in https://github.com/uber/causalml/pull/617
- Fix AttributeError in builds with ``numpy>=1.24`` and ``pandas>=2.0`` by @jeongyoonlee in https://github.com/uber/causalml/pull/631
- Pass on ``**kwargs`` in ``plot_shap_values`` of base meta leaner by @kklein in https://github.com/uber/causalml/pull/627
- Bump ``scipy`` from 1.4.1 to 1.10.0 by @dependabot in https://github.com/uber/causalml/pull/629
- Feature/ttest criterion by @volico in https://github.com/uber/causalml/pull/570
- Added Interaction Tree (IT), Causal Inference Tree (CIT), and Invariant DDP (IDDP) by @jroessler in https://github.com/uber/causalml/pull/562
- Causal trees option to return counterfactual outcomes by @alexander-pv in https://github.com/uber/causalml/pull/623

New contributors
~~~~~~~~~~~~~~~~
- @darthtrevino made their first contribution in https://github.com/uber/causalml/pull/553
- @ras44 made their first contribution in https://github.com/uber/causalml/pull/550
- @AbhishekVermaDH made their first contribution in https://github.com/uber/causalml/pull/558
- @joel-mcmurry made their first contribution in https://github.com/uber/causalml/pull/568
- @AlxClt made their first contribution in https://github.com/uber/causalml/pull/597
- @kklein made their first contribution in https://github.com/uber/causalml/pull/627
- @volico made their first contribution in https://github.com/uber/causalml/pull/570


0.13.0 (Sep 2022)
-----------------
- CausalML surpassed `1MM downloads <https://pepy.tech/project/causalml>`_ on PyPI and `3,200 stars <https://github.com/uber/causalml/stargazers>`_ on GitHub. Thanks for choosing CausalML and supporting us on GitHub.
- We have 7 new contributors @saiwing-yeung, @lixuan12315, @aldenrogers, @vincewu51, @AlkanSte, @enzoliao, and @alexander-pv. Thanks for your contributions!
- @alexander-pv revamped `CausalTreeRegressor` and added `CausalRandomForestRegressor` with more seamless integration with `scikit-learn`'s Cython tree module. He also added integration with `shap` for causal tree/ random forest interpretation. Please check out the `example notebook <https://github.com/uber/causalml/blob/master/docs/examples/causal_trees_interpretation.ipynb>`_.
- We dropped the support for Python 3.6 and removed its test workflow.

Updates
~~~~~~~
- Fix typo ``(% -> $)`` by @saiwing-yeung in https://github.com/uber/causalml/pull/488
- Add function for calculating PNS bounds by @t-tte in https://github.com/uber/causalml/pull/482
- Fix hard coding bug by @t-tte in https://github.com/uber/causalml/pull/492
- Update README of ``conda`` install and instruction of maintain in ``conda-forge`` by @ppstacy in https://github.com/uber/causalml/pull/485
- Update ``examples.rst`` by @lixuan12315 in https://github.com/uber/causalml/pull/496
- Fix incorrect ``effect_learner_objective`` in ``XGBRRegressor`` by @jeongyoonlee in https://github.com/uber/causalml/pull/504
- Fix Filter F doesn't work with latest ``statsmodels``' F test f-value format by @paullo0106 in https://github.com/uber/causalml/pull/505
- Exclude tests in ``setup.py`` by @aldenrogers in https://github.com/uber/causalml/pull/508
- Enabling higher orders feature importance for F filter and LR filter by @zhenyuz0500 in https://github.com/uber/causalml/pull/509
- Ate pretrain 0506 by @vincewu51 in https://github.com/uber/causalml/pull/511
- Update ``methodology.rst`` by @AlkanSte in https://github.com/uber/causalml/pull/518
- Fix the bug of incorrect result in qini for multiple models by @enzoliao in https://github.com/uber/causalml/pull/520
- Test ``get_qini()`` by @enzoliao in https://github.com/uber/causalml/pull/523
- Fixed typo in ``uplift_trees_with_synthetic_data.ipynb`` by @jroessler in https://github.com/uber/causalml/pull/531
- Remove Python 3.6 test from workflows by @jeongyoonlee in https://github.com/uber/causalml/pull/535
- Causal trees update by @alexander-pv in https://github.com/uber/causalml/pull/522
- Causal trees interpretation example by @alexander-pv in https://github.com/uber/causalml/pull/536


0.12.3 (Feb 2022)
-----------------
This patch is to release a version without the constraint for Shap to be abled to use for Conda.

Updates
~~~~~~~
- `#483 <https://github.com/uber/causalml/pull/483>`_ by @ppstacy: Modify the requirement version of Shap


0.12.2 (Feb 2022)
-----------------
This patch includes three updates by @tonkolviktor and @heiderich as follows. We also start using `black <https://black.readthedocs.io/en/stable/integrations/index.html>`_, a Python formatter. Please check out the updated `contribution guideline <https://github.com/uber/causalml/blob/master/CONTRIBUTING.md>`_ to learn how to use it.

Updates
~~~~~~~
- `#473 <https://github.com/uber/causalml/pull/477>`_ by @tonkolviktor: Open up the scipy dependency version
- `#476 <https://github.com/uber/causalml/pull/476>`_ by @heiderich: Use preferred backend for joblib instead of hard-coding it
- `#477 <https://github.com/uber/causalml/pull/477>`_ by @heiderich: Allow parallel prediction for UpliftRandomForestClassifier and make the joblib's preferred backend configurable


0.12.1 (Feb 2022)
-----------------
This patch includes two bug fixes for UpliftRandomForestClassifier as follows:

Updates
~~~~~~~
- `#462 <https://github.com/uber/causalml/pull/462>`_ by @paullo0106: Use the correct treatment_idx for fillTree() when applying validation data set
- `#468 <https://github.com/uber/causalml/pull/468>`_ by @jeongyoonlee: Switch the joblib backend for UpliftRandomForestClassifier to threading to avoid memory copy across trees


0.12.0 (Jan 2022)
-----------------
- CausalML surpassed `637K downloads <https://pepy.tech/project/causalml>`_ on PyPI and `2,500 stars <https://github.com/uber/causalml/stargazers>`_ on Github!
- We have 4 new community contributors, Luis (`@lgmoneda <https://github.com/lgmoneda>`_), Ravi (`@raviksharma <https://github.com/raviksharma>`_), Louis (`@LouisHernandez17 <https://github.com/LouisHernandez17>`_) and JackRab (`@JackRab <https://github.com/JackRab>`_). Thanks for the contribution!
- We refactored and speeded up UpliftTreeClassifier/UpliftRandomForestClassifier by 5x with Cython  (`#422 <https://github.com/uber/causalml/pull/422>`_ `#440 <https://github.com/uber/causalml/pull/440>`_ by @jeongyoonlee)
- We revamped our `API documentation <https://causalml.readthedocs.io/en/latest/about.html>`_, it now includes the latest methodology, references, installation, notebook examples, and graphs! (`#413 <https://github.com/uber/causalml/discussions/413>`_ by @huigangchen @t-tte @zhenyuz0500 @jeongyoonlee @paullo0106)
- Our team gave talks at `2021 Conference on Digital Experimentation @ MIT (CODE@MIT) <https://ide.mit.edu/events/2021-conference-on-digital-experimentation-mit-codemit/>`_, `Causal Data Science Meeting 2021 <https://www.causalscience.org/meeting/program/day-2/>`_,  and `KDD 2021 Tutorials <https://causal-machine-learning.github.io/kdd2021-tutorial/>`_ on CausalML introduction and applications. Please take a look if you missed them! Full list of publications and talks can be found here.

Updates
~~~~~~~
- Update documentation on Instrument Variable methods @huigangchen (`#447 <https://github.com/uber/causalml/pull/447>`_)
- Add benchmark simulation studies example notebook by @t-tte (`#443 <https://github.com/uber/causalml/pull/443>`_)
- Add sample_weight support for R-learner by @paullo0106 (`#425 <https://github.com/uber/causalml/pull/425>`_)
- Fix incorrect binning of numeric features in UpliftTreeClassifier by @jeongyoonlee (`#420 <https://github.com/uber/causalml/pull/420>`_)
- Update papers, talks, and publication info to README and refs.bib by @zhenyuz0500 (`#410 <https://github.com/uber/causalml/pull/410>`_ `#414 <https://github.com/uber/causalml/pull/414>`_ `#433 <https://github.com/uber/causalml/pull/433>`_)
- Add instruction for contributing.md doc by @jeongyoonlee (`#408 <https://github.com/uber/causalml/pull/408>`_)
- Fix incorrect feature importance calculation logic by @paullo0106 (`#406 <https://github.com/uber/causalml/pull/406>`_)
- Add parallel jobs support for NearestNeighbors search with n_jobs parameter by @paullo0106 (`#389 <https://github.com/uber/causalml/pull/389>`_)
- Fix bug in simulate_randomized_trial by @jroessler (`#385 <https://github.com/uber/causalml/pull/385>`_)
- Add GA pytest workflow by @ppstacy (`#380 <https://github.com/uber/causalml/pull/380>`_)



0.11.0 (2021-07-28)
-------------------
- CausalML surpassed `2K stars <https://github.com/uber/causalml/stargazers>`_!
- We have 3 new community contributors, Jannik (`@jroessler <https://github.com/jroessler>`_), Mohamed (`@ibraaaa <https://github.com/ibraaaa>`_), and Leo (`@lleiou <https://github.com/lleiou>`_). Thanks for the contribution!

Major Updates
~~~~~~~~~~~~~
- Make tensorflow dependency optional and add python 3.9 support by @jeongyoonlee (`#343 <https://github.com/uber/causalml/pull/343>`_)
- Add delta-delta-p (ddp) tree inference approach by @jroessler (`#327 <https://github.com/uber/causalml/pull/327>`_)
- Add conda env files for Python 3.6, 3.7, and 3.8 by @jeongyoonlee (`#324 <https://github.com/uber/causalml/pull/324>`_)

Minor Updates
~~~~~~~~~~~~~
- Fix inconsistent feature importance calculation in uplift tree by @paullo0106 (`#372 <https://github.com/uber/causalml/pull/372>`_)
- Fix filter method failure with NaNs in the data issue by @manojbalaji1 (`#367 <https://github.com/uber/causalml/pull/367>`_)
- Add automatic package publish by @jeongyoonlee (`#354 <https://github.com/uber/causalml/pull/354>`_)
- Fix typo in unit_selection optimization by @jeongyoonlee (`#347 <https://github.com/uber/causalml/pull/347>`_)
- Fix docs build failure by @jeongyoonlee (`#335 <https://github.com/uber/causalml/pull/335>`_)
- Convert pandas inputs to numpy in S/T/R Learners by @jeongyoonlee (`#333 <https://github.com/uber/causalml/pull/333>`_)
- Require scikit-learn as a dependency of setup.py by @ibraaaa (`#325 <https://github.com/uber/causalml/pull/325>`_)
- Fix AttributeError when passing in Outcome and Effect learner to R-Learner by @paullo0106 (`#320 <https://github.com/uber/causalml/pull/320>`_)
- Fix error when there is no positive class for KL Divergence filter by @lleiou (`#311 <https://github.com/uber/causalml/pull/311>`_)
- Add versions to cython and numpy in setup.py for requirements.txt accordingly by @maccam912 (`#306 <https://github.com/uber/causalml/pull/306>`_)



0.10.0 (2021-02-18)
-------------------
- CausalML surpassed `235,000 downloads <https://pepy.tech/project/causalml>`_!
- We have 5 new community contributors, Suraj (`@surajiyer <https://github.com/surajiyer>`_), Harsh (`@HarshCasper <https://github.com/HarshCasper>`_), Manoj (`@manojbalaji1 <https://github.com/manojbalaji1>`_), Matthew (`@maccam912 <https://github.com/maccam912>`_) and Václav (`@vaclavbelak <https://github.com/vaclavbelak>`_). Thanks for the contribution!

Major Updates
~~~~~~~~~~~~~
- Add Policy learner, DR learner, DRIV learner by @huigangchen (`#292 <https://github.com/uber/causalml/pull/292>`_)
- Add wrapper for CEVAE, a deep latent-variable and variational autoencoder based model by @ppstacy(`#276 <https://github.com/uber/causalml/pull/276>`_)

Minor Updates
~~~~~~~~~~~~~
- Add propensity_learner to R-learner by @jeongyoonlee (`#297 <https://github.com/uber/causalml/pull/297>`_)
- Add BaseLearner class for other meta-learners to inherit from without duplicated code by @jeongyoonlee (`#295 <https://github.com/uber/causalml/pull/295>`_)
- Fix installation issue for Shap>=0.38.1 by @paullo0106 (`#287 <https://github.com/uber/causalml/pull/287>`_)
- Fix import error for sklearn>= 0.24 by @jeongyoonlee (`#283 <https://github.com/uber/causalml/pull/283>`_)
- Fix KeyError issue in Filter method for certain dataset by @surajiyer (`#281 <https://github.com/uber/causalml/pull/281>`_)
- Fix inconsistent cumlift score calculation of multiple models by @vaclavbelak (`#273 <https://github.com/uber/causalml/pull/273>`_)
- Fix duplicate values handling in feature selection method by @manojbalaji1 (`#271 <https://github.com/uber/causalml/pull/271>`_)
- Fix the color spectrum of SHAP summary plot  for feature interpretations of meta-learners by @paullo0106 (`#269 <https://github.com/uber/causalml/pull/269>`_)
- Add IIA and value optimization related documentation by @t-tte (`#264 <https://github.com/uber/causalml/pull/264>`_)
- Fix StratifiedKFold arguments for propensity score estimation by @paullo0106 (`#262 <https://github.com/uber/causalml/pull/262>`_)
- Refactor the code with string format argument and is to compare object types, and change methods not using bound instance to static methods by @harshcasper (`#256 <https://github.com/uber/causalml/pull/256>`_, `#260 <https://github.com/uber/causalml/pull/260>`_)



0.9.0 (2020-10-23)
------------------
- CausalML won the 1st prize at the poster session in UberML'20
- DoWhy integrated CausalML starting v0.4 (`release note <https://github.com/microsoft/dowhy/releases/tag/v0.4>`_)
- CausalML team welcomes new project leadership, Mert Bay
- We have 4 new community contributors, Mario Wijaya (`@mwijaya3 <https://github.com/mwijaya3>`_), Harry Zhao (`@deeplaunch <https://github.com/deeplaunch>`_), Christophe (`@ccrndn <https://github.com/ccrndn>`_) and Georg Walther (`@waltherg <https://github.com/waltherg>`_). Thanks for the contribution!

Major Updates
~~~~~~~~~~~~~
- Add feature importance and its visualization to UpliftDecisionTrees and UpliftRF by @yungmsh (`#220 <https://github.com/uber/causalml/pull/220>`_)
- Add feature selection example with Filter methods by @paullo0106 (`#223 <https://github.com/uber/causalml/pull/223>`_)

Minor Updates
~~~~~~~~~~~~~
- Implement propensity model abstraction for common interface by @waltherg (`#223 <https://github.com/uber/causalml/pull/223>`_)
- Fix bug in BaseSClassifier and BaseXClassifier by @yungmsh and @ppstacy (`#217 <https://github.com/uber/causalml/pull/217>`_), (`#218 <https://github.com/uber/causalml/pull/218>`_)
- Fix parentNodeSummary for UpliftDecisionTrees by @paullo0106 (`#238 <https://github.com/uber/causalml/pull/238>`_)
- Add pd.Series for propensity score condition check by @paullo0106 (`#242 <https://github.com/uber/causalml/pull/242>`_)
- Fix the uplift random forest prediction output by @ppstacy (`#236 <https://github.com/uber/causalml/pull/236>`_)
- Add functions and methods to init for optimization module by @mwijaya3 (`#228 <https://github.com/uber/causalml/pull/228>`_)
- Install GitHub Stale App to close inactive issues automatically @jeongyoonlee (`#237 <https://github.com/uber/causalml/pull/237>`_)
- Update documentation by @deeplaunch, @ccrndn, @ppstacy(`#214 <https://github.com/uber/causalml/pull/214>`_, `#231 <https://github.com/uber/causalml/pull/231>`_, `#232 <https://github.com/uber/causalml/pull/232>`_)



0.8.0 (2020-07-17)
------------------
CausalML surpassed `100,000 downloads <https://pepy.tech/project/causalml>`_! Thanks for the support.

Major Updates
~~~~~~~~~~~~~
- Add value optimization to `optimize` by @t-tte (`#183 <https://github.com/uber/causalml/pull/183>`_)
- Add counterfactual unit selection to `optimize` by @t-tte (`#184 <https://github.com/uber/causalml/pull/184>`_)
- Add sensitivity analysis to `metrics` by @ppstacy (`#199 <https://github.com/uber/causalml/pull/199>`_, `#212 <https://github.com/uber/causalml/pull/212>`_)
- Add the `iv` estimator submodule and add 2SLS model to it by @huigangchen (`#201 <https://github.com/uber/causalml/pull/201>`_)

Minor Updates
~~~~~~~~~~~~~
- Add `GradientBoostedPropensityModel` by @yungmsh (`#193 <https://github.com/uber/causalml/pull/193>`_)
- Add covariate balance visualization by @yluogit (`#200 <https://github.com/uber/causalml/pull/200>`_)
- Fix bug in the X learner propensity model by @ppstacy (`#209 <https://github.com/uber/causalml/pull/209>`_)
- Update package dependencies by @jeongyoonlee (`#195 <https://github.com/uber/causalml/pull/195>`_, `#197 <https://github.com/uber/causalml/pull/197>`_)
- Update documentation by @jeongyoonlee, @ppstacy and @yluogit (`#181 <https://github.com/uber/causalml/pull/181>`_, `#202 <https://github.com/uber/causalml/pull/202>`_, `#205 <https://github.com/uber/causalml/pull/205>`_)



0.7.1 (2020-05-07)
------------------
Special thanks to our new community contributor, Katherine (`@khof312 <https://github.com/khof312>`_)!

Major Updates
~~~~~~~~~~~~~
- Adjust matching distances by a factor of the number of matching columns in propensity score matching by @yungmsh (`#157 <https://github.com/uber/causalml/pull/157>`_)
- Add TMLE-based AUUC/Qini/lift calculation and plotting by @ppstacy (`#165 <https://github.com/uber/causalml/pull/165>`_)

Minor Updates
~~~~~~~~~~~~~
- Fix typos and update documents by @paullo0106, @khof312, @jeongyoonlee (`#150 <https://github.com/uber/causalml/pull/150>`_, `#151 <https://github.com/uber/causalml/pull/151>`_, `#155 <https://github.com/uber/causalml/pull/155>`_, `#163 <https://github.com/uber/causalml/pull/163>`_)
- Fix error in `UpliftTreeClassifier.kl_divergence()` for `pk == 1 or 0` by @jeongyoonlee (`#169 <https://github.com/uber/causalml/pull/169>`_)
- Fix error in `BaseRRegressor.fit()` without propensity score input by @jeongyoonlee (`#170 <https://github.com/uber/causalml/pull/170>`_)


0.7.0 (2020-02-28)
------------------
Special thanks to our new community contributor, Steve (`@steveyang90 <https://github.com/steveyang90>`_)!

Major Updates
~~~~~~~~~~~~~
- Add a new `nn` inference submodule with `DragonNet` implementation by @yungmsh
- Add a new `feature selection` submodule with filter feature selection methods by @zhenyuz0500

Minor Updates
~~~~~~~~~~~~~
- Make propensity scores optional in all meta-learners by @ppstacy
- Replace `eli5` permutation importance with `sklearn`'s by @yluogit
- Replace `ElasticNetCV` with `LogisticRegressionCV` in `propensity.py` by @yungmsh
- Fix the normalized uplift curve plot with negative ATE by @jeongyoonlee
- Fix the TravisCI FOSSA error for PRs from forked repo by @steveyang90
- Add documentation about tree visualization by @zhenyuz0500

0.6.0 (2019-12-31)
------------------
Special thanks to our new community contributors, Fritz (`@fritzo <https://github.com/fritzo>`_), Peter (`@peterfoley <https://github.com/peterfoley>`_) and Tomasz (`@TomaszZamacinski <https://github.com/TomaszZamacinski>`_)!

- Improve `UpliftTreeClassifier`'s speed by 4 times by @jeongyoonlee
- Fix impurity computation in `CausalTreeRegressor` by @TomaszZamacinski
- Fix XGBoost related warnings by @peterfoley
- Fix typos and improve documentation by @peterfoley and @fritzo

0.5.0 (2019-11-26)
------------------
Special thanks to our new community contributors, Paul (`@paullo0106 <https://github.com/paullo0106>`_) and Florian (`@FlorianWilhelm <https://github.com/FlorianWilhelm>`_)!

- Add `TMLELearner`, targeted maximum likelihood estimator to `inference.meta` by @huigangchen
- Add an option to DGPs for regression to simulate imbalanced propensity distribution by @huigangchen
- Fix incorrect edge connections, and add more information in the uplift tree plot by @paullo0106
- Fix an installation error related to `Cython` and `numpy` by @FlorianWilhelm
- Drop Python 2 support from `setup.py` by @jeongyoonlee
- Update `causaltree.pyx` Cython code to be compatible with `scikit-learn>=0.21.0` by @jeongyoonlee

0.4.0 (2019-10-21)
------------------

- Add `uplift_tree_plot()` to `inference.tree` to visualize `UpliftTreeClassifier` by @zhenyuz0500
- Add the `Explainer` class to `inference.meta` to provide feature importances using `SHAP` and `eli5`'s `PermutationImportance` by @yungmsh
- Add bootstrap confidence intervals for the average treatment effect estimates of meta learners by @ppstacy

0.3.0 (2019-09-17)
------------------

- Extend meta-learners to support classification by @t-tte
- Extend meta-learners to support multiple treatments by @yungmsh
- Fix a bug in uplift curves and add Qini curves/scores to `metrics` by @jeongyoonlee
- Add `inference.meta.XGBRRegressor` with early stopping and ranking optimization by @yluogit

0.2.0 (2019-08-12)
------------------

- Add `optimize.PolicyLearner` based on Athey and Wager 2017 :cite:`athey2017efficient`
- Add the `CausalTreeRegressor` estimator based on Athey and Imbens 2016 :cite:`athey2016recursive` (experimental)
- Add missing imports in `features.py` to enable label encoding with grouping of rare values in `LabelEncoder()`
- Fix a bug that caused the mismatch between training and prediction features in `inference.meta.tlearner.predict()`

0.1.0 (unreleased)
------------------

- Initial release with the Uplift Random Forest, and S/T/X/R-learners.

---

### Choosing An Estimator (`docs/choosing_an_estimator.rst`)

=====================
Choosing an Estimator
=====================

CausalML implements many estimators because no single one dominates: they
differ in the outcome and treatment types they accept, the data they are
designed for, and what they report. This page maps a problem to a shortlist.
The mathematics of each method lives in the :doc:`methodology`; the API details
live in the :doc:`API Reference <causalml>`.

Start from the data
===================

**Was the treatment randomized?**

* **Yes, and compliance was perfect.** Any estimator below applies, and the
  assignment probability is known -- pass it as ``p`` instead of estimating it.
  For a binary conversion outcome where the goal is targeting segments or
  interpretable rules, start with the uplift trees. For per-unit CATE
  estimates with confidence intervals, start with the meta-learners.

* **Yes, but some units did not comply.** The randomized assignment is an
  instrument for the treatment actually received. Use the
  :ref:`DRIV learner <methodology:Doubly Robust Instrumental Variable (DRIV) learner>`
  (``BaseDRIVLearner``) to estimate the effect on compliers, or
  :ref:`2SLS <methodology:2-Stage Least Squares (2SLS)>` for a linear model.

* **No -- the data are observational.** Estimation requires that every
  confounder (a variable driving both treatment and outcome) is measured, and
  that treated and untreated units overlap (see
  :ref:`Checking Overlap <validation:Checking Overlap>`). Prefer the
  estimators that model the treatment assignment explicitly: the
  :ref:`X-Learner <methodology:X-Learner>`,
  :ref:`R-Learner <methodology:R-Learner>` and
  :ref:`DR learner <methodology:Doubly Robust (DR) learner>` all accept a
  propensity score ``p`` and estimate one internally when it is omitted.
  Validate with :doc:`sensitivity analysis <validation>` afterwards.

* **No, and an important confounder is unmeasured.** With an instrument, use
  the IV estimators above. With proxy variables for the hidden confounder,
  :ref:`CEVAE <methodology:CEVAE>` models it as a latent variable. Without
  either, no estimator in this package (or any other) identifies the effect.

**What do you need out of the model?**

* **Only the average effect (ATE)** -- :ref:`TMLE
  <methodology:Targeted maximum likelihood estimation (TMLE) for ATE>`,
  :ref:`IPTW <methodology:Inverse probability of treatment weighting>`,
  :ref:`matching <methodology:Matching>`, or any meta-learner's
  ``estimate_ate()``.
* **Per-unit effects (CATE)** -- meta-learners, causal trees and forests, or
  the neural models.
* **Segments and rules you can read** -- uplift trees, with
  :ref:`visualization <interpretation:Uplift Tree Visualization>`.
* **Who to treat under constraints** -- estimate CATE first, then use
  ``PolicyLearner`` or the
  :ref:`value optimization methods <methodology:Value optimization methods>`.

Capability matrix
=================

.. list-table::
   :header-rows: 1
   :widths: 26 17 15 16 16 10

   * - Estimator (classes)
     - Outcome type
     - Treatment
     - Observational data
     - Uncertainty
     - Extra install
   * - S/T/X/R meta-learners (``BaseSRegressor`` ... ``BaseRClassifier``)
     - continuous (``*Regressor``) or binary (``*Classifier``)
     - binary or multiple discrete
     - yes; X/R use a propensity score
     - ATE CI; bootstrap CATE CI
     - --
   * - DR learner (``BaseDRRegressor``/``BaseDRClassifier``)
     - continuous or binary
     - binary or multiple discrete
     - yes, doubly robust
     - ATE CI; bootstrap CATE CI
     - --
   * - DRIV learner (``BaseDRIVLearner``)
     - continuous
     - binary, with an instrument
     - yes, given an instrument
     - ATE CI; bootstrap CATE CI
     - --
   * - Uplift trees (``UpliftTreeClassifier``, ``UpliftRandomForestClassifier``)
     - binary or multi-class
     - binary or multiple discrete
     - designed for randomized data
     - --
     - --
   * - Causal trees (``CausalTreeRegressor``, ``CausalRandomForestRegressor``)
     - continuous
     - binary
     - yes
     - ATE CI (tree); per-prediction variance (forest)
     - --
   * - ``DragonNet``
     - continuous or binary
     - binary
     - yes
     - --
     - ``tf`` or ``jax``
   * - ``CEVAE``
     - continuous or binary
     - binary
     - yes, with proxies for a hidden confounder
     - --
     - ``torch`` or ``jax``
   * - 2SLS (``IVRegressor``)
     - continuous
     - continuous or binary, with an instrument
     - yes, given an instrument
     - coefficient SE
     - --
   * - ``TMLELearner`` (ATE only)
     - continuous
     - binary
     - yes, with a propensity score
     - ATE CI
     - --

How the uncertainty is computed, per estimator, is cataloged in
:doc:`inference`.

Two rules of thumb
==================

* **Start simple, then justify complexity.** A T-learner with a linear base
  learner is a transparent baseline; adopt a more flexible estimator when
  held-out evaluation (:doc:`validation`) shows it ranks units better.
* **Do not choose by in-sample fit.** CATE models cannot be scored against an
  observed label. Compare candidates with the validation losses and ranking
  metrics on held-out data, as described in
  :ref:`Model Selection with Validation Losses <validation:Model Selection with Validation Losses>`.

---

### Contributing (`docs/contributing.rst`)

============
Contributing
============

CausalML is developed in the open at `github.com/uber/causalml
<https://github.com/uber/causalml>`_, and contributions are welcome — bug
reports, documentation, new estimators, and reviews of open pull requests.

Setting up a development environment
------------------------------------

Follow :ref:`Install from source <installation:Install from source>`. The
editable install compiles the package's Cython extensions, so no separate build
step is needed. Then run the test suite as described in
:ref:`Running Tests <installation:Running Tests>`.

After editing any ``.pyx`` or ``.pxd`` file, reinstall so the extension is
rebuilt::

    pip install -e . --no-deps

Before opening a pull request
-----------------------------

* Format the diff with `black <https://black.readthedocs.io/>`_. It is the
  formatting gate CI enforces.
* Add or update tests covering the behavior you changed.
* Add an entry under ``Unreleased`` in :doc:`changelog` if the change is
  user-visible — a new feature, a bug fix, or a change in behavior.
* If you change an estimator's signature, check
  :doc:`migration` — the ``fit(X, y, treatment, ...)`` transition is in
  progress and new methods are expected to follow the target order.

Project documents
-----------------

* `Contributing guide <https://github.com/uber/causalml/blob/master/CONTRIBUTING.md>`_
* `Code of conduct <https://github.com/uber/causalml/blob/master/CODE_OF_CONDUCT.md>`_
* `Charter <https://github.com/uber/causalml/blob/master/CHARTER.md>`_
* `Governance <https://github.com/uber/causalml/blob/master/GOVERNANCE.md>`_
* `Maintainers <https://github.com/uber/causalml/blob/master/MAINTAINERS.md>`_
* `Steering committee <https://github.com/uber/causalml/blob/master/STEERING_COMMITTEE.md>`_
* `Security policy <https://github.com/uber/causalml/blob/master/SECURITY.md>`_

---

### Datasets (`docs/datasets.rst`)

==================
Benchmark Datasets
==================

CausalML ships loaders for three standard causal-inference benchmarks. Each is
downloaded from its original source the first time it is used and cached on disk;
none is redistributed with the package.

.. code-block:: python

    from causalml.dataset import fetch_lalonde, fetch_ihdp, fetch_twins

    lalonde = fetch_lalonde()                       # Bunch
    X, y, treatment = fetch_ihdp(replication=0, return_X_y_t=True)

Files are cached in ``~/causalml-data``, overridable with the ``CAUSALML_DATA``
environment variable or the ``data_home`` argument. Every download is verified
against a SHA256 digest, so a file that has been replaced or truncated raises
instead of being parsed as data. ``download_if_missing=False`` refuses to reach
the network, and ``causalml.dataset.clear_data_dir()`` empties the cache.

Which metric applies to which dataset
=====================================

What can be measured is a property of how the data was made, not a choice:

.. list-table::
   :header-rows: 1

   * - Dataset
     - Ground truth
     - Metrics that apply
   * - ``synthetic_data``, ``make_uplift_classification``
     - simulated per-unit ``tau``
     - :func:`~causalml.metrics.pehe`, :func:`~causalml.metrics.ate_error`
   * - IHDP
     - simulated ``mu0`` / ``mu1``
     - :func:`~causalml.metrics.pehe`, :func:`~causalml.metrics.ate_error`
   * - Twins
     - both twins observed
     - :func:`~causalml.metrics.pehe`, :func:`~causalml.metrics.ate_error`
   * - LaLonde
     - experimental ATE only
     - :func:`~causalml.metrics.ate_error` against that estimate
   * - Criteo, Hillstrom
     - none
     - AUUC, Qini, RATE

PEHE is only computable where the counterfactual was constructed. A ranking of
estimators on simulated outcomes is a statement about that simulation.

Datasets with a loader
======================

LaLonde (National Supported Work)
---------------------------------

A randomized job-training experiment: 445 rows, 185 treated, outcome ``re78``
(1978 earnings in dollars). Its experimental difference in means, about $1,794,
is the yardstick observational estimators are judged against
:cite:`lalonde1986evaluating`. The sample is the Dehejia-Wahba one
:cite:`dehejia1999causal`.

Source: the `causaldata <https://pypi.org/project/causaldata/>`_ package (MIT),
read at a pinned revision.

IHDP
----

Covariates from the Infant Health and Development Program randomized trial with
outcomes simulated on response surface B :cite:`hill2011bayesian`. The file holds
100 replications of the same 747 units. Each replication simulates its own
outcomes **and draws its own 672 / 75 train-test split**, so row ``i`` is a
different unit in each one and the treated count varies with it.

Results on IHDP are reported as a mean and standard error across replications::

    scores = [pehe(fetch_ihdp(replication=r).tau, predict(r)) for r in range(100)]

A single replication is not comparable to a published IHDP number. Neither,
exactly, is a mean over this file: the same release also exists in a
1,000-replication version, and that is what the published tables of the CEVAE
:cite:`louizos2017causal` and DragonNet :cite:`shi2019adapting` papers
aggregate over (with the 672 further split 63/27 into train and validation).
The data-generating process and split geometry match; the replication count
does not.

Source: the `clinicalml/cfrnet <https://github.com/clinicalml/cfrnet>`_ lineage
(MIT), files hosted at ``fredjo.com``.

Twins
-----

Same-sex twin births from the NBER linked birth / infant death records: 11,400
pairs, 30 covariates. Treatment is being the heavier twin and the outcome is
one-year mortality. Because both twins are observed, **both potential outcomes
are measured rather than simulated** — the only dataset here whose ground truth
is not a modelling assumption :cite:`louizos2017causal`.

Two things to know before using it. The raw outcome columns hold days survived
with ``9999`` standing for "survived the year", so mortality is
``outcome < 9999``; read as a number instead, the column averages about 8,000 and
means nothing. And revealing one twin per pair is what makes this observational —
``fetch_twins`` assigns at random, giving a trial with known counterfactuals,
while the confounded variants in the literature assign from a covariate and
differ between papers. Both potential outcomes are returned as ``y0`` and ``y1``
so a caller can construct their own and say which.

Source: NBER linked birth / infant death data (US federal, public domain), via
the van der Schaar lab mirror at a pinned revision.

Datasets without a loader
=========================

The remaining benchmarks named in the v1.0 roadmap are not shipped. The rule the
loaders follow is to ship one where the source carries an explicit permissive
license or is public-domain government data, fetch it at runtime, and never
vendor or mirror it. These do not meet that bar today:

.. list-table::
   :header-rows: 1

   * - Dataset
     - Source
     - Status
   * - ACIC (2016-2019 competitions)
     - `vdorie/aciccomp <https://github.com/vdorie/aciccomp>`_
     - No license file. Available through the R packages in that repository.
   * - Jobs
     - LaLonde lineage, distributed with the Shalit et al. code
     - No license file. The experimental subset overlaps ``fetch_lalonde``.
   * - Criteo-Uplift
     - `Criteo AI Lab <https://ailab.criteo.com/criteo-uplift-prediction-dataset/>`_
     - Non-commercial license. ``scikit-uplift`` ships ``fetch_criteo``.
   * - Hillstrom
     - `MineThatData <https://blog.minethatdata.com/2008/03/minethatdata-e-mail-analytics-and-data.html>`_
     - No license file. ``scikit-uplift`` ships ``fetch_hillstrom``.

No license is not the same as no permission, and these files have been
redistributed in the literature for years. It does mean CausalML does not
redistribute them. For Criteo and Hillstrom,
`scikit-uplift <https://github.com/maks-sh/scikit-uplift>`_ provides loaders.

Loading one yourself takes the same shape as the built-in loaders:

.. code-block:: python

    import pandas as pd

    df = pd.read_csv("your_download.csv")
    X = df[feature_columns].to_numpy()
    y = df["outcome"].to_numpy()
    treatment = df["treatment"].to_numpy()

    learner.fit(X=X, treatment=treatment, y=y)

---

### Examples (`docs/examples.rst`)

Examples
========

Working example notebooks are available in the `example folder <https://github.com/uber/causalml/tree/master/docs/examples>`_.

The notebooks are grouped by topic. Within each group they run from
introductory to advanced, so a group's first notebook is the place to start.

Getting started
---------------

The two canonical tours of the package: CATE estimation with the meta-learners,
and uplift modeling with the tree-based learners.

.. toctree::
    :maxdepth: 1

    examples/meta_learners_with_synthetic_data
    examples/uplift_trees_with_synthetic_data

Meta-learners
-------------

.. toctree::
    :maxdepth: 1

    examples/meta_learners_with_synthetic_data_multiple_treatment
    examples/dr_learner_with_synthetic_data

Uplift and causal trees
-----------------------

.. toctree::
    :maxdepth: 1

    examples/causal_trees_with_synthetic_data
    examples/causal_trees_with_synthetic_data_multiple_treatment_groups
    examples/causal_trees_interpretation
    examples/causal_tree_honesty_parity

Evaluation and validation
-------------------------

Validating a CATE model without ground truth: uplift and Qini curves, TMLE-based
evaluation, calibration, and sensitivity analysis.

.. toctree::
    :maxdepth: 1

    examples/validation_with_tmle
    examples/calibration
    examples/qini_curves_for_costly_treatment_arms
    examples/sensitivity_example_with_synthetic_data

Interpretation and feature selection
------------------------------------

.. toctree::
    :maxdepth: 1

    examples/feature_interpretations_example
    examples/uplift_tree_visualization
    examples/feature_selection

Neural models
-------------

DragonNet and CEVAE, each with a comparison of its available backends. These
need the optional ``tf``, ``torch`` or ``jax`` extras.

.. toctree::
    :maxdepth: 1

    examples/dragonnet_example
    examples/dragonnet_jax_vs_tf
    examples/cevae_example
    examples/cevae_jax_vs_torch

Instrumental variables
----------------------

.. toctree::
    :maxdepth: 1

    examples/iv_nlsym_synthetic_data

Policy and value optimization
-----------------------------

Turning effect estimates into treatment decisions: policy learning,
counterfactual unit selection and value optimization, and probabilities of
causation.

.. toctree::
    :maxdepth: 1

    examples/binary_policy_learner_example
    examples/counterfactual_unit_selection
    examples/counterfactual_value_optimization
    examples/necessary_and_sufficient

Datasets and benchmarks
-----------------------

The benchmark leaderboard regenerates every number published on the
:doc:`datasets` page; the simulation studies compare estimators on synthetic
and semi-synthetic data.

.. toctree::
    :maxdepth: 1

    examples/benchmark_leaderboard
    examples/benchmark_simulation_studies
    examples/benchmark_semi_synthetic_simulation_studies
    examples/logistic_regression_based_data_generation_for_uplift_classification

---

### Faq (`docs/faq.rst`)

==========================
Frequently Asked Questions
==========================

Importing CausalML fails with an XGBoost error on macOS
-------------------------------------------------------

``xgboost`` and ``lightgbm`` need the OpenMP runtime (``libomp``), which macOS
does not ship. Since v0.16, the import error names the fix directly: install it
with ``brew install libomp`` or ``conda install -c conda-forge llvm-openmp``,
then retry. See :ref:`Installation <installation:Installation>`.

Downloading a benchmark dataset fails with ``CERTIFICATE_VERIFY_FAILED``
------------------------------------------------------------------------

The dataset loaders (:doc:`datasets`) download over HTTPS with Python's
standard library, which needs a CA certificate bundle. Some Python
installations -- notably the python.org macOS installers -- do not wire one up
by default. Run the ``Install Certificates.command`` that ships with the
python.org installer, or point Python at the ``certifi`` bundle:

.. code-block:: bash

    export SSL_CERT_FILE=$(python -c "import certifi; print(certifi.where())")

My causal tree results changed after upgrading
----------------------------------------------

``CausalTreeRegressor`` and ``CausalRandomForestRegressor`` estimate leaves
honestly by default since #584: the tree structure is grown on one half of the
sample and leaf values are re-estimated on the other, which changes fitted
trees, leaf values and CATE estimates without any code edit. Pass
``honesty=False`` to reproduce the previous behavior, and see
:ref:`Honest estimation <methodology:Honest estimation>` for why the new
default is better on most data.

``UpliftRandomForestClassifier`` got slower after upgrading
-----------------------------------------------------------

Its default changed from ``n_jobs=-1`` (all cores) to ``n_jobs=None`` (one
worker) in #991, because each concurrent tree fit holds its own working set:
peak memory grew with the machine's core count -- 8.8x the input array at
``n_jobs=-1`` on 10 cores versus 1.4x single-threaded, in the benchmark that
motivated the change. Fitted models are identical either way. Pass
``n_jobs=-1`` explicitly to restore the previous speed if you have the memory.

My Qini or AUUC score is negative
---------------------------------

A negative score means the model's ranking performed *worse* than treating
units in random order on that data -- units it ranked as high-benefit gained
less than average. Before concluding the model is bad, compute the score on
held-out data with ``return_ci=True``: on small samples the confidence
interval is often wide enough that a negative point estimate is
indistinguishable from zero. The :doc:`validation` page gives the full
evaluation workflow, and Step 6 of the :doc:`tutorial` shows a real example.

My propensity scores pile up near 0 or 1
----------------------------------------

That signals an overlap (positivity) problem: some units essentially always or
never receive treatment given their covariates, so their counterfactual is not
represented in the data. Estimators that weight by inverse propensity become
unstable there. Common responses are trimming the non-overlapping region or
clipping the scores away from the boundaries -- and reconsidering whether the
treatment is really variable for those units. See
:ref:`Checking Overlap <validation:Checking Overlap>`.

The Twins dataset's outcome looks like earnings, not mortality
--------------------------------------------------------------

The Twins benchmark encodes survival as ``9999``, so mortality is
``outcome < 9999`` -- reading the column as a number produces a mean near
8,000 and every downstream statistic is garbage. The loader's docstring and
:doc:`datasets` record this; the loader's tests pin it.

My IHDP results differ across replications more than expected
-------------------------------------------------------------

Each IHDP replication draws its own train/test split of the same 747 units,
so rows are **not** aligned across replications -- averaging
predictions row-wise across replications compares different children. Evaluate
each replication independently and aggregate the metric, as the
:doc:`benchmark leaderboard <examples/benchmark_leaderboard>` does.

Do I need TensorFlow, PyTorch or JAX?
-------------------------------------

Only for the neural estimators: ``DragonNet`` needs the ``tf`` or ``jax``
extra and ``CEVAE`` needs ``torch`` or ``jax``. Everything else -- meta-learners,
trees, IV, evaluation -- runs without any of them. Install via
``pip install causalml[tf]`` etc.; see
:ref:`Installation <installation:Installation>`.

---

### Getting Started (`docs/getting_started.rst`)

===============
Getting Started
===============

What CausalML is, an introduction to causal machine learning for readers new to
it, how to install the package, a tour of the API in runnable snippets, an
end-to-end walkthrough of estimating and validating heterogeneous treatment
effects on a benchmark, and answers to common questions.

.. toctree::
    :maxdepth: 2

    about
    intro
    installation
    quickstart
    tutorial
    faq

---

### Index (`docs/index.rst`)

Welcome to Causal ML's documentation
====================================

.. toctree::
    :maxdepth: 2

    getting_started
    user_guide
    causalml
    contributing
    release_notes


Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

---

### Inference (`docs/inference.rst`)

==========================
Uncertainty Quantification
==========================

Every number CausalML produces is an estimate, and most of them can carry a
confidence interval. This page catalogs, per estimator family and per metric,
what uncertainty measure is available, how it is computed, and how to request
it. It consolidates an API surface that is otherwise spread across the
individual classes.

Average treatment effects
=========================

**Meta-learners.** ``estimate_ate()`` returns the triple ``(ate, lb, ub)``.
The default interval is analytic, based on the lower-bound formula (7) of
:cite:`imbens2009recent`; passing ``bootstrap_ci=True`` (with ``n_bootstraps``
and ``bootstrap_size``) replaces it with a bootstrap interval. The
:ref:`DRIV learner <methodology:Doubly Robust Instrumental Variable (DRIV) learner>`
follows the same interface.

**TMLE.** ``TMLELearner.estimate_ate(..., return_ci=True)`` reports the ATE
with a confidence interval; :ref:`the methodology section
<methodology:Targeted maximum likelihood estimation (TMLE) for ATE>` describes
the estimator.

**Causal trees.** ``CausalTreeRegressor.estimate_ate()`` also returns
``(ate, lb, ub)``.

CATE estimates
==============

**Meta-learners.** ``fit_predict(..., return_ci=True)`` returns per-unit CATE
estimates with lower and upper bounds from a bootstrap over refits:
``n_bootstraps`` controls the number of refits and ``bootstrap_size`` the
resample size, so the cost is roughly ``n_bootstraps`` times the single fit.

**Causal forests.** ``CausalRandomForestRegressor.calculate_error(X_train,
X_test)`` returns an unbiased sampling variance for each prediction, computed
with the infinitesimal jackknife of `Wager, Hastie and Efron (2014)
<https://arxiv.org/abs/1311.4555>`_ as implemented in `forestci
<https://github.com/scikit-learn-contrib/forest-confidence-interval>`_.

**Uplift trees.** ``UpliftTreeClassifier`` and
``UpliftRandomForestClassifier`` do not report uncertainty for their
estimates.

Evaluation metrics
==================

The evaluation functions accept ``return_ci=True`` and then return a DataFrame
with the score, its bootstrap standard error and confidence interval bounds
per model column (``n_bootstrap``, ``alpha`` and ``random_state`` control the
bootstrap). Pass ``random_state`` whenever the numbers will be reported: the
bootstrap otherwise draws from the global NumPy state and will not reproduce.

.. list-table::
   :header-rows: 1
   :widths: 30 40 30

   * - Function
     - Uncertainty reported
     - Notes
   * - ``auuc_score``
     - bootstrap SE and CI
     - no p-value by design; compare models by CI overlap
   * - ``qini_score``
     - bootstrap SE, CI and p-value
     - p-value tests ranking better than random
   * - ``rate_score``
     - half-sample bootstrap SE, CI and p-value
     - the heterogeneity test; see :ref:`RATE <methodology:RATE>`
   * - ``dr_score`` / ``plug_in_t_score``
     - bootstrap SE and CI on the loss
     - lower loss is better; see
       :ref:`Model Selection with Validation Losses <validation:Model Selection with Validation Losses>`

What an interval does and does not cover
========================================

These intervals quantify sampling uncertainty under each estimator's
assumptions -- most importantly unconfoundedness. They say nothing about bias
from an unmeasured confounder; that question belongs to
:ref:`sensitivity analysis <validation:Validation with Sensitivity Analysis>`.
And a per-unit CATE interval that excludes zero is not yet evidence of
targetable heterogeneity across units -- that claim needs the held-out tests
in :doc:`validation`.

---

### Installation (`docs/installation.rst`)

============
Installation
============

Installation with ``conda`` or ``pip`` is recommended.  Developers can follow the **Install from source** instructions below.  If building from source, consider doing so within a conda environment and then exporting the environment for reproducibility.

To use models under the ``inference.tf``, ``inference.torch`` or ``inference.jax`` module (e.g. ``DragonNet`` or ``CEVAE``), additional dependency of ``tensorflow``, ``torch`` or ``jax`` is required. For detailed instructions, see below.

System Requirements
-------------------

**Python Version:** Python 3.11 or later is required.

**Linux Distributions:** Pre-built binary wheels require a Linux distribution with glibc 2.28 or later:

* **Ubuntu:** 20.04 LTS or later
* **RHEL/CentOS:** 8 or later
* **Debian:** 10 (Buster) or later
* **Fedora:** 32 or later

.. note::
   For older Linux distributions (e.g., RHEL 7, Ubuntu 16.04, Ubuntu 18.04), you will need to build CausalML from source. See **Install from source** below.

**macOS and Windows:** All recent versions are supported.

.. note::
   On macOS, ``xgboost`` and ``lightgbm`` require the OpenMP runtime (``libomp``),
   which is not included by default. Without it, importing CausalML modules that
   use XGBoost can fail when the XGBoost native library cannot be loaded.
   Install it via Homebrew:

   .. code-block:: bash

      brew install libomp

   or via conda-forge:

   .. code-block:: bash

      conda install -c conda-forge llvm-openmp

Install using ``conda``
-----------------------

Install ``conda``
^^^^^^^^^^^^^^^^^

.. code-block:: bash

    wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
    bash Miniconda3-latest-Linux-x86_64.sh -b
    source miniconda3/bin/activate
    conda init
    source ~/.bashrc

Install from ``conda-forge``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Directly install from the ``conda-forge`` channel using ``conda``.

.. code-block:: bash

    conda install -c conda-forge causalml

Install from ``PyPI``
---------------------

.. code-block:: bash

    pip install causalml

Install ``causalml`` with ``tensorflow`` for ``DragonNet`` from ``PyPI``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

    pip install causalml[tf]

Install ``causalml`` with ``torch`` for ``CEVAE`` from ``PyPI``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

    pip install causalml[torch]

Install ``causalml`` with ``jax`` for ``DragonNet`` or ``CEVAE`` from ``PyPI``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

    pip install causalml[jax]


Install using `uv <https://github.com/astral-sh/uv/blob/main/README.md>`_
-------------------------------------------------------------------------

.. code-block:: bash

    uv init
    uv add causalml

Install ``causalml`` with ``tensorflow`` for ``DragonNet`` using `uv <https://github.com/astral-sh/uv/blob/main/README.md>`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

    uv add "causalml[tf]"

Install ``causalml`` with ``torch`` for ``CEVAE`` using `uv <https://github.com/astral-sh/uv/blob/main/README.md>`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

    uv add "causalml[torch]"

Install ``causalml`` with ``jax`` for ``DragonNet`` or ``CEVAE`` using `uv <https://github.com/astral-sh/uv/blob/main/README.md>`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

    uv add "causalml[jax]"
    




    
Install from source
-------------------

[Optional] If you don't have Graphviz installed, you can install it using ``conda``, ``brew`` (on MacOS), or ``apt`` (on Linux).

.. code-block:: bash

    conda install python-graphviz
    brew install graphviz           # MacOS
    sudo apt-get install graphviz   # Linux

First, clone the repository and install the package:

.. code-block:: bash

    git clone https://github.com/uber/causalml.git
    cd causalml
    pip install -e .

with ``tensorflow`` for ``DragonNet``:

.. code-block:: bash

    pip install -e ".[tf]"

with ``torch`` for ``CEVAE``:

.. code-block:: bash

    pip install -e ".[torch]"

with ``jax`` for ``DragonNet`` or ``CEVAE``:

.. code-block:: bash

    pip install -e ".[jax]"

=======

Windows
-------

See content in https://github.com/uber/causalml/issues/678


Running Tests
-------------

Make sure pytest is installed before attempting to run tests.

.. code-block:: bash

    pip install -e ".[test]"

Run all tests with:

.. code-block:: bash

    pytest -vs tests/ --cov causalml/

Add ``--runtf``, ``--runtorch`` and/or ``--runjax`` to run optional tensorflow/torch/jax tests which will be skipped by default.

You can also run tests via make:

.. code-block:: bash

    make test

---

### Interpretation (`docs/interpretation.rst`)

=======================
Interpretable Causal ML
=======================

Causal ML provides methods to interpret the treatment effect models trained, where we provide more sample code in `feature_interpretations_example.ipynb notebook <https://github.com/uber/causalml/blob/master/docs/examples/feature_interpretations_example.ipynb>`_.

Meta-Learner Feature Importances
--------------------------------

.. code-block:: python

    from causalml.inference.meta import BaseSRegressor, BaseTRegressor, BaseXRegressor, BaseRRegressor

    slearner = BaseSRegressor(LGBMRegressor(), control_name='control')
    slearner.estimate_ate(X=X, treatment=w_multi, y=y)
    slearner_tau = slearner.fit_predict(X=X, treatment=w_multi, y=y)

    model_tau_feature = RandomForestRegressor()  # specify model for model_tau_feature

    slearner.get_importance(X=X, tau=slearner_tau, model_tau_feature=model_tau_feature,
                            normalize=True, method='auto', features=feature_names)

    # Using the feature_importances_ method in the base learner (LGBMRegressor() in this example)
    slearner.plot_importance(X=X, tau=slearner_tau, normalize=True, method='auto')

    # Using scikit-learn's permutation_importance
    slearner.plot_importance(X=X, tau=slearner_tau, normalize=True, method='permutation')

    # Using SHAP
    shap_slearner = slearner.get_shap_values(X=X, tau=slearner_tau)

    # Plot shap values without specifying shap_dict
    slearner.plot_shap_values(X=X, tau=slearner_tau)

    # Plot shap values WITH specifying shap_dict
    slearner.plot_shap_values(X=X, shap_dict=shap_slearner)

    # interaction_idx set to 'auto' (searches for feature with greatest approximate interaction)
    slearner.plot_shap_dependence(treatment_group='treatment_A',
                                feature_idx=1,
                                X=X,
                                tau=slearner_tau,
                                interaction_idx='auto')

.. image:: ./_static/img/meta_feature_imp_vis.png
    :width: 629

.. image:: ./_static/img/meta_shap_vis.png
    :width: 629

.. image:: ./_static/img/meta_shap_dependence_vis.png
    :width: 629

Uplift Tree Visualization
-------------------------

.. code-block:: python

    from IPython.display import Image
    from causalml.inference.tree import UpliftTreeClassifier, UpliftRandomForestClassifier
    from causalml.inference.tree import uplift_tree_string, uplift_tree_plot
    from causalml.dataset import make_uplift_classification

    df, x_names = make_uplift_classification()
    uplift_model = UpliftTreeClassifier(max_depth=5, min_samples_leaf=200, min_samples_treatment=50,
                                        n_reg=100, evaluationFunction='KL', control_name='control')

    uplift_model.fit(df[x_names].values,
                    treatment=df['treatment_group_key'].values,
                    y=df['conversion'].values)

    graph = uplift_tree_plot(uplift_model.fitted_uplift_tree, x_names)
    Image(graph.create_png())

.. image:: ./_static/img/uplift_tree_vis.png
    :width: 629

Please see below for how to read the plot, and `uplift_tree_visualization.ipynb example notebook <https://github.com/uber/causalml/blob/master/docs/examples/uplift_tree_visualization.ipynb>`_ is provided in the repo.

- feature_name > threshold: For non-leaf node, the first line is an inequality indicating the splitting rule of this node to its children nodes.
- impurity: the impurity is defined as the value of the split criterion function (such as KL, Chi, or ED) evaluated at this current node
- total_sample: sample size in this node.
- group_sample: sample sizes by treatment groups
- uplift score: treatment effect in this node, if there are multiple treatment, it indicates the maximum (signed) of the treatment effects across all treatment vs control pairs.
- uplift p_value: p value of the treatment effect in this node
- validation uplift score:  all the information above is static once the tree is trained (based on the trained trees), while the validation uplift score represents the treatment effect of the testing data when the method fill() is used. This score can be used as a comparison to the training uplift score, to evaluate if the tree has an overfitting issue.

Uplift Tree Feature Importances
-------------------------------

.. code-block:: python

    pd.Series(uplift_model.feature_importances_, index=x_names).sort_values().plot(kind='barh', figsize=(12,8))

.. image:: ./_static/img/uplift_tree_feature_imp_vis.png
    :width: 629

---

### Intro (`docs/intro.rst`)

================================
Intro to Causal Machine Learning
================================

What is Causal Machine Learning?
--------------------------------

Causal machine learning is a branch of machine learning that focuses on understanding the cause and effect relationships in data. It goes beyond just predicting outcomes based on patterns in the data, and tries to understand how changing one variable can affect an outcome.
Suppose we are trying to predict a student's test score based on how many hours they study and how much sleep they get. Traditional machine learning models would find patterns in the data, like students who study more or sleep more tend to get higher scores.
But what if you want to know what would happen if a student studied an extra hour each day? Or slept an extra hour each night? Modeling these potential outcomes or counterfactuals is where causal machine learning comes in. It tries to understand cause-and-effect relationships - how much changing one variable (like study hours or sleep hours) will affect the outcome (the test score).
This is useful in many fields, including economics, healthcare, and policy making, where understanding the impact of interventions is crucial.
While traditional machine learning is great for prediction, causal machine learning helps us understand the difference in outcomes due to interventions.


Difference from Traditional Machine Learning
--------------------------------------------

Traditional machine learning and causal machine learning are both powerful tools, but they serve different purposes and answer different types of questions.
Traditional Machine Learning is primarily concerned with prediction. Given a set of input features, it learns a function from the data that can predict an outcome. It's great at finding patterns and correlations in large datasets, but it doesn't tell us about the cause-and-effect relationships between variables. It answers questions like "Given a patient's symptoms, what disease are they likely to have?"
On the other hand, Causal Machine Learning is concerned with understanding the cause-and-effect relationships between variables. It goes beyond prediction and tries to answer questions about intervention: "What will happen if we change this variable?" For example, in a medical context, it could help answer questions like "What will happen if a patient takes this medication?"
In essence, while traditional machine learning can tell us "what is", causal machine learning can help us understand "what if". This makes causal machine learning particularly useful in fields where we need to make decisions based on data, such as policy making, economics, and healthcare.


Measuring Causal Effects
------------------------

**Randomized Control Trials (RCT)** are the gold standard for causal effect measurements.  Subjects are randomly exposed to a treatment and the Average Treatment Effect (ATE) is measured as the difference between the mean effects in the treatment and control groups.  Random assignment removes the effect of any confounders on the treatment.

If an RCT is available and the treatment effects are heterogeneous across covariates, measuring the conditional average treatment effect (CATE) can be of interest.  The CATE is an estimate of the treatment effect conditioned on the available covariates.  We call these Heterogeneous Treatment Effects (HTEs).

Why a Predictive Model Is Not Enough
------------------------------------

Outside an RCT, the units that received treatment differ from those that did
not, and any variable that drives both the treatment and the outcome -- a
*confounder* -- makes the naive comparison wrong. The simulation below makes
this concrete. Data are drawn from the first synthetic mechanism of
:cite:`nie2017quasi` (see :ref:`Validation with Synthetic Data Sets
<validation:Validation with Synthetic Data Sets>`), where the same covariates
drive both the probability of treatment and the outcome, and the true average
treatment effect is 0.5. Two hundred datasets are simulated; on each, the
treatment-minus-control difference in means and a T-learner's ATE estimate
(:ref:`T-Learner <methodology:T-Learner>` with gradient boosting) are recorded:

.. code-block:: python

    import numpy as np
    from sklearn.ensemble import GradientBoostingRegressor
    from causalml.dataset import synthetic_data
    from causalml.inference.meta import BaseTRegressor

    naive, tlearn = [], []
    for i in range(200):
        np.random.seed(i)
        y, X, treatment, tau, b, e = synthetic_data(mode=1, n=2000, p=5, sigma=1.0)
        naive.append(y[treatment == 1].mean() - y[treatment == 0].mean())
        learner = BaseTRegressor(learner=GradientBoostingRegressor(random_state=i))
        ate, _, _ = learner.estimate_ate(X=X, treatment=treatment, y=y)
        tlearn.append(float(ate))

.. image:: ./_static/img/intro_confounding_sim.png
    :width: 629
    :alt: Histograms of 200 ATE estimates: the difference in means centers far from the true ATE, the T-learner centers close to it.

The difference in means centers at 0.90 -- a bias of +0.40, nearly the size of
the true effect itself -- because treated units would have had higher outcomes
even without treatment. The T-learner, which models the outcome separately in
each group and differences the predictions, centers at 0.59. Its remaining bias
comes from regularization in the outcome models; estimators designed to be
robust to it, such as the :ref:`R-Learner <methodology:R-Learner>` and
:ref:`Doubly Robust (DR) learner <methodology:Doubly Robust (DR) learner>`,
are covered in the :doc:`methodology`.


Example Use Cases
-----------------

- **Campaign Targeting Optimization**: An important lever to increase ROI in an advertising campaign is to target the ad to the set of customers who will have a favorable response in a given KPI such as engagement or sales. CATE identifies these customers by estimating the effect of ad exposure on the KPI at the individual level from A/B experiment or historical observational data.

- **Personalized Engagement**: A company might have multiple options to interact with its customers such as different product choices in up-sell or different messaging channels for communications. One can use CATE to estimate the heterogeneous treatment effect for each customer and treatment option combination for an optimal personalized engagement experience.

---

### Methodology (`docs/methodology.rst`)

===========
Methodology
===========

In this section we dive more deeply into the algorithms implemented in CausalML.  To provide a basis for the discussion, we review some of the frameworks and definitions used in the literature.

We use the Neyman-Rubin potential outcomes framework and assume Y represents the outcome, W represents the treatment assignment, and X_i the observed covariates.

If you are deciding which of these methods to use on a given problem, start
with :doc:`choosing_an_estimator`; this page documents how each method works.


Supported Algorithms
--------------------
CausalML currently supports the following methods:

- Tree-based algorithms
    - :ref:`Uplift Random Forests <methodology:Uplift Tree>` on KL divergence, Euclidean Distance, and Chi-Square
    - :ref:`Uplift Random Forests <methodology:CTS>` on Contextual Treatment Selection
    - :ref:`Uplift Random Forests <methodology:DDP>` on delta-delta-p (:math:`\Delta\Delta P`) criterion (only for binary trees and two-class problems)
    - :ref:`Uplift Random Forests <methodology:IDDP>` on IDDP (only for binary trees and two-class problems)
    - :ref:`Interaction Tree <methodology:IT>` (only for binary trees and two-class problems)
    - :ref:`Causal Inference Tree <methodology:CIT>` (only for binary trees and two-class problems)
- Meta-learner algorithms
    - :ref:`S-Learner <methodology:S-Learner>`
    - :ref:`T-Learner <methodology:T-Learner>`
    - :ref:`X-Learner <methodology:X-Learner>`
    - :ref:`R-Learner <methodology:R-Learner>`
    - :ref:`Doubly Robust (DR) learner <methodology:Doubly Robust (DR) learner>`
- Instrumental variables algorithms
    - :ref:`2-Stage Least Squares (2SLS) <methodology:2-Stage Least Squares (2SLS)>`
    - :ref:`Doubly Robust Instrumental Variable (DRIV) learner <methodology:Doubly Robust Instrumental Variable (DRIV) learner>`
- Neural network based algorithms
    - :ref:`CEVAE <methodology:CEVAE>`
    - :ref:`DragonNet <methodology:DragonNet>`
- Treatment optimization algorithms
    - :ref:`Counterfactual Unit Selection <methodology:Counterfactual Unit Selection>`
    - :ref:`Counterfactual Value Estimator <methodology:Counterfactual Value Estimator>`


Meta-Learner Algorithms
-----------------------

*Relevant classes:* ``BaseSRegressor``/``BaseSClassifier``,
``BaseTRegressor``/``BaseTClassifier``, ``BaseXRegressor``/``BaseXClassifier``,
``BaseRRegressor``/``BaseRClassifier`` and
``BaseDRRegressor``/``BaseDRClassifier`` in ``causalml.inference.meta``, each
taking any scikit-learn-compatible model as its base learner. Preconfigured
variants (``XGBTRegressor``, ``LRSRegressor``, ...) are listed in the
:doc:`API Reference <causalml>`.

A meta-algorithm (or meta-learner) is a framework to estimate the Conditional Average Treatment Effect (CATE) using any machine learning estimators (called base learners) :cite:`kunzel2019metalearners`.

A meta-algorithm uses either a single base learner while having the treatment indicator as a feature (e.g. S-learner), or multiple base learners separately for each of the treatment and control groups (e.g. T-learner, X-learner and R-learner).

Confidence intervals of average treatment effect estimates are calculated based on the lower bound formula (7) from :cite:`imbens2009recent`.

S-Learner
~~~~~~~~~

S-learner estimates the treatment effect using a single machine learning model as follows:

**Stage 1**

Estimate the average outcomes :math:`\mu(x)` with covariates :math:`X` and an indicator variable for treatment :math:`W`:

.. math::
  \mu(x,w) = E[Y \mid X=x, W=w]

using a machine learning model.

**Stage 2**

Define the CATE estimate as:

.. math::
   \hat\tau(x) = \hat\mu(x, W=1) - \hat\mu(x, W=0)

Including the propensity score in the model can reduce bias from regularization induced confounding :cite:`hahn2017bayesian`.

When the control and treatment groups are very different in covariates, a single linear model is not sufficient to encode the different relevant dimensions and smoothness of features for the control and treatment groups :cite:`alaa2018limits`.

T-Learner
~~~~~~~~~

T-learner :cite:`kunzel2019metalearners` consists of two stages as follows:

**Stage 1**

Estimate the average outcomes :math:`\mu_0(x)` and :math:`\mu_1(x)`:

.. math::
   \mu_0(x) = E[Y(0)|X=x] \\
   \mu_1(x) = E[Y(1)|X=x]

using machine learning models.

**Stage 2**

Define the CATE estimate as:

.. math::
   \hat\tau(x) = \hat\mu_1(x) - \hat\mu_0(x)

X-Learner
~~~~~~~~~

X-learner :cite:`kunzel2019metalearners` is an extension of T-learner, and consists of three stages as follows:

**Stage 1**

Estimate the average outcomes :math:`\mu_0(x)` and :math:`\mu_1(x)`:

.. math::
   \mu_0(x) = E[Y(0)|X=x] \\
   \mu_1(x) = E[Y(1)|X=x]

using machine learning models.

**Stage 2**

Impute the user level treatment effects, :math:`D^1_i` and :math:`D^0_j` for user :math:`i` in the treatment group based on :math:`\mu_0(x)`, and user :math:`j` in the control groups based on :math:`\mu_1(x)`:

.. math::
   D^1_i = Y^1_i - \hat\mu_0(X^1_i) \\
   D^0_i = \hat\mu_1(X^0_i) - Y^0_i

then estimate :math:`\tau_1(x) = E[D^1|X=x]`, and :math:`\tau_0(x) = E[D^0|X=x]` using machine learning models.

**Stage 3**

Define the CATE estimate by a weighted average of :math:`\tau_1(x)` and :math:`\tau_0(x)`:

.. math::
   \tau(x) = g(x)\tau_0(x) + (1 - g(x))\tau_1(x)

where :math:`g \in [0, 1]`. We can use propensity scores for :math:`g(x)`.

R-Learner
~~~~~~~~~

R-learner :cite:`nie2017quasi` uses the cross-validation out-of-fold estimates of outcomes :math:`\hat{m}^{(-i)}(x_i)` and propensity scores :math:`\hat{e}^{(-i)}(x_i)`. It consists of two stages as follows:

**Stage 1**

Fit :math:`\hat{m}(x)` and :math:`\hat{e}(x)` with machine learning models using cross-validation.

**Stage 2**

Estimate treatment effects by minimising the R-loss, :math:`\hat{L}_n(\tau(x))`:

.. math::
   \hat{L}_n(\tau(x)) = \frac{1}{n} \sum^n_{i=1}\big(\big(Y_i - \hat{m}^{(-i)}(X_i)\big) - \big(W_i - \hat{e}^{(-i)}(X_i)\big)\tau(X_i)\big)^2

where :math:`\hat{e}^{(-i)}(X_i)`, etc. denote the out-of-fold held-out predictions made without using the :math:`i`-th training sample.

Doubly Robust (DR) learner

DR-learner :cite:kennedy2020optimal estimates the CATE via cross-fitting a doubly-robust score function in two stages as follows. We start by randomly split the data :math:\{Y, X, W\} into 3 partitions :math:\{Y^i, X^i, W^i\}, i=\{1,2,3\}.

Stage 1

Fit a propensity score model :math:\hat{e}(x) with machine learning using :math:\{X^1, W^1\}, and fit outcome regression models :math:\hat{m}_0(x) and :math:\hat{m}_1(x) for treated and untreated users with machine learning using :math:\{Y^2, X^2, W^2\}.

Stage 2

Use machine learning to fit the CATE model, :math:\hat{\tau}(X) from the pseudo-outcome

.. math::
\phi = \frac{W-\hat{e}(X)}{\hat{e}(X)(1-\hat{e}(X))}\left(Y-\hat{m}_W(X)\right)+\hat{m}_1(X)-\hat{m}_0(X)

with :math:\{Y^3, X^3, W^3\}

Stage 3

Repeat Stage 1 and Stage 2 again twice, rotating the three partitions so that each one trains the CATE model exactly once. First use :math:\{Y^2, X^2, W^2\}, :math:\{Y^3, X^3, W^3\}, and :math:\{Y^1, X^1, W^1\} for the propensity score model, the outcome models, and the CATE model. Then use :math:\{Y^3, X^3, W^3\}, :math:\{Y^1, X^1, W^1\}, and :math:\{Y^2, X^2, W^2\} for the propensity score model, the outcome models, and the CATE model. The final CATE model is the average of the 3 CATE models.

Doubly Robust Instrumental Variable (DRIV) learner

text
We combine the idea from DR-learner :cite:`kennedy2020optimal` with the doubly robust score function for LATE described in :cite:`10.1111/ectj.12097` to estimate the conditional LATE. Towards that end, we start by randomly split the data :math:`\{Y, X, W, Z\}` into 3 partitions :math:`\{Y^i, X^i, W^i, Z^i\}, i=\{1,2,3\}`.

**Stage 1**

Fit propensity score models :math:`\hat{e}_0(x)` and :math:`\hat{e}_1(x)` for assigned and unassigned users using :math:`\{X^1, W^1, Z^1\}`, and fit outcome regression models :math:`\hat{m}_0(x)` and :math:`\hat{m}_1(x)` for assigned and unassigned users with machine learning using :math:`\{Y^2, X^2, Z^2\}`. Assignment probability, :math:`p_Z`, can either be user provided or come from a simple model, since in most use cases assignment is random by design.

**Stage 2**

Use machine learning to fit the conditional :ref:`LATE <methodology:LATE>` model, :math:`\hat{\tau}(X)` by minimizing the following loss function

.. math::
   L(\hat{\tau}(X)) = \hat{E} &\left[\left(\hat{m}_1(X)-\hat{m}_0(X)+\frac{Z(Y-\hat{m}_1(X))}{p_Z}-\frac{(1-Z)(Y-\hat{m}_0(X))}{1-p_Z} \right.\right.\\
   &\left.\left.\quad -\Big(\hat{e}_1(X)-\hat{e}_0(X)+\frac{Z(W-\hat{e}_1(X))}{p_Z}-\frac{(1-Z)(W-\hat{e}_0(X))}{1-p_Z}\Big) \hat{\tau}(X) \right)^2\right]

with :math:`\{Y^3, X^3, W^3\}`

**Stage 3**

Similar to the DR-Learner Repeat Stage 1 and Stage 2 again twice with different permutations of partitions for estimation. The final conditional LATE model is the average of the 3 conditional LATE models.

CATE Model Evaluation
----------------------

Cross-validation cannot be used to select among fitted CATE models because the
ground-truth treatment effect is never observed for any unit -- only one of the
two potential outcomes is realized. The methods below are surrogate metrics that
score fitted CATE models using only observed data, so that models such as the
S/T/X/R/DR-learners above can be compared and selected among.

These metrics evaluate *effect-magnitude accuracy* -- how close a model's CATE
estimate is to the truth -- which is a distinct question from *targeting
quality* -- whether a model correctly ranks units by benefit -- addressed by
:ref:`RATE <methodology:RATE>`.

DR (Doubly Robust) pseudo-outcome loss
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Following :cite:`kennedy2020optimal`, we construct the same doubly-robust
pseudo-outcome used in the DR-learner above,

.. math::
   \phi = \frac{W-\hat{e}(X)}{\hat{e}(X)(1-\hat{e}(X))}\left(Y-\hat{m}_W(X)\right)+\hat{m}_1(X)-\hat{m}_0(X)

with :math:`\hat{e}(x)`, :math:`\hat{m}_0(x)`, and :math:`\hat{m}_1(x)` cross-fit
so that :math:`\phi_i` never depends on a model that saw unit :math:`i` during
training. Under either correct propensity or correct outcome-model
specification, :math:`E[\phi \mid X]` is an unbiased estimate of the true CATE,
which lets :math:`\phi` stand in for the unobserved ground truth when scoring a
fitted CATE model :math:`\hat\tau(x)`:

.. math::
   \text{DR loss}(\hat\tau) = \frac{1}{n}\sum_i \left(\hat\tau(X_i) - \phi_i\right)^2

Lower is better. :cite:`mahajan2024empirical` found DR-based metrics dominate
across 78 benchmark datasets for CATE model selection.

Plug-in T-learner loss
~~~~~~~~~~~~~~~~~~~~~~

A simpler, non-doubly-robust baseline: fit a cross-fit :ref:`T-Learner <methodology:T-Learner>`, and use
its held-out estimate :math:`\hat\mu_1(x) - \hat\mu_0(x)` directly as a proxy
for the true CATE:

.. math::
   \text{T-loss}(\hat\tau) = \frac{1}{n}\sum_i \left(\hat\tau(X_i) - (\hat\mu_1(X_i) - \hat\mu_0(X_i))\right)^2

This is biased under a misspecified outcome model, unlike the DR loss above.
Despite that, :cite:`mahajan2024empirical` found it is never dominated across
their benchmark datasets, making it a useful complement to DR-based scoring
rather than a replacement.

RATE
~~~~

The Rank-Weighted Average Treatment Effect (RATE) of
:cite:`yadlowsky2021evaluating` scores a model as a *targeting rule*: not how
close :math:`\hat\tau(x)` is to the truth, but whether the units it ranks
highest are the ones that benefit most.

Rank units by a prioritization score :math:`\hat\tau(x)`, treat the top
:math:`q` fraction, and measure the gain over treating everyone. That is the
Targeting Operator Characteristic (TOC) curve:

.. math::
   \text{TOC}(q) = E\left[Y(1)-Y(0) \mid \hat\tau(X) \ge F^{-1}_{\hat\tau}(1-q)\right] - E\left[Y(1)-Y(0)\right]

where :math:`F^{-1}_{\hat\tau}(1-q)` is the :math:`(1-q)` quantile of the
score, so the conditioning set is exactly the top-:math:`q` fraction.
:math:`\text{TOC}(1) = 0` by construction, since selecting everyone gives back
the overall ATE, and a curve that is positive at small :math:`q` means the rule
has found units with above-average benefit. RATE reduces the curve to a single
number with a weight function :math:`\alpha(q)`:

.. math::
   \text{RATE} = \int_0^1 \alpha(q)\,\text{TOC}(q)\,dq

Two weightings are implemented, and the choice between them is one of
statistical power rather than correctness:

- ``weighting="autoc"`` (the default) sets :math:`\alpha(q) = 1/q`. It weights
  the highest-priority units most heavily and is most powerful when the benefit
  is concentrated in a small subgroup.
- ``weighting="qini"`` sets :math:`\alpha(q) = q`, which recovers the Qini
  coefficient. It is more powerful when treatment effects are diffuse across
  the population.

``causalml.metrics.rate_score`` returns the scalar;
``causalml.metrics.get_toc`` and ``causalml.metrics.plot_toc`` return and draw
the curve behind it. The integral is evaluated on the discrete quantile grid as
a midpoint-weighted mean whose weights are normalized to sum to one, so the
absolute scale can differ slightly from the continuous definition above while
the ordering of models is unchanged.

Passing ``return_ci=True`` adds a standard error, a confidence interval and a
p-value for :math:`H_0: \text{RATE} = 0` -- the null that the ranking is no
better than random -- from a half-sample bootstrap that draws :math:`n/2` units
without replacement, the resampling scheme covered by the functional central
limit theorem in :cite:`yadlowsky2021evaluating`.

One assumption is easy to miss: without a ground-truth effect column, the
subset ATE inside the TOC is a plain difference in mean outcomes between the
treated and control units of each band, which is unbiased under randomization
but not under confounding. For observational data, pass cross-fit AIPW
pseudo-outcomes -- the same :math:`\phi` as in the DR loss above -- as
``treatment_effect_col``, so that the ranking is scored against a
doubly-robust estimate rather than a raw difference in means.

Tree-Based Algorithms
---------------------

*Relevant classes:* the uplift trees ``UpliftTreeClassifier`` and
``UpliftRandomForestClassifier``, which split on the divergence criteria below,
and the causal trees ``CausalTreeRegressor`` and
``CausalRandomForestRegressor``, regression trees that split on
treatment-effect heterogeneity and estimate leaves honestly by default (see
:ref:`Honest estimation <methodology:Honest estimation>`). All live in
``causalml.inference.tree``.

Uplift Tree
~~~~~~~~~~~

The Uplift Tree approach consists of a set of methods that use a tree-based algorithm where the splitting criterion is based on differences in uplift. :cite:`Rzepakowski2012-br` proposed three different ways to quantify the gain in divergence as the result of splitting :cite:`Gutierrez2016-co`:

.. math::
   D_{gain} = D_{after_{split}} (P^T, P^C) - D_{before_{split}}(P^T, P^C)

where :math:`D` measures the divergence and :math:`P^T` and :math:`P^C` refer to the probability distribution of the outcome of interest in the treatment and control groups, respectively. Three different ways to quantify the divergence, KL, ED and Chi, are implemented in the package.

Throughout this subsection the outcome is binary, so :math:`P^T` and :math:`P^C`
are each described by a single number: let :math:`p` be the sample mean of the
outcome in the treatment group and :math:`q` the sample mean in the control
group, both computed within the node being scored. Each divergence below sums
over the two outcome classes :math:`\{1, 0\}` -- which is where the second term
in each expression comes from -- and :math:`D_{gain}` then combines the parent's
divergence with its children's as above :cite:`Gutierrez2016-co`.

KL
~~~
The Kullback-Leibler (KL) divergence is given by:

.. math::
   KL(P^T : P^C) = p\log\frac{p}{q} + (1-p)\log\frac{1-p}{1-q}

At :math:`p = 0` and :math:`p = 1` the expression is taken at its limit,
:math:`-\log(1-q)` and :math:`-\log q` respectively.

ED
~~~
The Euclidean Distance is given by:

.. math::
   ED(P^T : P^C) = (p - q)^2 + \big((1-p) - (1-q)\big)^2 = 2(p - q)^2

where the notation is the same as above.

Chi
~~~
Finally, the :math:`\chi^2`-divergence is given by:

.. math::
   \chi^2(P^T : P^C) = \frac{(p - q)^2}{q} + \frac{(p - q)^2}{1 - q}

where the notation is again the same as above.

DDP
~~~

Another Uplift Tree algorithm that is implemented is the delta-delta-p (:math:`\Delta\Delta P`) approach by :cite:`hansotia2002ddp`, where the sample splitting criterion is defined as follows:

.. math::
    \Delta\Delta P=|(P^T(y|a_0)-P^C(y|a_0) - (P^T(y|a_1)-P^C(y|a_1)))|

where :math:`a_0` and :math:`a_1` are the outcomes of a Split A, :math:`y` is the selected class, and :math:`P^T` and :math:`P^C` are the response rates of treatment and control group, respectively. In other words, we first calculate the difference in the response rate in each branch (:math:`\Delta P_{left}` and :math:`\Delta P_{right}`), and subsequently, calculate their differences (:math:`\Delta\Delta P = |\Delta P_{left} - \Delta P_{right}|`).

IDDP
~~~~

Build upon the :math:`\Delta\Delta P` approach, the IDDP approach by :cite:`rossler2022the` is implemented, where the sample splitting
criterion is defined as follows:

.. math::
    IDDP = \frac{\Delta\Delta P^*}{I(\phi, \phi_l, \phi_r)}

where :math:`\Delta\Delta P^*` is defined as :math:`\Delta\Delta P - |E[Y(1) - Y(0) \mid X \in \phi]|` and
:math:`I(\phi, \phi_l, \phi_r)` is defined as:

.. math::
    I(\phi, \phi_l, \phi_r) = H(\frac{n_t(\phi)} {n(\phi)}, \frac{n_c(\phi)}{n(\phi)}) * 2 \frac{1+\Delta\Delta P^*}{3} + \frac{n_t(\phi)}{n(\phi)} H(\frac{n_t(\phi_l)}{n(\phi)}, \frac{n_t(\phi_r)}{n(\phi)}) \\
    + \frac{n_c(\phi)}{n(\phi)} * H(\frac{n_c(\phi_l)}{n(\phi)}, \frac{n_c(\phi_r)}{n(\phi)}) + \frac{1}{2}

where the entropy H is defined as :math:`H(p,q)=(-p*log_2(p)) + (-q*log_2(q))` and where :math:`\phi` is a subset of the feature space
associated with the current decision node, and :math:`\phi_l` and :math:`\phi_r` are the left and right child nodes, respectively.
:math:`n_t(\phi)` is the number of treatment samples, :math:`n_c(\phi)` the number of control samples, and :math:`n(\phi)` the number
of all samples in the current (parent) node.

IT
~~

Further, the package implements the Interaction Tree (IT) proposed by :cite:`su2009subgroup`, where the sample splitting criterion
maximizes the G statistic among all permissible splits:

.. math::
    G(s^*) = \max_s G(s)

where :math:`G(s)=t^2(s)` and :math:`t(s)` is defined as:

.. math::
    t(s) = \frac{(y^L_1 - y^L_0) - (y^R_1 - y^R_0)}{\sigma \sqrt{1/n_1 + 1/n_2 + 1/n_3 + 1/n_4}}

where :math:`\sigma=\sqrt{\sum_{i=1}^4 w_i s_i^2}` is a pooled estimator of the constant standard deviation, and :math:`w_i=(n_i-1)/\sum_{j=1}^4(n_j-1)`.
Further, :math:`y^L_1`, :math:`s^2_1`, and :math:`n_1` are the sample mean, the sample variance, and the sample size
for the treatment group in the left child node, respectively. Similar notation applies to the other quantities.

Note that this implementation deviates from the original implementation in that (1) the pruning techniques and (2) the validation method
for determining the best tree size are different.

CIT
~~~

Also, the package implements the Causal Inference Tree (CIT) by :cite:`su2012facilitating`, where the sample splitting
criterion calculates the likelihood ratio test statistic:

.. math::
    LRT(s) = -n_{\tau L}/2 * ln(n_{\tau L} SSE_{\tau L}) -n_{\tau R}/2 * ln(n_{\tau R} SSE_{\tau R}) + \\
    n_{\tau L1} ln n_{\tau L1} + n_{\tau L0} ln n_{\tau L0} + n_{\tau R1} ln n_{\tau R1} + n_{\tau R0} ln n_{\tau R0}

where :math:`n_{\tau}`, :math:`n_{\tau 0}`, and :math:`n_{\tau 1}` are the total number of observations in node :math:`\tau`,
the number of observations in node :math:`\tau` that are assigned to the control group, and the number of observations in node :math:`\tau`
that are assigned to the treatment group, respectively. :math:`SSE_{\tau}` is defined as:

.. math::
    SSE_{\tau} = \sum_{i \in \tau: t_i=1}(y_i - \hat{y_{t1}})^2 + \sum_{i \in \tau: t_i=0}(y_i - \hat{y_{t0}})^2

and :math:`\hat{y_{t0}}` and :math:`\hat{y_{t1}}` are the sample average responses of the control and treatment groups in node
:math:`\tau`, respectively.

Note that this implementation deviates from the original implementation in that (1) the pruning techniques and (2) the validation method
for determining the best tree size are different.

CTS
~~~

The final Uplift Tree algorithm that is implemented is the Contextual Treatment Selection (CTS) approach by :cite:`Zhao2017-kg`, where the sample splitting criterion is defined as follows:

.. math::
   \hat{\Delta}_{\mu}(s) = \hat{p}(\phi_l \mid \phi) \times \max_{t=0, ..., K}\hat{y}_t(\phi_l) + \hat{p}(\phi_r \mid \phi) \times \max_{t=0, ..., K}\hat{y}_t(\phi_r) -  \max_{t=0, ..., K}\hat{y}_t(\phi)

where :math:`\phi_l` and :math:`\phi_r` refer to the feature subspaces in the left leaf and the right leaves respectively, :math:`\hat{p}(\phi_j \mid \phi)` denotes the estimated conditional probability of a subject's being in :math:`\phi_j` given :math:`\phi`, and :math:`\hat{y}_t(\phi_j)` is the conditional expected response under treatment :math:`t`.

Honest estimation
~~~~~~~~~~~~~~~~~

A tree that estimates each leaf's treatment effect on the same rows that chose
its splits inherits the selection bias of the split search: the search favors
partitions whose in-sample effects look large, so re-using those rows makes the
leaf estimates systematically too extreme. *Honest estimation*
:cite:`athey2016recursive` removes this bias by splitting the sample in two --
the tree structure is grown on one half and each leaf's per-group outcome means
are re-estimated on the other -- at the cost of fitting each half with less
data.

``CausalTreeRegressor`` and ``CausalRandomForestRegressor`` estimate honestly
by default (``honesty=True``), matching the defaults of ``grf`` and EconML.
``UpliftTreeClassifier`` and ``UpliftRandomForestClassifier`` accept the same
parameters with ``honesty=False`` by default. ``estimation_sample_size``
(default ``0.5``) sets the held-out fraction, and the split is stratified on
treatment.

The trade is variance: each half sees only part of the data, so an honest tree
is shallower and noisier per leaf. Pass ``honesty=False`` when halving the
sample leaves too little to split on.

On ``CausalRandomForestRegressor``, honesty composes with the bootstrap: each
tree's bootstrap counts weight both the structure fit and the leaf
re-estimation. This is not the subsample-without-replacement scheme of
generalized random forests :cite:`athey2019generalized`.

Pruning
~~~~~~~

Both tree families grow greedily and can overfit. Each offers a
validation-based way to cut the fitted tree back; both are off by default
because they change fitted trees.

``UpliftTreeClassifier`` prunes against a held-out sample:
``prune(X, treatment, y)`` collapses an internal node whose split does not
improve the treatment-effect estimate on that sample, controlled by
``min_gain`` and ``prune_rule``, cascading upward until no further merge helps.
Passing ``prune_fraction`` to ``fit`` runs the same procedure as part of the
fit: that fraction of the rows is held out, stratified on treatment and
outcome, the tree grows on the remainder, and pruning uses the holdout.

``CausalTreeRegressor`` uses cost-complexity pruning, the scheme behind
scikit-learn's ``ccp_alpha``: a penalty per leaf defines a sequence of nested
subtrees, and a larger ``ccp_alpha`` keeps a smaller one. Setting
``ccp_alpha="cv"`` selects the subtree by ``cv_folds``-fold cross-validation
instead, scoring each candidate with the honest splitting objective on the
held-out fold; it requires ``honesty=True`` and completes the algorithm of
:cite:`athey2016recursive`. Prefer it on a single tree, where the gains grow
with the noise level -- on ``CausalRandomForestRegressor`` the same option
measured no gain, since averaging across trees already removes the variance
that pruning targets.


Neural Network Algorithms
-------------------------

*Relevant classes:* ``DragonNet`` in ``causalml.inference.tf`` or
``causalml.inference.jax``, and ``CEVAE`` in ``causalml.inference.torch`` or
``causalml.inference.jax``.

Both methods below are optional backends. Install them with the ``tf``, ``torch``
or ``jax`` extras, e.g. ``pip install causalml[tf]``.

CEVAE
~~~~~

The Causal Effect Variational Autoencoder (CEVAE) :cite:`louizos2017causal`
targets the case where the confounders are never observed directly. Instead of
assuming that the covariates :math:`X` satisfy unconfoundedness, it assumes
:math:`X` are noisy **proxies** for a latent confounder :math:`Z` — for example,
a zip code standing in for socio-economic status — and recovers the treatment
effect by inferring :math:`Z`.

The generative model is

.. math::
   Z \sim p(Z), \quad
   X \sim p(X \mid Z), \quad
   W \sim p(W \mid Z), \quad
   Y \sim p(Y \mid W, Z)

where each distribution is parameterised by a neural network. The outcome model
uses a disjoint pair of networks for :math:`p(Y \mid W=0, Z)` and
:math:`p(Y \mid W=1, Z)`, so a treatment arm with few observations does not have
its outcome head diluted by the other arm.

Because :math:`Z` is latent, the posterior :math:`p(Z \mid X, W, Y)` is
intractable. CEVAE approximates it with an inference network
:math:`q(Z \mid X, W, Y)` trained jointly with the generative model by
maximising the evidence lower bound (ELBO). Two auxiliary networks,
:math:`q(W \mid X)` and :math:`q(Y \mid X, W)`, are trained alongside it, because
at prediction time only :math:`X` is available: the treatment and outcome that
the inference network expects have to be supplied by the auxiliary heads before
:math:`Z` can be inferred for a new unit.

The CATE estimate marginalises the outcome heads over the inferred latent:

.. math::
   \hat\tau(x) = \mathbb{E}_{Z \sim q(Z \mid X=x)}
       \big[\mathbb{E}[Y \mid W=1, Z] - \mathbb{E}[Y \mid W=0, Z]\big]

CausalML wraps Pyro's implementation in ``causalml.inference.torch.CEVAE`` and
also ships a JAX/``flax.nnx`` backend.

DragonNet
~~~~~~~~~

DragonNet :cite:`shi2019adapting` starts from the sufficiency of the propensity
score: if adjusting for :math:`X` suffices to identify the treatment effect, then
adjusting for :math:`e(X) = P(W=1 \mid X)` alone also suffices. A representation
of :math:`X` therefore only has to retain what predicts treatment, and a
representation trained purely to predict the outcome may discard exactly that.

The architecture makes this explicit with three heads on a shared trunk
:math:`Z(x)` of three fully-connected layers:

- a **propensity head** :math:`\hat{e}(x)`, a single sigmoid unit on
  :math:`Z(x)`;
- two **outcome heads** :math:`\hat{Q}(0, x)` and :math:`\hat{Q}(1, x)`, each its
  own stack of layers.

Attaching the propensity head directly to :math:`Z(x)` — with no hidden layer of
its own — is what forces the shared trunk to keep the propensity information
rather than discarding it in favour of outcome prediction.

The second contribution is **targeted regularization**, which adds a trainable
scalar :math:`\epsilon` and an extra loss term. Writing
:math:`\hat{y} = w\hat{Q}(1,x) + (1-w)\hat{Q}(0,x)` and

.. math::
   h(w, x) = \frac{w}{\hat{e}(x)} - \frac{1-w}{1 - \hat{e}(x)}

the perturbed prediction is :math:`\tilde{y} = \hat{y} + \epsilon \, h(w, x)`, and
the training objective becomes

.. math::
   \mathcal{L} = \mathcal{L}_{\text{DragonNet}}
       + \beta \sum_i \big(y_i - \tilde{y}_i\big)^2

with :math:`\beta` the ``ratio`` argument. This is a one-step correction in the
spirit of :ref:`TMLE <methodology:Targeted maximum likelihood estimation (TMLE) for ATE>`:
at the optimum the fitted model satisfies the estimating equation for the ATE,
which is what gives the resulting estimator its asymptotic guarantees. It is
enabled by default via ``targeted_reg=True``.

The CATE estimate is the difference of the two outcome heads:

.. math::
   \hat\tau(x) = \hat{Q}(1, x) - \hat{Q}(0, x)

CausalML ships DragonNet as ``causalml.inference.tf.DragonNet`` (TensorFlow) and
a JAX/``flax.nnx`` backend.

Value optimization methods
--------------------------

*Relevant classes:* ``CounterfactualUnitSelector`` and
``CounterfactualValueEstimator`` in ``causalml.optimize``. ``PolicyLearner``
in the same module learns a treatment-assignment policy directly.

The package supports methods for assigning treatment groups when treatments are costly. To understand the problem, it is helpful to divide the population into four categories according to how a unit's *outcome* responds to being treated:

* **Persuadables**. Those who will have a favourable outcome if and only if they are treated.
* **Sure things**. Those who will have a favourable outcome whether or not they are treated.
* **Lost causes**. Those who will never have a favourable outcome whether or not they are treated.
* **Sleeping dogs**. Those who will have a favourable outcome if and only if they are not treated.

These four response types are also referred to as compliers, always-takers, never-takers and defiers. Note that those names are standard in the instrumental variables literature for a **different** partition of the population -- one defined by whether a unit *takes* the treatment in response to its *assignment*, which is the sense used in the :ref:`LATE <methodology:LATE>` section below (see :cite:`angrist2008mostly` for that one). The two partitions do not coincide, so the uplift names are used here.

Counterfactual Unit Selection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:cite:`ijcai2019-248` propose a method for selecting units for treatments using counterfactual logic. Suppose the following benefits for selecting units belonging to the different categories above:

* Persuadables: :math:`\beta`
* Sure things: :math:`\gamma`
* Lost causes: :math:`\theta`
* Sleeping dogs: :math:`\delta`

If :math:`X` denotes the set of individual's features, the unit selection problem can be formulated as follows:

.. math::
   \operatorname*{argmax}_X \beta P(\text{persuadable} \mid X) + \gamma P(\text{sure thing} \mid X) + \theta P(\text{lost cause} \mid X) + \delta P(\text{sleeping dog} \mid X)

The problem can be reformulated using counterfactual logic. Suppose :math:`W = w` indicates that an individual is treated and :math:`W = w'` indicates he or she is untreated. Similarly, let :math:`F = f` denote a favourable outcome for the individual and :math:`F = f'` an unfavourable outcome. Then the optimization problem becomes:

.. math::
   \operatorname*{argmax}_X \beta P(f_w, f'_{w'} \mid X) + \gamma P(f_w, f_{w'} \mid X) + \theta P(f'_w, f'_{w'} \mid X) + \delta P(f_{w'}, f'_{w} \mid X)

Note that the above simply follows from the definitions of the relevant user segments. :cite:`ijcai2019-248` then use counterfactual logic (:cite:`pearl2009causality`) to solve the above optimization problem under certain conditions.

N.B. The current implementation in the package is highly experimental.

Counterfactual Value Estimator
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The counterfactual value estimation method implemented in the package predicts the outcome for a unit under different treatment conditions using a standard machine learning model. The expected value of assigning a unit into a particular treatment is then given by

.. math::
   \mathbb{E}[(v - cc_w)Y_w - ic_w]

where :math:`Y_w` is the probability of a favourable event (such as conversion) under a given treatment :math:`w`, :math:`v` is the value of the favourable event, :math:`cc_w` is the cost of the treatment triggered in case of a favourable event, and :math:`ic_w` is the cost associated with the treatment whether or not the outcome is favourable. This method builds upon the ideas discussed in :cite:`zhao2019uplift`.

Probabilities of causation
--------------------------

A cause is said to be *necessary* for an outcome if the outcome would not have occurred in the absence of the cause. A cause is said to be *sufficient* for an outcome if the outcome would have occurred in the presence of the cause. A cause is said to be *necessary and sufficient* if both of the above two conditions hold. :cite:`tian2000probabilities` show that we can calculate bounds for the probability that a cause is of each of the above three types.

To understand how the bounds for the probabilities of causation are calculated, we need special notation to represent counterfactual quantities. Let :math:`y_t` represent the proposition “:math:`y` would occur if the treatment group was set to ‘treatment’”, :math:`y^{\prime}_c` represent the proposition “:math:`y` would not occur if the treatment group was set to ‘control’”, and similarly for the remaining two combinations of the (by assumption) binary outcome and treatment variables.

Then the probability that the treatment is *sufficient* for :math:`y` to occur can be defined as

.. math::

    PS = P(y_t \mid c, y^{\prime})

This is the probability that the :math:`y` would occur if the treatment was set to :math:`t` when in fact the treatment was set to control and the outcome did not occur.

The probability that the treatment is *necessary* for :math:`y` to occur can be defined as

.. math::
    PN = P(y^{\prime}_c \mid t, y)

This is the probability that :math:`y` would not occur if the treatment was set to control, while in actuality both :math:`y` occurs and the treatment takes place.

Finally, the probability that the treatment is both necessary and sufficient is defined as 

.. math::
    PNS = P(y_t, y^{\prime}_c)

and states that :math:`y` would occur if the treatment took place; and :math:`y` would not occur if the treatment did not take place. PNS is related with PN and PS as follows:

.. math::
    PNS = P(t, y)PN + P(c, y^{\prime})PS

In bounding the above three quantities, we utilize observational data in addition to experimental data. The observational data is characterized in terms of the joint probabilities:

.. math::
    P_{TY} = {P(t, y),  P(c, y), P(t, y^{\prime}), P(c, y^{\prime})}

Given this, :cite:`tian2000probabilities` use the program developed in :cite:`balke1995probabilistic` to obtain sharp bounds of the above three quantities. The main idea in this program is to turn the bounding task into a linear programming problem (for a modern implementation of their approach see `here <https://cran.r-project.org/web/packages/causaloptim/vignettes/vertexenum-speed.html>`_).

Using the linear programming approach and given certain constraints together with observational data, :cite:`tian2000probabilities` find that the sharp lower bound for PNS is given by

.. math::
    \max\{0, P(y_t) - P(y_c), P(y) - P(y_c), P(y_t) - P(y)\}

and the sharp upper bound is given by

.. math::
    \min\{P(y_t), P(y^{\prime}_c), P(t, y) + P(c, y^{\prime}), P(y_t) - P(y_c) + P(t, y^{\prime}) + P(c, y)\}

They use a similar routine to find the bounds for PS and PN. The `get_pns_bounds()` function calculates the bounds for each of the three probabilities of causation using the results in :cite:`tian2000probabilities`.

Selected traditional methods
----------------------------

*Relevant classes:* ``NearestNeighborMatch`` and ``MatchOptimizer`` in
``causalml.match``, the propensity models in ``causalml.propensity``,
``IVRegressor`` in ``causalml.inference.iv`` for 2SLS, and ``TMLELearner`` in
``causalml.inference.meta``.

The package supports selected traditional causal inference methods. These are usually used to conduct causal inference with observational (non-experimental) data. In these types of studies, the observed difference between the treatment and the control is in general not equal to the difference between "potential outcomes" :math:`\mathbb{E}[Y(1) - Y(0)]`. Thus, the methods below try to deal with this problem in different ways.


Matching
~~~~~~~~
The general idea in matching is to find treated and non-treated units that are as similar as possible in terms of their relevant characteristics. As such, matching methods can be seen as part of the family of causal inference approaches that try to mimic randomized controlled trials.

While there are a number of different ways to match treated and non-treated units, the most common method is to use the propensity score:

.. math::
   e_i(X_i) = P(W_i = 1 \mid X_i)

Treated and non-treated units are then matched in terms of :math:`e(X)` using some criterion of distance, such as :math:`k:1` nearest neighbours. Because matching is usually between the treated population and the control, this method estimates the average treatment effect on the treated (ATT):

.. math::
   \mathbb{E}[Y(1) \mid W = 1] - \mathbb{E}[Y(0) \mid W = 1]

See :cite:`stuart2010matching` for a discussion of the strengths and weaknesses of the different matching methods.

Inverse probability of treatment weighting
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The inverse probability of treatment weighting (IPTW) approach uses the propensity score :math:`e` to weigh the treated and non-treated populations by the inverse of the probability of the actual treatment :math:`W`. For a binary treatment :math:`W \in \{1, 0\}`:

.. math::
   \frac{W}{e} + \frac{1 - W}{1 - e}

In this way, the IPTW approach can be seen as creating an artificial population in which the treated and non-treated units are similar in terms of their observed features :math:`X`.

One of the possible benefits of IPTW compared to matching is that less data may be discarded due to lack of overlap between treated and non-treated units. A known problem with the approach is that extreme propensity scores can generate highly variable estimators. Different methods have been proposed for trimming and normalizing the IPT weights (:cite:`https://doi.org/10.1111/1468-0262.00442`). An overview of the IPTW approach can be found in :cite:`https://doi.org/10.1002/sim.6607`.

2-Stage Least Squares (2SLS)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

One of the basic requirements for identifying the treatment effect of :math:`W` on :math:`Y` is that :math:`W` is orthogonal to the potential outcome of :math:`Y`, conditional on the covariates :math:`X`. This may be violated if both :math:`W` and :math:`Y` are affected by an unobserved variable, the error term after removing the true effect of :math:`W` from :math:`Y`, that is not in :math:`X`. In this case, the instrumental variables approach attempts to estimate the effect of :math:`W` on :math:`Y` with the help of a third variable :math:`Z` that is correlated with :math:`W` but is uncorrelated with the error term. In other words, the instrument :math:`Z` is only related with :math:`Y` through the directed path that goes through :math:`W`. If these conditions are satisfied, in the case without covariates, the effect of :math:`W` on :math:`Y` can be estimated using the sample analog of:

.. math::
   \frac{Cov(Y_i, Z_i)}{Cov(W_i, Z_i)}

The most common method for instrumental variables estimation is the two-stage least squares (2SLS). In this approach, the cause variable :math:`W` is first regressed on the instrument :math:`Z`. Then, in the second stage, the outcome of interest :math:`Y` is regressed on the predicted value from the first-stage model. Intuitively, the effect of :math:`W` on :math:`Y` is estimated by using only the proportion of variation in :math:`W` due to variation in :math:`Z`. Specifically, assume that we have the linear model

.. math::
   Y = W \alpha + X \beta + u = \Xi \gamma + u

Here for convenience we let :math:`\Xi=[W, X]` and :math:`\gamma=[\alpha', \beta']'`. Assume that we have instrumental variables :math:`Z` whose number of columns is at least the number of columns of :math:`W`, let :math:`\Omega=[Z, X]`, 2SLS estimator is as follows

.. math::
   \hat{\gamma}_{2SLS} = \left[\Xi'\Omega (\Omega'\Omega)^{-1} \Omega' \Xi\right]^{-1}\left[\Xi'\Omega(\Omega'\Omega)^{-1}\Omega'Y\right].

See :cite:`10.1257/jep.15.4.69` for a detailed discussion of the method.

LATE
~~~~

In many situations the treatment :math:`W` may depend on subject's own choice and cannot be administered directly in an experimental setting. However one can randomly assign users into treatment/control groups so that users in the treatment group can be nudged to take the treatment. This is the case of noncompliance, where users may fail to comply with their assignment status, :math:`Z`, as to whether to take treatment or not. In general there are 3 types of users in this situation, classified by whether they *take* the treatment in response to their assignment:

* **Compliers** Those who will take the treatment if and only if they are assigned to the treatment group.
* **Always-Takers** Those who will take the treatment regardless of which group they are assigned to.
* **Never-Takers** Those who will not take the treatment regardless of which group they are assigned to.

However one assumes that there is no Defier for identification purposes, i.e. those who will only take the treatment if they are assigned to the control group.

In this case one can measure the treatment effect of Compliers,

.. math::
   \hat{\tau}_{Complier}=\frac{E[Y|Z=1]-E[Y|Z=0]}{E[W|Z=1]-E[W|Z=0]}

This is Local Average Treatment Effect (LATE). The estimator is also equivalent to 2SLS if we take the assignment status, :math:`Z`, as an instrument.


Targeted maximum likelihood estimation (TMLE) for ATE
-----------------------------------------------------

Targeted maximum likelihood estimation (TMLE) :cite:`tmle` provides a doubly robust semiparametric method that "targets" directly on the average treatment effect with the aid from machine learning algorithms. Compared to other methods including outcome regression and inverse probability of treatment weighting, TMLE usually gives better performance especially when dealing with skewed treatment and outliers.

Given binary treatment :math:`W`, covariates :math:`X`, and outcome :math:`Y`, the TMLE for ATE is performed in the following steps

**Step 1**

Use cross fit to estimate the propensity score :math:`\hat{e}(x)`, the predicted outcome for treated :math:`\hat{m}_1(x)`, and predicted outcome for control :math:`\hat{m}_0(x)` with machine learning.

**Step 2**

Scale :math:`Y` into :math:`\tilde{Y}=\frac{Y-\min Y}{\max Y - \min Y}` so that :math:`\tilde{Y} \in [0,1]`. Use the same scale function to transform :math:`\hat{m}_i(x)` into :math:`\tilde{m}_i(x)`, :math:`i=0,1`. Clip the scaled functions so that their values stay in the unit interval.

**Step 3**

Let :math:`Q=\log(\tilde{m}_W(X)/(1-\tilde{m}_W(X)))`. Maximize the following pseudo log-likelihood function

.. math::
   \max_{h_0, h_1} -\frac{1}{N} \sum_i & \left[ \tilde{Y}_i \log \left(1+\exp\left(-Q_i-h_0 \frac{1-W}{1-\hat{e}(X_i)}-h_1 \frac{W}{\hat{e}(X_i)}\right) \right) \right. \\
   &\quad\left.+(1-\tilde{Y}_i)\log\left(1+\exp\left(Q_i+h_0\frac{1-W}{1-\hat{e}(X_i)}+h_1\frac{W}{\hat{e}(X_i)}\right)\right)\right]

**Step 4**

Let

.. math::
   \tilde{Q}_0 &= \frac{1}{1+\exp\left(-Q-h_0 \frac{1}{1-\hat{e}(X)}\right)},\\
   \tilde{Q}_1 &= \frac{1}{1+\exp\left(-Q-h_1 \frac{1}{\hat{e}(X)}\right)}.

The ATE estimate is the sample average of the differences of :math:`\tilde{Q}_1` and :math:`\tilde{Q}_0` after rescale to the original range.

---

### Migration (`docs/migration.rst`)

=================
Migration Guide
=================

.. _fit-argument-order:

``fit()`` argument order changes in v1.0
========================================

**If you only read one thing:** pass ``treatment`` and ``y`` as keyword
arguments, starting now.

.. code-block:: python

    # Before — breaks silently at v1.0
    learner.fit(X, treatment, y)

    # After — works identically before and after v1.0
    learner.fit(X=X, treatment=treatment, y=y)

That single edit is the whole migration. Keyword arguments are
order-independent, so a call written this way behaves the same in every version
and needs no second edit when the flip lands.

Why this is changing
--------------------

CausalML learners have always taken ``fit(X, treatment, y, ...)``, which puts
``y`` third. ``sklearn.pipeline.Pipeline`` calls its final estimator's
``fit(X, y)`` positionally, so a CausalML learner cannot be a pipeline step
without an adapter — the problem reported in `#854
<https://github.com/uber/causalml/issues/854>`_.

In v1.0 the positional order becomes ``fit(X, y, treatment, ...)``, matching the
scikit-learn convention.

.. warning::

    This change cannot be detected at runtime. ``y`` and ``treatment`` are both
    same-length arrays, so a call left in the old positional form will **not**
    raise ``TypeError`` after the flip — it will train on swapped arguments and
    return plausible-looking, wrong numbers.

    This is why the deprecation warns for a full release cycle before anything
    moves, and why keyword arguments are the recommended fix rather than
    "reorder your positional calls at v1.0".

What happens when
-----------------

**0.18.0 (October 2026), the release that ships this guide.** Positional order
is *unchanged*. Passing ``treatment`` or ``y`` positionally emits a
``FutureWarning`` pointing here. Nothing breaks.

**0.19.0 (December 2026) and 0.20.0 (March 2027).** Unchanged — the warning
stays, the signatures do not move.

**v1.0 (June 2027).** The signatures are reordered and the warning is removed.
Positional calls in the old order start silently mis-training; keyword calls are
unaffected.

That is **three minor releases and roughly nine months** of warning before
anything moves. If that is not enough for your codebase, say so on the `v1.0
roadmap discussion <https://github.com/uber/causalml/discussions/938>`_.

Release dates are targets rather than commitments; the *ordering* is the part
you can rely on, and no signature moves before v1.0.

The rule
--------

One rule, applied uniformly across the package:

    **X, then** ``y``, **then** ``treatment``\ **, and every other parameter
    keeps its relative position.**

So the two arguments move to the front in scikit-learn order, and everything
else — ``p``, ``sample_weight``, ``return_ci``, ``verbose`` — shifts right
without being reshuffled among itself.

The rule is implemented as ``v1_order()`` in ``causalml/inference/_arg_order.py``
and derived per method from that method's own signature, so the target for every
method is machine-checkable rather than hand-maintained.

Finding the calls you need to change
------------------------------------

Run your test suite or a representative script with the warning made fatal, so
each call site surfaces as a traceback:

.. code-block:: bash

    python -W error::FutureWarning your_script.py

That also promotes unrelated ``FutureWarning``\ s from pandas, NumPy and
scikit-learn, so on a large codebase it is usually easier to collect only this
deprecation and keep going:

.. code-block:: python

    import warnings

    with warnings.catch_warnings(record=True) as caught:
        warnings.simplefilter("always")
        ...  # your code
    for w in caught:
        if issubclass(w.category, FutureWarning) and "argument order" in str(w.message):
            print(f"{w.filename}:{w.lineno}")

The warning is raised with ``stacklevel=2``, so ``filename`` and ``lineno``
point at *your* call site, not at CausalML internals.

A grep is less reliable than it looks: ``fit(X, a, b)`` is also how you call a
plain scikit-learn estimator, and those are unaffected. Prefer the warning.

What changes, by family
-----------------------

Every public method whose positional order changes is listed below. ``predict``
is included deliberately: its current order does not block ``Pipeline``, since
``predict(X)`` already works, but flipping only ``fit`` would leave one class
with the same two arguments in opposite positional orders on two methods.

Meta-learners
~~~~~~~~~~~~~

``causalml.inference.meta`` — the S/T/X/R/DR learners and their classifier
variants, ``XGBTRegressor``, ``XGBRRegressor``, ``XGBTClassifier``,
``XGBRClassifier``, ``LRSRegressor``, ``TMLELearner``.

Methods: ``fit``, ``predict``, ``fit_predict``, ``estimate_ate``, ``bootstrap``,
``fit_bootstrap_ensemble``.

.. code-block:: text

    now:  (X, treatment, y, p, ...)
    v1.0: (X, y, treatment, p, ...)

Causal trees and forests
~~~~~~~~~~~~~~~~~~~~~~~~

``CausalTreeRegressor``, ``CausalRandomForestRegressor``.

Methods: ``fit``, ``fit_predict``, ``estimate_ate``, ``bootstrap``,
``bootstrap_pool``.

.. code-block:: text

    now:  (X, treatment, y, sample_weight, ...)
    v1.0: (X, y, treatment, sample_weight, ...)

Uplift trees and forests
~~~~~~~~~~~~~~~~~~~~~~~~

``UpliftTreeClassifier``, ``UpliftRandomForestClassifier``.

Methods: ``fit``, ``fill``, ``prune``.

``UpliftTreeClassifier.fit`` also takes a validation triple, which is reordered
in place so it stays consistent with the main one:

.. code-block:: text

    now:  (X, treatment, y, X_val, treatment_val, y_val, sample_weight, check_input)
    v1.0: (X, y, treatment, X_val, y_val, treatment_val, sample_weight, check_input)

Instrumental variables
~~~~~~~~~~~~~~~~~~~~~~

Two signatures here are **not** a plain ``treatment``/``y`` swap.

``IVRegressor.fit`` — the instrument ``w`` keeps its relative position and
shifts right:

.. code-block:: text

    now:  (X, treatment, y, w)
    v1.0: (X, y, treatment, w)

``BaseDRIVLearner`` (``fit``, ``fit_predict``, ``estimate_ate``, ``bootstrap``)
— ``assignment`` currently sits *second*, and lands fourth:

.. code-block:: text

    now:  (X, assignment, treatment, y, p, pZ, ...)
    v1.0: (X, y, treatment, assignment, p, pZ, ...)

``BaseDRIVLearner.predict`` takes no ``assignment`` and follows the ordinary
rule: ``(X, treatment, y, ...)`` becomes ``(X, y, treatment, ...)``.

Neural estimators
~~~~~~~~~~~~~~~~~

``DragonNet`` (TensorFlow and JAX backends) and ``CEVAE`` (PyTorch and JAX
backends). Methods: ``fit``, ``predict``, ``fit_predict``.

.. code-block:: text

    now:  (X, treatment, y, p)
    v1.0: (X, y, treatment, p)

Optional backends are only importable with the matching extra installed, so
these calls may not warn in an environment where the backend is missing. They
change all the same.

Policy learning
~~~~~~~~~~~~~~~

``PolicyLearner.fit``:

.. code-block:: text

    now:  (X, treatment, y, p, dhat)
    v1.0: (X, y, treatment, p, dhat)

Sensitivity analysis helpers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``causalml.metrics.sensitivity.Sensitivity`` — ``get_prediction``,
``get_ate_ci``, ``get_potential_outcome_predictions``. These are a third shape,
with ``p`` currently second; it shifts right:

.. code-block:: text

    now:  (X, p, treatment, y)
    v1.0: (X, y, treatment, p)

``Sensitivity.summary`` and ``sensitivity_analysis`` read ``treatment`` and
``y`` from a DataFrame by column name and are unaffected.

Deliberately unchanged
----------------------

A method is in scope only if **both** ``y`` and ``treatment`` appear after the
first position. Anything else keeps its current signature at v1.0:

* ``BaseRLearner.predict(X, p, return_components)`` — takes neither, so there is
  nothing to reorder. After the flip, ``BaseRLearner.fit`` and ``.predict``
  legitimately differ; that is not an oversight.
* ``ElasticNetPropensityModel`` and the other ``PropensityModel`` classes —
  ``fit(X, y)`` and ``fit_predict(X, y)`` take no ``treatment`` and already
  follow the scikit-learn convention.
* Dataset helpers that name their treatment vector something other than
  ``treatment``, such as ``SemiSynthDataGenerator.fit(X, w, y)``.
* Any plain scikit-learn estimator you pass *into* a CausalML learner. Only
  CausalML's own signatures change.

If a method takes only ``y`` and no ``treatment``, this deprecation does not
apply to it.

---

### Quickstart (`docs/quickstart.rst`)

API Quickstart
==============

Working example notebooks are available in the `example folder <https://github.com/uber/causalml/tree/master/docs/examples>`_.

Propensity Score
----------------

Propensity Score Estimation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from causalml.propensity import ElasticNetPropensityModel

    pm = ElasticNetPropensityModel(n_fold=5, random_state=42)
    # The second argument is the treatment indicator, not the outcome: a
    # propensity score is P(W = 1 | X).
    ps = pm.fit_predict(X, treatment)

Propensity Score Matching
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from causalml.match import NearestNeighborMatch, create_table_one

    psm = NearestNeighborMatch(replace=False,
                               ratio=1,
                               random_state=42)
    matched = psm.match_by_group(data=df,
                                 treatment_col=treatment_col,
                                 score_cols=score_cols,
                                 groupby_col=groupby_col)

    create_table_one(data=matched,
                     treatment_col=treatment_col,
                     features=covariates)

Average Treatment Effect (ATE) Estimation
-----------------------------------------

Meta-learners and Uplift Trees
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In addition to the Methodology section, you can find examples in the links below for :ref:`Meta-Learner Algorithms <methodology:Meta-Learner Algorithms>` and :ref:`Tree-Based Algorithms <methodology:Tree-Based Algorithms>`

- Meta-learners (S/T/X/R): `meta_learners_with_synthetic_data.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/meta_learners_with_synthetic_data.ipynb>`_
- Meta-learners (S/T/X/R) with multiple treatment: `meta_learners_with_synthetic_data_multiple_treatment.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/meta_learners_with_synthetic_data_multiple_treatment.ipynb>`_
- Comparing meta-learners across simulation setups: `benchmark_simulation_studies.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/benchmark_simulation_studies.ipynb>`_
- Doubly Robust (DR) learner: `dr_learner_with_synthetic_data.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/dr_learner_with_synthetic_data.ipynb>`_
- TMLE learner: `validation_with_tmle.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/validation_with_tmle.ipynb>`_
- Uplift Trees: `uplift_trees_with_synthetic_data.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/uplift_trees_with_synthetic_data.ipynb>`_

.. code-block:: python

    from causalml.inference.meta import LRSRegressor
    from causalml.inference.meta import XGBTRegressor, MLPTRegressor
    from causalml.inference.meta import BaseXRegressor
    from causalml.inference.meta import BaseRRegressor
    from xgboost import XGBRegressor
    from causalml.dataset import synthetic_data

    y, X, treatment, _, _, e = synthetic_data(mode=1, n=1000, p=5, sigma=1.0)

    lr = LRSRegressor()
    te, lb, ub = lr.estimate_ate(X=X, treatment=treatment, y=y)
    print('Average Treatment Effect (Linear Regression): {:.2f} ({:.2f}, {:.2f})'.format(te[0], lb[0], ub[0]))

    xg = XGBTRegressor(random_state=42)
    te, lb, ub = xg.estimate_ate(X=X, treatment=treatment, y=y)
    print('Average Treatment Effect (XGBoost): {:.2f} ({:.2f}, {:.2f})'.format(te[0], lb[0], ub[0]))

    nn = MLPTRegressor(hidden_layer_sizes=(10, 10),
                     learning_rate_init=.1,
                     early_stopping=True,
                     random_state=42)
    te, lb, ub = nn.estimate_ate(X=X, treatment=treatment, y=y)
    print('Average Treatment Effect (Neural Network (MLP)): {:.2f} ({:.2f}, {:.2f})'.format(te[0], lb[0], ub[0]))

    xl = BaseXRegressor(learner=XGBRegressor(random_state=42))
    te, lb, ub = xl.estimate_ate(X=X, treatment=treatment, y=y, p=e)
    print('Average Treatment Effect (BaseXRegressor using XGBoost): {:.2f} ({:.2f}, {:.2f})'.format(te[0], lb[0], ub[0]))

    rl = BaseRRegressor(learner=XGBRegressor(random_state=42))
    te, lb, ub =  rl.estimate_ate(X=X, p=e, treatment=treatment, y=y)
    print('Average Treatment Effect (BaseRRegressor using XGBoost): {:.2f} ({:.2f}, {:.2f})'.format(te[0], lb[0], ub[0]))


More algorithms
----------------

Treatment optimization algorithms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We have developed :ref:`Counterfactual Unit Selection <methodology:Counterfactual Unit Selection>` and :ref:`Counterfactual Value Estimator <methodology:Counterfactual Value Estimator>` methods, please find the code snippet below and details in the following notebooks:

- `counterfactual_unit_selection.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/counterfactual_unit_selection.ipynb>`_
- `counterfactual_value_optimization.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/counterfactual_value_optimization.ipynb>`_

.. code-block:: python

    from causalml.optimize import CounterfactualValueEstimator
    from causalml.optimize import get_treatment_costs, get_actual_value

    # load data set and train test split
    df_train, df_test = train_test_split(df)
    train_idx = df_train.index
    test_idx = df_test.index
    # some more code here to initiate and train the Model, and produce tm_pred
    # please refer to the counterfactual_value_optimization notebook for complete example

    # run the counterfactual calculation with TwoModel prediction
    cve = CounterfactualValueEstimator(treatment=df_test['treatment_group_key'],
                                       control_name='control',
                                       treatment_names=conditions[1:],
                                       y_proba=y_proba,
                                       cate=tm_pred,
                                       value=conversion_value_array[test_idx],
                                       conversion_cost=cc_array[test_idx],
                                       impression_cost=ic_array[test_idx])

    cve_best_idx = cve.predict_best()
    cve_best = [conditions[idx] for idx in cve_best_idx]
    actual_is_cve_best = df.loc[test_idx, 'treatment_group_key'] == cve_best
    cve_value = actual_value.loc[test_idx][actual_is_cve_best].mean()

    labels = [
        'Random allocation',
        'Best treatment',
        'T-Learner',
        'CounterfactualValueEstimator'
    ]
    values  = [
        random_allocation_value,
        best_ate_value,
        tm_value,
        cve_value
    ]
    # plot the result
    plt.bar(labels, values)

.. image:: ./_static/img/counterfactual_value_optimization.png
    :width: 629

Instrumental variables algorithms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- 2-Stage Least Squares (2SLS): `iv_nlsym_synthetic_data.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/iv_nlsym_synthetic_data.ipynb>`_


Neural network based algorithms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- CEVAE: `cevae_example.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/cevae_example.ipynb>`_
- DragonNet: `dragonnet_example.ipynb <https://github.com/uber/causalml/blob/master/docs/examples/dragonnet_example.ipynb>`_


Interpretation
----------------
Please see :ref:`Interpretable Causal ML ` section

Validation
----------

Please see :ref:`Validation <validation:Validation>` section


Synthetic Data Generation Process
---------------------------------

Single Simulation
~~~~~~~~~~~~~~~~~

.. code-block:: python

  from causalml.dataset import *

  # Generate synthetic data for single simulation
  y, X, treatment, tau, b, e = synthetic_data(mode=1)
  y, X, treatment, tau, b, e = simulate_nuisance_and_easy_treatment()

  # Generate predictions for single simulation
  single_sim_preds = get_synthetic_preds(simulate_nuisance_and_easy_treatment, n=1000)

  # Generate multiple scatter plots to compare learner performance for a single simulation
  scatter_plot_single_sim(single_sim_preds)

  # Visualize distribution of learner predictions for a single simulation
  distr_plot_single_sim(single_sim_preds, kind='kde')

.. image:: ./_static/img/synthetic_dgp_scatter_plot.png
    :width: 629


Multiple Simulations
~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

  from causalml.dataset import *

  # Generalize performance summary over k simulations
  num_simulations = 12
  preds_summary = get_synthetic_summary(simulate_nuisance_and_easy_treatment, n=1000, k=num_simulations)

  # Generate scatter plot of performance summary
  scatter_plot_summary(preds_summary, k=num_simulations)

  # Generate bar plot of performance summary
  bar_plot_summary(preds_summary, k=num_simulations)


.. image:: ./_static/img/synthetic_dgp_scatter_plot_multiple.png
    :width: 629

.. image:: ./_static/img/synthetic_dgp_bar_plot_multiple.png
    :width: 629

Sensitivity Analysis
---------------------------

For more details, please refer to the `sensitivity_example_with_synthetic_data.ipynb notebook <https://github.com/uber/causalml/blob/master/docs/examples/sensitivity_example_with_synthetic_data.ipynb>`_.

.. code-block:: python

    from causalml.metrics.sensitivity import Sensitivity
    from causalml.metrics.sensitivity import SensitivitySelectionBias
    from causalml.inference.meta import BaseXLearner
    from sklearn.linear_model import LinearRegression

    # Calling the Base XLearner class and return the sensitivity analysis summary report
    learner_x = BaseXLearner(LinearRegression())
    sens_x = Sensitivity(df=df, inference_features=INFERENCE_FEATURES, p_col='pihat',
                         treatment_col=TREATMENT_COL, outcome_col=OUTCOME_COL, learner=learner_x)
    # Here for Selection Bias method will use default one-sided confounding function and alpha (quantile range of outcome values) input
    sens_summary_x = sens_x.sensitivity_analysis(methods=['Placebo Treatment',
                                                         'Random Cause',
                                                         'Subset Data',
                                                         'Random Replace',
                                                         'Selection Bias'], sample_size=0.5)

    # Selection Bias: Alignment confounding Function
    sens_x_bias_alignment = SensitivitySelectionBias(df, INFERENCE_FEATURES, p_col='pihat', treatment_col=TREATMENT_COL,
                                                 outcome_col=OUTCOME_COL, learner=learner_x, confound='alignment',
                                                 alpha_range=None)
    # Plot the results by rsquare with partial r-square results by each individual features
    sens_x_bias_alignment.plot(lls_x_bias_alignment, partial_rsqs_x_bias_alignment, type='r.squared', partial_rsqs=True)


.. image:: ./_static/img/sensitivity_selection_bias_r2.png
    :width: 629

Feature Selection
---------------------------

For more details, please refer to the `feature_selection.ipynb notebook <https://github.com/uber/causalml/blob/master/docs/examples/feature_selection.ipynb>`_ and the associated paper, :cite:`zhao2020feature`.

.. code-block:: python

    from causalml.feature_selection.filters import FilterSelect
    from causalml.dataset import make_uplift_classification

    # define parameters for simulation
    y_name = 'conversion'
    treatment_group_keys = ['control', 'treatment1']
    n = 100000
    n_classification_features = 50
    n_classification_informative = 10
    n_classification_repeated = 0
    n_uplift_increase_dict = {'treatment1': 8}
    n_uplift_decrease_dict = {'treatment1': 4}
    delta_uplift_increase_dict = {'treatment1': 0.1}
    delta_uplift_decrease_dict = {'treatment1': -0.1}

    # make a synthetic uplift data set
    random_seed = 20200808
    df, X_names = make_uplift_classification(
        treatment_name=treatment_group_keys,
        y_name=y_name,
        n_samples=n,
        n_classification_features=n_classification_features,
        n_classification_informative=n_classification_informative,
        n_classification_repeated=n_classification_repeated,
        n_uplift_increase_dict=n_uplift_increase_dict,
        n_uplift_decrease_dict=n_uplift_decrease_dict,
        delta_uplift_increase_dict = delta_uplift_increase_dict,
        delta_uplift_decrease_dict = delta_uplift_decrease_dict,
        random_seed=random_seed
    )

    # Feature selection with Filter method
    filter_f = FilterSelect()
    method = 'F'
    f_imp = filter_f.get_importance(df, X_names, y_name, method,
                          treatment_group = 'treatment1')
    print(f_imp)

    # Use likelihood ratio test method
    method = 'LR'
    lr_imp = filter_f.get_importance(df, X_names, y_name, method,
                          treatment_group = 'treatment1')
    print(lr_imp)

    # Use KL divergence method
    method = 'KL'
    kl_imp = filter_f.get_importance(df, X_names, y_name, method,
                          treatment_group = 'treatment1',
                          n_bins=10)
    print(kl_imp)

---

### References (`docs/references.rst`)

References
==========

Open Source Software Projects
-----------------------------

Python Packages
~~~~~~~~~~~~~~~

- `DoWhy <https://github.com/Microsoft/dowhy>`_: a package for causal inference based on causal graphs.
- `CausalLift <https://github.com/Minyus/causallift/>`_: a package for uplift modeling based on T-learner :cite:`kunzel2019metalearners`.
- `PyLift <https://github.com/wayfair/pylift>`_: a package for uplift modeling based on the transformed outcome method in :cite:`athey2016recursive`.
- `EconML <https://github.com/Microsoft/EconML>`_: a package for treatment effect estimation with orthogonal random forest :cite:`oprescu2018orthogonal`, DeepIV :cite:`hartford2017deep` and other ML methods.

R Packages
~~~~~~~~~~

- `uplift <https://cran.r-project.org/web/packages/uplift/index.html>`_: a package for treatment effect estimation with ML.
- `grf <https://github.com/grf-labs/grf>`_: a package for forest-based honest estimation from :cite:`athey2019generalized`.

Papers
------

.. bibliography:: refs.bib
    :style: plain

---

### Release Notes (`docs/release_notes.rst`)

=============
Release Notes
=============

What changed in each release, and the guides for transitions that need code
changes on the caller's side.

.. toctree::
    :maxdepth: 2

    changelog
    migration

---

### Tutorial (`docs/tutorial.rst`)

=========================================================
Estimating and Validating Heterogeneous Treatment Effects
=========================================================

This walkthrough trains four CATE estimators from different families -- two
meta-learners, a causal forest, and a neural network -- on the same data and
then does the part that is genuinely hard in causal ML: deciding which of them
to believe. The data is the IHDP benchmark, where the per-unit ground truth is
known -- so the validation methods you would use on real data can themselves
be checked against the right answer. Each step names the User Guide page that
covers it in depth. The dataset downloads on first use and is cached locally
(see :doc:`datasets`).

Step 1: State the question
==========================

The Infant Health and Development Program (IHDP) benchmark
:cite:`hill2011bayesian` starts from a real randomized trial of home visits
for premature infants, with a child's cognitive test score as the outcome. Two
modifications made it the standard testbed for heterogeneous-effect
estimation: a nonrandom subset of the treated group was removed, so treatment
assignment is confounded the way observational data is, and the outcomes are
simulated from the real covariates -- so both potential outcomes, and
therefore every unit's true effect, are known. Each of its 100 replications
simulates new outcomes for the same 747 units and draws its own 672/75
train-test split.

Step 2: Load the data
=====================

.. code-block:: python

    import numpy as np
    import pandas as pd
    from causalml.dataset import fetch_ihdp

    RS = 42
    np.random.seed(RS)

    train = fetch_ihdp(replication=0, split="train")
    test = fetch_ihdp(replication=0, split="test")
    X_tr = pd.DataFrame(train.data, columns=train.feature_names)
    w_tr, y_tr, tau_tr = train.treatment, train.target, train.tau
    X_te = pd.DataFrame(test.data, columns=test.feature_names)
    w_te, y_te, tau_te = test.treatment, test.target, test.tau

    print(X_tr.shape, w_tr.sum(), X_te.shape, w_te.sum())
    print("true ATE: %.3f | sd of tau: %.3f" % (tau_tr.mean(), tau_tr.std()))

.. code-block:: text

    (672, 25) 123 (75, 25) 16
    true ATE: 4.012 | sd of tau: 0.866

672 training units with 123 treated, 25 covariates, and -- because this is a
benchmark -- the true effect of every unit, held aside for scoring.

Step 3: State the identification and check overlap
==================================================

Before any estimation, state what makes the effect identifiable at all: we
assume treatment assignment is *unconfounded* given the 25 observed covariates
-- adjusting for them blocks every path by which assignment and outcome are
jointly determined, so the causal effect can be recovered from observational
data. Every estimator below stands on this assumption. On this benchmark it
holds by construction, because the outcomes were simulated from the observed
covariates alone; on real data it is untestable, which is why Step 8 stresses
it rather than verifying it.

Identification also needs *overlap*: treated and untreated units throughout
the covariate space (see
:ref:`Checking Overlap <validation:Checking Overlap>`). Estimate each unit's
probability of treatment -- the propensity score -- and compare its
distribution across groups. One practical note: on this data the model's
default cross-validation grid selects a penalty that collapses every score to
the treated share, so the grid is widened explicitly.

.. code-block:: python

    from causalml.propensity import ElasticNetPropensityModel

    pm = ElasticNetPropensityModel(Cs=np.logspace(0, 3, 8), random_state=RS)
    pm.fit(X_tr, w_tr)
    p_tr, p_te = pm.predict(X_tr), pm.predict(X_te)

.. image:: ./_static/img/tutorial_overlap.png
    :width: 629
    :alt: Propensity-score distributions with the treatment group shifted right and a mass of control units near zero.

The two distributions overlap over most of the range -- estimation is possible
-- but they are far from identical: the confounding introduced by removing
part of the treated group is exactly what this picture shows, and a spike of
near-zero-propensity controls marks a region with almost no treated
counterparts. On a randomized experiment this plot is flat.

Step 4: Train four estimators
=============================

One estimator from each corner of the library: an
:ref:`X-Learner <methodology:X-Learner>` and an
:ref:`R-Learner <methodology:R-Learner>` wrapping the same gradient-boosted
base learner, a :ref:`causal random forest <methodology:Honest estimation>`
with its default honest estimation, and :ref:`DragonNet
<methodology:DragonNet>`, a neural network built for exactly this kind of
problem. (DragonNet needs an optional extra -- ``pip install causalml[jax]``
for the implementation used here, or ``causalml[tf]`` for the TensorFlow one;
see :ref:`Installation <installation:Installation>`.)

.. code-block:: python

    from xgboost import XGBRegressor
    from causalml.inference.meta import BaseXRegressor, BaseRRegressor
    from causalml.inference.tree import CausalRandomForestRegressor
    from causalml.inference.jax import DragonNet

    learners = {
        "X-learner": BaseXRegressor(learner=XGBRegressor(random_state=RS)),
        "R-learner": BaseRRegressor(learner=XGBRegressor(random_state=RS)),
        "Causal RF": CausalRandomForestRegressor(random_state=RS),
        "DragonNet": DragonNet(verbose=False, seed=RS),
    }

    def fit_all(learners, X, w, y, p):
        for name, m in learners.items():
            if name in ("X-learner", "R-learner"):
                m.fit(X=X, treatment=w, y=y, p=p)
            else:
                m.fit(X=X, treatment=w, y=y)

    def predict_all(learners, X, p):
        out = {}
        for name, m in learners.items():
            if name == "X-learner":
                out[name] = m.predict(X=X, p=p).flatten()
            elif name == "DragonNet":
                out[name] = np.asarray(m.predict_tau(X)).flatten()
            else:
                out[name] = m.predict(X).flatten()
        return out

    fit_all(learners, X_tr, w_tr, y_tr, p_tr)

    print("ATE (truth: %.3f):" % tau_tr.mean())
    ate_x = learners["X-learner"].estimate_ate(X=X_tr, treatment=w_tr, y=y_tr,
                                               p=p_tr, pretrain=True)
    print("  X-learner  %.2f (%.2f, %.2f)" % ate_x)
    ate_r = learners["R-learner"].estimate_ate(X=X_tr, treatment=w_tr, y=y_tr,
                                               p=p_tr, pretrain=True)
    print("  R-learner  %.2f (%.2f, %.2f)" % ate_r)
    cate_tr = predict_all(learners, X_tr, p_tr)
    print("  Causal RF  %.2f (point estimate)" % cate_tr["Causal RF"].mean())
    print("  DragonNet  %.2f (point estimate)" % cate_tr["DragonNet"].mean())

.. code-block:: text

    ATE (truth: 4.012):
      X-learner  4.16 (4.07, 4.24)
      R-learner  4.14 (4.13, 4.15)
      Causal RF  3.99 (point estimate)
      DragonNet  3.87 (point estimate)

All four land near the truth. What each family reports differs -- the
meta-learners return an ATE interval from ``estimate_ate()``, the forest
offers per-prediction variances instead, and DragonNet reports a point
estimate (the full catalog is :doc:`inference`). Note the R-learner: the
narrowest interval on the table, and the only one that excludes the true
value. Precision is not accuracy, and nothing here says which estimator to
trust -- that takes the next two steps.

Step 5: Evaluate against the ground truth
=========================================

Predict each unit's effect on the held-out split and score it against the
truth: PEHE (precision in estimating heterogeneous effects -- the root mean
squared error of the per-unit estimates) and :func:`~causalml.metrics.ate_error`:

.. code-block:: python

    from causalml.metrics import pehe, ate_error

    cate_te = predict_all(learners, X_te, p_te)
    for name, c in cate_te.items():
        print("  %-10s PEHE %.3f  ate_error %+.3f"
              % (name, pehe(tau_te, c, squared=False), ate_error(tau_te, c)))

.. code-block:: text

      X-learner  PEHE 0.883  ate_error +0.164
      R-learner  PEHE 2.132  ate_error +0.395
      Causal RF  PEHE 0.797  ate_error +0.064
      DragonNet  PEHE 0.455  ate_error +0.034

With ground truth, evaluation is just measurement: DragonNet -- a network
whose architecture was designed against benchmarks like this one -- is the
most accurate, the forest and the X-learner follow, and the R-learner (with
this base learner and these defaults) is failing. On real data there is no
such measurement, which is the situation the next step simulates.

Step 6: Validate as if the truth were unknown
=============================================

Everything in this step uses only what real data provides: covariates,
treatment, outcome, and the models' predictions. The validation losses score
each model's predictions against a proxy for the true effect built by
cross-fitting on the held-out data -- the doubly robust (DR) pseudo-outcome
loss and the plug-in T-learner loss (see
:ref:`Model Selection with Validation Losses <validation:Model Selection with Validation Losses>`):

.. code-block:: python

    from causalml.metrics import dr_score, plug_in_t_score, rate_score

    df = pd.DataFrame({"y": y_te, "w": w_te, **cate_te})
    print(dr_score(df, X=X_te, outcome_col="y", treatment_col="w", p=p_te,
                   learner=XGBRegressor(random_state=RS),
                   return_ci=True, random_state=RS).round(3))
    print(plug_in_t_score(df, X=X_te, outcome_col="y", treatment_col="w",
                          learner=XGBRegressor(random_state=RS),
                          return_ci=True, random_state=RS).round(3))

.. code-block:: text

               dr_loss      se  ci_lower  ci_upper
    model
    X-learner   37.947  22.495    -6.142    82.036
    R-learner   44.465  24.535    -3.622    92.553
    Causal RF   38.482  22.476    -5.570    82.534
    DragonNet   40.687  24.687    -7.700    89.074

               plug_in_t_loss     se  ci_lower  ci_upper
    model
    X-learner           1.773  0.272     1.241     2.305
    R-learner           5.400  0.934     3.569     7.231
    Causal RF           1.396  0.244     0.917     1.875
    DragonNet           1.654  0.451     0.769     2.538

The plug-in loss, knowing nothing of the truth, reproduces its main verdict:
the R-learner is worst by a wide margin, and the other three sit within each
other's uncertainty. (It cannot resolve the top group's internal order -- it
puts the forest first where the truth puts DragonNet -- but it reliably
catches the failing model. The DR loss agrees on the ordering's tail but is
too noisy at this sample size to separate anything.)

The ranking metrics tell a sharper cautionary tale:

.. code-block:: python

    print(rate_score(df, outcome_col="y", treatment_col="w",
                     return_ci=True, random_state=RS).round(3))

.. code-block:: text

                rate     se  ci_lower  ci_upper  p_value
    model
    X-learner -0.166  0.191    -0.541     0.208    0.385
    R-learner  0.612  0.301     0.022     1.201    0.042
    Causal RF -0.143  0.291    -0.713     0.428    0.624
    DragonNet -0.080  0.183    -0.439     0.279    0.662

On 75 validation rows, the only nominally significant
:ref:`RATE <methodology:RATE>` belongs to the *worst* model in the lineup --
with four models tested at once, one accidental p < 0.05 is exactly what
noise produces. Small validation sets do not merely weaken rank-based
metrics; they can hand a significant-looking verdict to the wrong model. The
next step gives these metrics the sample size they need.

Step 7: Visualize the ranking with gain and TOC curves
======================================================

To see what the ranking metrics measure, pool the replication's 747 units and
re-split them in half, so the validation side has 374 rows instead of 75, and
refit the four learners on the other half:

.. code-block:: python

    import matplotlib.pyplot as plt
    from sklearn.model_selection import train_test_split
    from causalml.metrics import auuc_score, plot_gain, plot_toc

    X_all = pd.DataFrame(np.vstack([train.data, test.data]),
                         columns=train.feature_names)
    w_all = np.concatenate([train.treatment, test.treatment])
    y_all = np.concatenate([train.target, test.target])

    fit_idx, val_idx = train_test_split(np.arange(len(y_all)), test_size=0.5,
                                        random_state=RS, stratify=w_all)
    pm_v = ElasticNetPropensityModel(Cs=np.logspace(0, 3, 8), random_state=RS)
    pm_v.fit(X_all.iloc[fit_idx], w_all[fit_idx])
    p_fit = pm_v.predict(X_all.iloc[fit_idx])
    p_val = pm_v.predict(X_all.iloc[val_idx])

    learners_v = {
        "X-learner": BaseXRegressor(learner=XGBRegressor(random_state=RS)),
        "R-learner": BaseRRegressor(learner=XGBRegressor(random_state=RS)),
        "Causal RF": CausalRandomForestRegressor(random_state=RS),
        "DragonNet": DragonNet(verbose=False, seed=RS),
    }
    fit_all(learners_v, X_all.iloc[fit_idx], w_all[fit_idx],
            y_all[fit_idx], p_fit)
    cate_val = predict_all(learners_v, X_all.iloc[val_idx], p_val)

    df_val = pd.DataFrame({"y": y_all[val_idx], "w": w_all[val_idx], **cate_val})
    print(auuc_score(df_val, outcome_col="y", treatment_col="w",
                     return_ci=True, random_state=RS).round(3))

.. code-block:: text

                auuc     se  ci_lower  ci_upper
    model
    X-learner  0.527  0.014     0.500     0.554
    R-learner  0.525  0.013     0.499     0.552
    Causal RF  0.499  0.017     0.465     0.532
    DragonNet  0.540  0.013     0.514     0.566

The AUUC (area under the cumulative gain curve, normalized so 0.5 is random
targeting) now separates a little: DragonNet holds the largest margin over
random and is the only one whose interval clears 0.5, while the causal
forest's ranking is indistinguishable from random ordering. The gain curves
show how thin these margins are:

.. code-block:: python

    fig, ax = plt.subplots(figsize=(7, 4.2))
    plot_gain(df_val, outcome_col="y", treatment_col="w", ax=ax)

.. image:: ./_static/img/tutorial_gain.png
    :width: 629
    :alt: Cumulative gain curves with DragonNet slightly above the random diagonal and the causal forest tracking it.

Every curve stays close to the random diagonal, because most units share a
similar effect: ranking cannot beat random by much when there is little
spread to exploit. Where the advantage sits matters, and that is what the TOC
curve isolates -- TOC(q) is the excess effect among the top-q fraction over
the overall ATE:

.. code-block:: python

    fig, ax = plt.subplots(figsize=(7, 4.2))
    plot_toc(df_val, outcome_col="y", treatment_col="w", ax=ax)

.. image:: ./_static/img/tutorial_toc.png
    :width: 629
    :alt: TOC curves with the X-learner spiking early, DragonNet strongest through the middle, and the causal forest near zero.

The X-learner's curve spikes to about 1.5 in the top few percent and decays;
DragonNet's advantage is smaller there but spread across the middle of the
ranking; the forest's curve sits near zero throughout. :ref:`RATE
<methodology:RATE>` with its default ``autoc`` weighting integrates the TOC
with weight :math:`1/q`, so it rewards exactly the early concentration the
X-learner has:

.. code-block:: python

    print(rate_score(df_val, outcome_col="y", treatment_col="w",
                     return_ci=True, random_state=RS).round(3))

.. code-block:: text

                rate     se  ci_lower  ci_upper  p_value
    model
    X-learner  0.381  0.191     0.006     0.757    0.046
    R-learner -0.012  0.148    -0.301     0.277    0.935
    Causal RF -0.009  0.277    -0.553     0.534    0.974
    DragonNet  0.277  0.267    -0.246     0.800    0.299

With five times the validation rows of Step 6, the spurious R-learner signal
is gone and the X-learner's RATE excludes zero (p = 0.046): its top-ranked
units demonstrably benefit more than average. Note the division of labor the
two metrics just displayed -- AUUC favored DragonNet's broad, thin margin;
RATE favored the X-learner's concentrated one. Neither is wrong; they answer
different targeting questions, which is why both exist.

Step 8: Stress the assumptions
==============================

Every estimate above leans on unconfoundedness: that the 25 covariates capture
everything driving both treatment and outcome. Sensitivity analysis (see
:ref:`Validation with Sensitivity Analysis <validation:Validation with Sensitivity Analysis>`)
perturbs the analysis and re-estimates. Replacing the treatment with random
noise -- the placebo test -- should destroy the effect; adding a random
covariate or halving the sample should not:

.. code-block:: python

    from causalml.metrics.sensitivity import Sensitivity

    df_s = X_tr.assign(treatment=w_tr, outcome=y_tr, p=p_tr)
    sens = Sensitivity(df=df_s, inference_features=list(X_tr.columns),
                       p_col="p", treatment_col="treatment",
                       outcome_col="outcome",
                       learner=BaseXRegressor(learner=XGBRegressor(random_state=RS)))
    print(sens.sensitivity_analysis(
        methods=["Placebo Treatment", "Random Cause", "Subset Data"],
        sample_size=0.5).to_string())

.. code-block:: text

                              Method     ATE   New ATE  New ATE LB  New ATE UB
    0              Placebo Treatment  4.1561 -0.208613   -0.313350   -0.103875
    1                   Random Cause  4.1561  4.013964    3.933172    4.094756
    2  Subset Data(sample size @0.5)  4.1561  3.939882    3.817171    4.062592

The placebo collapses the estimate by 95% -- not exactly to zero, since a
flexible learner finds some structure even in noise, but to the far side of
negligible -- while the other two perturbations barely move it. The analysis
is behaving the way a real effect should.

Step 9: The replication protocol
================================

IHDP results are published as a mean and standard error *across* replications
-- a single replication is not comparable to a published number. The loop is
the unit of comparison:

.. code-block:: python

    rows = []
    for rep in range(10):
        train = fetch_ihdp(replication=rep, split="train")
        test = fetch_ihdp(replication=rep, split="test")
        ...  # refit the four learners, score PEHE on the test split
    print(pd.DataFrame(rows).groupby("model")["pehe"].agg(["mean", "sem"]).round(3))

.. code-block:: text

                mean    sem
    model
    Causal RF  5.458  3.343
    DragonNet  0.599  0.054
    R-learner  4.729  1.843
    X-learner  3.524  2.031

The loop rewrites the single-replication story. A few replications simulate
heavy-tailed outcomes, and they blow up every estimator except DragonNet,
whose mean stays at 0.6 with a standard error fifty times smaller than the
others' -- while the causal forest, second-best on replication 0, has the
worst mean of all. A ranking read off one replication does not survive the
protocol, which is the reason the protocol exists. The
:doc:`benchmark leaderboard <examples/benchmark_leaderboard>` is the canonical
version of this loop, running all 100 replications of the file CausalML ships.

One caution when comparing against the literature: the CEVAE
:cite:`louizos2017causal` and DragonNet :cite:`shi2019adapting` papers report
IHDP as a mean and standard error over the *1,000-replication* release of this
same data-generating process (and DragonNet reports only the ATE error, not
PEHE), so their table values are not directly comparable to numbers computed
on the 100-replication file.

Summary
=======

The workflow, in the order this page ran it: state the identification and
check overlap before estimating; estimate the ATE and treat interval width
with suspicion; measure per-unit accuracy against ground truth where it
exists; validate blind with the DR and plug-in losses, which reliably catch a
failing model; give rank-based metrics enough validation data before trusting
them -- at 75 rows RATE crowned the worst model, at 374 it found the real
signal; read gain and TOC curves together, because AUUC and RATE reward
different shapes of heterogeneity; stress unconfoundedness with sensitivity
analysis; and never report a benchmark number from a single replication.

On this data: DragonNet was the most accurate and most stable per-unit
estimator, the X-learner had the most concentrated targeting signal, the
causal forest was accurate on average but ranked no better than random, and
the R-learner's confident-looking ATE interval concealed the worst per-unit
estimates in the lineup. Every one of those verdicts required a different
tool, and none of them is visible in Step 4's table of four plausible ATEs.

Where to next
=============

* Which estimator fits your problem: :doc:`choosing_an_estimator`
* The mathematics of each method: :doc:`methodology`
* The full evaluation workflow: :doc:`validation`
* What uncertainty each estimator reports: :doc:`inference`
* Interpreting a fitted model: :doc:`interpretation`

---

### User Guide (`docs/user_guide.rst`)

==========
User Guide
==========

How to choose among the methods CausalML implements and the theory behind
them, how to interpret and validate what a fitted model produces and with what
uncertainty, the benchmark datasets with known ground truth, and worked
examples as runnable notebooks.

.. toctree::
    :maxdepth: 2

    choosing_an_estimator
    methodology
    interpretation
    validation
    inference
    datasets
    examples
    references

---

### Validation (`docs/validation.rst`)

==========
Validation
==========

Estimation of the treatment effect cannot be validated the same way as regular ML predictions because the true value is not available except for the experimental data. Here we focus on the internal validation methods under the assumption of unconfoundedness of potential outcomes and the treatment status conditioned on the feature set available to us.

This page is ordered the way an analysis uses these checks. Before fitting,
confirm the data can support causal estimation at all by checking overlap.
After fitting, evaluate on held-out data: ranking metrics (AUUC and Qini) show
whether the model orders units by benefit, :ref:`RATE <methodology:RATE>` tests
whether that ordering beats random targeting, and validation losses compare
candidate models against a proxy for the unobservable true effect. Where ground
truth exists -- synthetic data and the :doc:`benchmark datasets <datasets>` --
error can be measured directly. Finally, stress the unconfoundedness assumption
itself with sensitivity analysis.

Checking Overlap
----------------

Every method on this page assumes *overlap* (also called positivity): each unit
must have a non-zero probability of receiving each treatment, so that both
potential outcomes are represented at every point of the covariate space where
effects are estimated. Where one group is absent, a CATE estimate is
extrapolation, and no held-out metric can detect that it is wrong.

Estimate propensity scores (e.g. with ``ElasticNetPropensityModel``; see the
:ref:`Propensity Score <quickstart:Propensity Score>` section of the
quickstart) and compare their distributions in the treatment and control
groups. Scores piling up near 0 or 1 signal weak overlap: estimators that
weight by inverse propensity become unstable there, which is why methods that
consume propensity scores commonly clip them away from the boundaries (e.g.
the ``p_clip_bounds`` argument of ``dr_score()``).

Validation with Uplift Curve (AUUC)
-----------------------------------

We can validate the estimation by evaluating and comparing the uplift gains with AUUC (Area Under Uplift Curve), it calculates cumulative gains. Please find more details in `meta_learners_with_synthetic_data.ipynb example notebook <https://github.com/uber/causalml/blob/master/docs/examples/meta_learners_with_synthetic_data.ipynb>`_.

Compute ranking metrics on data the model was not fitted on: the split search
inside any learner favors partitions whose in-sample effects look large, so
in-sample curves overstate targeting quality. For the same reason, comparing
average effects between subgroups that were themselves formed from the model's
in-sample scores does not validate the model -- the groups were chosen by the
very estimates being tested.

.. code-block:: python

    from causalml.dataset import *
    from causalml.metrics import *
    # Single simulation
    train_preds, valid_preds = get_synthetic_preds_holdout(simulate_nuisance_and_easy_treatment,
                                                           n=50000,
                                                           valid_size=0.2)
    # Cumulative Gain AUUC values for a Single Simulation of Validation Data
    get_synthetic_auuc(valid_preds)


.. image:: ./_static/img/auuc_table_vis.png
    :width: 629

.. image:: ./_static/img/auuc_vis.png
    :width: 629

``auuc_score()`` and ``qini_score()`` accept ``return_ci=True`` to report a
bootstrap standard error and confidence interval alongside each score, so two
models can be compared with their uncertainty in view rather than by point
estimates alone.

For data with skewed treatment, it is sometimes advantageous to use :ref:`Targeted maximum likelihood estimation (TMLE) for ATE <methodology:Targeted maximum likelihood estimation (TMLE) for ATE>` to generate the AUUC curve for validation, as TMLE provides a more accurate estimation of ATE. Please find `validation_with_tmle.ipynb example notebook <https://github.com/uber/causalml/blob/master/docs/examples/validation_with_tmle.ipynb>`_ for details.

Testing for Heterogeneity with RATE
-----------------------------------

A model can produce a plausible-looking uplift curve on data with no real
heterogeneity. ``rate_score()`` turns the ranking into a hypothesis test: the
Rank-Weighted Average Treatment Effect is the weighted area under the Targeting
Operator Characteristic curve, and with ``return_ci=True`` a confidence
interval that excludes zero is evidence the model's prioritization beats
treating at random. Use ``weighting="autoc"`` when effects are expected to
concentrate in a small subgroup and ``weighting="qini"`` when they are diffuse.
The mathematics, the weighting trade-off and the bootstrap are documented in
:ref:`RATE <methodology:RATE>`.

Model Selection with Validation Losses
--------------------------------------

To choose among candidate CATE models, score each against a proxy for the true
effect built on held-out folds. ``dr_score()`` measures the mean squared error
against a cross-fitted doubly robust (AIPW) pseudo-outcome, and
``plug_in_t_score()`` measures it against a cross-fitted T-learner proxy; both
are losses, so lower is better, and both accept ``return_ci=True``. The
constructions and their assumptions are documented in
:ref:`DR pseudo-outcome loss <methodology:DR (Doubly Robust) pseudo-outcome loss>`
and :ref:`Plug-in T-learner loss <methodology:Plug-in T-learner loss>`.

Validation with Ground Truth
----------------------------

Where the true effect is known, error can be measured directly instead of
through proxies: ``pehe()`` (precision in estimating heterogeneous effects, the
mean squared error of the CATE estimates), ``ate_error()`` for the average
effect, and ``policy_risk()`` for the treat-if-positive policy on randomized
data. These apply to synthetic data, to the semi-synthetic and randomized
:doc:`benchmark datasets <datasets>`, and nowhere else -- none of them can be
computed on ordinary observational data. The
:doc:`benchmark leaderboard notebook <examples/benchmark_leaderboard>` runs
CausalML's estimators through these metrics end to end.

Validation with Synthetic Data Sets
-----------------------------------

We can test the methodology with simulations, where we generate data with known causal and non-causal links between the outcome, treatment and some of confounding variables.

We implemented the following sets of synthetic data generation mechanisms based on :cite:`nie2017quasi`:

Mechanism 1
~~~~~~~~~~~

| This generates a complex outcome regression model with easy treatment effect with input variables :math:`X_i \sim Unif(0, 1)^d`.
| The treatment flag is a binomial variable, whose d.g.p. is:
|
|   :math:`P(W_i = 1 | X_i) = trim_{0.1}(sin(\pi X_{i1} X_{i2}))`
|
| With :
|   :math:`trim_\eta(x)=\max (\eta,\min (x,1-\eta))`
|
| The outcome variable is:
|
|   :math:`y_i = sin(\pi X_{i1} X_{i2}) + 2(X_{i3} - 0.5)^2 + X_{i4} + 0.5 X_{i5} + (W_i - 0.5)(X_{i1} + X_{i2})/ 2 + \epsilon_i`
|

Mechanism 2
~~~~~~~~~~~

| This simulates a randomized trial. The input variables are generated by :math:`X_i \sim N(0, I_{d\times d})`
|
| The treatment flag is generated by a fair coin flip:
|
|   :math:`P(W_i = 1|X_i) = 0.5`
|
| The outcome variable is
|
|   :math:`y_i = max(X_{i1} + X_{i2}, X_{i3}, 0) + max(X_{i4} + X_{i5}, 0) + (W_i - 0.5)(X_{i1} + \log(1 + e^{X_{i2}}))`
|

Mechanism 3
~~~~~~~~~~~

| This one has an easy propensity score but a difficult control outcome. The input variables follow :math:`X_i \sim N(0, I_{d\times d})`
|
| The treatment flag is a binomial variable, whose d.g.p is:
|
|   :math:`P(W_i = 1 | X_i) = \frac{1}{1+\exp{X_{i2} + X_{i3}}}`
|
| The outcome variable is:
|
|   :math:`y_i = 2\log(1 + e^{X_{i1} + X_{i2} + X_{i3}}) + (W_i - 0.5)`
|

Mechanism 4
~~~~~~~~~~~

| This contains an unrelated treatment arm and control arm, with input data generated by :math:`X_i \sim N(0, I_{d\times d})`.
|
| The treatment flag is a binomial variable whose d.g.p. is:
|
|   :math:`P(W_i = 1 | X_i) = \frac{1}{1+\exp{-X_{i1}} + \exp{-X_{i2}}}`
|
| The outcome variable is:
|
|   :math:`y_i = \frac{1}{2}\big(max(X_{i1} + X_{i2} + X_{i3}, 0) + max(X_{i4} + X_{i5}, 0)\big) + (W_i - 0.5)(max(X_{i1} + X_{i2} + X_{i3}, 0) - max(X_{i4} + X_{i5}, 0))`
|

Validation with Multiple Estimates
----------------------------------

We can validate the methodology by comparing the estimates with other approaches, checking the consistency of estimates across different levels and cohorts.

Model Robustness for Meta Algorithms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In meta-algorithms we can assess the quality of user-level treatment effect estimation by comparing estimates from different underlying ML algorithms. We will report MSE, coverage (overlapping 95% confidence interval), uplift curve. In addition, we can split the sample within a cohort and compare the result from out-of-sample scoring and within-sample scoring.

User Level/Segment Level/Cohort Level Consistency
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We can also evaluate user-level/segment level/cohort level estimation consistency by conducting T-test.

Stability between Cohorts
~~~~~~~~~~~~~~~~~~~~~~~~~

Treatment effect may vary from cohort to cohort but should not be too volatile. For a given cohort, we will compare the scores generated by model fit to another score with the ones generated by its own model.

Validation with Sensitivity Analysis
------------------------------------
Sensitivity analysis aims to check the robustness of the unconfoundedness assumption. If there is hidden bias (unobserved confounders), it determines how severe that bias would have to be to change the conclusion, by examining the average treatment effect estimation.

We implemented the following methods to conduct sensitivity analysis. The heading of each is the string to pass to ``Sensitivity.sensitivity_analysis(methods=[...])``:

Placebo Treatment
~~~~~~~~~~~~~~~~~

| Replace treatment with a random variable.

Random Cause
~~~~~~~~~~~~

| Add a random common cause variable.

Subset Data
~~~~~~~~~~~

| Remove a random subset of the data.

Random Replace
~~~~~~~~~~~~~~

| Randomly replace a covariate with an irrelevant variable.

Selection Bias
~~~~~~~~~~~~~~

| `Blackwell (2013) <https://www.mattblackwell.org/files/papers/sens.pdf>`_ introduced an approach to sensitivity analysis for causal effects that directly models confounding or selection bias.
|
| One Sided Confounding Function: here as the name implies, this function can detect sensitivity to one-sided selection bias, but it would fail to detect other deviations from ignorability. That is, it can only determine the bias resulting from the treatment group being on average better off or the control group being on average better off.
|
| Alignment Confounding Function: this type of bias is likely to occur when units select into treatment and control based on their predicted treatment effects
|
| The sensitivity analysis is rigid in this way because the confounding function is not identified from the data, so that the causal model in the last section is only identified conditional on a specific choice of that function. The goal of the sensitivity analysis is not to choose the “correct” confounding function, since we have no way of evaluating this correctness. By its very nature, unmeasured confounding is unmeasured. Rather, the goal is to identify plausible deviations from ignorability and test sensitivity to those deviations. The main harm that results from the incorrect specification of the confounding function is that hidden biases remain hidden.

---

--- METRICS ---
- Files Extracted: 23
- Estimated Token Budget: ~54300 tokens
- Canonical Reference: https://codewiki.google/github.com/uber/causalml

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#causal-inference Browse causal-inference catalog → #incubation Browse incubation catalog → #machine-learning Browse machine-learning catalog → #uplift-modeling Browse uplift-modeling catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →