Helpful SciPy utilities and recipes
Go to file
RunasSudo b7a66849ff
Refactor and add test for correlation ratio (eta) in yli.auto_correlations
2024-05-16 23:11:43 +10:00
docs Add maxiter, tolerance parameters to IntervalCensoredCox 2023-07-16 16:20:39 +10:00
tests Refactor and add test for correlation ratio (eta) in yli.auto_correlations 2024-05-16 23:11:43 +10:00
yli Refactor and add test for correlation ratio (eta) in yli.auto_correlations 2024-05-16 23:11:43 +10:00
.coveragerc Update unit tests 2023-04-17 15:16:09 +10:00
.gitignore Update unit tests 2023-04-17 15:16:09 +10:00
COPYING Publish beta ratio distribution 2022-10-05 20:08:04 +11:00 Use hpstat for yli.turnbull to enable computation of confidence intervals 2023-10-20 21:12:55 +11:00

Helpful SciPy utilities and recipes

This is a collection of helpful utilities and recipes for biostatistics analyses in Python/SciPy. The objective is to (where possible) wrap existing implementations of statistical functions to provide a convenient and consistent interface, which reduces boilerplate, and reports results in a user-friendly standard format for medical research (cf. APA, AMA, NEJM style).

For example, compare the standard statsmodels code:

import statsmodels.api as sm
d1 = sm.stats.DescrStatsW(df[df['Type'] == 'Fresh']['Potency'])
d2 = sm.stats.DescrStatsW(df[df['Type'] == 'Stored']['Potency'])

cm = sm.stats.CompareMeans(d1, d2)
print(d1.mean - d2.mean)
(4.236832825824121, 0.0004959477708633275, 18.0)
(0.2722297224438056, 0.8077702775561927)

With the following:

yli.ttest_ind(df, 'Potency', 'Type')

t(18) = 4.24; p < 0.001*
Δμ (95% CI) = 0.54 (0.27–0.81), Fresh > Stored


This is a personal helper library, whose scope is limited to statistical functions and applications which are useful to me. It is not intended or expected that you will necessarily be able to import the library wholesale for your purposes. Rather, in the spirit of collaboration it is hoped that this library may contain examples and generally standalone implementations (hence ‘utilities and recipes’) which may be helpful for you to use or adapt.


The mandatory dependencies of this library are:

Optional dependencies are:

  • hpstat, for turnbull and IntervalCensoredCox
  • matplotlib and seaborn, for plotting functions
  • mpmath, for beta_ratio and beta_oddsratio
  • PyCryptodome, for pickle_write_encrypted and pickle_read_encrypted
  • rpy2, with R packages:
    • BFpack, for bayesfactor_afbf (RegressionModel.bayesfactor_beta_zero)
    • logistf, for PenalisedLogit
  • shap, for RegressionModel.shap


Relevant statistical functions are all directly available from the top-level yli namespace:

  • Significance testing:
    • anova_oneway: One-way ANOVA
    • chi2: Pearson χ2 test
    • mannwhitney: Mann–Whitney U test
    • pearsonr: Pearson correlation coefficient r
    • ttest_ind: Independent 2-sample t test
  • Regression:
    • IntervalCensoredCox: Model for interval-censored Cox regression
    • PenalisedLogit: Model for Firth penalised logistic regression
    • regress: Fit arbitrary regression models
    • vif: Compute the variance inflation factor for independent variables in regression
  • Survival analysis:
    • kaplanmeier: Kaplan–Meier plot
    • logrank: Log-rank test
    • turnbull: Turnbull estimator plot, including pointwise confidence intervals, for interval-censored data
  • Input/output:
    • pickle_write_compressed, pickle_read_compressed: Pickle a pandas DataFrame and compress using LZMA
    • pickle_write_encrypted, pickle_read_encrypted: Pickle a pandas DataFrame, compress using LZMA, and encrypt
  • Distributions:
    • beta_oddsratio: SciPy distribution for the odds ratio of 2 independent beta-distributed variables
    • beta_ratio: SciPy distribution for the ratio of 2 independent beta-distributed variables
    • hdi: Find the highest density interval (e.g. highest posterior density, HPD/HDI) for a SciPy distribution
    • transformed_dist: SciPy distribution for an arbitrary transformation of a SciPy distribution
  • Bayesian inference:
    • bayesfactor_afbf: Adjusted fractional Bayes factor for a hypothesis on parameters from regression

Documentation and examples

Each function is documented in the respective docstring within the source code, and Sphinx documentation is buildable from the docs directory. Hosted documentation is available at

Examples can be found in the unit tests in the tests directory.


No warranty is made as to the correctness of any function in this library. While the library is unit tested, the validation is not extensive. This applies particularly for functions which are more than simple wrappers for existing implementations. Please apply appropriate caution.