mutual_info module

Contains functions relating to calculating and plotting mutual information.

mutual_info.calculate_all_mi(data_path_fs, data_path_ma, n_fields, n_bandpowers, save_path, data_label='obs_bp', no_whiten=False, dilution_fac=1)

Calculate the mutual information (after whitening) for all pair populations within the full-sky and cut-sky simulated bandpowers output by simulation.combine_simbp_batches. Outputs one file per pair population.

Parameters

  • data_path_fs (str) – Path to full-sky simulated bandpowers.

  • data_path_ma (str) – Path to cut-sky simulated bandpowers.

  • n_fields (int) – Number of fields, such that the number of spectra = n_fields * (n_fields + 1) / 2.

  • n_bandpowers (int) – Number of bandpowers.

  • save_path (str) – Path to save output to, containing a placeholder {pop} which will be replaced with the population label.

  • data_label (str, optional) – Label of the simulated data within the input npz files. Defaults to ‘obs_bp’, but for Gaussian samples output by simulation.gaussian_samples this should be replaced by ‘gauss_bp’.

  • no_whiten (bool, optional) – If False, whitening is applied before mutual information estimation (default False).

  • dilution_fac (float, optional) – If > 1, the data sample will be randomly diluted by a factor of dilution_fac for faster but less accurate mutual information estimation. Default 1 (no dilution).

mutual_info.calculate_mi_slics_gauss(slics_filemask, gauss_path, n_tomo, n_bp, n_real, save_path, no_whiten=False, dilution_fac=1)

Calculate the mutual information (after whitening) for all pair populations within the SLICS and Gaussian field simulated bandpowers output by simulation.combine_slics and simulation.sim_flat. Outputs one file per pair population.

Parameters

  • slics_filemask (str) – Path to bandpowers from SLICS simulations, with {tomo} as a placeholder for tomographic bin.

  • gauss_path (str) – Path to bandpowers from Gaussian fields simulations.

  • n_tomo (int) – Number of redshift bins.

  • n_bp (int) – Number of bandpowers.

  • n_real (int) – Number of realisations.

  • save_path (str) – Path to save output to, containing a placeholder {pop} which will be replaced with the population label.

  • no_whiten (bool, optional) – If False, whitening is applied before mutual information estimation (default False).

  • dilution_fac (float, optional) – If > 1, the data sample will be randomly diluted by a factor of dilution_fac for faster but less accurate mutual information estimation. Default 1 (no dilution).

mutual_info.combine_mi_files(input_filemask, save_path)

Combine single-population mutual information files output by calculation_all_mi to a single file.

Parameters

  • input_filemask (str) – glob-compatible filemask matching single-population files, e.g. ‘mi_*.npz’.

  • save_path (str) – Path to save output to.

mutual_info.combine_mi_slics_gauss(input_filemask, save_path)

Alternative version of combine_mi_files for the output of calculate_mi_slics_gauss. Combine single-population mutual information files to a single file.

Parameters

  • input_filemask (str) – glob-compatible filemask matching single-population files, e.g. ‘mi_*.npz’.

  • save_path (str) – Path to save output to.

mutual_info.get_pair_idxs(spectra, bandpower, n_fields, n_bandpowers, spectrum_table=None, skip_dupe_check=False)

Return spectrum and bandpower indices of pairs of a given pair population.

Possible values for spectra are:

  • same-auto = same spectrum, auto-spectra only

  • same-cross = same spectrum, cross-spectra only

  • auto-auto = different auto-spectra

  • auto-cross-parent = a cross-spectrum and one of its parent auto-spectra

  • auto-cross-nonparent = a cross-spectrum and an auto-spectrum that is not its parent

  • cross-cross-sibling = two cross-spectra sharing one parent auto-spectrum

  • cross-cross-nonsibling = two cross-spectra sharing no parent auto-spectra

Possible values for bandpower are:

  • same

  • different (including duplicate pairs unless spectra is same-auto or same-cross, such that no two rows in the output table are complete duplicates)

Parameters

  • spectra (str) – Description of the relationship between spectra included in this pair population (see list above).

  • bandpower (str) – Description of the relationship between bandpowers included in this pair population (see list above).

  • n_fields (int) – Number of fields, such that the number of spectra = n_fields * (n_fields + 1) / 2.

  • n_bandpowers (int) – Number of bandpowers.

  • spectrum_table (2D numpy array, optional) – Precomputed spectrum lookup table produced by get_spectrum_table, which is generated if not supplied.

  • skip_dupe_check (bool, optional) – If True, skip the check for duplicate rows in the output table (default False).

Returns

List of 2D nested tuples giving the indices of all the pairs meeting the required description. Each is: ((spectrum idx 1, spectrum idx 2), (bandpower idx 1, bandpower idx 2)).

Return type

list

mutual_info.get_pair_idxs_slics_gauss(spectra, bandpowers, n_fields, n_bandpowers)

Alternative version of get_pair_idxs for the SLICS vs Gaussian fields comparison. Return spectrum and bandpower indices of pairs of a given pair population.

Possible values for spectra are:

  • same-spec

  • different-spec

Possible values for bandpower are:

  • same-bp

  • different-bp (including duplicate pairs unless spectra is same-spec, such that no two rows in the output table are complete duplicates)

Parameters

  • spectra (str) – Description of the relationship between spectra included in this pair population (see list above).

  • bandpower (str) – Description of the relationship between bandpowers included in this pair population (see list above).

  • n_fields (int) – Number of fields, such that the number of spectra = n_fields * (n_fields + 1) / 2.

  • n_bandpowers (int) – Number of bandpowers.

Returns

List of 2D nested tuples giving the indices of all the pairs meeting the required description. Each is: ((spectrum idx 1, spectrum idx 2), (bandpower idx 1, bandpower idx 2)).

Return type

list

mutual_info.get_spectrum_table(n_fields)

Generate the lookup table for power spectra, which gives the two ‘parent’ auto-spectra for any cross-spectrum.

Parameters

n_fields (int) – Number of fields, such that the number of spectra = n_fields * (n_fields + 1) / 2.

Returns

Table of shape (n_cross_spectra, 3) where the three columns are: cross-spectrum index, parent 1 (row) index, parent 2 (column) index. Row and column refer to laying out all power spectra in the upper triangle of a (n_fields x n_fields) matrix.

Return type

2D numpy array

mutual_info.get_transcov(x, y, n_bins=100, cov_fac=None, check=True, pit=False)

Calculate additive transcovariance after whitening, using a histogram method. Implements equation 7 of Sellentin & Heavens 2018.

Parameters

  • x (1D numpy array) – Samples of the first variable.

  • y (1D numpy array) – Samples of the second variable.

  • n_bins (int or (int, int), optional) – Number of bins to use along each axis in the histogram, default 100.

  • cov_fac (float, optional) – Correction factor to apply to the inverse of an unbiased estimate of the data covariance to obtain an unbiased estimate of the inverse covariance, following Hartlap et al. (2007) eqn 17. If not supplied it is calculated.

  • check (bool, optional) – Check that the whitened data has the correct mean and covariance (default True).

  • pit (bool, optional) – Apply a probability integral transform to map the marginals to a standard normal distribution (default False).

Returns

Estimate of additive transcovariance of x and y after whitening.

Return type

float

mutual_info.pit_stdnorm(data)

Apply a probability integral transform to map the marginals of data to a standard normal distribution.

Parameters

data (2D numpy array) – Bivariate input data with shape (2, n) where n is the number of samples and the first axis indexes the two variables.

Returns

Input data with each variable mapped to a standard normal distribution. Shape (2, n’), where n’ <= n as some samples may be removed if they map to non-finite values.

Return type

2D numpy array

mutual_info.plot_all_pairs(input_path, save_path=None)

Plot full-sky and cut-sky mutual information distributions for all populations, as output by combine_mi_files.

Parameters

  • input_path (str) – Path to MI estimates as output by combine_mi_files.

  • save_path (str, optional) – Path to save plot to, if supplied. If not supplied plot will be displayed.

mutual_info.plot_matrix_slics_gauss(input_path, save_path=None)

Plot SLICS vs Gaussian fields mutual information matrix for same-spectrum pairs.

Parameters

  • input_path (str) – Path to mutual information estimates as output by combine_mi_slics_gauss.

  • save_path (str, optional) – Path to save plot to, if supplied. If not supplied plot will be displayed.

mutual_info.plot_parents(input_path, save_path=None)

Plot full-sky and cut-sky mutual information distribution for auto-cross-parent pairs, with same bandpower and different bandpower pairs side-by-side.

Parameters

  • input_path (str) – Path to MI estimates as output by combine_mi_files.

  • save_path (str, optional) – Path to save plot to, if supplied. If not supplied plot will be displayed.

mutual_info.plot_samespec_slics_gauss(input_path, save_path=None)

Plot all pairs and same-spec pairs side-by-side, for SLICS and Gaussian field bandpowers.

Parameters

  • input_path (str) – Path to mutual information estimates as output by combine_mi_slics_gauss.

  • save_path (str, optional) – Path to save plot to, if supplied. If not supplied plot will be displayed.

mutual_info.plot_sim_gauss(sim_path, gauss_path, save_path=None)

Plot mutual information distributions for simulations and Gaussian samples side-by-side.

Parameters

  • sim_path (str) – Path to mutual information estimates from simulated bandpowers as output by combine_mi_files.

  • gauss_path (str) – Path to mutual information estimates from Gaussian samples as output by combine_mi_files.

  • save_path (str, optional) – Path to save plot to, if supplied. If not supplied plot will be displayed.

mutual_info.plot_transcov(input_path, save_path=None)

Plot full-sky and cut-sky distributions of transcovariance, as output by sim_bp_transcov.

Parameters

  • input_path (str) – Path to transcovariance estimates produced by sim_bp_transcov.

  • save_path (str, optional) – Path to save plot to, if supplied. If not supplied plot will be displayed.

mutual_info.plot_vs_l(input_path, pbl_path, lmin, save_path=None)

Plot full-sky and cut-sky mutual information vs l for all same-bandpower pairs.

Parameters

  • input_path (str) – Path to mutual information estimates as output by combine_mi_files.

  • pbl_path (str) – Path to bandpower binning matrix.

  • lmin (int) – l corresponding to lower bound of first bandpower.

  • save_path (str, optional) – Path to save plot to, if supplied. If not supplied plot will be displayed.

mutual_info.plot_whitening(nowhiten_path, whiten_path, n_fields, n_bandpowers, save_path=None)

Plot full-sky and cut-sky mutual information distributions for adjacent bandpowers, with and without whitening.

Parameters

  • nowhiten_path (str) – Path to mutual information estimates without whitening, as output by combine_mi_files.

  • whiten_path (str) – Path to mutual information estimates with whitening, as output by combine_mi_files.

  • n_fields (int) – Number of fields, such that the number of power spectra = n_fields * (n_fields + 1) / 2.

  • n_bandpowers (int) – Number of bandpowers.

  • save_path (str, optional) – Path to save plot to, if supplied. If not supplied plot will be displayed.

mutual_info.sim_bp_transcov(data_path_fs, data_path_ma, n_fields, n_bandpowers, save_path, pit=False)

Calculate whitened trancovariance of full- and cut-sky simulated bandpowers output by simulation.combine_sim_bp_batches.

Parameters

  • data_path_fs (str) – Path to full-sky simulated bandpowers.

  • data_path_ma (str) – Path to cut-sky simulated bandpowers.

  • n_fields (int) – Number of fields, such that n_spectra = n_fields * (n_fields + 1) / 2.

  • n_bandpowers (int) – Number of bandpowers.

  • save_path (str) – Path to save transcovariance matrices to.

  • pit (bool, optional) – If True, map the marginals to standard normal distributions before calculating transcovariance (default False).