:py:mod:`cubnm.sim.base` ####################### .. py:module:: cubnm.sim.base .. autoapi-nested-parse:: Simulation of the models .. autoapisummary:: cubnm.sim.base.SimGroup cubnm.sim.base.MultiSimGroupMixin .. autoapisummary:: cubnm.sim.base.create_multi_sim_group .. py:class:: SimGroup(duration, TR, sc, sc_dist=None, out_dir=None, dt='0.1', bw_dt='1.0', states_ts=False, states_sampling=None, noise_out=False, do_fc=True, do_fcd=True, window_size=30, window_step=5, sim_seed=0, exc_interhemispheric=False, force_cpu=False, force_gpu=False, gof_terms=['+fc_corr', '-fcd_ks'], bw_params='friston2003', bold_remove_s=30, fcd_drop_edges=True, noise_segment_length=30, sim_verbose=False, progress_interval=500) Group of simulations that are executed in parallel (as possible depending on the available hardware) on GPU/CPU. Parameters --------- duration: :obj:`float` simulation duration (in seconds) It must not have more than 3 decimal places. TR: :obj:`float` BOLD TR (in seconds) sc: :obj:`str` or :obj:`np.ndarray` path to structural connectome strengths (as an unlabled .txt/.npy) or a numpy array. Shape: (nodes, nodes) If asymmetric, rows are sources and columns are targets. sc_dist: :obj:`str` or :obj:`np.ndarray` path to structural connectome distances (as an unlabled .txt) or a numpy array. Shape: (nodes, nodes) If provided ``'v'`` (velocity) will be a free parameter and there will be delay in inter-regional connections. If asymmetric, rows are sources and columns are targets. out_dir: {:obj:`str` or 'same' or None} output directory - :obj:`str`: will create a directory in the provided path - ``'same'``: will create a directory named based on sc (only should be used when sc is a path and not a numpy array) - ``None``: will not have an output directory (and cannot save outputs) dt: :obj:`decimal.Decimal` or :obj:`str` model integration time step (in msec) bw_dt: :obj:`decimal.Decimal` or :obj:`str` Ballon-Windkessel integration time step (in msec) states_ts: :obj:`bool` return time series of model states to self.sim_states Note that this will increase the memory usage and is not recommended for large number of simulations (e.g. in a grid search) states_sampling: :obj:`float` sampling rate of model states in seconds. Default is None, which uses BOLD TR as the sampling rate. noise_out: :obj:`bool` return noise time series do_fc: :obj:`bool` calculate simulated functional connectivity (FC) do_fcd: :obj:`bool` calculate simulated functional connectivity dynamics (FCD) window_size: :obj:`int` dynamic FC window size (in seconds) will be converted to N TRs (nearest even number) The actual window size is number of TRs + 1 (including center) window_step: :obj:`int` dynamic FC window step (in seconds) will be converted to N TRs sim_seed: :obj:`int` seed used for the noise simulation exc_interhemispheric: :obj:`bool` excluded interhemispheric connections from sim FC and FCD calculations force_cpu: :obj:`bool` use CPU for the simulations (even if GPU is available). If set to False the program might use GPU or CPU depending on GPU availability force_gpu: :obj:`bool` on some HPC/HTC systems occasionally GPUtil might not detect an available GPU device. Use this if there is a GPU available but is not being used for the simulation. If set to True but a GPU is not available will lead to errors. gof_terms: :obj:`list` of :obj:`str` list of goodness-of-fit terms to be used for scoring. May include: - ``'-fcd_ks'``: negative Kolmogorov-Smirnov distance of FCDs - ``'+fc_corr'``: Pearson correlation of FCs - ``'-fc_diff'``: negative absolute difference of FC means - ``'-fc_normec'``: negative Euclidean distance of FCs divided by max EC [sqrt(n_pairs*4)] bw_params: {'friston2003' or 'heinzle2016-3T' or :obj:`dict`} see :func:`cubnm.utils.get_bw_params` for details bold_remove_s: :obj:`float` remove the first bold_remove_s seconds from the simulated BOLD in FC, FCD and mean state calculations (but the entire BOLD will be returned to .sim_bold) fcd_drop_edges: :obj:`bool` drop the edge windows in FCD calculations noise_segment_length: :obj:`float` or None in seconds, length of the noise segments in the simulations The noise segment will be repeated after shuffling of nodes and time points. To generate noise for the entire simulation without repetition, set this to None. Note that varying the noise segment length will result in a different noise array even if seed is fixed (but fixed combination of seed and noise_segment_length will result in reproducible noise) sim_verbose: :obj:`bool` verbose output of the simulation including details of simulations and a progress bar. progress_interval: :obj:`int` msec; interval of progress updates in the simulation Only used if ``sim_verbose`` is ``True`` Attributes --------- param_lists: :obj:`dict` of :obj:`np.ndarray` dictionary of parameter lists, including - global parameters with shape (N_SIMS,) - regional parameters with shape (N_SIMS, nodes) - ``'v'``: conduction velocity. Shape: (N_SIMS,) Additional attributes will be added after running the simulations. See :func:`cubnm.sim.base.SimGroup._process_out` for details. Notes ----- Derived classes must set the following attributes: model_name: :obj:`str` name of the model used in the simulations global_param_names: :obj:`list` of :obj:`str` names of global parameters regional_param_names: :obj:`list` of :obj:`str` names of regional parameters state_names: :obj:`list` of :obj:`str` names of the state variables sel_state_var: :obj:`str` name of the state variable used in the tests n_noise: :obj:`int` number of noise elements per node per time point (e.g. 2 if there are noise to E and I neuronal populations) default_params: :obj:`dict` default parameter values for the simulations. If a parameter's default is set to None, it will be a required (fixed or free) parameter. This is often the case with 'G' (global coupling). .. py:method:: post_init() Post-initilaization hook that normally does nothing, but can be overridden in derived classes to add custom post-initialization steps. .. py:method:: _check_dt() Check if integrations steps are valid .. py:method:: _set_default_params(missing=True) Set default parameters for the simulations. Parameters ---------- missing: :obj:`bool`, optional If True (default), only sets parameters that are currently None or have an incorrect length in self.param_lists. If False, always overwrites all listed defaults. .. py:method:: get_config(include_N=False, for_reinit=False) Get the configuration of the simulation group Parameters ---------- include_N: :obj:`bool` include N in the output config is ignored when for_reinit is True for_reinit: :obj:`bool` include the parameters that need reinitialization of the simulation core session if changed Returns ------- config: :obj:`dict` dictionary of simulation group configuration .. py:method:: run(force_reinit=False) Run the simulations in parallel (as possible) on GPU/CPU through the :func:`cubnm._core.run_simulations` function which runs compiled C++/CUDA code. Parameters ---------- force_reinit: :obj:`bool` force reinitialization of the session. At the beginning of each session (when `cubnm` is imported) some variables are initialized on CPU/GPU and reused in every run. Set this to True if you want to reinitialize these variables. This is rarely needed. .. py:method:: _process_out(out) Assigns model outputs (as arrays) to object attributes with correct shapes, names and types. Parameters ---------- out: :obj:`tuple` output of ``cubnm._core.run_simulations`` function Notes ----- The simulation outputs are assigned to the following object attributes: - init_time: :obj:`float` initialization time of the simulations - run_time: :obj:`float` run time of the simulations - sim_bold: :obj:`np.ndarray` simulated BOLD time series. Shape: (N_SIMS, duration/TR, nodes) - sim_states: :obj:`dict` of :obj:`np.ndarray` simulated state variables with keys as state names and values as arrays with the shape (N_SIMS, nodes) when ``states_ts`` is False, and (N_SIMS, duration/TR, nodes) when ``states_ts`` is True If ``do_fc`` is ``True``, additionally includes: - sim_fc_trils: :obj:`np.ndarray` simulated FC lower triangle. Shape: (N_SIMS, n_pairs) If ``do_fcd`` is ``True``, additionally includes: - sim_fcd_trils: :obj:`np.ndarray` simulated FCD lower triangle. Shape: (N_SIMS, n_window_pairs) .. py:method:: get_state_averages() Get the averages of state variables across time and nodes for each simulation. Returns ------- state_averages: :obj:`pd.DataFrame` DataFrame of state averages with columns as state names and rows as simulations .. py:method:: get_noise() Get the (recreated) noise time series. This requires recreation of the noise array if noise segmenting is on. Noise will be recreated based on shuffling indices of nodes and time steps, similar to how it is done in the core code. Returns ------- noise: :obj:`np.ndarray` Noise segment array. Shape: (n_noise, nodes, bw_it, inner_it) .. py:method:: get_sim_fc(idx) Get the simulated FC of a given simulation ``idx`` as a square matrix. Parameters ---------- idx: :obj:`int` index of the simulation to get the FC for Returns ------- fc: :obj:`np.ndarray` simulated FC matrix of shape (nodes, nodes) for the simulation with index ``idx`` .. py:method:: get_sim_fcd(idx) Get the simulated FCD of a given simulation ``idx`` as a square matrix. Parameters ---------- idx: :obj:`int` index of the simulation to get the FC for Returns ------- fcd: :obj:`np.ndarray` simulated FC matrix of shape (n_windows, n_windows) for the simulation with index ``idx`` .. py:method:: slice(key, inplace=False) Slice the simulation group to a single simulation Parameters ---------- key: :obj:`int` index of the simulation to slice inplace: :obj:`bool` the object will be sliced in place and therefore the data of other simulations will be removed. Otherwise a new object copied from the current object will be returned. Returns ------- obj: :obj:`cubnm.sim.SimGroup` sliced simulation group .. py:method:: clear() Clear the simulation outputs .. py:method:: _problem_init(problem) Extends BNMProblem initialization if needed. By default it doesn't do anything. Parameters ---------- problem: :obj:`cubnm.optimize.BNMProblem` optimization problem object .. py:method:: _problem_evaluate(problem, X, out, *args, **kwargs) Extends BNMProblem evaluation if needed. By default it doesn't do anything. Parameters ---------- X: :obj:`np.ndarray` the normalized parameters of current population in range [0, 1]. Shape: (N, ndim) out: :obj:`dict` the output dictionary to store the results with keys 'F' and 'G'. Currently only 'F' (cost) is used. *args, **kwargs .. py:method:: score(emp_fc_tril=None, emp_fcd_tril=None, emp_bold=None, force_cpu=False, usable_mem=None) Calcualates individual goodness-of-fit terms and aggregates them. .. note:: If `emp_bold` is provided, `emp_fc_tril` and `emp_fcd_tril` will be ignored. .. note:: For each measure, if the value is NaN, it will be set to the "worst" possible value. NaNs may occur in simulated FCD or FC. For example, in the rWWEx model, when excitation is too high and noise is low, `S` and in turn `BOLD` in some areas may become saturated and show no variability. This can result in correlations of their BOLD signals with other nodes (within certain dynamic windows) being NaN. Parameters -------- emp_fc_tril: :obj:`np.ndarray` or None 1D array of empirical FC lower triangle. Shape: (edges,) emp_fcd_tril: :obj:`np.ndarray` or None 1D array of empirical FCD lower triangle. Shape: (window_pairs,) emp_bold: :obj:`np.ndarray` or None cleaned and parcellated empirical BOLD time series. Shape: (nodes, volumes) Motion outliers should either be excluded (not recommended as it disrupts the temporal structure) or replaced with zeros. If provided emp_fc_tril and emp_fcd_tril will be ignored. force_cpu: :obj:`bool` force CPU for the calculations. Otherwise if GPU is available some of the scores (including "+fc_corr", "-fcd_ks") will be calculated on GPU. usable_mem: :obj:`int` amount of available GPU memory to be used in bytes. If None, 80% of the free memory will be used. Returns ------- scores: :obj:`pd.DataFrame` The goodness of fit measures (columns) of each simulation (rows) .. py:method:: _get_save_data() Get the simulation outputs and parameters to be saved to disk Returns ------- out_data: :obj:`dict` dictionary of simulation outputs and parameters .. py:method:: save() Save simulation outputs to disk as an npz file. .. py:method:: _get_test_configs(cpu_gpu_identity=False) :classmethod: Get configs for testing the simulations Parameters ---------- cpu_gpu_identity: :obj:`bool` indicates whether configs are for CPU/GPU identity tests in which case force_cpu is not included in the configs since tests will be done on both CPU and GPU Returns ------- configs: :obj:`dict` of :obj:`list` .. py:method:: _get_test_instance(opts) :classmethod: Initializes an instance that is used in tests Parameters ---------- opts: :obj:`dict` dictionary of test options Returns ------- sim_group: :obj:`cubnm.sim.SimGroup` simulation group object of the test simulation which is not run yet .. py:class:: MultiSimGroupMixin(sim_groups) Mixin for combining multiple simulation groups into one, which is intended for batch optimization, i.e. running multiple optimizations at the same time on GPU. .. py:method:: run(**kwargs) Runs merged simulations of all children by concatenating SCs and parameters and then running all simulations in parallel as a single merged simulation group. Parameters ---------- **kwargs keyword arguments to be passed to `cubnm.sim.SimGroup.run` method .. py:method:: _process_out(out) Divides simulations outputs across individual children SimGroup objects which will respectively convert the output to attributes with correct shapes, names and types. See :func:`cubnm.sim.base.SimGroup._process_out` for details. Parameters ---------- out: :obj:`tuple` output of `run_simulations` function .. py:function:: create_multi_sim_group(sim_group_cls) Dynamically creates a MultiSimGroup class by combining a model's specific ``SimGroup`` class with :class:`cubnm.sim.base.MultiSimGroupMixin`, which can be used in batch optimization. Parameters ---------- sim_group_cls: :obj:`type` :class:`cubnm.sim.base.SimGroup` subclass, e.g. :class:`cubnm.sim.rww.rWWSimGroup` Returns ------- MultiSimGroup: :obj:`type` :class:`MultiSimGroup` class that can be used in batch optimization