:py:mod:`cubnm.optimize` ####################### .. py:module:: cubnm.optimize .. autoapi-nested-parse:: Optimizers of the model free parameters .. autoapisummary:: cubnm.optimize.BNMProblem cubnm.optimize.Optimizer cubnm.optimize.GridOptimizer cubnm.optimize.PymooOptimizer cubnm.optimize.CMAESOptimizer cubnm.optimize.NSGA2Optimizer cubnm.optimize.BayesOptimizer .. autoapisummary:: cubnm.optimize.batch_optimize .. py:class:: BNMProblem(model, params, emp_fc_tril=None, emp_fcd_tril=None, emp_bold=None, het_params=[], maps=None, maps_coef_range='auto', node_grouping=None, multiobj=False, **kwargs) Bases: :py:obj:`pymoo.core.problem.Problem` Brain network model problem. A :class:`pymoo.core.problem.Problem` that defines the model, free parameters and their ranges, and target empirical data (FC and FCD), and the simulation configurations (through :class:`cubnm.sim.SimGroup`). :class:`cubnm.optimize.Optimizer` classes can be used to optimize the free parameters of this problem. Parameters ---------- model: :obj:`str`, {'rWW', 'rWWEx', 'Kuramoto'} model name params: :obj:`dict` of :obj:`tuple` or :obj:`float` a dictionary including parameter names as keys and their fixed values (:obj:`float`) or continuous range of values (:obj:`tuple` of (min, max)) as values. emp_fc_tril: :obj:`np.ndarray` or :obj:`None` lower triangular part of empirical FC. Shape: (edges,) emp_fcd_tril: :obj:`np.ndarray` or :obj:`None` lower triangular part of empirical FCD. Shape: (window_pairs,) emp_bold: :obj:`np.ndarray` or None cleaned and parcellated empirical BOLD time series. Shape: (nodes, volumes) Motion outliers can either be excluded (not recommended as it disrupts the temporal structure) or replaced with zeros. If provided emp_fc_tril and emp_fcd_tril will be ignored. het_params: :obj:`list` of :obj:`str`, optional which regional parameters are heterogeneous across nodes maps: :obj:`str`, optional path to heterogeneity maps as a text file or a numpy array. Shape: (n_maps, nodes). If provided one free parameter per regional parameter per each map will be added. maps_coef_range: 'auto' or :obj:`tuple` or :obj:`list` of :obj:`tuple` - 'auto': uses (-1/max, -1/min) for maps with positive and negative values (assuming they are z-scored) and (0, 1) otherwise - :obj:`tuple`: uses the same range for all maps - :obj:`list` of :obj:`tuple`: n-map element list specifying the range of coefficients for each map node_grouping: {None, 'node', 'sym', :obj:`str`, :obj:`np.ndarray`}, optional - None: does not use region-/group-specific parameters - 'node': each node has its own regional free parameters - 'sym': uses the same regional free parameters for each pair of symmetric nodes (e.g. L and R hemispheres). Assumes symmetry of parcels between L and R hemispheres. - :obj:`str`: path to a text file including node grouping array. Shape: (nodes,) - :obj:`np.ndarray`: a numpy array. Shape: (nodes,) multiobj: :obj:`bool`, optional instead of combining the objectives into a single objective function (via summation) defines each objective separately. This must not be used with single-objective optimizers **kwargs Keyword arguments passed to :class:`cubnm.sim.SimGroup` .. py:method:: get_config(include_sim_group=True, include_N=False) Get the problem configuration Parameters ---------- include_sim_group: :obj:`bool`, optional whether to include the configuration of the associated :class:`cubnm.sim.SimGroup` include_N: :obj:`bool`, optional whether to include the current population size in the configuration Returns ------- :obj:`dict` the configuration of the problem .. py:method:: _get_Xt(X) Transforms normalized parameters in range [0, 1] to the actual parameter ranges Parameters ---------- X: :obj:`np.ndarray` the normalized parameters of current population. Shape: (N, ndim) Returns ------- :obj:`np.ndarray` the transformed parameters of current population. Shape: (N, ndim) .. py:method:: _get_X(Xt) Normalizes parameters to range [0, 1] Parameters ---------- Xt: :obj:`np.ndarray` the parameters of current population. Shape: (N, ndim) Returns ------- :obj:`np.ndarray` the normalized parameters current population. Shape: (N, ndim) .. py:method:: _set_sim_params(X) Sets the global and regional parameters of the problem's :class:`cubnm.sim.SimGroup` based on the problem free and fixed parameters and type of regional parameter heterogeneity (map-based, group-based or none). Parameters ---------- X: :obj:`np.ndarray` the normalized parameters of current population in range [0, 1]. Shape: (N, ndim) .. py:method:: _evaluate(X, out, *args, **kwargs) Ovewrites the :meth:`pymoo.core.problem.Problem._evaluate` method to evaluate the goodness of fit of the simulations based on parameters `X` and store the results in `out`. Parameters ---------- X: :obj:`np.ndarray` the normalized parameters of current population in range [0, 1]. Shape: (N, ndim) out: :obj:`dict` the output dictionary to store the results with keys 'F' and 'G'. Currently only 'F' (cost) is used. *args, **kwargs additional arguments passed to the evaluation function kwargs may include: - scores: :obj:`list` an empty list passed on to evaluation function to store the individual goodness of fit measures - skip_run: :obj:`bool`, optional will only be true in batch optimization where the simulations are already run and only the GOF calculation is needed .. py:method:: eval(X, skip_run=False) Runs the simulations based on normalized candidate free parameters `X` and evaluates their goodness of fit to the empirical FC and FCD of the problem. Parameters ---------- X: :obj:`np.ndarray` the normalized parameters of current population in range [0, 1]. Shape: (N, ndim) skip_run: :obj:`bool`, optional will only be true in batch optimization where the simulations are already run and only the GOF calculation is needed Returns ------- :obj:`pd.DataFrame` The goodness of fit measures (columns) of each simulation (rows) .. py:class:: Optimizer(**kwargs) Bases: :py:obj:`abc.ABC` Base class for evolutionary optimizers .. py:method:: optimize() :abstractmethod: .. py:method:: save(save_opt=True, save_obj=False) Saves the output of the optimizer, including history of particles, history of optima, the optimal point, and its simulation data. The output will be saved to `out_dir` of the problem's :class:`cubnm.sim.SimGroup`. If a directory with the same type of optimizer already exists, a new directory with a new index will be created. Parameters --------- save_opt: :obj:`bool`, optional reruns and saves the optimal simulation(s) data save_obj: :obj:`bool`, optional saves the optimizer object which also includes the simulation data of all simulations and therefore can be large file. Warning: this file is very large. .. py:method:: get_config() Get the optimizer configuration Returns ------- :obj:`dict` the configuration of the optimizer .. py:class:: GridOptimizer(**kwargs) Bases: :py:obj:`Optimizer` Grid search optimizer Example ------- Run a grid search of rWW model with 10 G and 10 wEE values with fixed wEI: :: from cubnm import datasets, optimize problem = optimize.BNMProblem( model = 'rWW', params = { 'G': (0.5, 2.5), 'wEE': (0.05, 0.75), 'wEI': 0.21, }, duration = 60, TR = 1, window_size=10, window_step=2, sc = datasets.load_sc('strength', 'schaefer-100'), emp_bold = datasets.load_bold('schaefer-100'), ) go = optimize.GridOptimizer() go.optimize(problem, grid_shape=10) go.save() .. py:method:: optimize(problem, grid_shape) Runs a grid search optimization on the given problem. Parameters ---------- problem: :obj:`cubnm.optimizer.BNMProblem` The problem to be set up with the algorithm. grid_shape: :obj:`int` or :obj:`dict` Shape of the grid search. If an integer is provided the same number of points are used for each parameter. If a dictionary is provided, the keys should be the parameter names and the values should be the number of points within the range of each parameter. .. py:method:: save(save_avg_states=True, **kwargs) Saves the output of the optimizer, including history of particles, history of optima, the optimal point, and its simulation data. The output will be saved to `out_dir` of the problem's :class:`cubnm.sim.SimGroup`. Parameters --------- save_avg_states: :obj:`bool`, optional saves the average states of all simulations **kwargs additional keyword arguments passed to the :meth:`cubnm.optimizer.Optimizer.save` .. py:method:: get_config() Get the optimizer configuration Returns ------- :obj:`dict` the configuration of the optimizer .. py:class:: PymooOptimizer(termination=None, n_iter=2, seed=0, print_history=True, save_history_sim=False, **kwargs) Bases: :py:obj:`Optimizer` Generic wrapper for `pymoo` optimizers. Parameters: ---------- termination: :obj:`pymoo.termination.Termination`, optional The termination object that defines the stopping criteria for the optimization process. If not provided, the termination criteria will be based on the number of iterations (`n_iter`). n_iter: :obj:`int`, optional The maximum number of iterations for the optimization process. This parameter is only used if `termination` is not provided. seed: :obj:`int`, optional The seed value for the random number generator used by the optimizer. print_history: :obj:`bool`, optional Flag indicating whether to print the optimization history during the optimization process. save_history_sim: :obj:`bool`, optional Flag indicating whether to save the simulation data of each iteration. Default is False to avoid consuming too much memory across iterations. **kwargs Additional keyword arguments that can be passed to the `pymoo` optimizer. .. py:method:: setup_problem(problem, pymoo_verbose=False, **kwargs) Registers a :class:`cubnm.optimizer.BNMProblem` with the optimizer, so that the optimizer can optimize its free parameters. Parameters ---------- problem: :obj:`cubnm.optimizer.BNMProblem` The problem to be set up with the algorithm. pymoo_verbose: :obj:`bool`, optional Flag indicating whether to enable verbose output from pymoo. Default is False. **kwargs Additional keyword arguments to be passed to the algorithm setup method. .. py:method:: optimize() Optimizes the associated :class:`cubnm.optimizer.BNMProblem` free parameters through an evolutionary optimization approach by running multiple generations of parallel simulations until the termination criteria is met or maximum number of iterations is reached. .. py:method:: save(save_opt=True, save_obj=False) Saves the output of the optimizer, including history of particles, history of optima, the optimal point, and its simulation data. The output will be saved to `out_dir` of the problem's :class:`cubnm.sim.SimGroup`. If a directory with the same type of optimizer already exists, a new directory with a new index will be created. Parameters --------- save_opt: :obj:`bool`, optional reruns and saves the optimal simulation(s) data save_obj: :obj:`bool`, optional saves the optimizer object which also includes the simulation data of all simulations and therefore can be large file. Warning: this file is very large. .. py:method:: get_config() Get the optimizer configuration Returns ------- :obj:`dict` the configuration of the optimizer .. py:class:: CMAESOptimizer(popsize, x0=None, sigma=0.5, use_bound_penalty=False, algorithm_kws={}, **kwargs) Bases: :py:obj:`PymooOptimizer` Covariance Matrix Adaptation Evolution Strategy (CMA-ES) optimizer Parameters ---------- popsize: :obj:`int` The population size for the optimizer x0: array-like, optional The initial guess for the optimization. If None (default), the initial guess will be estimated based on `popsize` random samples as the first generation sigma: :obj:`float`, optional The initial step size for the optimization use_bound_penalty: :obj:`bool`, optional Whether to use a bound penalty for the optimization algorithm_kws: :obj:`dict`, optional Additional keyword arguments for the CMAES algorithm **kwargs Additional keyword arguments Example ------- Run a CMAES optimization for 10 iterations with a population size of 20: :: from cubnm import datasets, optimize problem = optimize.BNMProblem( model = 'rWW', params = { 'G': (0.5, 2.5), 'wEE': (0.05, 0.75), 'wEI': 0.15, }, emp_bold = datasets.load_bold('schaefer-100'), duration = 60, TR = 1, sc_path = datasets.load_sc('strength', 'schaefer-100'), states_ts = True ) cmaes = optimize.CMAESOptimizer(popsize=20, n_iter=10, seed=1) cmaes.setup_problem(problem) cmaes.optimize() cmaes.save() .. py:attribute:: max_obj :value: 1 .. py:method:: setup_problem(problem, **kwargs) Extends :meth:`cubnm.optimizer.PymooOptimizer.setup_problem` to set up the optimizer with the problem and set the bound penalty option based on the optimizer's `use_bound_penalty` attribute. Parameters ---------- problem: :obj:`cubnm.optimizer.BNMProblem` The problem to be set up with the algorithm. **kwargs Additional keyword arguments to be passed to :meth:`cubnm.optimizer.PymooOptimizer.setup_problem` .. py:method:: optimize() Optimizes the associated :class:`cubnm.optimizer.BNMProblem` free parameters through an evolutionary optimization approach by running multiple generations of parallel simulations until the termination criteria is met or maximum number of iterations is reached. .. py:method:: save(save_opt=True, save_obj=False) Saves the output of the optimizer, including history of particles, history of optima, the optimal point, and its simulation data. The output will be saved to `out_dir` of the problem's :class:`cubnm.sim.SimGroup`. If a directory with the same type of optimizer already exists, a new directory with a new index will be created. Parameters --------- save_opt: :obj:`bool`, optional reruns and saves the optimal simulation(s) data save_obj: :obj:`bool`, optional saves the optimizer object which also includes the simulation data of all simulations and therefore can be large file. Warning: this file is very large. .. py:method:: get_config() Get the optimizer configuration Returns ------- :obj:`dict` the configuration of the optimizer .. py:class:: NSGA2Optimizer(popsize, algorithm_kws={}, **kwargs) Bases: :py:obj:`PymooOptimizer` Non-dominated Sorting Genetic Algorithm II (NSGA-II) optimizer Parameters ---------- popsize: int The population size for the optimizer algorithm_kws: dict, optional Additional keyword arguments for the NSGA2 algorithm kwargs: dict Additional keyword arguments for the base class .. py:attribute:: max_obj :value: 3 .. py:method:: setup_problem(problem, pymoo_verbose=False, **kwargs) Registers a :class:`cubnm.optimizer.BNMProblem` with the optimizer, so that the optimizer can optimize its free parameters. Parameters ---------- problem: :obj:`cubnm.optimizer.BNMProblem` The problem to be set up with the algorithm. pymoo_verbose: :obj:`bool`, optional Flag indicating whether to enable verbose output from pymoo. Default is False. **kwargs Additional keyword arguments to be passed to the algorithm setup method. .. py:method:: optimize() Optimizes the associated :class:`cubnm.optimizer.BNMProblem` free parameters through an evolutionary optimization approach by running multiple generations of parallel simulations until the termination criteria is met or maximum number of iterations is reached. .. py:method:: save(save_opt=True, save_obj=False) Saves the output of the optimizer, including history of particles, history of optima, the optimal point, and its simulation data. The output will be saved to `out_dir` of the problem's :class:`cubnm.sim.SimGroup`. If a directory with the same type of optimizer already exists, a new directory with a new index will be created. Parameters --------- save_opt: :obj:`bool`, optional reruns and saves the optimal simulation(s) data save_obj: :obj:`bool`, optional saves the optimizer object which also includes the simulation data of all simulations and therefore can be large file. Warning: this file is very large. .. py:method:: get_config() Get the optimizer configuration Returns ------- :obj:`dict` the configuration of the optimizer .. py:class:: BayesOptimizer(popsize, n_iter, seed=0) Bases: :py:obj:`Optimizer` Bayesian optimizer Parameters ---------- popsize: :obj:`int` The population size for the optimizer n_iter: :obj:`int` The number of iterations for the optimization process seed: :obj:`int`, optional The seed value for the random number generator used by the optimizer. .. py:attribute:: max_obj :value: 1 .. py:method:: setup_problem(problem, **kwargs) Sets up the optimizer with the problem Parameters ---------- problem: :obj:`cubnm.optimizer.BNMProblem` The problem to be set up with the algorithm. **kwargs Additional keyword arguments to be passed to :class:`skopt.Optimizer` .. py:method:: optimize() Optimizes the associated :class:`cubnm.optimizer.BNMProblem` free parameters through an evolutionary optimization approach by running multiple generations of parallel simulations until the termination criteria is met or maximum number of iterations is reached. .. py:method:: save(save_opt=True, save_obj=False) Saves the output of the optimizer, including history of particles, history of optima, the optimal point, and its simulation data. The output will be saved to `out_dir` of the problem's :class:`cubnm.sim.SimGroup`. If a directory with the same type of optimizer already exists, a new directory with a new index will be created. Parameters --------- save_opt: :obj:`bool`, optional reruns and saves the optimal simulation(s) data save_obj: :obj:`bool`, optional saves the optimizer object which also includes the simulation data of all simulations and therefore can be large file. Warning: this file is very large. .. py:method:: get_config() Get the optimizer configuration Returns ------- :obj:`dict` the configuration of the optimizer .. py:function:: batch_optimize(optimizers, problems, save=True, setup_kwargs={}) Optimize a batch of optimizers in parallel (without requiring multiple CPU cores when using GPUs). Parameters ---------- optimizers: :obj:`list` of :obj:`cubnm.optimize.PymooOptimizer` or :obj:`cubnm.optimize.PymooOptimizer` A (list of) optimizer instance(s) to be run in parallel. If not a list, the same optimizer will be used for all problems and problems must be a list. problems: :obj:`list` of :obj:`cubnm.optimize.BNMProblem` or :obj:`cubnm.optimize.BNMProblem` A (list of) problem instance(s) to be set up with the optimizers. Will be mapped one-to-one with the optimizers. If not a list, the same problem will be used in all optimizers and optimizers must be a list. save: :obj:`bool`, optional save the optimizers and their results. This is more efficient than saving each optimizer separately as saving involves rerunning the optimal simulations, which is done in a batch in this function. setup_kwargs: :obj:`dict`, optional kwargs passed on to :meth:`cubnm.optimize.PymooOptimizer.setup_problem` Returns ------- optimizers: :obj:`list` of :obj:`cubnm.optimize.PymooOptimizer` A list of optimizer instances which have been run in parallel Example ------- Run CMAES for two subjects (with different SC and functional data) in a batch: :: from cubnm import datasets, optimize # assuming sub1 and sub2 SC and BOLD are available as # numpy arrays `sc_sub1`, `sc_sub2`, `bold_sub1`, `bold_sub2` # shared problem configuration problem_kwargs = dict( model = 'rWW', params = { 'G': (1.0, 3.0), 'wEE': (0.05, 0.5), 'wEI': 0.15, }, duration = 60, TR = 1, ) # problem for subject 1 p_sub1 = optimize.BNMProblem( sc = sc_sub1, emp_bold = bold_sub1, **problem_kwargs ) # problem for subject 2 p_sub2 = optimize.BNMProblem( sc = sc_sub2, emp_bold = bold_sub2, **problem_kwargs ) # optimizer cmaes = optimize.CMAESOptimizer(popsize=20, n_iter=10, seed=1) # batch optimization optimizers = optimize.batch_optimize(cmaes, [p_sub1, p_sub2]) # print optima print(optimizers[0].opt) print(optimizers[1].opt)