Dataset Pool

A class called DatasetPool is design to maintain a 'pool' of datasets. It is mainly used when computing variables. Therefore in most cases, it will be only needed if the Dataset method compute_variables() is used (for which an instance of DatasetPool is passed as an argument) in order to have external datasets available that are required by the various variable implementations (see an example in Section 6.3.1).

The class is initialized by passing arguments:

package_order

- a list of packages that are used for finding the corresponding dataset class. Default is an empty list.

package_order_exception

- a dictionary of exceptions from the package_order as pairs dataset name and list of packages for that dataset. Default is an empty dictionary.

storage

- an object of class Storage that contains data of datasets to be included in pool. Default is None.

One can add a dataset into the pool using the method
_add_dataset(dataset_name, dataset),
or by passing multiple datasets in one dictionary using the method
add_datasets_if_not_included(datasets_dict).

A dataset can be accessed by the method
get_dataset(dataset_name, dataset_arguments)
where the second argument is optional. If a dataset of the given name is included in the pool, it is returned. Otherwise, the class creates a Dataset object with data from the storage that was passed in the initialization. In order to create the appropriate Dataset class, it is searched for a module called dataset_name_dataset.py which should contain a class called DatasetNameDataset. The DatasetPool class searches for this module in the directory datasets of packages given in the initialization argument 'package_order'.