Run Manager Configuration

As described in Sections 20.1.1 and 20.2, running UrbanSim simulation or estimation is controlled by a user-defined configuration. The following code, baseline.py, contains a fully specified configuration that influences the run management. Mandatory entries and default values for optional entries are marked in the comments. The actual values for the listed entries are only examples.

from opus_core.configuration import Configuration
from opus_core.database_management.configurations.scenario_database_configuration \
    import ScenarioDatabaseConfiguration
from opus_core.database_management.configurations.estimation_database_configuration \
    import EstimationDatabaseConfiguration
from opus_core.configurations.baseyear_cache_configuration \
    import BaseyearCacheConfiguration
from urbansim.configurations.creating_baseyear_cache_configuration \
    import CreatingBaseyearCacheConfiguration
    
class Baseline(Configuration):
    def __init__(self):
        config = {
          'project_name':'urbansim_gridcell',
          'description':'baseline',  
          'model_system':'urbansim.model_coordinators.model_system', # mandatory
          'base_year': 2000,   # default: read from table 'base_year' in cache
          'years': (2001, 2030),                                     # mandatory
          'cache_directory':'d:/urbansim_cache/',                    # mandatory
          
          'scenario_database_configuration': ScenarioDatabaseConfiguration(
              database_name = 'my_baseyear_database'), # mandatory for simulation
              
          'estimation_database_configuration': EstimationDatabaseConfiguration(
              database_name = 'my_estimation_database'), # mandatory for estim.
              
          'creating_baseyear_cache_configuration': CreatingBaseyearCacheConfiguration(
              default: 'opus_tmp'+random string
              cache_directory_root = 'd:/urbansim_cache',
              
              cache_from_database = False, # default: True
              
              # mandatory if 'cache_from_database' is False
              baseyear_cache = BaseyearCacheConfiguration(
                 # mandatory for this block
                 existing_cache_to_copy = 'd:/urbansim_cache/run_397.2006_05_23_18_21',
            
                 # default: all years in 'existing_cache_to_copy'
                 years_to_cache = range(1996,2001)
              ),
              tables_to_cache = [ # default: []
                      'gridcells', 'households', 'jobs', 'zones'
                                 ]
              tables_to_cache_nchunks = { # default: each table defaults to 1
                      'gridcells':2,
                   },
              tables_to_copy_to_previous_years = { # default: no copied tables
                 'development_type_groups':1996, # table name and year to put it in
                 'development_types':1996,
                 'development_type_group_definitions':1996,
                 'urbansim_constants': 1996,
                   },
              unroll_gridcells = True  # default: True
           ),
          Configuration.__init__(self, config)

The 'model_system' entry is the full Opus path to the model system that will be used by the run manager to run/estimate a set of models.

Entry 'years' determines for what years the simulation should run as a tuple with first and last year to run.

Entry 'cache_directory' is used by the scripts create_baseyear_cache.py and start_estimation.py only. It is not used for simulations.

Entry 'creating_baseyear_cache_configuration' contains a configuration for creating the simulation cache. Entry 'cache_directory_root' is the root directory where data should be cached during processing. The actual cache directory is created as a subdirectory of this location.

The entry 'tables_to_cache' is used by the script create_baseyear_cache.py. Only tables listed here are cached from database into the baseyear cache.

If a database table is so large that Python runs out of memory when copying it to cache, you can reduce memory usage (but increase the time it takes to cache the data) by increasing the number of ``chunks'' in which the dataset's attributes are read from the table. By default, all attributes of a table are read in a single chunk. Setting the 'tables_to_cache_nchunks' configuration will tell the caching code to use that many chunks. For instance, if a dataset has 11 attributes, setting 'tables_to_chunk_nchunks' to 3 will use three chunks loading 4, 4, and 3 attributes, in each chunk.

In the 'baseyear_cache' block, the directory with the already cached data should be put into the entry existing_cache_to_copy. The run manager then copies data from that directory into the simulation cache for this run. If you want to copy only selected years, they can be specified in the entry years_to_cache as a list of those years; by default all years are copied. Note that this behaviour can be alternatively controlled directly from the command line (see start_run.py -h) which has priority over entries in this configuration.

The 'tables_to_copy_to_previous_years' entry is used when a lag variable needs to compute data for before the base year. If this is the case, add those tables to this list, and indicate the year to which to copy the tables. In general, it is safe to copy the tables to the earliest year created by the unroll gridcell process. You can determine what this year is by examining the year directories created in your baseyear cache.

The entry unroll_gridcells is specific to the urbansim gridcell project. It controls if gridcells are unrolled into years before the base year.

There are several run manager configurations in Opus. See for example the directory psrc/configs for configuration of different PSRC runs.