Re: [UrbanSim-Users] my_estimation_config (P.S.)

Siva,

I haven't seen this error before. It can have two reasons:

- Your Opus version is older and does not work with my suggestion.
- The error appears because your estimation wasn't successful and didn't
produce any reasonable result. You probably have a problem with
multicollinearity. Also one of the explanatory variables might have the
same value for all observations.

Hana

sivakarthik wrote:
> Hi Hana,
> I've made the changes you asked me to make but I am getting the
> following error message:
> C:\opus\eugene\estimation>run_estimation.py
> Cache Directory set to: C:/opus/threecounty/2000/flatfile
> Start simulation run: started on Wed Mar 05 16:31:49 2008
> random seed = 10
> Starting simulation for year 2000: started on Wed Mar 05 16:31:49 2008
> Closing log file:
> C:/opus/threecounty/2000/flatfile\run_multiprocess.log
>
> Logging to file: C:/opus/threecounty/2000/flatfile\year_2000_log.txt
> Simulate year 2000: started on Wed Mar 05 16:31:49 2008
> Running Household Relocation Model (from
> urbansim.models.agent_reloc
> ation_model): started on Wed Mar 05 16:31:49 2008
> compute_probabilities ...
> compute_choices ...
> Number of movers: 927
> Running Household Relocation Model (from
> urbansim.models.agent_reloc
> ation_model): completed...0.5 sec
> Unplace 77 agents.
> Estimating Household Location Choice Model (from
> urbansim.models.hou
> sehold_location_choice_model): started on Wed Mar 05 16:31:49 2008
> Number of agents for estimation: 6944
> Sampling locations for estimation ...
> Sampling done in 1 chunk(s).
> Choice set size: 30
> Compute variables ...
> urbansim.household_x_gridcell.cost_to_income_ratio
> urbansim.gridcell.total_annual_rent
> urbansim.gridcell.total_residential_value........0.0
> sec
>
> urbansim.gridcell.total_annual_rent: completed.......0.1
> sec
>
> urbansim.household_x_gridcell.cost_to_income_ratio:
> completed...
> 0.1 sec
> urbansim.household_x_gridcell.income_and_year_built......0.0
> sec
>
>
> urbansim.household_x_gridcell.percent_high_income_households_wit
> hin_walking_distance_if_low_income
>
> urbansim.gridcell.percent_high_income_households_within_walk
> ing_distance
>
> urbansim.gridcell.number_of_high_income_households_withi
> n_walking_distance
>
> urbansim.gridcell.number_of_high_income_households
> urbansim.household.is_high_income........0.0
> sec
>
>
> urbansim.gridcell.number_of_high_income_households:
> completed...0.1 sec
>
> urbansim.gridcell.number_of_high_income_households_withi
> n_walking_distance: completed...0.4 sec
>
> urbansim.gridcell.number_of_households_within_walking_di
> stance
> urbansim.gridcell.number_of_households.......0.1
> sec
>
>
> urbansim.gridcell.number_of_households_within_walking_di
> stance: completed...0.4 sec
>
> urbansim.gridcell.percent_high_income_households_within_walk
> ing_distance: completed...0.9 sec
> urbansim.household.is_low_income.....................0.0
> sec
>
>
> urbansim.household_x_gridcell.percent_high_income_households_wit
> hin_walking_distance_if_low_income: completed...0.9 sec
>
> urbansim.household_x_gridcell.percent_low_income_households_with
> in_walking_distance_if_high_income
>
> urbansim.gridcell.percent_low_income_households_within_walki
> ng_distance
>
> urbansim.gridcell.number_of_low_income_households_within
> _walking_distance
>
> urbansim.gridcell.number_of_low_income_households...
> .0.1 sec
>
> urbansim.gridcell.number_of_low_income_households_within
> _walking_distance: completed...0.4 sec
>
> urbansim.gridcell.percent_low_income_households_within_walki
> ng_distance: completed...0.5 sec
>
> urbansim.household_x_gridcell.percent_low_income_households_with
> in_walking_distance_if_high_income: completed...0.5 sec
>
> urbansim.household_x_gridcell.percent_minority_households_within
> _walking_distance_if_minority
>
> urbansim.gridcell.percent_minority_households_within_walking
> _distance
>
> urbansim.gridcell.number_of_minority_households_within_w
> alking_distance
> urbansim.gridcell.number_of_minority_households
> urbansim.household.is_minority...........0.0
> sec
>
> urbansim.gridcell.number_of_minority_households:
> com
> pleted...0.1 sec
>
> urbansim.gridcell.number_of_minority_households_within_w
> alking_distance: completed...0.4 sec
>
> urbansim.gridcell.percent_minority_households_within_walking
> _distance: completed...0.5 sec
>
> urbansim.household_x_gridcell.percent_minority_households_within
> _walking_distance_if_minority: completed...0.5 sec
>
> urbansim.household_x_gridcell.residential_units_when_household_h
> as_children
> urbansim.household.is_without_children...............0.0
> sec
>
>
> urbansim.household_x_gridcell.residential_units_when_household_h
> as_children: completed...0.0 sec
>
> urbansim.household_x_gridcell.young_household_in_high_density_re
> sidential
> urbansim.gridcell.is_in_development_type_group_SSS...0.0
> sec
>
> urbansim.household.is_young..........................0.0
> sec
>
>
> urbansim.household_x_gridcell.young_household_in_high_density_re
> sidential: completed...0.1 sec
> urbansim.household_x_gridcell.young_household_in_mixed_use
> urbansim.gridcell.is_in_development_type_group_SSS...0.0
> sec
>
> urbansim.household_x_gridcell.young_household_in_mixed_use:
> comp
> leted...0.1 sec
> Estimate ...
> submodel: -2
> WARNING: Estimation led to singular matrix. No results.
> Estimating Household Location Choice Model (from
> urbansim.models.hou
> sehold_location_choice_model): completed...4.4 sec
> Simulate year 2000: completed....................................5.1
> sec
>
> Closing log file:
> C:/opus/threecounty/2000/flatfile\year_2000_log.txt
> Starting simulation for year 2000: completed.........................5.2
> sec
>
> Start simulation run: completed..........................................5.2
> sec
>
> Closing log file: C:/opus/threecounty/2000/flatfile\run_multiprocess.log
> Traceback (most recent call last):
> File "C:\opus\eugene\estimation\run_estimation.py", line 31, in ?
> estimator.estimate()
> File "c:\opus\urbansim\estimation\estimator.py", line 71, in estimate
> self.save_results(out_storage=out_storage)
> File "c:\opus\urbansim\estimation\estimator.py", line 147, in save_results
>
> self.specification.write(out_storage=AttributeCache(),out_table_name=specifi
> cation_table)
> File "c:\opus\opus_core\equation_specification.py", line 152, in write
> self.out_storage.write_dataset(local_resources)
> File "c:\opus\opus_core\store\old\attribute_cache.py", line 62, in
> write_datas
> et
> attrtype = write_resources['attrtype']
> File "c:\opus\opus_core\resources.py", line 30, in __getitem__
> return SessionConfiguration().get_dataset_from_pool(key)
> File "c:\opus\opus_core\session_configuration.py", line 113, in
> get_dataset_fr
> om_pool
> return self.get_dataset_pool().get_dataset(dataset_name)
> File "c:\opus\opus_core\dataset_pool.py", line 56, in get_dataset
> self._load_new_dataset(dataset_name, dataset_arguments)
> File "c:\opus\opus_core\dataset_pool.py", line 129, in _load_new_dataset
> raise Exception("Dataset '%s' not found in any of the "
> Exception: Dataset 'attrtype' not found in any of the packages: 'urbansim',
> 'opus_core'.
>
>
> When I tried looking for attrtype in the urbansim.org. It is mentioned that
> attrtype is a dictionary giving the type for each attribute. Should this be
> user specified? Can you please help me with this error. Also you have
> mentioned that my simulation was not successful earlier because a singular
> matrix has been formed. So, does this mean that there is an error in the
> explanatory variables I have chosen to estimate HLCM? I am referring to
> The multi-collinearity problem.
>
> Thanks and Regards,
> Siva
>
>
>
>
>
>
>
>
>
>
>
>
>
> -----Original Message-----
> From: Hana Sevcikova [mailto:hana_at_cs.washington.edu]
> Sent: Monday, March 03, 2008 1:55 PM
> To: sivakarthik
> Cc: users_at_urbansim.org
> Subject: Re: my_estimation_config (P.S.)
>
> Siva, you've got an unwanted line break there when pasting. Check the
> syntax.
>
> Hana
>
> sivakarthik wrote:
>> Hi Hana,
>> I've made the changes you asked me but I am receiving a syntax
>> error.
>>
>> C:\opus\eugene\estimation>python run_estimation.py
>> Traceback (most recent call last):
>> File "run_estimation.py", line 16, in ?
>> from urbansim.estimation.estimation_runner import EstimationRunner
>> File "c:\opus\urbansim\estimation\estimation_runner.py", line 15, in ?
>> from urbansim.estimation.estimator import Estimator
>> File "c:\opus\urbansim\estimation\estimator.py", line 144
>> self.specification.write(out_storage=Attribute Cache(),
>> out_table_name=speci
>> fication_table)
>> ^
>> SyntaxError: invalid syntax
>>
>> Thanks and Regards,
>> Siva
>>
>>
>> -----Original Message-----
>> From: Hana Sevcikova [mailto:hana_at_cs.washington.edu]
>> Sent: Monday, March 03, 2008 12:44 PM
>> To: sivakarthik
>> Subject: Re: my_estimation_config (P.S.)
>>
>> Siva, I just saw, you need to do a little bit more changes:
>>
>> - remove the output_configuration entry from your
>> my_estimation_config.py file
>>
>> - in the 'save_results' method replace the line:
>>
>> raise StandardError, "No output_configuration given."
>>
>> by:
>>
>> pass
>>
>> Let me know if it works,
>>
>> Hana
>>
>>
>> Hana Sevcikova wrote:
>>> Siva,
>>>
>>> You're right - right now the code requires SQL in order to store your
>>> estimation results. But there is no reason for doing so and I will
>>> change it. For now, please make the following changes in the file
>>> urbansim/estimation/estimator.py:
>>>
>>> Comment out the last three lines in the method 'save_results':
>>>
>>>
> #self.specification.write(out_storage=out_storage,out_table_name=specificati
>> on_table)
> #self.coefficients.write(out_storage=out_storage,out_table_name=coefficients
>> _table)
> #self.cache_specification_and_coefficients(out_storage,specification_table,c
>> oefficients_table)
>>> and add these two lines:
>>>
>>>
> self.specification.write(out_storage=AttributeCache(),out_table_name=specifi
>> cation_table)
> self.coefficients.write(out_storage=AttributeCache(),out_table_name=coeffici
>> ents_table)
>>> This will only write results into your cache without accessing the
>>> database. And I will go ahead and make this as an option in the
>> repository.
>>> BTW, your estimation was not successful, since it led to a singular
>>> matrix. Once it is successful, you will see the estimated coefficients
>>> in the output. Check values of your predictors, if some of them has the
>>> same value for all observations.
>>>
>>> Hana
>>>
>>>
>>>
>>> sivakarthik wrote:
>>>> Hello,
>>>>
>>>> From the output below I feel that my estimation was
>>>> successful but I am unable to print the output of it. I am not using
>>>> SQL for running UrbanSim. In my_estimation_config file in Eugeneà
>>>> estimation folder the output_configuration is linked with SQL
>>>> database. I want to know if there is any solution to get the output to
>>>> a particular folder without using SQL database.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> C:\opus\eugene\estimation>run_estimation.py
>>>>
>>>> Cache Directory set to: C:/opus/threecounty/2000/flatfile
>>>>
>>>> Start simulation run: started on Sat Mar 01 21:45:47 2008
>>>>
>>>> random seed = 10
>>>>
>>>> Starting simulation for year 2000: started on Sat Mar 01 21:45:47
>>>> 2008
>>>>
>>>> Closing log file:
>>>> C:/opus/threecounty/2000/flatfile\run_multiprocess.log
>>>>
>>>>
>>>>
>>>> Logging to file:
>>>> C:/opus/threecounty/2000/flatfile\year_2000_log.txt
>>>>
>>>> Simulate year 2000: started on Sat Mar 01 21:45:47 2008
>>>>
>>>> Running Household Relocation Model (from
>>>> urbansim.models.agent_reloc
>>>>
>>>> ation_model): started on Sat Mar 01 21:45:47 2008
>>>>
>>>> compute_probabilities ...
>>>>
>>>> compute_choices ...
>>>>
>>>> Number of movers: 927
>>>>
>>>> Running Household Relocation Model (from
>>>> urbansim.models.agent_reloc
>>>>
>>>> ation_model): completed...0.6 sec
>>>>
>>>> Unplace 77 agents.
>>>>
>>>> Estimating Household Location Choice Model (from
>>>> urbansim.models.hou
>>>>
>>>> sehold_location_choice_model): started on Sat Mar 01 21:45:48 2008
>>>>
>>>> Number of agents for estimation: 6944
>>>>
>>>> Sampling locations for estimation ...
>>>>
>>>> Sampling done in 1 chunk(s).
>>>>
>>>> Choice set size: 30
>>>>
>>>> Compute variables ...
>>>>
>>>> urbansim.household_x_gridcell.cost_to_income_ratio
>>>>
>>>> urbansim.gridcell.total_annual_rent
>>>>
>>>>
>>>> urbansim.gridcell.total_residential_value........0.0 sec
>>>>
>>>>
>>>>
>>>> urbansim.gridcell.total_annual_rent:
>>>> completed.......0.1 sec
>>>>
>>>>
>>>>
>>>> urbansim.household_x_gridcell.cost_to_income_ratio:
>>>> completed...
>>>>
>>>> 0.1 sec
>>>>
>>>>
>>>> urbansim.household_x_gridcell.income_and_year_built......0.0 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.household_x_gridcell.percent_high_income_households_wit
>>>>
>>>> hin_walking_distance_if_low_income
>>>>
>>>>
>>>> urbansim.gridcell.percent_high_income_households_within_walk
>>>>
>>>> ing_distance
>>>>
>>>>
>>>> urbansim.gridcell.number_of_high_income_households_withi
>>>>
>>>> n_walking_distance
>>>>
>>>>
>>>> urbansim.gridcell.number_of_high_income_households
>>>>
>>>>
>>>> urbansim.household.is_high_income........0.0 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.gridcell.number_of_high_income_households:
>>>>
>>>> completed...0.1 sec
>>>>
>>>>
>>>> urbansim.gridcell.number_of_high_income_households_withi
>>>>
>>>> n_walking_distance: completed...0.4 sec
>>>>
>>>>
>>>> urbansim.gridcell.number_of_households_within_walking_di
>>>>
>>>> stance
>>>>
>>>>
>>>> urbansim.gridcell.number_of_households.......0.1 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.gridcell.number_of_households_within_walking_di
>>>>
>>>> stance: completed...0.4 sec
>>>>
>>>>
>>>> urbansim.gridcell.percent_high_income_households_within_walk
>>>>
>>>> ing_distance: completed...0.9 sec
>>>>
>>>>
>>>> urbansim.household.is_low_income.....................0.0 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.household_x_gridcell.percent_high_income_households_wit
>>>>
>>>> hin_walking_distance_if_low_income: completed...1.0 sec
>>>>
>>>>
>>>> urbansim.household_x_gridcell.percent_low_income_households_with
>>>>
>>>> in_walking_distance_if_high_income
>>>>
>>>>
>>>> urbansim.gridcell.percent_low_income_households_within_walki
>>>>
>>>> ng_distance
>>>>
>>>>
>>>> urbansim.gridcell.number_of_low_income_households_within
>>>>
>>>> _walking_distance
>>>>
>>>>
>>>> urbansim.gridcell.number_of_low_income_households...
>>>>
>>>> .0.1 sec
>>>>
>>>>
>>>> urbansim.gridcell.number_of_low_income_households_within
>>>>
>>>> _walking_distance: completed...0.4 sec
>>>>
>>>>
>>>> urbansim.gridcell.percent_low_income_households_within_walki
>>>>
>>>> ng_distance: completed...0.5 sec
>>>>
>>>>
>>>> urbansim.household_x_gridcell.percent_low_income_households_with
>>>>
>>>> in_walking_distance_if_high_income: completed...0.5 sec
>>>>
>>>>
>>>> urbansim.household_x_gridcell.percent_minority_households_within
>>>>
>>>> _walking_distance_if_minority
>>>>
>>>>
>>>> urbansim.gridcell.percent_minority_households_within_walking
>>>>
>>>> _distance
>>>>
>>>>
>>>> urbansim.gridcell.number_of_minority_households_within_w
>>>>
>>>> alking_distance
>>>>
>>>>
>>>> urbansim.gridcell.number_of_minority_households
>>>>
>>>>
>>>> urbansim.household.is_minority...........0.0 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.gridcell.number_of_minority_households: com
>>>>
>>>> pleted...0.1 sec
>>>>
>>>>
>>>> urbansim.gridcell.number_of_minority_households_within_w
>>>>
>>>> alking_distance: completed...0.4 sec
>>>>
>>>>
>>>> urbansim.gridcell.percent_minority_households_within_walking
>>>>
>>>> _distance: completed...0.5 sec
>>>>
>>>>
>>>> urbansim.household_x_gridcell.percent_minority_households_within
>>>>
>>>> _walking_distance_if_minority: completed...0.5 sec
>>>>
>>>>
>>>> urbansim.household_x_gridcell.residential_units_when_household_h
>>>>
>>>> as_children
>>>>
>>>>
>>>> urbansim.household.is_without_children...............0.0 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.household_x_gridcell.residential_units_when_household_h
>>>>
>>>> as_children: completed...0.0 sec
>>>>
>>>>
>>>> urbansim.household_x_gridcell.young_household_in_high_density_re
>>>>
>>>> sidential
>>>>
>>>>
>>>> urbansim.gridcell.is_in_development_type_group_SSS...0.0 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.household.is_young..........................0.0 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.household_x_gridcell.young_household_in_high_density_re
>>>>
>>>> sidential: completed...0.1 sec
>>>>
>>>>
>>>> urbansim.household_x_gridcell.young_household_in_mixed_use
>>>>
>>>>
>>>> urbansim.gridcell.is_in_development_type_group_SSS...0.0 sec
>>>>
>>>>
>>>>
>>>>
>>>> urbansim.household_x_gridcell.young_household_in_mixed_use: comp
>>>>
>>>> leted...0.1 sec
>>>>
>>>> Estimate ...
>>>>
>>>> submodel: -2
>>>>
>>>> WARNING: Estimation led to singular matrix. No results.
>>>>
>>>> Estimating Household Location Choice Model (from
>>>> urbansim.models.hou
>>>>
>>>> sehold_location_choice_model): completed...4.5 sec
>>>>
>>>> Simulate year 2000:
>>>> completed....................................5.5 sec
>>>>
>>>>
>>>>
>>>> Closing log file:
>>>> C:/opus/threecounty/2000/flatfile\year_2000_log.txt
>>>>
>>>> Starting simulation for year 2000:
>>>> completed.........................5.5 sec
>>>>
>>>>
>>>>
>>>> Start simulation run:
>>>> completed..........................................5.5 sec
>>>>
>>>>
>>>>
>>>> Closing log file: C:/opus/threecounty/2000/flatfile\run_multiprocess.log
>>>>
>>>> Traceback (most recent call last):
>>>>
>>>> File "C:\opus\eugene\estimation\run_estimation.py", line 31, in ?
>>>>
>>>> estimator.estimate()
>>>>
>>>> File "c:\opus\urbansim\estimation\estimator.py", line 71, in estimate
>>>>
>>>> self.save_results(out_storage=out_storage)
>>>>
>>>> File "c:\opus\urbansim\estimation\estimator.py", line 122, in
>>>> save_results
>>>>
>>>> db_server = MysqlDatabaseServer(self.config["output_configuration"])
>>>>
>>>> File "c:\opus\opus_core\store\mysql_database_server.py", line 39, in
>>>> __init__
>>>>
>>>> raise MySqlDbImportException(e)
>>>>
>>>> opus_core.exception.mysqldb_import_exception.MySqlDbImportException:
>>>> Unable to i
>>>>
>>>> mport the MySQLdb package, which is the connection between the MySql
>>>> database an
>>>>
>>>> d Python. The underlying exception was: No module named MySQLdb
>>>>
>>>>
>>>>
>>>> Thanks and Regards,
>>>>
>>>> Siva
>>>>
Received on Fri Mar 07 2008 - 17:12:23 PST