Re: [UrbanSim-Users] Memory error in HLCM estimation

To my knowledge we have not tested 64 bit XP at all. I believe we only have
one 64 bit workstation and it runs linux. If anyone has a 64 bit XP machine
and is willing to install OPUS, we would appreciate the feedback!
 
--Jesse Ayers
--Center for Urban Simulation and Policy Analysis
--www.urbansim.org
 
 

-----Original Message-----
From: David Kruse [mailto:dkruse_at_aacog.com]
Sent: Thursday, January 31, 2008 7:50 AM
To: Jesse Ayers
Subject: RE: [UrbanSim-Users] Memory error in HLCM estimation

Hi Jesse,

Saw the post below and was wondering if you support Windows XP Pro X64 in 64
bit mode? If not, do the same problems exist with the 64 bit OS?

David

  _____

From: users-bounces_at_urbansim.org [mailto:users-bounces_at_urbansim.org] On
Behalf Of Jesse Ayers
Sent: Thursday, January 31, 2008 9:36 AM
To: 'Mark Kryvobokov'; users_at_urbansim.org
Subject: Re: [UrbanSim-Users] Memory error in HLCM estimation

Hi Mark-

I am not sure what platform you are working on or how much memory your
system has, but if you are using Windows, this may help. 32 bit Windows has
some limits on the amount of memory a single process can use, but there is a
way around this limit by following these tips:

http://trondheim.cs.washington.edu/cgi-bin/trac.cgi/wiki/WindowsMemoryUsage

Hopefully that is helpful to yourself or others. If that does not turn out
to be applicable, perhaps one of the other folks here can chime in with
options that may be available in the code.

--Jesse Ayers

--Center for Urban Simulation and Policy Analysis

--www.urbansim.org

-----Original Message-----
From: users-bounces_at_urbansim.org [mailto:users-bounces_at_urbansim.org] On
Behalf Of Mark Kryvobokov
Sent: Thursday, January 31, 2008 7:27 AM
To: users_at_urbansim.org
Subject: [UrbanSim-Users] Memory error in HLCM estimation

Hello,

I receive a memory error while trying to run estimation for HLCM. We have
data on 662249 households, but no data on residential units. I experimented
with different numbers of residential units, considering the ratios of
households-to-residential units in each gridcell as 1:1, 1:2, 1:3 and even
1:1000 and 3:1. The result is the same memory error:

Start simulation run: started on Thu Jan 31 16:11:26 2008

random seed = (10, 10)

Starting simulation for year 1980: started on Thu Jan 31 16:11:26 2008

Closing log file: /tmp/opus\run_multiprocess.log

Logging to file: /tmp/opus\year_1980_log.txt

Simulate year 1980: started on Thu Jan 31 16:11:26 2008

Running Household Relocation Model (from
urbansim.models.agent_relocation_model): started on Thu Jan 31 16:11:26 2008

compute_probabilities ...

compute_choices ...

Number of movers: 394660

Running Household Relocation Model (from
urbansim.models.agent_relocation_model): completed...8.9 sec

Unplace 32888 agents.

Estimating Household Location Choice Model (from
urbansim.models.household_location_choice_model): started on Thu Jan 31
16:11:38 2008

Number of agents for estimation: 662249

Sampling locations for estimation ...

Sampling done in 1 chunk(s).

Choice set size: 30

Compute variables ...

urbansim.gridcell.ln_residential_units...................0.0 sec

urbansim.gridcell.travel_time_to_CBD.....................0.0 sec

Estimate ...

submodel: -2

Estimating Household Location Choice Model (from
urbansim.models.household_location_choice_model): completed...1 min, 7.5 sec

Simulate year 1980: completed............................1 min, 19.8 sec

Closing log file: /tmp/opus\year_1980_log.txt

Traceback (most recent call last):

Starting simulation for year 1980: completed.................1 min, 19.9 sec

Start simulation run: completed..................................1 min, 19.9
sec

Closing log file: /tmp/opus\run_multiprocess.log

File "C:\MyNew\Opus41\lyonprototype41\estimation\run_estimation.py", line
69, in <module>

estimator.estimate()

File "C:\MyNew\Opus41\opus-4.1\urbansim\estimation\estimator.py", line 67,
in estimate

self.model_system.run(self.config,
write_datasets_to_cache_at_end_of_year=False)

File "C:\MyNew\Opus41\opus-4.1\urbansim\model_coordinators\model_system.py",
line 137, in run

write_datasets_to_cache_at_end_of_year=write_datasets_to_cache_at_end_of_yea
r)

File "C:\MyNew\Opus41\opus-4.1\urbansim\model_coordinators\model_system.py",
line 363, in _run_year

self.vardict[outputvar] = self.do_process(locals())

File "C:\MyNew\Opus41\opus-4.1\urbansim\model_coordinators\model_system.py",
line 441, in do_process

return eval(ev)

File "<string>", line 1, in <module>

File "C:\MyNew\Opus41\opus-4.1\opus_core\model.py", line 60, in
logged_estimate_method

results = estimate_method(*req_args, **opt_args)

File "C:\MyNew\Opus41\opus-4.1\urbansim\models\location_choice_model.py",
line 195, in estimate

agents_index, procedure, estimate_config=estimate_config,
debuglevel=debuglevel)

File "C:\MyNew\Opus41\opus-4.1\opus_core\choice_model.py", line 311, in
estimate

result = self.estimate_step()

File "C:\MyNew\Opus41\opus-4.1\opus_core\choice_model.py", line 339, in
estimate_step

result[submodel] = self.estimate_submodel(self.get_all_data(submodel),
submodel)

File "C:\MyNew\Opus41\opus-4.1\opus_core\choice_model.py", line 366, in
estimate_submodel

return self.procedure.run(data, upc_sequence=self.upc_sequence,
resources=self.estimate_config)

File "C:\MyNew\Opus41\opus-4.1\opus_core\configurable.py", line 38, in
config_run_method

results = run_method(*req_args, **opt_args)

File "C:\MyNew\Opus41\opus-4.1\opus_core\bhhh_mnl_estimation.py", line 54,
in run

l_2=self.mnl_loglikelihood(data, b2, depm).sum()

File "C:\MyNew\Opus41\opus-4.1\opus_core\bhhh_mnl_estimation.py", line 115,
in mnl_loglikelihood

self.upc_sequence.compute_utilities(data, b, self.resources)

File "C:\MyNew\Opus41\opus-4.1\opus_core\upc_sequence.py", line 58, in
compute_utilities

self.utilities = self.utility_class.run(data, coefficients, resources)

File "C:\MyNew\Opus41\opus-4.1\opus_core\configurable.py", line 38, in
config_run_method

results = run_method(*req_args, **opt_args)

File "C:\MyNew\Opus41\opus-4.1\opus_core\linear_utilities.py", line 37, in
run

return sum(utility,axis=2)

File "C:\Python25\Lib\site-packages\numpy\core\fromnumeric.py", line 866, in
sum

return sum(axis, dtype, out)

MemoryError
Received on Thu Jan 31 2008 - 07:53:55 PST