Re: Estimation, Calibration, Validation

Jen

Yes, the estimation tutorial is on the to-do list...

Regarding calibration. Two points. First, most of the location choice
models use very large choice sets, so typically use a random sampling of
alternatives. This means that the alternatives are not fixed, and that
alternative-specific constants are not in the model. So the kind of
calibration that travel modelers are used to, involving adjusting the
alternative-specific constants to match aggregate mode shares, for
example, is not relevant or appropriate. One can of course put in dummy
variables for larger geographies, to attempt to impose some of this kind
of capacity to calibrate constants for spatial bias, but it is not clear
whether those constants should remain fixed over time, and if they are
used excessively, then any model using them would become relatively
constrained, and therefore less responsive to policy changes. So - one
should be careful with using and calibrating constants in the way travel
models have generally approached cross-sectional calibration.

Second, we have developed a new method for calibrating the uncertainty
in the entire model system using a statistical technique developed by
Adrian Raftery, called Bayesian Melding. It is described in a new paper
by Hana Sevcikova, Adrian Raftery and me in Transportation Research B,
2007, titled 'Assessing Uncertainty in Urban Simulations Using Bayesian
Melding'. We have it implemented in UrbanSim4, though it is not yet
documented to the point that it would be widely usable without
considerable expertise and effort. We need to make some assessment of
whether we can generalize it a bit and put a user-interface on it to
make it broadly usable. Oh, and write more documentation and a tutorial...

Paul

Jen Duthie wrote:
> Hello,
>
> Here in Austin, we haven't yet estimated, calibrated, or validated our
> model, but we're starting to explore how we will do this in the near
> future.
>
> If anyone on this list can share their experiences in any of these three
> areas with me, I'd greatly appreciate it. I'm especially interested in
> what methods were used and what parameters were most difficult to
> estimate/calibrate.
>
> From what I can tell so far, UrbanSim 4 has an internal mechanism for
> estimating the coefficients of choice models and regression models. I'm
> delighted to hear Paul's mention of creating tutorials for estimation
> that will help with the details.
>
> I'm not as sure how to approach calibration. I read an older (1999?)
> report on the calibration of the Eugene model, where simulated annealing
> was used. Without a widely available tool for this, I have trouble
> seeing many MPOs using such an approach.
>
> Thanks!
> Jen
> _______________________________________________
> Users mailing list
> Users_at_urbansim.org
> http://www.urbansim.org/mailman/listinfo/users
>
Received on Fri Jul 20 2007 - 08:17:06 PDT