Diagram
AMOSInterface depicts the steps in AMOS that need to interface with
UrbanSim? and MALTA for data.
Given these data requirement, the diagram below shows the data exchange between
UrbanSim? - AMOS - MALTA. AMOS directly accesses household and person attributes from
UrbanSim?, which indirectly links to parcels. AMOS generates three tables: person activities, person trips, and vehicle trips and pass them to MALTA for routing and dynamic assignment and gets arrival time for each person trips in return. In this particular data flow chart, we propose to run an independent process for fast query to Time-space Prim of the whole day from the previous iteration of MALTA (TSP Graph archives).
Previous Discussion:
AMOS - MALTA Interface
Currently we envision that AMOS models will be implemented within OPUS, using existing model templates, possibly with some new additions. These models read and write to Python Numpy arrays in memory.
What are the major options for interfacing the models?
Loose Coupling - file exchange between stand-alone programs
- This could be either files or database tables
- OPUS Model Coordinator invokes AMOS and MALTA once per minute, but does not access internal classes or methods of either
- Benefits: lightweight interface (easy to implement), also more tuned to heterogeneous computing environment (computers can locate at different locations)
- Costs: slow performance, especially considering the need to iterate to convergence
Tight Coupling - in-memory data exchange with API calls
- Python and C++ can pass data in memory, translating between C++ objects in MALTA and Numpy arrays used in Python
- OPUS Model Coordinator calls AMOS and MALTA via their API, using SWIG to access MALTA
- Benefits: Performance is high since the data is shared in memory and not written to database until convergence
- Costs: More implementation effort for the interface (SWIG wrapper, clean APIs), but these have benefits as well, co-location of computers, depending on the communication frequencies (YC).
General Questions Regarding the Model Interface
Does Malta need to communicate with a particular node (MPI) within MALTA?
- No - it should be sufficient to have a master (combined) array of vehicle and person trips passed to MALTA, and retrieve vehicle arrival times and skims.
How should the translation of data from internal data structures within MALTA to Numpy Arrays used in AMOS occur?
- Option 1: write a Python wrapper (e.g. SWIG) for AMOS that allows the data to be passed to Python as a Numpy Array in memory (fastest)
- Option 2: write a table in Postgres and have MALTA read the table (avoids writing wrapper, but is much slower)
Do MALTA and AMOS need to be running on the same computer?
- Ideally, we should be able to have them run on the same or different computers. We need to identify the most plausible architectures for this.
- Having MALTA and AMOS running on remote locations is plausible, but may not be practical due to frequent communication. Ideally, they are in the same cluster to reduce the communication overhead (YC).
What data needs to be passed from AMOS to MALTA?
Every One Minute AMOS passes to MALTA:
What data needs to be passed from MALTA to AMOS?
Every One Minute MALTA passes to AMOS:
- Vehicle Arrival Times: [vehicle_trip_id, arrival_time]
- These are relatively small arrays of varying length depending on time of day
AMOS will read archived results from prior day (or from bootstrap), not minute-by-minute from MALTA (?)
-- PaulWaddell - 03 Nov 2009
Topic revision: r12 - 17 Feb 2010 - 03:32:57 -
LimingWang