export_grid in data_manager sometimes has memory allocation error

[anonymous]

When using export_grid to export multiple quantities from one sww file, often a memory allocation error is thrown. This seems more common with big sww files.

Traceback (most recent call last):
  File "run_broome_urs.py", line 268, in ?
    
  File "run_broome_urs.py", line 252, in run_model
    buildings_filename_out = project_urs.buildings_filename_out
  File "/d/cit/1/cit/unixhome/nbartzis/anuga/anuga_core/source/anuga/shallow_water/data_manager.py", line 1687, in export_grid
    format)
  File "/d/cit/1/cit/unixhome/nbartzis/anuga/anuga_core/source/anuga/shallow_water/data_manager.py", line 1859, in sww2dem
    quantity_dict[name] = fid.variables[name][:] 
MemoryError: can't allocate memory for array

Looking at the code the memory should not be increasing each time it outputs a different quantities' grid but thats what seems to happen. As the error always occurs after at least one grid has been written.

J:\inundation\data\western_australia\broome_tsunami_scenario_2006\anuga\outputs\20070615_063022_run_final_4.9_exmouth_nbartzis

So we need to look in to where the memory is being allocated and minimise it.

[nick]

Here are two examples of export_grid one that works and another that doesn't and throws a memory allocation error. I have simply included the directory name where you can find the screen_output file and hence the details of the model.

Firstly the one that works, the resulting sww file is about 500Mb in size J:\inundation\data\western_australia\exmouth_tsunami_scenario\anuga\outputs\20070619_042140_run_final_0_onslow_nbartzis

and secondly the one that doesn't, with sww file size of near 2Gb J:\inundation\data\western_australia\broome_tsunami_scenario_2006\anuga\outputs\20070618_064014_run_final_4.9_onslow_nbartzis

[duncan]

I've modified how quad uses its memory. Nick, could you check to see if this has fixed the problem.

[nick]

Hi Duncan There is still a problem. Which is: exporting grids from large sww file (around 1.9 Gigs) on cyclone causes the script to crash because too much swap memory gets used. However it doesn't seem to throw an error message anymore, so something might of changed.

You can try 'run_broome_urs_edit.py' on cyclone in: \inundation\data\western_australia\broome_tsunami_scenario_2006\anuga\outputs\20070618_064014_run_final_4.9_onslow_nbartzis

and then on either bombora or tornado and you should find cyclone fails and tornado/bombora passes.

'run_broome_urs_edit.py' simply sets up the right parameter for file names and runs 'export_grid' on the sww file in the directory above.

[anonymous]

doing this:

You can try 'run_broome_urs_edit.py' on cyclone in: \inundation\data\western_australia\broome_tsunami_scenario_2006\anuga\outputs\20070618_064014_run_final_4.9_onslow_nbartzis

Returns a message of Killed to the command prompt.

[anonymous]

Current feedback; (it's doing 1 quantity. Failing on the second.)

dgray@compute-0-4:/d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsun
ami_scenario_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis$ p
 run_broome_urs_edit.py
WARNING: Could not import pypar - defining sequential interface
I am processor 0 of 1 on node compute-0-4.local
final
----------------------------------------------------------------------------
Polygon   Max triangle area (m^2)   Total area (km^2)   Estimated #triangles
----------------------------------------------------------------------------
Interior  20000                     567.50              28375
Interior  5000                      21.31               4261
Interior  5000                      140.73              28146
Interior  2000                      95.84               47917
Interior  500                       95.84               191671
Bounding  150000                    18771.02            125140
Estimated total number of triangles: 607876
Note: This is generally about 20% less than the final amount
min number triangles 607876
tide 4.9
output_dir /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami_sce
nario_2006/anuga/outputs/20080306_231100_run_final_4.9_onslow_dgray/
Make directory /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami
_scenario_2006/anuga/outputs/20080306_231100_run_final_4.9_onslow_dgray/
Files run_broome_urs_edit.py and /d/xrd/gem/5/nhi/inundation/data/western_austra
lia/broome_tsunami_scenario_2006/anuga/outputs/20070618_064014_run_final_4.9_ons
low_nbartzis/project_urs.py copied
SWWFILE /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami_scenar
io_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/broome
iterate over ['broome', 'broome_time_19320_0']
Reading from /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami_s
cenario_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/broome.
sww
Output directory is /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_ts
unami_scenario_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/
broome_elevation_town_edit
------------------------------------------------
Statistics of SWW file:
  Name: /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami_scenar
io_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/broome.sww
  Reference:
    Lower left corner: [236945.900000, 7897453.160000]
    Start time: 4000.000000
  Extent:
    x [m] in [0.000000, 210284.656250], len(x) == 297213
    y [m] in [0.000000, 194475.453125], len(y) == 297213
    t [s] in [0.000000, 19260.000000], len(t) == 278
  Quantities [SI units]:
Processing quantity elevation
Processed values for elevation are in [-169.056900, 53.436504]
Creating grid
FitInterpolate: Building mesh
FitInterpolate: Building quad tree
Interpolating
Interpolated values are in [-102.167983, 36.589247]
Writing /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami_scenar
io_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/broome_eleva
tion_town_edit.prj
Writing /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami_scenar
io_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/broome_eleva
tion_town_edit.asc
Doing row 0 of 601
Doing row 61 of 601
Doing row 122 of 601
Doing row 183 of 601
Doing row 244 of 601
Doing row 305 of 601
Doing row 366 of 601
Doing row 427 of 601
Doing row 488 of 601
Doing row 549 of 601
Reading from /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami_s
cenario_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/broome.
sww
Output directory is /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_ts
unami_scenario_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/
broome_stage_town_edit
------------------------------------------------
Statistics of SWW file:
  Name: /d/xrd/gem/5/nhi/inundation/data/western_australia/broome_tsunami_scenar
io_2006/anuga/outputs/20070618_064014_run_final_4.9_onslow_nbartzis/broome.sww
  Reference:
    Lower left corner: [236945.900000, 7897453.160000]
    Start time: 4000.000000
  Extent:
    x [m] in [0.000000, 210284.656250], len(x) == 297213
    y [m] in [0.000000, 194475.453125], len(y) == 297213
    t [s] in [0.000000, 19260.000000], len(t) == 278
  Quantities [SI units]:
Processing quantity stage
Traceback (most recent call last):
  File "run_broome_urs_edit.py", line 113, in ?
    dir_comment=project_urs.dir_comment)
  File "run_broome_urs_edit.py", line 97, in run_model
    format = 'asc')
  File "/d/xrd/gem/5/nhi/inundation/sandpits/dgray/inundation/anuga_core/source/anuga/shallow_water/data_manager.py", line 1706, in export_grid
    format)
  File "/d/xrd/gem/5/nhi/inundation/sandpits/dgray/inundation/anuga_core/source/anuga/shallow_water/data_manager.py", line 1965, in sww2dem
    quantity_dict[name] = fid.variables[name][:]
MemoryError: can't allocate memory for array