{11} Active Tickets by Priority (Full Description) (46 matches)

Domain can safely be pickled, but boundaries that use functions or lambdas cannot be pickled. We need an alternative means of checkpointing ANUGA runs, possibly by separating the boundaries from the Domain so they will not be pickled along with the Domain.

The calculation of boundary flows is not accurate. The current implementation uses the x and y momentum on the boundary edge to calculate the flux of water. Actually the flux should be calculated via the numerical flux method. This info will be available in the well balanced implementation.

Shows up if you use all reflective boundaries. Then there should be zero boundary flows!

ANUGA was tested for Python2.6 on 13th July 2009 but we need to move towards Python3.0 which the goal of Py2.6.

The way to see what needs to be done is to run ANUGA under Python2.6 using the -3 flag, e.g.

python -3 test_all.py

I started this process in changeset:7309 (the caching module).

This was suggested by Joaquim Luis and Rajaraman in Sourceforge postings November 2007. They had an sww file used with Field boundary which didn't quite cover the boundary segment. The old error message just said that NANs were found, but now why. This was confusing since the sww didn't contain NANs.

The georeferencing object isn't used consistently yet. Some code in source:inundation/pyvolution/data_manager.py is working directly with xllcorner and yllcorner

Also make sure that georeferencing is tested everywhere (esp least squares)

Currently the set of points in geospatial_data are assumed to be in the Southern hemisphere. There should be an attribute so the hemisphere is known.

ted Rigby has shown that ANUGA isn't performing very well if one wants to convert 18 million points in a csv file to the pts netcdf format. Currently, we do this by creating a Geospatial data object from the csv file and then exporting it to the pts format. The problem is that all data is stored in memory and this causes thrashing for large datasets. This problem is holding Ted back in running a validation against the Macquarie-Rivulaet catchment.

One solution would be to make use of blocking so that the csv file isn't read until the export code is run - and then only in blocks.

Another, and perhaps simpler, approach would be to write a conversion script, csv2pts, which does the conversion without ever having to store all data in memory.

Now that large NetCDF format has been in place for a good while and the viewer has been update to use this, we can remove the need for splitting into chunks less than 2GB. I think this is in store_timestep()

I propose doing this in two steps:

1: Increase the variable controlling the splitting to a number large enough to prevent it. Verify all tests and validations work. 2: Carefully remove the splitting algorithm in store_timestep() and the codes that were modified to observe splitted sww files. Verify all tests and validations.

This supersedes ticket:257 which I have closed.

It would be useful to have a boundary condition that absorbs everything without being transmissive and without reflecting anything.

Attached is a paper from former advisor Jesper Larsen which outlines a technique for creating an absorbing boundary condition.

As sww files tend to get very big, ANUGA could perhaps compress them on the fly (much like Google Earth can create kmz instead of kml). The file format could be called swz.

A test showed that they do compress Original sww 684,150kb Compressed folder by Windows XP 337,443kb Zipped by Zip Genius 236,883kb So there is some file size to save, but compressed sww files need to be handled by the other products like ANUGA Viewers, WateRide? and the Linux and 64 bit versions of ANUGA.

When generating a mesh, very small (~0.0001) and very large(~1e12) maximum areas are being set to a max area of 1.

It would be good to allow ANUGA direct access to GIS files such as shape files, GML, KML and so on. This should probably be done through the GDAL library.

Other tools that are based on GDAL are Quantum GIS (​http://www.qgis.org/) and ogr2ogr (​http://www.gdal.org/ogr/ogr2ogr.html).

Quantum GIS is very promising as it allows very easy creation of polygons and has a light simple interface. It is unknown how well qgis handles projections.

Ogr2ogr is commandline driven and can convert most formats. Two important examples are

ogr2ogr -f "GML" map_area.gml map_area.shp

which creates a gml file from a shape file. That file can easily be converted to csv if needed (however, it would be better to read the GDAL formats directly).

ogr2ogr -f "KML" -t_srs WGS84 bay_area.kml bay_area.shp

which converts a shape file in UTM coordinates to kml in WGS84 for use with Google Earth.

First task in this ticket will be to see if Python can get direct access to the underlying libraries. One, called pymod, ships with ogr2ogr. DGAL may be the same thing, but I am not sure.

This code needs a revamp. The semantics of 'location' and 'indices' isn't clear and we also need to have the option of restricting the area by specifying a polygon as requested by rajaraman 22 April 2008.

I suggest we get rid of edge_values as an option and also to rename 'unique vertices' to the more intuitive name 'nodes' and let that be the default. This is what is set using point sets by least squares anyway. I the user needs to set vertex_values by triangle e.g. to include discontinuities it may be better to imply that by the shape of input value in case it is an array.

I also suggest that we allow restrictions by region_tag in set_quantity to roll set_region in seemlessly. In fact set_region could probably be refactored using set_quantity with the polygon in question allowing for more code reuse.

This ticket should be done in conjunction with ticket:250 and ticket:254

It seems that domains don't get the zone info from data so sww files have zone=-1. It can easily be overridden by setting domain.geo_reference.zone manually, but it'd be best to get it from mesh and or data points.

Need to run all demos used in the manual. This will also check things like loading from points file and realistic fitting procedures. This could be added to the validation suite.

Also make fast version of something like the Cairns example (two timesteps, and smaller points file) to run within the unit test suite.

Ted Rigby discovered that a hydrograph used with Transmissive_Momentum_Set_stage didn't use the real model time when domain.starttime > 0

Changing domain.time to domain.get_time in the evaluate method solves the problem, but opens the question about the general architecture. Currently, relative times are passed in (e.g. file_boundary) but changing this will touch many modules and will have to be done carefully.

The overriding principle, though, should be that whenever a modeller references time it should be the absolute time (internal time + starttime) just as we intend with spatial coordinates.

Implement a forcing function for variable rainfall.

Suggested requirements:

Take as input an arbitrary number of time series each containing time (s) and rainfall (mm) in two columns. This is the same format we used for the rainfall test using one rainfall gauge.

The time intervals might be different for each time series, i.e. some might be sampled every 300s while others could be sampled at other times. The intervals can also vary within one time series

Then build a forcing function based on this data such that for each timestep t in ANUGA:

Values at t be established for each timeseries through linear interpolation. Values be established across the domain at time t using the built-in smoothing code Those values be applied as we did in the test case except they will be spatially varying.

To support the storm surge inundation modelling capability.

See report from WRL (draft delivered to GA, see TRIM 2010/1135)

From Gareth Davies:

At present in the code, the user inputs polygons by typing something like

Bounding_polygon = anuga.read_polygon(‘my_extent.csv’)

where ‘my_extent.csv’ is a file with xy coordinate pairs (and no header), like this:

x1, y1 x2, y2 x3, y3 x4, y4

What we would like is for the user to be able to type:

bounding_polygon = anuga.read_polygon_shapefile(‘myshapefile.shp’)

[where ‘myshapefile.shp’ is a polygon shapefile which ONLY contains a single polygon, which the user wants to use as a bounding polygon],

and for the above command to have the same effect as the previous command.

## Things to consider are:

1) Should we support shapefiles with multiple polygons in them? I suggest no, because it will make it more complex to code and there is more potential for user error. However, this could be done. Perhaps better to have a warning or error if there is more than 1 polygon in the shapefile.

2) Should we check the orientation of the polygon? At present, I believe that ANUGA can have errors if all polygons are not oriented in the same way (i.e. clockwise or counter-clockwise). We could potentially ensure that the polygon was oriented clockwise (using a simple test like this one: ​http://paulbourke.net/geometry/clockwise/index.html, and re-ordering if needed). I suggest that this should either not be done, or optionally be done, with a warning if the vertices are being re-ordered. The reason is that one could imagine a situation where a user entered some polygons as csv files in a counterclockwise fashion, and then entered another polygon using a shapefile which had been ordered counter-clockwise – and the user would not wish for this polygon to be automatically re-ordered without a warning.

3) Should we support shapefiles that are not polygons (e.g. lines or points)? I think we should do this with a warning, however, we must assume that the ordering of the points is the same as it would be for a polygon.

When running anuga_core/test_all.py on the NCI and getting thos error:

MKL FATAL ERROR: Cannot load neither libmkl_mc3.so nor libmkl_def.so

load up the appropriate matplotlib module for your version of python e.g. for python 2.6 add:

module load python/2.6-matplotlib

to your logon script.

(module load python/2.7.2-matplotlib for python 2.7.2 etc. Make sure to check with the command "module avail" is your python version has an associated matplotlib module.)

Parallel domains are equipped with 'ghost' triangles. It would be useful to have access to the true boundary and somehow hook get_boundary_polygon() into that rather than giving the extended boundary.

One way would be to build the local meshes temporarily (as is done already) and generate a boundary_polygon to be stored with the parallel_domain. Then override get_boundary_polygon() to return that.

Here's information from Linda (18/10/6) on how to get access to the individual meshes:

Hi,

 	The triangles, nodes etc are stored in a dictionary, eg
     submesh["full_nodes"], submesh["full_triangles"], 
submesh["full_boundary"]. To obtain, say the triangles for submesh 1 use 
submesh["full_triangles"][1]. The full_... entries are the submeshes 
without the ghosts. The triangle structure is the same as the standard 
structure, but the node structure has been extended to include the global 
ID to local ID mapping (I have not removed any information just extended 
it). If you plan to pass the these boundary polygons around as an 
additional structure, you will need to also update build_commun.py and 
change build_local.py to change the global ID to a local ID once the 
information has been sent to the processor.

 	If all that you need is the nodes and triangles (you do not need 
to know any neighbour information) you may want to look at build_local.py. build_local is suppose to be run on each processor so the 
triangles and nodes for that processor are stored in submesh["full_nodes"] 
and submesh["full_triangles"]. The advange here is that you will not need 
to update the communication calls. You still have to be careful becasue 
the information in submesh["full_.."] is originally stored in terms of the 
global ID, it is not until I get to build_local_GA that the global IDs are mapped to the local IDs.

 								Linda




On Wed, 18 Oct 2006, Ole.Nielsen@ga.gov.au wrote:

> Hi Linda
>
> Thanks for mail.
>
> I understand that you are busy.
> Perhaps all I need is a hint on how to do it.
>
> If you can tell me where in the code I have access to the partitioned 
> meshes in terms of their nodes (Nx2, Float) and triangles (Mx3, Int). 
> I was looking in build_submesh, I am unsure if all the partitioned 
> triangles are lumped into one structure. How can I get access to the 
> individual submeshes (without ghosts) in terms of nodes and triangles?
>
>
> If you can tell me that, I'll be right to do it.
>
> Cheers
> Ole
>
>
> ---------------------------------------------------------------------
> Dr. Ole Nielsen                           |  Computational Scientist
> Geospatial and Earth Monitoring Division  |  E: Ole.Nielsen@ga.gov.au
> Geoscience Australia                      |  P: +61 2 6249 9048
> Canberra, Australia                       |  F: +61 2 6249 9986
> ---------------------------------------------------------------------
>
>
> -----Original Message-----
> From: Linda Stals [mailto:stals@maths.anu.edu.au]
> Sent: Wednesday, 18 October 2006 9:36
> To: Nielsen Ole
> Cc: stephen.roberts@anu.edu.au; u4112456@anu.edu.au
> Subject: RE: Mesh partitioning
>
> Hi,
>
> 	I will put it on my TODO list, but I have the end-of-semester and 
> paper due so may take a while to bubble up to the top.
>
> 								Linda
>
>
> On Tue, 17 Oct 2006, Ole.Nielsen@ga.gov.au wrote:
>
>> Hi again Linda
>>
>> >EE 361 1 -0.475
>> >Ee 361 1 -0.455140483872
>>
>> >Am I reading something wrong?
>>
>> Probably a flow on effect.
>> Anyways, I works when I moved the boundary allocation to AFTER the 
>> distribution which makes heaps of sense in retrospect. I know I can 
>> make boundaries independent of the domain and will do so later on.
>>
>>
>> Meanwhile I have one request for some help.
>>
>> I would love to be able to get the true boundary polygon for each 
>> mesh (available as get_boundary_polygon() in the normal meshes). I 
>> have started doing in build_submesh.py but am unsure how to get that 
>> for EACH submesh. Can you help?
>>
>> Cheers and thanks
>> Ole
>>

It is evident in zero flow situations that the flux and gravity terms are perhaps not well balanced, or that the method used to deal with wet and dry interface, induces unrealistic flows.

Problem should be investigated to ascertain solution and implement solution.

A proper 1-D flow model would supersede previous (and unfinished) work on culverts. See closed ticket:145 for more info