Changeset 7276 for anuga_validation


Ignore:
Timestamp:
Jun 30, 2009, 2:07:41 PM (16 years ago)
Author:
ole
Message:

Merged numpy branch back into the trunk.

In ~/sandpit/anuga/anuga_core/source
svn merge -r 6246:HEAD ../../branches/numpy .

In ~/sandpit/anuga/anuga_validation
svn merge -r 6417:HEAD ../branches/numpy_anuga_validation .

In ~/sandpit/anuga/misc
svn merge -r 6809:HEAD ../branches/numpy_misc .

For all merges, I used numpy version where conflicts existed

The suites test_all.py (in source/anuga) and validate_all.py passed using Python2.5 with numpy on my Ubuntu Linux box.

Location:
anuga_validation
Files:
38 edited
1 copied

Legend:

Unmodified
Added
Removed

  • anuga_validation/automated_validation_tests/README.txt

    r3706 r7276  
    44
    55The validation tests take the form of automated unit tests except that they
    6 all test the whole system rather than individual components and that they generally take considerably longer time to execute than normal unittests.
     6all test the whole system rather than individual components and that they
     7generally take considerably longer time to execute than normal unittests.
    78
    89The master script, validate_all.py, will run through all tests. 
     10
     11NOTE: In some circumstances there may be problems running validate_all.py
     12IF MORE THAN ONE PYTHON IS INSTALLED.  The workaround is to:
     131. Decide which python you want to use, and get the absolute path to that
     14   executable ('which python2.5', for example)
     152. Create a local 'bin' directory:
     16       mkdir ~/bin
     173. Create a 'python' link in that directory to the python executable:
     18       ln -s ~/bin/python <path_from_step_1>
     194. Create or modify your PATH variable in ~/.bashrc to include:
     20       export PATH=$HOME/bin:$PATH
     21   That is, ensure your 'local' bin is searched first
     225. Remove any 'python' alias you may have defined in ~/.bashrc (or elsewhere)
     236. Ensure all the above changes are in effect by opening a new terminal
     24
     25The above workaround was tested under Linux.  Not sure about Windows.

  • anuga_validation/automated_validation_tests/UQ_runup_2006/Dam break_240406_0011.lic

    r5337 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
    3   <ga_license_file>
     1<?xml version='1.0' encoding='iso-8859-1'?>
     2<ga_license_file>
    43    <metadata>
    54      <author>Duncan Gray</author>

  • anuga_validation/automated_validation_tests/UQ_runup_2006/actual.lic

    r5105 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
    3   <ga_license_file>
     1<?xml version='1.0' encoding='iso-8859-1'?>
     2<ga_license_file>
    43    <metadata>
    54      <author>Duncan Gray</author>

  • anuga_validation/automated_validation_tests/fitting/validate_benchmark_fit.py

    r7259 r7276  
    2424mem 30200
    2525
     262009 June 22 - RW
     27Changed the time/memory values to be exactly those found on the hardware.
     28Took 10 measurements, used maximum in found values.
     29
    2630"""
    2731
    … …  
    3438    def setUp(self):
    3539        pass
    36        
    3740
    3841    def tearDown(self):
    … …  
    4548    # That's because v4872 is using geo-ref, whereas the
    4649    # previous version did not.
     50
    4751    def test_fit_time_and_mem(self):
    4852        import socket
     53
     54        # get hostname - for *nix and Windows
    4955        host =  socket.gethostname()
    50         #print "host", host
     56
     57        # define dictionary of expected time/memory usage per machine.
     58        # key must be unique *prefix* of machine name, lowercase.
     59        # value is tuple: (time, memory) in seconds and KiB.
     60        expected_results = {'tornado':  (10.8, 40468),  # tornado headnode
     61                            'cyclone':  (7.4,  40468),  # cyclone headnode
     62                            'compute':  (10.8, 40468),  # cluster computenode
     63                            'nautilus': (8.1,  16104),  # Ole's 32bit Ubuntu
     64                            'bogong':   (14.2, 30000),  # ANU?
     65                            'pc-31569': (31.6, 15000),  # DSG's PC?
     66                            'pc-32572': (12.8, 15788),  # Ross' 32bit work Ubuntu
     67                            'saturn':   (12.8, 39404)   # Ross' 64bit home Ubuntu
     68                           }
     69
     70        # run trial, report on time and memory
    5171        ben = BenchmarkLeastSquares()
    52         time, mem, num_tri, one_t, more_t, quad_t = ben.trial(
    53             num_of_points=1000
    54             ,maxArea=0.0001
    55             ,is_fit=True
    56             ,segments_in_mesh=False
    57             ,use_file_type='pts'
    58             ,save=False
    59             )
    60            
    61            
    62         #print "time", time
    63         #print "mem", mem
     72        (time, mem, num_tri, one_t,
     73         more_t, quad_t) = ben.trial(num_of_points=1000,
     74                                     maxArea=0.0001,
     75                                     is_fit=True,
     76                                     segments_in_mesh=False,
     77                                     use_file_type='pts',
     78                                     save=False
     79                                    )
    6480
    65         #Defaults
    66         time_standard = 120.
    67         mem_standard = 50000
    68        
    69         if host.find('tornado') == 0:
    70             # Tornado headnode
    71             #time_standard = 14.5
    72             time_standard = 24.
    73             mem_standard = 40468
    74            
    75         elif host.find('compute') == 0: # cyclone or tornado node
    76             time_standard = 19.0
    77             mem_standard = 40468
     81        # Get expected machine values, else a default set of values
     82        time_standard = 15.0        # max of above, plus a little
     83        mem_standard = 40000
     84        for key in expected_results:
     85            if host.lower().startswith(key):
     86                (time_standard, mem_standard) = expected_results[key]
     87                break
    7888
    79         elif host.find('cyclone') == 0: # cyclone headnode
    80             time_standard = 13.3
    81             mem_standard = 40468
    82             #mem_standard = 29424
    83 
    84         elif host.find('nautilus') == 0:
    85             time_standard = 27.6
    86            
    87             # v 4910 is giving a mem of 15572
    88             mem_standard = 16104
    89            
    90 
    91         elif host.find('bogong') == 0:
    92             time_standard = 14.2
    93             mem_standard = 30000 # Updated by Ole 20080507
    94 
    95 
    96         elif host.find('pc-31569') == 0: # DSG's PC
    97             time_standard = 31.6
    98             mem_standard = 15000 #?
    99 
    100         elif host.find('PC-32572') == 0: # Ross' Ubuntu box - nump trunk
    101             time_standard = 12.8
    102             mem_standard = 15788 #?
    103 
    104             """
    105             test_fit_time_and_mem (__main__.Test_uq) ...  very start mem_usage() 98076
    106  before fitting mem_usage() 120012
    107  after fitting mem_usage() 150212
    108 time 15.19490695
    109 mem 30200
    110 
    111 test_fit_time_and_mem (__main__.Test_uq) ...  very start mem_usage() 98108
    112  before fitting mem_usage() 134696
    113  after fitting mem_usage() 155820
    114 
    115             """
    116        
    117         # Do the assertions here
     89        # don't want exception here, report on time *and* memory
     90        got_error = False
    11891        msg = ('Time used was %.1f s, standard is %.1f s +20%% (%.1f s)'
    11992               % (time, time_standard, int(time_standard*1.2)))
    120         assert time < time_standard*1.2, msg
    121 
    122 
     93        #print msg
     94        #assert time < time_standard*1.2, msg
     95        if time > time_standard*1.2:
     96            print msg
     97            got_error = True
    12398
    12499        if sys.platform == 'win32':
    … …  
    135110        msg = ('Memory used was %d KiB, standard is %d KiB +20%% (%d KiB)'
    136111               % (mem, mem_standard, int(mem_standard*1.2)))
     112        #print msg
    137113        assert mem < mem_standard*1.2, msg           
     114
     115        if got_error:
     116            raise RuntimeError
    138117           
    139118#-------------------------------------------------------------

  • anuga_validation/automated_validation_tests/flow_tests/test_inflow_using_flowline.py

    r7144 r7276  
    99
    1010verbose = True
    11 import Numeric as num
     11import numpy as num
    1212
    1313#------------------------------------------------------------------------------

  • anuga_validation/automated_validation_tests/okushiri_tank_validation/Benchmark_2_Bathymetry.lic

    r5025 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>

  • anuga_validation/automated_validation_tests/okushiri_tank_validation/Benchmark_2_input.lic

    r5025 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>

  • anuga_validation/automated_validation_tests/okushiri_tank_validation/compare_timeseries_with_measures.py

    r6160 r7276  
    66"""
    77
    8 import Numeric as num
     8import numpy as num
    99from Scientific.IO.NetCDF import NetCDFFile
    1010

  • anuga_validation/automated_validation_tests/okushiri_tank_validation/create_okushiri.py

    r6160 r7276  
    33
    44
    5 import Numeric as num
     5import numpy as num
    66
    77from anuga.pmesh.mesh import *
    … …  
    99from anuga.coordinate_transforms.geo_reference import Geo_reference
    1010from anuga.geospatial_data import Geospatial_data
     11from anuga.config import netcdf_float
    1112
    1213import project
    … …  
    111112
    112113    N = len(lines)
    113     T = num.zeros(N, num.Float)  #Time
    114     Q = num.zeros(N, num.Float)  #Values
     114    T = num.zeros(N, num.float)  #Time
     115    Q = num.zeros(N, num.float)  #Values
    115116
    116117    for i, line in enumerate(lines):
    … …  
    128129    fid.starttime = 0.0
    129130    fid.createDimension('number_of_timesteps', len(T))
    130     fid.createVariable('time', num.Float, ('number_of_timesteps',))
     131    fid.createVariable('time', netcdf_float, ('number_of_timesteps',))
    131132    fid.variables['time'][:] = T
    132133
    133     fid.createVariable('stage', num.Float, ('number_of_timesteps',))
     134    fid.createVariable('stage', netcdf_float, ('number_of_timesteps',))
    134135    fid.variables['stage'][:] = Q[:]
    135136
    136     fid.createVariable('xmomentum', num.Float, ('number_of_timesteps',))
     137    fid.createVariable('xmomentum', netcdf_float, ('number_of_timesteps',))
    137138    fid.variables['xmomentum'][:] = 0.0
    138139
    139     fid.createVariable('ymomentum', num.Float, ('number_of_timesteps',))
     140    fid.createVariable('ymomentum', netcdf_float, ('number_of_timesteps',))
    140141    fid.variables['ymomentum'][:] = 0.0
    141142

  • anuga_validation/automated_validation_tests/okushiri_tank_validation/output_ch5-7-9.lic

    r5025 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>
    … …  
    87  <datafile>
    98    <filename>output_ch5-7-9.txt</filename>
    10     <checksum>-1044604103</checksum>
     9    <checksum>3250363193</checksum>
    1110    <publishable>Yes</publishable>
    1211    <accountable>Ole Nielsen</accountable>

  • anuga_validation/automated_validation_tests/okushiri_tank_validation/test_caching_of_set_quantity.py

    r6708 r7276  
    1313from anuga.fit_interpolate.fit import _fit_to_mesh
    1414import project
    15 import Numeric as num
     15import numpy as num
    1616import time
    1717

  • anuga_validation/automated_validation_tests/patong_beach_validation/build_urs_boundary.py

    r6844 r7276  
    1111
    1212from Scientific.IO.NetCDF import NetCDFFile
    13 import Numeric as num
     13import numpy as num
    1414
    1515from anuga.shallow_water.data_manager import urs2sts
    16 
    17 import Numeric as num
    1816
    1917
    … …  
    148146
    149147        weight_factor = 1.0
    150         mux_weights = weight_factor*num.ones(len(mux_filenames), num.Float)
     148        mux_weights = weight_factor*num.ones(len(mux_filenames), num.float)
    151149           
    152150        order_filename = project.urs_order

  • anuga_validation/automated_validation_tests/patong_beach_validation/cmpsww.py

    r7040 r7276  
    1313import getopt
    1414from Scientific.IO.NetCDF import NetCDFFile
    15 import Numeric as num
     15import numpy as num
    1616from anuga.config import netcdf_mode_r
    1717
    … …  
    2121#####
    2222
    23 # allowable 'slop' when testing two float values
    24 epsilon = 1.0e-9
     23# Global for the '-q' quiet flag
     24quiet = None
     25
     26# default tolerances - same as numpy defaults
     27default_abs_tolerance = 1.0e-08
     28default_rel_tolerance = 1.0000000000000001e-05
    2529
    2630# Global attributes that should exist and be same in both files
    … …  
    6973# @param variables A list of variable names to compare.
    7074# @return Returns if files 'equal', else raises RuntimeError.
    71 def files_are_the_same(files, globals=None, timesteps=None, variables=None):
     75def files_are_the_same(files, globals=None, timesteps=None, variables=None,
     76                       rel_tolerance=default_rel_tolerance,
     77                       abs_tolerance=default_abs_tolerance):
    7278    # split out the filenames and check they exist
    7379    (file1, file2) = files
    … …  
    222228    #####
    223229
     230    error_msg = ''
     231    glob_vars_bad = {}
     232    data_vars_bad = {}
     233
    224234    # check values of global attributes
    225235    for glob_name in globals:
    226236        if getattr(fid1, glob_name) != getattr(fid2, glob_name):
    227             error_msg += ("\nFiles differ in global '%s':\n"
    228                           "%s: '%s',\n"
    229                           "%s: '%s'\n"
    230                           % (glob_name, filename1, str(g1), filename2, str(g2)))
     237            print("\nFiles differ in global '%s':\n"
     238                  "%s: '%s',\n"
     239                  "%s: '%s'"
     240                  % (glob_name, filename1, str(g1), filename2, str(g2)))
     241            glob_vars_bad[glob_name] = glob_vars_bad.get(glob_name, 0) + 1
    231242            error = True
    232243
    233244    # check data variables, be clever with time series data
     245    max_rel_difference = -1
     246    diff_count = 0
    234247    for var_name in variables:
    235248        var_dims = expected_variables[var_name]
    … …  
    239252                var1 = num.array(fid1.variables[var_name][t,:])
    240253                var2 = num.array(fid2.variables[var_name][t,:])
    241                 if var1 != var2:
     254                if not num.allclose(var1, var2,
     255                                    rtol=rel_tolerance, atol=abs_tolerance):
     256                    error = True
    242257                    for i in xrange(len(var1)):
    243                         if var1[i] != var2[i]:
    244                             error_msg += ('\nFiles differ in variable '
    245                                               '%s[%d,%d]:\n'
    246                                           '%s: %f\n'
    247                                           '%s: %f\n'
    248                                           'difference=%f\n'
    249                                           % (var_name, t, i,
    250                                              filename1, var1[i],
    251                                              filename2, var2[i],
    252                                              var1[i]-var2[i]))
    253                             break
    254                     error = True
     258                        if not num.allclose(var1[i], var2[i],
     259                                            rtol=rel_tolerance,
     260                                            atol=abs_tolerance):
     261                            abs_difference = num.abs(var1[i]-var2[i])
     262                            max_a_b = num.max(num.abs(var1[i]),
     263                                              num.abs(var2[i]))
     264                            rel_difference = num.abs(abs_difference/max_a_b)
     265
     266                            if not quiet:
     267                                print('\nFiles differ in variable '
     268                                      '%s[%d,%d]:\n'
     269                                      '%s: %f\n'
     270                                      '%s: %f\n'
     271                                      'abs. difference=%e, rel. difference=%e\n'
     272                                      % (var_name, t, i,
     273                                         filename1, var1[i],
     274                                         filename2, var2[i],
     275                                         abs_difference,
     276                                         rel_difference))
     277
     278                            if rel_difference > max_rel_difference:
     279                                max_rel_difference = rel_difference
     280                                max_rel_difference_abs = abs_difference
     281                                max_rel_difference_a = var1[i]
     282                                max_rel_difference_b = var2[i]
     283
     284                            data_vars_bad[var_name] = data_vars_bad.get(var_name, 0) + 1
     285                            diff_count += 1
    255286        else:
    256287            # simple data, check whole thing at once
    257288            var1 = num.array(fid1.variables[var_name][:])
    258289            var2 = num.array(fid2.variables[var_name][:])
    259             if not num.allclose(var1, var2):
     290            if not num.allclose(var1, var2,
     291                                rtol=rel_tolerance, atol=abs_tolerance):
    260292                for j in xrange(len(var1)):
    261                     if abs(var1[j] - var2[j]) > epsilon:
    262                         error_msg += ('\nFiles differ in variable '
    263                                       '%s[%d]:\n'
    264                                       '%s: %f\n'
    265                                       '%s: %f\n'
    266                                       'difference=%f\n'
    267                                        % (var_name, j, 
    268                                           filename1, var1[j],
    269                                           filename2, var2[j],
    270                                           var1[j]-var2[j]))
    271                         break
     293                    if not num.allclose(var1[j], var2[j],
     294                                          rtol=rel_tolerance, atol=abs_tolerance):
     295                        abs_difference = num.abs(var1[j]-var2[j])
     296                        max_a_b = num.max(num.abs(var1[j]),
     297                                          num.abs(var2[j]))
     298                        rel_difference = num.abs(abs_difference/max_a_b)
     299
     300                        if not quiet:
     301                            print('\nFiles differ in variable '
     302                                  '%s[%d]:\n'
     303                                  '%s: %f\n'
     304                                  '%s: %f\n'
     305                                  'abs. difference=%e, rel. difference=%e\n'
     306                                   % (var_name, j, 
     307                                      filename1, var1[j],
     308                                      filename2, var2[j],
     309                                      abs_difference,
     310                                      rel_difference))
     311
     312                        if rel_difference > max_rel_difference:
     313                            max_rel_difference = rel_difference
     314                            max_rel_difference_abs = abs_difference
     315                            max_rel_difference_a = var1[j]
     316                            max_rel_difference_b = var2[j]
     317
     318                        data_vars_bad[var_name] = data_vars_bad.get(var_name, 0) + 1
     319                        diff_count += 1
    272320                error = True
    273321
    … …  
    280328
    281329    if error:
     330        error_msg += ('\nNumber of data differences=%d\n'
     331                      'Maximum relative data difference=%e\n'
     332                      'associated absolute difference=%e\n'
     333                      "associated 'a' value=%e\n"
     334                      "associated 'b' value=%e\n"
     335                      % (diff_count, max_rel_difference, max_rel_difference_abs,
     336                         max_rel_difference_a, max_rel_difference_b))
     337        error_msg += ('\nglob_vars bad=%s\n' % str(glob_vars_bad))
     338        error_msg += ('\ndata_vars bad=%s\n' % str(data_vars_bad))
    282339        raise RuntimeError, error_msg
    283340
    … …  
    293350    a('where <options> is zero or more of:\n')
    294351    a('                   -h        print this help\n')
     352    a('                   -q        be quiet, print only summary of differences\n')
    295353    a("                   -a <val>  set absolute threshold of 'equivalent'\n")
    296354    a("                   -r <val>  set relative threshold of 'equivalent'\n")
    … …  
    319377# @return The status code the program will exit with.
    320378def main(argv=None):
     379    global quiet
     380
    321381    if argv is None:
    322382        argv = sys.argv
    … …  
    324384    try:
    325385        try:
    326             opts, args = getopt.getopt(argv[1:], 'hg:t:v:',
     386            opts, args = getopt.getopt(argv[1:], 'hqa:g:r:t:v:',
    327387                                       ['help', 'globals',
    328388                                        'variables', 'timesteps'])
    … …  
    338398    timesteps = None
    339399    variables = None
     400    quiet = False
     401    rel_tolerance = default_rel_tolerance
     402    abs_tolerance = default_abs_tolerance
    340403    for opt, arg in opts:
    341404        if opt in ('-h', '--help'):
    342405            print usage()
    343406            sys.exit(0)
     407        elif opt in ('-q', '--quiet'):
     408            quiet = True
     409        elif opt in ('-a', '--absolute'):
     410            abs_tolerance = float(arg)
     411        elif opt in ('-r', '--relative'):
     412            rel_tolerance = float(arg)
    344413        elif opt in ('-g', '--globals'):
    345414            globals = arg.split(',')
    … …  
    357426    try:
    358427        files_are_the_same(args, globals=globals,
    359                            timesteps=timesteps, variables=variables)
     428                           timesteps=timesteps, variables=variables,
     429                           rel_tolerance=rel_tolerance,
     430                           abs_tolerance=abs_tolerance)
    360431    except RuntimeError, msg:
    361432         print msg

  • anuga_validation/automated_validation_tests/patong_beach_validation/run_model.py

    r6927 r7276  
    2828# Related major packages
    2929from Scientific.IO.NetCDF import NetCDFFile
    30 import Numeric as num
     30#import numpy as num
    3131
    3232from anuga.interface import create_domain_from_regions
    … …  
    4444from anuga.utilities.polygon import read_polygon, Polygon_function
    4545from anuga.caching import cache
     46
     47import anuga.utilities.log as log
    4648
    4749# Application specific imports
    … …  
    7072#-------------------------------------------------------------------------------
    7173
    72 print 'Create computational domain'
     74log.critical('Create computational domain')
    7375
    7476# Create the STS file
    … …  
    7678# We need to use caching instead!
    7779
    78 print 'project.mux_data_folder=%s' % project.mux_data_folder
     80log.critical( 'project.mux_data_folder=%s' % project.mux_data_folder)
    7981if not os.path.exists(project.event_sts + '.sts'):
    8082    bub.build_urs_boundary(project.mux_input_filename, project.event_sts)
    … …  
    111113                                    use_cache=True,
    112114                                    verbose=True)
    113 print domain.statistics()
     115log.critical(domain.statistics())
    114116
    115117# FIXME(Ole): How can we make this more automatic?
    … …  
    125127#-------------------------------------------------------------------------------
    126128
    127 print 'Setup initial conditions'
     129log.critical('Setup initial conditions')
    128130
    129131# Set the initial stage in the offcoast region only
    … …  
    144146if project.use_buildings:
    145147    # Add buildings from file
    146     print 'Reading building polygons'   
     148    log.critical('Reading building polygons')
    147149    building_polygons, building_heights = csv2building_polygons(project.building_polygon)
    148150    #clipping_polygons=project.building_area_polygons)
    149151
    150     print 'Creating %d building polygons' % len(building_polygons)
     152    log.critical('Creating %d building polygons' % len(building_polygons))
    151153    def create_polygon_function(building_polygons, geo_reference=None):
    152154        L = []
    153155        for i, key in enumerate(building_polygons):
    154             if i%100==0: print i
     156            if i%100==0: log.critical(i)
    155157            poly = building_polygons[key]
    156158            elev = building_heights[key]
    … …  
    161163        return buildings
    162164
    163     print 'Creating %d building polygons' % len(building_polygons)
     165    log.critical('Creating %d building polygons' % len(building_polygons))
    164166    buildings = cache(create_polygon_function,
    165167                      building_polygons,
    … …  
    167169                      verbose=True)
    168170
    169     print 'Adding buildings'
     171    log.critical('Adding buildings')
    170172    domain.add_quantity('elevation',
    171173                        buildings,
    … …  
    178180#-------------------------------------------------------------------------------
    179181
    180 print 'Set boundary - available tags:', domain.get_boundary_tags()
     182log.critical('Set boundary - available tags:' % domain.get_boundary_tags())
    181183
    182184Br = Reflective_boundary(domain)
    … …  
    204206for t in domain.evolve(yieldstep=2000,
    205207                       finaltime=6000):
    206     print domain.timestepping_statistics()
    207     print domain.boundary_statistics(tags='ocean')
     208    log.critical(domain.timestepping_statistics())
     209    log.critical(domain.boundary_statistics(tags='ocean'))
    208210
    209211# Start detailed model
    … …  
    211213                       finaltime=project.finaltime,
    212214                       skip_initial_step=True):
    213     print domain.timestepping_statistics()
    214     print domain.boundary_statistics(tags='ocean')
     215    log.critical(domain.timestepping_statistics())
     216    log.critical(domain.boundary_statistics(tags='ocean'))
    215217   
    216 print 'Simulation took %.2f seconds' %(time.time()-t0)
     218log.critical('Simulation took %.2f seconds' %(time.time()-t0))
    217219     

  • anuga_validation/automated_validation_tests/patong_beach_validation/validate.py

    r7267 r7276  
    1313import time
    1414import shutil
    15 import platform
    1615
    1716from anuga.utilities.system_tools import get_web_file, untar_file, file_length
    … …  
    3332           'http://dfn.dl.sourceforge.net/sourceforge/anuga/'             # de
    3433          ]
     34
    3535### for testing
    3636##MIRRORS = ['http://10.7.64.243/patong_validation_data/']       # local linux box
    … …  
    4646# these names must be of the form <scene>.sww.<type>.tgz
    4747# as code below depends upon it.
    48 Optional_Data_Objects = ('patong.sww.TRIAL.tgz',
     48Optional_Data_Objects = (
     49                         'patong.sww.TRIAL.tgz',
    4950                         'patong.sww.BASIC.tgz',
    5051                         'patong.sww.FINAL.tgz'
    … …  
    6667OUTPUT_SWW = 'patong.sww'
    6768
    68 # default name of python to run
    69 PythonName = 'python'
    70 
    7169
    7270def setup():
    73     '''Prepare for the validation run.
    74 
    75     Check we have required data set in project.py.
    76     '''
     71    '''Prepare for the validation run.'''
    7772   
    7873    pass
    … …  
    302297    # run the simulation, produce SWW file
    303298    log.info('Running the simulation ...')
    304     cmd = '%s run_model.py > %s' % (PythonName, RUNMODEL_STDOUT)
     299    cmd = 'python run_model.py > %s' % RUNMODEL_STDOUT
    305300    log.debug("run_simulation: doing '%s'" % cmd)
    306301    res = os.system(cmd)
    … …  
    349344    # compare SWW files here and there
    350345    new_output_sww = os.path.join(output_directory, expected_sww)
    351     cmd = '%s cmpsww.py %s %s > cmpsww.stdout' % (PythonName, local_sww, new_output_sww)
     346    cmd = 'python cmpsww.py %s %s > cmpsww.stdout' % (local_sww, new_output_sww)
    352347    log.debug("check_that_output_is_as_expected: doing '%s'" % cmd)
    353348    res = os.system(cmd)
    … …  
    381376################################################################################
    382377
    383 # determine what python we are, initialize PythonName
    384 py_tuple = platform.python_version_tuple()
    385 PythonName = 'python%s.%s' % (py_tuple[0], py_tuple[1])
    386 
    387378# set logging levels
    388379log.console_logging_level = log.INFO
    389380log.log_logging_level = log.DEBUG
    390 log_filename = log.log_filename
    391381
    392382setup()
    393383
    394384# prepare user for what is about to happen
    395 
    396 msg = '''
     385log.critical('''
    397386Please note that this validation test is accurate only on 64bit Linux or
    398387Windows.  Running the validation on a 32bit operating system will result in
    … …  
    412401if you wish.  If not supplied in environment variables you will be prompted for
    413402the information.
    414 '''
    415 
    416 log.critical(msg)
     403''')
     404
    417405
    418406# make sure local data is up to date
    … …  
    443431for odo in Optional_Data_Objects:
    444432    start_time = time.time()
     433
    445434    (_, vtype, _) = odo.rsplit('.', 2)
    446435    vtype = vtype.lower()
    … …  
    452441        (expected_sww, _) = valid_sww.rsplit('.', 1)
    453442        check_that_output_is_as_expected(expected_sww, valid_sww)
    454     shutil.move(log_filename, '%s.%s' % (log_filename, vtype))
     443
    455444    stop_time = time.time()
    456     log.critical("'%s' validation took %.1fs" % (vtype, stop_time - start_time))
     445    log.critical("'%s' validation took %.1fs\n\n\n" % (vtype, stop_time - start_time))
    457446
    458447# clean up
     448log.critical('Tearing down ...')
    459449teardown()

  • anuga_validation/automated_validation_tests/urs_mux_files_validation/boundary_gauges.lic

    r5109 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
    3   <ga_license_file>
     1<?xml version='1.0' encoding='iso-8859-1'?>
     2<ga_license_file>
    43    <metadata>
    54      <author>Duncan Gray</author>
    … …  
    76    <datafile>
    87      <filename>boundary_gauges.txt</filename>
    9       <checksum>-1553233046</checksum>
     8      <checksum>2741734250</checksum>
    109      <publishable>Yes</publishable>
    1110      <accountable>Duncan Gray</accountable>

  • anuga_validation/automated_validation_tests/urs_mux_files_validation/gauges.lic

    r5109 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
    3   <ga_license_file>
     1<?xml version='1.0' encoding='iso-8859-1'?>
     2<ga_license_file>
    43    <metadata>
    54      <author>Duncan Gray</author>

  • anuga_validation/automated_validation_tests/urs_mux_files_validation/o_test-e-mux.lic

    r5505 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
    3   <ga_license_file>
     1<?xml version='1.0' encoding='iso-8859-1'?>
     2<ga_license_file>
    43    <metadata>
    54      <author>Duncan Gray</author>
    … …  
    76    <datafile>
    87      <filename>o_test-e-mux</filename>
    9       <checksum>-1664514244</checksum>
     8      <checksum>2630453052</checksum>
    109      <publishable>Yes</publishable>
    1110      <accountable>Duncan Gray</accountable>

  • anuga_validation/automated_validation_tests/urs_mux_files_validation/o_test-n-mux.lic

    r5505 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
    3   <ga_license_file>
     1<?xml version='1.0' encoding='iso-8859-1'?>
     2<ga_license_file>
    43    <metadata>
    54      <author>Duncan Gray</author>

  • anuga_validation/automated_validation_tests/urs_mux_files_validation/o_test-z-mux.lic

    r5505 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
    3   <ga_license_file>
     1<?xml version='1.0' encoding='iso-8859-1'?>
     2<ga_license_file>
    43    <metadata>
    54      <author>Duncan Gray</author>
    … …  
    76    <datafile>
    87      <filename>o_test-z-mux</filename>
    9       <checksum>-1449856062</checksum>
     8      <checksum>2845111234</checksum>
    109      <publishable>Yes</publishable>
    1110      <accountable>Duncan Gray</accountable>

  • anuga_validation/automated_validation_tests/urs_mux_files_validation/run_Bf.py

    r6160 r7276  
    2020# Related major packages
    2121from anuga.shallow_water import Domain,Dirichlet_boundary,File_boundary,Transmissive_boundary, Field_boundary
    22 import Numeric as num
     22import numpy as num
    2323from anuga.pmesh.mesh_interface import create_mesh_from_regions
    2424from anuga.abstract_2d_finite_volumes.util import start_screen_catcher, copy_code_files, sww2timeseries, get_data_from_file

  • anuga_validation/automated_validation_tests/validate_all.py

    r6885 r7276  
    3939    os.chdir(path)
    4040    s = 'python %s' %(filename)
    41     print
    4241    print s
    4342    os.system(s)

  • anuga_validation/circular_island/get_compared_graphs.py

    r5142 r7276  
    1212#from anuga.utilities.polygon import read_polygon#, plot_polygons
    1313#from math import cos,pi,sin,tan#,sqrt
    14 #from Numeric import array, zeros, Float, allclose,resize,sqrt
    1514#from anuga.shallow_water.data_manager import csv2dict
    1615#from time import localtime, strftime, gmtime

  • anuga_validation/circular_island/run_circular.py

    r5442 r7276  
    1515from anuga.abstract_2d_finite_volumes.util import file_function, sww2csv_gauges,csv2timeseries_graphs
    1616from anuga.pmesh.mesh_interface import create_mesh_from_regions
    17 from anuga.utilities.polygon import read_polygon#, plot_polygons
    18 from math import cos,pi,sin,tan#,sqrt
    19 from Numeric import array, zeros, Float, allclose,resize,sqrt
     17from anuga.utilities.polygon import read_polygon
     18from math import cos,pi,sin,tan
     19import numpy as num
    2020from anuga.shallow_water.data_manager import csv2dict
    2121from time import localtime, strftime, gmtime
    2222from anuga.utilities.system_tools import get_user_name, get_host_name
    2323from os import sep, environ, getenv
     24from anuga.config import netcdf_float
     25from Scientific.IO.NetCDF import NetCDFFile
     26
    2427#-------------------------
    2528# Create Domain from mesh
    … …  
    5760
    5861    print 'Preparing time boundary from %s' %textversion
    59     from Numeric import array
    6062
    6163    fid = open(textversion)
    … …  
    7072
    7173    N = len(lines)
    72     T = zeros(N, Float)  #Time
    73     Q = zeros(N, Float)  #Values
     74    T = num.zeros(N, num.float)  #Time
     75    Q = num.zeros(N, num.float)  #Values
    7476
    7577    for i, line in enumerate(lines):
    … …  
    8284
    8385    #Create tms file
    84     from Scientific.IO.NetCDF import NetCDFFile
    85 
    8686    print 'Writing to', filename
    8787    fid = NetCDFFile(filename[:-4] + '.tms', 'w')
    … …  
    9191    fid.starttime = 0.0
    9292    fid.createDimension('number_of_timesteps', len(T))
    93     fid.createVariable('time', Float, ('number_of_timesteps',))
     93    fid.createVariable('time', netcdf_float, ('number_of_timesteps',))
    9494    fid.variables['time'][:] = T
    9595
    96     fid.createVariable('stage', Float, ('number_of_timesteps',))
     96    fid.createVariable('stage', netcdf_float, ('number_of_timesteps',))
    9797    fid.variables['stage'][:] = Q[:]
    9898
    99     fid.createVariable('xmomentum', Float, ('number_of_timesteps',))
     99    fid.createVariable('xmomentum', netcdf_float, ('number_of_timesteps',))
    100100    fid.variables['xmomentum'][:] = 0.0
    101101
    102     fid.createVariable('ymomentum', Float, ('number_of_timesteps',))
     102    fid.createVariable('ymomentum', netcdf_float, ('number_of_timesteps',))
    103103    fid.variables['ymomentum'][:] = 0.0
    104104
    … …  
    181181def circular_island_elevation(x,y):
    182182    water_depth = 0.32
    183     list_xy = sqrt((center_x-x)**2+(center_y-L-y)**2)
     183    list_xy = num.sqrt((center_x-x)**2+(center_y-L-y)**2)
    184184    print 'x',min(x),max(x)
    185185    print 'y',min(y),max(y)

  • anuga_validation/circular_island/sqrt_table_run_circular.py

    r5442 r7276  
    1717from anuga.utilities.polygon import read_polygon#, plot_polygons
    1818from math import cos,pi,sin,tan,sqrt
    19 from Numeric import array, zeros, Float, allclose,resize
     19import numpy as num
    2020from anuga.shallow_water.data_manager import csv2dict
    2121from time import localtime, strftime, gmtime
    2222from anuga.utilities.system_tools import get_user_name, get_host_name
    2323from os import sep, environ, getenv
     24from anuga.config import netcdf_float
     25from Scientific.IO.NetCDF import NetCDFFile
     26
     27
    2428#-------------------------
    2529# Create Domain from mesh
    … …  
    5660
    5761    print 'Preparing time boundary from %s' %textversion
    58     from Numeric import array
    5962
    6063    fid = open(textversion)
    … …  
    6972
    7073    N = len(lines)
    71     T = zeros(N, Float)  #Time
    72     Q = zeros(N, Float)  #Values
     74    T = num.zeros(N, num.float)  #Time
     75    Q = num.zeros(N, num.float)  #Values
    7376
    7477    for i, line in enumerate(lines):
    … …  
    8184
    8285    #Create tms file
    83     from Scientific.IO.NetCDF import NetCDFFile
    84 
    8586    print 'Writing to', filename
    8687    fid = NetCDFFile(filename[:-4] + '.tms', 'w')
    … …  
    9091    fid.starttime = 0.0
    9192    fid.createDimension('number_of_timesteps', len(T))
    92     fid.createVariable('time', Float, ('number_of_timesteps',))
     93    fid.createVariable('time', netcdf_float, ('number_of_timesteps',))
    9394    fid.variables['time'][:] = T
    9495
    95     fid.createVariable('stage', Float, ('number_of_timesteps',))
     96    fid.createVariable('stage', netcdf_float, ('number_of_timesteps',))
    9697    fid.variables['stage'][:] = Q[:]
    9798
    98     fid.createVariable('xmomentum', Float, ('number_of_timesteps',))
     99    fid.createVariable('xmomentum', netcdf_float, ('number_of_timesteps',))
    99100    fid.variables['xmomentum'][:] = 0.0
    100101
    101     fid.createVariable('ymomentum', Float, ('number_of_timesteps',))
     102    fid.createVariable('ymomentum', netcdf_float, ('number_of_timesteps',))
    102103    fid.variables['ymomentum'][:] = 0.0
    103104

  • anuga_validation/okushiri_2005/Benchmark_2_Bathymetry.lic

    r5023 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>
    … …  
    87  <datafile>
    98    <filename>Benchmark_2_Bathymetry.txt</filename>
    10     <checksum>-1310724450</checksum>
     9    <checksum>2984242846</checksum>
    1110    <publishable>Yes</publishable>
    1211    <accountable>Ole Nielsen</accountable>

  • anuga_validation/okushiri_2005/Benchmark_2_input.lic

    r5023 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>

  • anuga_validation/okushiri_2005/compare_timeseries.py

    r4557 r7276  
    55"""
    66
    7 from Numeric import allclose, argmin, argmax
     7import numpy as num
    88from Scientific.IO.NetCDF import NetCDFFile
    99
    … …  
    6868assert reference_time[0] == 0.0
    6969assert reference_time[-1] == finaltime
    70 assert allclose(reference_time, input_time)
     70assert num.allclose(reference_time, input_time)
    7171
    7272for key in gauge_names:
    … …  
    144144
    145145    # Locations of extrema
    146     i0 = argmax(observed_timeseries)
    147     i1 = argmax(model)
     146    i0 = num.argmax(observed_timeseries)
     147    i1 = num.argmax(model)
    148148    res = abs(reference_time[i1] - reference_time[i0])
    149149    print 'Timelag between maxima = %.18e' %res
    150150   
    151151
    152     i0 = argmin(observed_timeseries)
    153     i1 = argmin(model)
     152    i0 = num.argmin(observed_timeseries)
     153    i1 = num.argmin(model)
    154154    res = abs(reference_time[i1] - reference_time[i0])
    155155    print 'Timelag between minima = %.18e' %res

  • anuga_validation/okushiri_2005/create_okushiri.py

    r3915 r7276  
    11"""Create mesh and time boundary for the Okushiri island validation
    22"""
    3 
    4 
    5 from Numeric import array, zeros, Float, allclose
    63
    74from anuga.pmesh.mesh import *
    … …  
    96from anuga.coordinate_transforms.geo_reference import Geo_reference
    107from anuga.geospatial_data import Geospatial_data
     8
     9from Scientific.IO.NetCDF import NetCDFFile
    1110
    1211import project
    … …  
    5150    """
    5251
    53     from Scientific.IO.NetCDF import NetCDFFile
    54     from Numeric import array
    55 
    56 
    5752    print 'Creating', filename
    5853
    … …  
    6964
    7065    N = len(lines)
    71     T = zeros(N, Float)  #Time
    72     Q = zeros(N, Float)  #Values
     66    T = num.zeros(N, num.float)  #Time
     67    Q = num.zeros(N, num.float)  #Values
    7368
    7469    for i, line in enumerate(lines):
    … …  
    8681    fid.starttime = 0.0
    8782    fid.createDimension('number_of_timesteps', len(T))
    88     fid.createVariable('time', Float, ('number_of_timesteps',))
     83    fid.createVariable('time', netcdf_float, ('number_of_timesteps',))
    8984    fid.variables['time'][:] = T
    9085
    91     fid.createVariable('stage', Float, ('number_of_timesteps',))
     86    fid.createVariable('stage', netcdf_float, ('number_of_timesteps',))
    9287    fid.variables['stage'][:] = Q[:]
    9388
    94     fid.createVariable('xmomentum', Float, ('number_of_timesteps',))
     89    fid.createVariable('xmomentum', netcdf_float, ('number_of_timesteps',))
    9590    fid.variables['xmomentum'][:] = 0.0
    9691
    97     fid.createVariable('ymomentum', Float, ('number_of_timesteps',))
     92    fid.createVariable('ymomentum', netcdf_float, ('number_of_timesteps',))
    9893    fid.variables['ymomentum'][:] = 0.0
    9994

  • anuga_validation/okushiri_2005/create_okushiri_original.py

    r3913 r7276  
    22"""
    33
    4 from Numeric import array, zeros, Float, allclose
     4import numpy as num
     5from Scientific.IO.NetCDF import NetCDFFile
    56
    67from anuga.pmesh.mesh import *
    78from anuga.coordinate_transforms.geo_reference import Geo_reference
     9from anuga.config import netcdf_float
    810
    911import project
    … …  
    1921
    2022    print 'Preparing time boundary from %s' %textversion
    21     from Numeric import array
    2223
    2324    fid = open(textversion)
    … …  
    3233
    3334    N = len(lines)
    34     T = zeros(N, Float)  #Time
    35     Q = zeros(N, Float)  #Values
     35    T = num.zeros(N, num.float)  #Time
     36    Q = num.zeros(N, num.float)  #Values
    3637
    3738    for i, line in enumerate(lines):
    … …  
    4344
    4445    #Create tms file
    45     from Scientific.IO.NetCDF import NetCDFFile
    46 
    4746    print 'Writing to', filename
    4847    fid = NetCDFFile(filename, 'w')
    … …  
    5251    fid.starttime = 0.0
    5352    fid.createDimension('number_of_timesteps', len(T))
    54     fid.createVariable('time', Float, ('number_of_timesteps',))
     53    fid.createVariable('time', netcdf_float, ('number_of_timesteps',))
    5554    fid.variables['time'][:] = T
    5655
    57     fid.createVariable('stage', Float, ('number_of_timesteps',))
     56    fid.createVariable('stage', netcdf_float, ('number_of_timesteps',))
    5857    fid.variables['stage'][:] = Q[:]
    5958
    60     fid.createVariable('xmomentum', Float, ('number_of_timesteps',))
     59    fid.createVariable('xmomentum', netcdf_float, ('number_of_timesteps',))
    6160    fid.variables['xmomentum'][:] = 0.0
    6261
    63     fid.createVariable('ymomentum', Float, ('number_of_timesteps',))
     62    fid.createVariable('ymomentum', netcdf_float, ('number_of_timesteps',))
    6463    fid.variables['ymomentum'][:] = 0.0
    6564
    … …  
    226225    m.generateMesh('pzq28.0za1000000a')
    227226
    228     import project
    229227    m.export_mesh_file(project.mesh_filename)
    230228   

  • anuga_validation/okushiri_2005/output_ch5-7-9.lic

    r5020 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>
    … …  
    1918  <datafile>
    2019    <filename>output_ch5-7-9.txt</filename>
    21     <checksum>-1044604103</checksum>
     20    <checksum>3250363193</checksum>
    2221    <publishable>Yes</publishable>
    2322    <accountable>Ole Nielsen</accountable>

  • anuga_validation/okushiri_2005/problem02.lic

    r5023 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>
    … …  
    87  <datafile>
    98    <filename>problem02.doc</filename>
    10     <checksum>-1811884659</checksum>
     9    <checksum>2483082637</checksum>
    1110    <publishable>Yes</publishable>
    1211    <accountable>Ole Nielsen</accountable>

  • anuga_validation/okushiri_2005/test_caching_of_set_quantity.py

    r7040 r7276  
    1616from anuga.fit_interpolate.fit import _fit_to_mesh
    1717import project
    18 import Numeric as num
     18import numpy as num
    1919import time
    2020

  • anuga_validation/okushiri_2005/timings.lic

    r5023 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>
    … …  
    87  <datafile>
    98    <filename>timings.txt</filename>
    10     <checksum>-1935951304</checksum>
     9    <checksum>2359015992</checksum>
    1110    <publishable>Yes</publishable>
    1211    <accountable>Ole Nielsen</accountable>

  • anuga_validation/performance_tests/okushiri/Benchmark_2.lic

    r5026 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>

  • anuga_validation/performance_tests/okushiri/Benchmark_2_Bathymetry.lic

    r5026 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>

  • anuga_validation/performance_tests/okushiri/Benchmark_2_input.lic

    r5026 r7276  
    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 
     1<?xml version='1.0' encoding='iso-8859-1'?>
    32<ga_license_file>
    43  <metadata>
    … …  
    87  <datafile>
    98    <filename>Benchmark_2_input.tms</filename>
    10     <checksum>-725631838</checksum>
     9    <checksum>3569335458</checksum>
    1110    <publishable>Yes</publishable>
    1211    <accountable>Ole Nielsen</accountable>

  • anuga_validation/performance_tests/run_profile.py

    r7181 r7276  
    88from anuga.shallow_water import Domain, Reflective_boundary
    99from mesh_factory import rectangular
     10#import numpy as num     # not used?
    1011
    1112
Note: See TracChangeset for help on using the changeset viewer.