Introduce large file support for NetCDF to allow larger than 2 GByte files

[ole]

The barrier of 2GB can be removed by moving 2 v3.6 and 64bit storage (even on 32 bit platforms). See

​http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#Large%20File%20Support4

[ole]

Duncan, would you mind putting this on *your* list of things todo?

[ole]

Currently, there is some functionality for splitting Netcdf files exceeding 2GB in data_manager.py but it does not have a unit test. In fact the max_size argument to the class Data_format_sww which is suppose to control the split isn't even passed down from get_dataobject or store_connectivity where one would like to control this. Especially if one were to write a small test with a small value for max_size.

[rwilson]

It appears that recent changes in NetCDF (beyond the 64-bit offset format changes) allow large dataset file sizes:

With the netCDF-4/HDF5 format size limitations are further relaxed, and files can be as large as the underlying file system supports.

​http://www.unidata.ucar.edu/software/netcdf/docs/netcdf.html#index-limitations-of-netCDF-61