Least squares memory issue (A vs AtA)

[ole]

memory issue: !

If LS is used for fitting

compute self.Atz and self AtA directly and avoid self.A. need alpha

If LS is to be used for interpolation

compute self.A no alpha needed

Rethink design and API so that this is addressed without duplicating code.

[anonymous]

done changes, unchecked in, to least squares. after 20 hrs on the test_data_management test it is at this stage; N:\inundation\pyvolution>python test_data_manager.py Reading from karratha_100m.sww Reading quantity depth Reducing quantity depth Creating grid Building mesh Building interpolation matrix Number of datapoints: 12061352 Number of basis functions: 65227 Doing 0 of 12061352 Traceback (most recent call last):

File "test_data_manager.py", line 2225, in ?

runner.run(suite)

File "C:\Python23\lib\unittest.py", line 658, in run

test(result)

File "C:\Python23\lib\unittest.py", line 389, in call

test(result)

File "C:\Python23\lib\unittest.py", line 229, in call

testMethod()

File "test_data_manager.py", line 2212, in xxxtestz_sww2ers_real

verbose = True)

File "N:\inundation\pyvolution\data_manager.py", line 2876, in sww2ers

verbose = verbose)

File "N:\inundation\pyvolution\least_squares.py", line 355, in init

precrop = precrop)

File "N:\inundation\pyvolution\least_squares.py", line 545, in build_interpola

tion_matrix_A

element_found, sigma0, sigma1, sigma2, k = \

File "N:\inundation\pyvolution\least_squares.py", line 614, in search_triangle

s_of_vertices

sigma1 = dot((x-xi2), n1)/dot((xi1-xi2), n1)

KeyboardInterrupt?

Note, it was not using all of the availabe memory. Seems like a slow algorithm issue

[anonymous]

ran same test as 29 sept. Same results, longer time (160 hrs). meemory usage 399,708 K, cpu at 100%. What was to screen when I stopped it: I:\inundation\pyvolution>test_data_manager.py Reading from karratha_100m.sww Reading quantity depth Reducing quantity depth Creating grid Building mesh Building interpolation matrix Number of datapoints: 12061352 Number of basis functions: 65227 Doing 0 of 12061352

Therefore not a memory problem, but a slow algorithm problem.