Opened 20 years ago
Closed 19 years ago
#8 closed enhancement (fixed)
Least squares memory issue (A vs AtA)
Reported by: | ole | Owned by: | duncan |
---|---|---|---|
Priority: | high | Milestone: | least_squares module finished |
Component: | Efficiency and optimisation | Version: | |
Severity: | normal | Keywords: | |
Cc: | ole |
Description
memory issue: !
If LS is used for fitting
compute self.Atz and self AtA directly and avoid self.A. need alpha
If LS is to be used for interpolation
compute self.A no alpha needed
Rethink design and API so that this is addressed without duplicating code.
Change History (8)
comment:1 Changed 20 years ago by
Priority: | normal → high |
---|
comment:2 Changed 20 years ago by
Component: | pyvolution → least_squares |
---|
comment:3 Changed 20 years ago by
comment:4 Changed 20 years ago by
ran same test as 29 sept. Same results, longer time (160 hrs). meemory usage 399,708 K, cpu at 100%. What was to screen when I stopped it: I:\inundation\pyvolution>test_data_manager.py Reading from karratha_100m.sww Reading quantity depth Reducing quantity depth Creating grid Building mesh Building interpolation matrix Number of datapoints: 12061352 Number of basis functions: 65227 Doing 0 of 12061352
Therefore not a memory problem, but a slow algorithm problem.
comment:5 Changed 20 years ago by
Severity: | normal → enhancement |
---|
comment:6 Changed 20 years ago by
Component: | Testing and validation → Efficiency and optimisation |
---|
comment:7 Changed 20 years ago by
Status: | new → assigned |
---|
comment:8 Changed 19 years ago by
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
done changes, unchecked in, to least squares. after 20 hrs on the test_data_management test it is at this stage; N:\inundation\pyvolution>python test_data_manager.py Reading from karratha_100m.sww Reading quantity depth Reducing quantity depth Creating grid Building mesh Building interpolation matrix Number of datapoints: 12061352 Number of basis functions: 65227 Doing 0 of 12061352 Traceback (most recent call last):
tion_matrix_A
s_of_vertices
KeyboardInterrupt?
Note, it was not using all of the availabe memory. Seems like a slow algorithm issue