[math] MATH-1385 OutOfMemoryError: GC overhead limit exceeded

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[math] MATH-1385 OutOfMemoryError: GC overhead limit exceeded

Rob Tompkins
Hello all,

TL;DR - curious if anyone is familiar with the issue, not sure my diagnosis is correct.
---------
We have an individual reporting that their FastFourierTransform is causing OutOfMemoryErrors, which seems to me to stem from the mechanics of the data handling as opposed to the calculations themselves.

My thought is that the issue at hand stems from there being no upper limits on the size of the array returned by TransformUtils.createComplexArray, and this causes the potential for memory profile limitations.

I was thinking that maybe we could somehow perform the transform in portions as opposed to doing a complete copy of the input data here: https://github.com/apache/commons-math/blob/MATH_3_6_1/src/main/java/org/apache/commons/math3/transform/FastFourierTransformer.java#L370-L372 <https://github.com/apache/commons-math/blob/MATH_3_6_1/src/main/java/org/apache/commons/math3/transform/FastFourierTransformer.java#L370-L372>

or minimally not make a second copy of the data in the TransformUtils.

Let me know if anyone has any ideas here.

Cheers,
-Rob