# Why not BigDecimal? Classic List Threaded 7 messages Open this post in threaded view
|

## Why not BigDecimal?

 Apache Commons-Math seems like an excellent library, but what I don't understand is why we are using 'double' everywhere instead of BigDecimal. I wrote a simple program to run a Multiple Regression Analysis followed by Rank, and compared my results to those from R stats package and because of lack of precision the 'ranks' are way off.  I mean I am assuming that if we had used BigDecimal the ranks would have matched to the ones from R. Is there something I am missing?
Open this post in threaded view
|

## Re: Why not BigDecimal?

 Doesn't R use doubles under the covers?  Note this quote from the manual: *R has no single precision data type. All real numbers are stored in double precision format*. (from http://stat.ethz.ch/R-manual/R-patched/library/base/html/double.html) Any difference in the results that you saw is likely due to different algorithms.  If you mean rank as in the rank of a matrix, then the exact value is very much a matter of judgment since it involves an implicit comparison of a numerical value to zero.  Using BigDecimal is very unlikely to have significantly affected your results. On Thu, Feb 11, 2010 at 9:29 PM, Something Something < [hidden email]> wrote: > I wrote a simple program to run a Multiple Regression Analysis followed by > Rank, and compared my results to those from R stats package and because of > lack of precision the 'ranks' are way off.  I mean I am assuming that if we > had used BigDecimal the ranks would have matched to the ones from R. > > Is there something I am missing? > -- Ted Dunning, CTO DeepDyve
Open this post in threaded view
|

## RE: Why not BigDecimal?

 Interesting that this is a precision issue. I'm not surprised depending on what you are doing, double precision may not be enough. It depends a lot on how the calculations are broken into smaller parts. BigDecimal is fantastically useful... Andy http://www.geog.leeds.ac.uk/people/a.turner/  -----Original Message----- From: Ted Dunning [mailto:[hidden email]] Sent: 12 February 2010 05:59 To: Commons Users List Subject: Re: Why not BigDecimal? Doesn't R use doubles under the covers?  Note this quote from the manual: *R has no single precision data type. All real numbers are stored in double precision format*. (from http://stat.ethz.ch/R-manual/R-patched/library/base/html/double.html) Any difference in the results that you saw is likely due to different algorithms.  If you mean rank as in the rank of a matrix, then the exact value is very much a matter of judgment since it involves an implicit comparison of a numerical value to zero.  Using BigDecimal is very unlikely to have significantly affected your results. On Thu, Feb 11, 2010 at 9:29 PM, Something Something < [hidden email]> wrote: > I wrote a simple program to run a Multiple Regression Analysis followed by > Rank, and compared my results to those from R stats package and because of > lack of precision the 'ranks' are way off.  I mean I am assuming that if we > had used BigDecimal the ranks would have matched to the ones from R. > > Is there something I am missing? > -- Ted Dunning, CTO DeepDyve
Open this post in threaded view
|

## Re: Why not BigDecimal?

 In reply to this post by Something Something Something Something a écrit : > Apache Commons-Math seems like an excellent library, but what I don't > understand is why we are using 'double' everywhere instead of BigDecimal. Commons-math is a low level library intended to be used by many different types of applications. Using primitive double as the standard type is a fair bet to integrate smoothly with a priori unknown applications. Using BigDecimal would greatly restrict the audience. Also note that BigDecimal lacks many functions (sin, cos, sqrt, cbrt, exp, log ...). I also doubt performances would be on par with primitive doubles with repset to speed, but this is only a personal guess that would need to be verified. > > I wrote a simple program to run a Multiple Regression Analysis followed by > Rank, and compared my results to those from R stats package and because of > lack of precision the 'ranks' are way off.  I mean I am assuming that if we > had used BigDecimal the ranks would have matched to the ones from R. There are many other things that could explain differences. Even with BigDecimal, it is difficult to set the proper scale, so your assumtion needs to be verified. Luc > > Is there something I am missing? > --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]
Open this post in threaded view
|

## Re: Why not BigDecimal?

 In reply to this post by Andy Turner It is not a precision issue.  R and commons-math use different algorithms with the same underlying numerical implementation. It is even an open question which result is better.  R has lots of credibility, but I have found cases where it lacked precision (and I coded up a patch that was accepted). Unbounded precision integers and rationals are very useful, but not usually for large scale numerical programming.  Except in a very few cases, if you need more than 17 digits of precision, you have other very serious problems that precision won't help. On Fri, Feb 12, 2010 at 1:40 AM, Andy Turner <[hidden email]>wrote: > Interesting that this is a precision issue. I'm not surprised depending on > what you are doing, double precision may not be enough. It depends a lot on > how the calculations are broken into smaller parts. BigDecimal is > fantastically useful... > -- Ted Dunning, CTO DeepDyve