[jira] Created: (MATH-186) test results depend on java version

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (MATH-186) test results depend on java version

JIRA jira@apache.org
test results depend on java version
-----------------------------------

                 Key: MATH-186
                 URL: https://issues.apache.org/jira/browse/MATH-186
             Project: Commons Math
          Issue Type: Bug
    Affects Versions: 1.2
         Environment: GNU/Linux (ubuntu Gutsy Gibbon),  java 1.3: eclipse compiler, java 1.4: Blackdown-1.4.2-02, java 1.6:  SUN 1.6.0_03-b05,  AMD athlon XP2000+
            Reporter: Luc Maisonobe
            Priority: Minor
             Fix For: 1.2


Running the tests with maven and changing the java version used by changing JAVA_HOME in the ~/.mavenrc file, I get different results.

With the Eclipse compiler set to 1.3 compatibility and with blackdown jvm (1.4), the tests succed.
With the Sun jvm (1.6), SummaryStatisticsAbstractTest.testEqualsAndHashCode (which is used both by SummaryStatisticsTest and SynchronizedSummaryStatisticsTest) fails.

The error is related to geometric mean computation, which lead to slightly different results depending on the order of added elements. One instance returns 2.213363839400643 and the other returns 2.2133638394006434. Both results are consistent with IEEE754 arithmetic (they differ in the last two bits).

Using Sun 1.6.0_03 jvm, the different values induce a test failure when SummaryStatistics.equals() method is called (it checks for exact equality). If this part of the test is commented out, another failure occurs when the SummaryStatistics.hashcode() method is called.

Changing the equals method would be possible, but would be a change of semantics and would imply choosing some threshold which would never suit everybody needs. Changing the hashcode method simply does not seem realistic to me. So I would like to keep these methods as they are now. So the main conclusion would be that the test is too sensitive to jvm implementation (which are consistent with IEEE754 arithmetic in this case).

I don't know what to do about this issue.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (MATH-186) test results depend on java version

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/MATH-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565231#action_12565231 ]

Phil Steitz commented on MATH-186:
----------------------------------

I think the test should be modified to add data in the same order in each case, which will ensure equals semantics works.  The assumption that arithmetic operations are commutative is not true and the test in its current form depends on that.

> test results depend on java version
> -----------------------------------
>
>                 Key: MATH-186
>                 URL: https://issues.apache.org/jira/browse/MATH-186
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 1.2
>         Environment: GNU/Linux (ubuntu Gutsy Gibbon),  java 1.3: eclipse compiler, java 1.4: Blackdown-1.4.2-02, java 1.6:  SUN 1.6.0_03-b05,  AMD athlon XP2000+
>            Reporter: Luc Maisonobe
>            Priority: Minor
>             Fix For: 1.2
>
>
> Running the tests with maven and changing the java version used by changing JAVA_HOME in the ~/.mavenrc file, I get different results.
> With the Eclipse compiler set to 1.3 compatibility and with blackdown jvm (1.4), the tests succed.
> With the Sun jvm (1.6), SummaryStatisticsAbstractTest.testEqualsAndHashCode (which is used both by SummaryStatisticsTest and SynchronizedSummaryStatisticsTest) fails.
> The error is related to geometric mean computation, which lead to slightly different results depending on the order of added elements. One instance returns 2.213363839400643 and the other returns 2.2133638394006434. Both results are consistent with IEEE754 arithmetic (they differ in the last two bits).
> Using Sun 1.6.0_03 jvm, the different values induce a test failure when SummaryStatistics.equals() method is called (it checks for exact equality). If this part of the test is commented out, another failure occurs when the SummaryStatistics.hashcode() method is called.
> Changing the equals method would be possible, but would be a change of semantics and would imply choosing some threshold which would never suit everybody needs. Changing the hashcode method simply does not seem realistic to me. So I would like to keep these methods as they are now. So the main conclusion would be that the test is too sensitive to jvm implementation (which are consistent with IEEE754 arithmetic in this case).
> I don't know what to do about this issue.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (MATH-186) test results depend on java version

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/MATH-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Luc Maisonobe resolved MATH-186.
--------------------------------

    Resolution: Fixed

Changed slightly the test to be less sensitive to JVM issues.
The change was simply to add the points in a different order. So instead of having one instance handling values 2, 1, 3, 4 and the other handling values 4, 2, 3, 1, the second instance now handles 2, 3, 1, 4.
It works, but it is definitely not a clean solution. The problem may happen again with other jvm implementations.

> test results depend on java version
> -----------------------------------
>
>                 Key: MATH-186
>                 URL: https://issues.apache.org/jira/browse/MATH-186
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 1.2
>         Environment: GNU/Linux (ubuntu Gutsy Gibbon),  java 1.3: eclipse compiler, java 1.4: Blackdown-1.4.2-02, java 1.6:  SUN 1.6.0_03-b05,  AMD athlon XP2000+
>            Reporter: Luc Maisonobe
>            Priority: Minor
>             Fix For: 1.2
>
>
> Running the tests with maven and changing the java version used by changing JAVA_HOME in the ~/.mavenrc file, I get different results.
> With the Eclipse compiler set to 1.3 compatibility and with blackdown jvm (1.4), the tests succed.
> With the Sun jvm (1.6), SummaryStatisticsAbstractTest.testEqualsAndHashCode (which is used both by SummaryStatisticsTest and SynchronizedSummaryStatisticsTest) fails.
> The error is related to geometric mean computation, which lead to slightly different results depending on the order of added elements. One instance returns 2.213363839400643 and the other returns 2.2133638394006434. Both results are consistent with IEEE754 arithmetic (they differ in the last two bits).
> Using Sun 1.6.0_03 jvm, the different values induce a test failure when SummaryStatistics.equals() method is called (it checks for exact equality). If this part of the test is commented out, another failure occurs when the SummaryStatistics.hashcode() method is called.
> Changing the equals method would be possible, but would be a change of semantics and would imply choosing some threshold which would never suit everybody needs. Changing the hashcode method simply does not seem realistic to me. So I would like to keep these methods as they are now. So the main conclusion would be that the test is too sensitive to jvm implementation (which are consistent with IEEE754 arithmetic in this case).
> I don't know what to do about this issue.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (MATH-186) test results depend on java version

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/MATH-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565234#action_12565234 ]

Luc Maisonobe commented on MATH-186:
------------------------------------

Sorry Phil, I did not see your proposal before sending my last comment.
You are right, I will commit a new fix for this using exactly the same order (2, 1, 3, 4) for both instances. This is a real solution, not a dirty trick like mine.
Thanks

> test results depend on java version
> -----------------------------------
>
>                 Key: MATH-186
>                 URL: https://issues.apache.org/jira/browse/MATH-186
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 1.2
>         Environment: GNU/Linux (ubuntu Gutsy Gibbon),  java 1.3: eclipse compiler, java 1.4: Blackdown-1.4.2-02, java 1.6:  SUN 1.6.0_03-b05,  AMD athlon XP2000+
>            Reporter: Luc Maisonobe
>            Priority: Minor
>             Fix For: 1.2
>
>
> Running the tests with maven and changing the java version used by changing JAVA_HOME in the ~/.mavenrc file, I get different results.
> With the Eclipse compiler set to 1.3 compatibility and with blackdown jvm (1.4), the tests succed.
> With the Sun jvm (1.6), SummaryStatisticsAbstractTest.testEqualsAndHashCode (which is used both by SummaryStatisticsTest and SynchronizedSummaryStatisticsTest) fails.
> The error is related to geometric mean computation, which lead to slightly different results depending on the order of added elements. One instance returns 2.213363839400643 and the other returns 2.2133638394006434. Both results are consistent with IEEE754 arithmetic (they differ in the last two bits).
> Using Sun 1.6.0_03 jvm, the different values induce a test failure when SummaryStatistics.equals() method is called (it checks for exact equality). If this part of the test is commented out, another failure occurs when the SummaryStatistics.hashcode() method is called.
> Changing the equals method would be possible, but would be a change of semantics and would imply choosing some threshold which would never suit everybody needs. Changing the hashcode method simply does not seem realistic to me. So I would like to keep these methods as they are now. So the main conclusion would be that the test is too sensitive to jvm implementation (which are consistent with IEEE754 arithmetic in this case).
> I don't know what to do about this issue.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (MATH-186) test results depend on java version

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/MATH-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565235#action_12565235 ]

Phil Steitz commented on MATH-186:
----------------------------------

For this to happen with other JVM impls with the change just made, the jvm would have to produce different results for the exact same computations, which would be odd.  Equals is strictly defined for this class, which I think is OK and the test is really supposed to just be testing equals and hashcode.

> test results depend on java version
> -----------------------------------
>
>                 Key: MATH-186
>                 URL: https://issues.apache.org/jira/browse/MATH-186
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 1.2
>         Environment: GNU/Linux (ubuntu Gutsy Gibbon),  java 1.3: eclipse compiler, java 1.4: Blackdown-1.4.2-02, java 1.6:  SUN 1.6.0_03-b05,  AMD athlon XP2000+
>            Reporter: Luc Maisonobe
>            Priority: Minor
>             Fix For: 1.2
>
>
> Running the tests with maven and changing the java version used by changing JAVA_HOME in the ~/.mavenrc file, I get different results.
> With the Eclipse compiler set to 1.3 compatibility and with blackdown jvm (1.4), the tests succed.
> With the Sun jvm (1.6), SummaryStatisticsAbstractTest.testEqualsAndHashCode (which is used both by SummaryStatisticsTest and SynchronizedSummaryStatisticsTest) fails.
> The error is related to geometric mean computation, which lead to slightly different results depending on the order of added elements. One instance returns 2.213363839400643 and the other returns 2.2133638394006434. Both results are consistent with IEEE754 arithmetic (they differ in the last two bits).
> Using Sun 1.6.0_03 jvm, the different values induce a test failure when SummaryStatistics.equals() method is called (it checks for exact equality). If this part of the test is commented out, another failure occurs when the SummaryStatistics.hashcode() method is called.
> Changing the equals method would be possible, but would be a change of semantics and would imply choosing some threshold which would never suit everybody needs. Changing the hashcode method simply does not seem realistic to me. So I would like to keep these methods as they are now. So the main conclusion would be that the test is too sensitive to jvm implementation (which are consistent with IEEE754 arithmetic in this case).
> I don't know what to do about this issue.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.