[MATH] what a commons-TLP split could look like

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[MATH] what a commons-TLP split could look like

Eric Barnhill
Here's a proposed draft for how o.a.c.m might be split into a commons
component, and more sophisticated spin-out components, around which we
might develop a new Math TLP or incubator project in which components of
the project that find backers continue on, and those that do not are frozen
at their current state for now but not discarded (henceforth I just call
this TLP).

As a general strategy, I think it works to move the Field and Field Element
superclasses into the TLP. Methods that use these elements or other
math-specific data classes (e.g. DerivativeStructure, Neuron,
AlternativeHypothesis) belong in the TLP. Array-based classes belong in
commons. I see two exceptions to this. One is Complex objects which seem to
me to belong in the commons remit. The other is the array of statistics
objects (like Summary Statistics) as those statistics operations are right
for commons. I think some refactoring of the stats packages may enable a
clearer separation of the stats packages into those appropriate for commons
and those appropriate for a TLP stats package, but I leave that for another
thread.

Also I call a package "dormant" when to my untrained eye it appears to have
been underdeveloped and should perhaps be put out of its misery.

Here we go:
--
o.a.c.m. Field, FieldElement, RealFieldElement -> TLP
o.a.c.m. Analysis -> TLP
o.a.c.m. complex.Complex, ComplexFormat, ComplexUtils -> commons
o.a.c.m. complex.ComplexField, Quaternion, RootsOfUnity -> TLP
o.a.c.m. dfp -> dormant
o.a.c.m. exception -> commons
o.a.c.m. filter -> dormant
o.a.c.m. fraction -> all except Big FractionField to commons.
BigFractionField to TLP
o.a.c.m. genetics -> TLP
o.a.c.m. geometry -> TLP
o.a.c.m. linear -> TLP
o.a.c.m. ml -> TLP
o.a.c.m. ode -> TLP
o.a.c.m. optim and optimization -> combine into one TLP component
o.a.c.m. random and distribution -> combine into one TLP component
o.a.c.m. primes -> commons
o.a.c.m. special -> dormant
o.a.c.m. stat.frequency -> perhaps belongs with random and distribution?
o.a.c.m. stat.clustering -> dormant
o.a.c.m. stat.correlation -> array based methods to commons, Matrix methods
to TLP
o.a.c.m. stat.descriptive, inferences, interval, ranking, regression ->
Matrix methods to TLP. Array methods to commons. Statistics objects should
be divided between both after a refactoring.
o.a.c.m.transform -> commons
o.a.c.m.util -> commons, perhaps refactored so the classes are in more
informative package names
--

Eric
Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

sebb-2-2
On 20 June 2016 at 10:31, Eric Barnhill <[hidden email]> wrote:

> Here's a proposed draft for how o.a.c.m might be split into a commons
> component, and more sophisticated spin-out components, around which we
> might develop a new Math TLP or incubator project in which components of
> the project that find backers continue on, and those that do not are frozen
> at their current state for now but not discarded (henceforth I just call
> this TLP).
>
> As a general strategy, I think it works to move the Field and Field Element
> superclasses into the TLP. Methods that use these elements or other
> math-specific data classes (e.g. DerivativeStructure, Neuron,
> AlternativeHypothesis) belong in the TLP. Array-based classes belong in
> commons. I see two exceptions to this. One is Complex objects which seem to
> me to belong in the commons remit. The other is the array of statistics
> objects (like Summary Statistics) as those statistics operations are right
> for commons. I think some refactoring of the stats packages may enable a
> clearer separation of the stats packages into those appropriate for commons
> and those appropriate for a TLP stats package, but I leave that for another
> thread.
>
> Also I call a package "dormant" when to my untrained eye it appears to have
> been underdeveloped and should perhaps be put out of its misery.
>
> Here we go:
> --
> o.a.c.m. Field, FieldElement, RealFieldElement -> TLP
> o.a.c.m. Analysis -> TLP
> o.a.c.m. complex.Complex, ComplexFormat, ComplexUtils -> commons
> o.a.c.m. complex.ComplexField, Quaternion, RootsOfUnity -> TLP
> o.a.c.m. dfp -> dormant

That does not seem right.

This is needed for FastMath testing

AFAICT, there are no unresolved bug reports relating to dfp; and those
that were raised were dealt with in a reasonable time frame.

So why drop it?

> o.a.c.m. exception -> commons
> o.a.c.m. filter -> dormant
> o.a.c.m. fraction -> all except Big FractionField to commons.
> BigFractionField to TLP
> o.a.c.m. genetics -> TLP
> o.a.c.m. geometry -> TLP
> o.a.c.m. linear -> TLP
> o.a.c.m. ml -> TLP
> o.a.c.m. ode -> TLP
> o.a.c.m. optim and optimization -> combine into one TLP component
> o.a.c.m. random and distribution -> combine into one TLP component
> o.a.c.m. primes -> commons
> o.a.c.m. special -> dormant
> o.a.c.m. stat.frequency -> perhaps belongs with random and distribution?
> o.a.c.m. stat.clustering -> dormant
> o.a.c.m. stat.correlation -> array based methods to commons, Matrix methods
> to TLP
> o.a.c.m. stat.descriptive, inferences, interval, ranking, regression ->
> Matrix methods to TLP. Array methods to commons. Statistics objects should
> be divided between both after a refactoring.
> o.a.c.m.transform -> commons
> o.a.c.m.util -> commons, perhaps refactored so the classes are in more
> informative package names
> --
>
> Eric

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Eric Barnhill
Like I said, my untrained eye. Sounds like it should definitely be kept.

Eric

On Mon, Jun 20, 2016 at 11:47 AM, sebb <[hidden email]> wrote:

> On 20 June 2016 at 10:31, Eric Barnhill <[hidden email]> wrote:
> > Here's a proposed draft for how o.a.c.m might be split into a commons
> > component, and more sophisticated spin-out components, around which we
> > might develop a new Math TLP or incubator project in which components of
> > the project that find backers continue on, and those that do not are
> frozen
> > at their current state for now but not discarded (henceforth I just call
> > this TLP).
> >
> > As a general strategy, I think it works to move the Field and Field
> Element
> > superclasses into the TLP. Methods that use these elements or other
> > math-specific data classes (e.g. DerivativeStructure, Neuron,
> > AlternativeHypothesis) belong in the TLP. Array-based classes belong in
> > commons. I see two exceptions to this. One is Complex objects which seem
> to
> > me to belong in the commons remit. The other is the array of statistics
> > objects (like Summary Statistics) as those statistics operations are
> right
> > for commons. I think some refactoring of the stats packages may enable a
> > clearer separation of the stats packages into those appropriate for
> commons
> > and those appropriate for a TLP stats package, but I leave that for
> another
> > thread.
> >
> > Also I call a package "dormant" when to my untrained eye it appears to
> have
> > been underdeveloped and should perhaps be put out of its misery.
> >
> > Here we go:
> > --
> > o.a.c.m. Field, FieldElement, RealFieldElement -> TLP
> > o.a.c.m. Analysis -> TLP
> > o.a.c.m. complex.Complex, ComplexFormat, ComplexUtils -> commons
> > o.a.c.m. complex.ComplexField, Quaternion, RootsOfUnity -> TLP
> > o.a.c.m. dfp -> dormant
>
> That does not seem right.
>
> This is needed for FastMath testing
>
> AFAICT, there are no unresolved bug reports relating to dfp; and those
> that were raised were dealt with in a reasonable time frame.
>
> So why drop it?
>
> > o.a.c.m. exception -> commons
> > o.a.c.m. filter -> dormant
> > o.a.c.m. fraction -> all except Big FractionField to commons.
> > BigFractionField to TLP
> > o.a.c.m. genetics -> TLP
> > o.a.c.m. geometry -> TLP
> > o.a.c.m. linear -> TLP
> > o.a.c.m. ml -> TLP
> > o.a.c.m. ode -> TLP
> > o.a.c.m. optim and optimization -> combine into one TLP component
> > o.a.c.m. random and distribution -> combine into one TLP component
> > o.a.c.m. primes -> commons
> > o.a.c.m. special -> dormant
> > o.a.c.m. stat.frequency -> perhaps belongs with random and distribution?
> > o.a.c.m. stat.clustering -> dormant
> > o.a.c.m. stat.correlation -> array based methods to commons, Matrix
> methods
> > to TLP
> > o.a.c.m. stat.descriptive, inferences, interval, ranking, regression ->
> > Matrix methods to TLP. Array methods to commons. Statistics objects
> should
> > be divided between both after a refactoring.
> > o.a.c.m.transform -> commons
> > o.a.c.m.util -> commons, perhaps refactored so the classes are in more
> > informative package names
> > --
> >
> > Eric
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Patrick Meyer
In reply to this post by Eric Barnhill
After reading more about the proposed changes, I am now comfortable with
dividing the library into separate components. However, I have two
concerns. First, a package or class that passes unit tests should not be
abandoned or depricated just because there is no one to support it. It
should only be dropped if it fails a test AND there is no one to support
it. Second, we should be careful about dividing the library into too many
parts. For example, I don't agree with separating array based stats from
matrix or online based stats. That's my two cents. Thanks.
On Jun 20, 2016 5:31 AM, "Eric Barnhill" <[hidden email]> wrote:

> Here's a proposed draft for how o.a.c.m might be split into a commons
> component, and more sophisticated spin-out components, around which we
> might develop a new Math TLP or incubator project in which components of
> the project that find backers continue on, and those that do not are frozen
> at their current state for now but not discarded (henceforth I just call
> this TLP).
>
> As a general strategy, I think it works to move the Field and Field Element
> superclasses into the TLP. Methods that use these elements or other
> math-specific data classes (e.g. DerivativeStructure, Neuron,
> AlternativeHypothesis) belong in the TLP. Array-based classes belong in
> commons. I see two exceptions to this. One is Complex objects which seem to
> me to belong in the commons remit. The other is the array of statistics
> objects (like Summary Statistics) as those statistics operations are right
> for commons. I think some refactoring of the stats packages may enable a
> clearer separation of the stats packages into those appropriate for commons
> and those appropriate for a TLP stats package, but I leave that for another
> thread.
>
> Also I call a package "dormant" when to my untrained eye it appears to have
> been underdeveloped and should perhaps be put out of its misery.
>
> Here we go:
> --
> o.a.c.m. Field, FieldElement, RealFieldElement -> TLP
> o.a.c.m. Analysis -> TLP
> o.a.c.m. complex.Complex, ComplexFormat, ComplexUtils -> commons
> o.a.c.m. complex.ComplexField, Quaternion, RootsOfUnity -> TLP
> o.a.c.m. dfp -> dormant
> o.a.c.m. exception -> commons
> o.a.c.m. filter -> dormant
> o.a.c.m. fraction -> all except Big FractionField to commons.
> BigFractionField to TLP
> o.a.c.m. genetics -> TLP
> o.a.c.m. geometry -> TLP
> o.a.c.m. linear -> TLP
> o.a.c.m. ml -> TLP
> o.a.c.m. ode -> TLP
> o.a.c.m. optim and optimization -> combine into one TLP component
> o.a.c.m. random and distribution -> combine into one TLP component
> o.a.c.m. primes -> commons
> o.a.c.m. special -> dormant
> o.a.c.m. stat.frequency -> perhaps belongs with random and distribution?
> o.a.c.m. stat.clustering -> dormant
> o.a.c.m. stat.correlation -> array based methods to commons, Matrix methods
> to TLP
> o.a.c.m. stat.descriptive, inferences, interval, ranking, regression ->
> Matrix methods to TLP. Array methods to commons. Statistics objects should
> be divided between both after a refactoring.
> o.a.c.m.transform -> commons
> o.a.c.m.util -> commons, perhaps refactored so the classes are in more
> informative package names
> --
>
> Eric
>
Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Gilles Sadowski
In reply to this post by Eric Barnhill
Hi.

On Mon, 20 Jun 2016 11:31:23 +0200, Eric Barnhill wrote:

> Here's a proposed draft for how o.a.c.m might be split into a commons
> component, and more sophisticated spin-out components, around which
> we
> might develop a new Math TLP or incubator project in which components
> of
> the project that find backers continue on, and those that do not are
> frozen
> at their current state for now but not discarded (henceforth I just
> call
> this TLP).
>
> As a general strategy, I think it works to move the Field and Field
> Element
> superclasses into the TLP. Methods that use these elements or other
> math-specific data classes (e.g. DerivativeStructure, Neuron,
> AlternativeHypothesis) belong in the TLP. Array-based classes belong
> in
> commons. I see two exceptions to this. One is Complex objects which
> seem to
> me to belong in the commons remit. The other is the array of
> statistics
> objects (like Summary Statistics) as those statistics operations are
> right
> for commons. I think some refactoring of the stats packages may
> enable a
> clearer separation of the stats packages into those appropriate for
> commons
> and those appropriate for a TLP stats package, but I leave that for
> another
> thread.
>
> Also I call a package "dormant" when to my untrained eye it appears
> to have
> been underdeveloped and should perhaps be put out of its misery.

Here is a list of usage count of CM packages (Github search[1]):
  o.a.c.math3.analysis      2
  o.a.c.math3.distribution 39
  o.a.c.math3.exception     3
  o.a.c.math3.linear       10
  o.a.c.math3.primes        1
  o.a.c.math3.random       25
  o.a.c.math3.special       2
  o.a.c.math3.stat         29
  o.a.c.math3.util         25

[Does someone know how to make this list more representative by adding
data from other sources?]


> Here we go:
> --
> o.a.c.m. Field, FieldElement, RealFieldElement -> TLP

+1

> o.a.c.m. Analysis -> TLP

Mostly yes, but some utilities are used by packages that would
be good components (e.g. "o.a.c.m.distribution").  For example,
an efficient and robust root solver ("BrentSolver").

> o.a.c.m. complex.Complex, ComplexFormat, ComplexUtils -> commons

+1

> o.a.c.m. complex.ComplexField, Quaternion, RootsOfUnity -> TLP

Not sure about "Quaternion"; could be in the "Complex" component.

> o.a.c.m. dfp -> dormant

Not really (as Sebb pointed out already).
To be included in an "altmath" component together with "FastMath",
and other utilities for high precision testing.

> o.a.c.m. exception -> commons

-1
Each component use its own exception set.
No hope to get a consensus on this subject. :-}

It would not be a good idea for a "core" package to have an exception
library as its sole dependency.
Also, for core packages, the exception machinery of CM is not
necessary.
[Personally, I don't think that it is ever necessary.]

> o.a.c.m. filter -> dormant

+0
There is an open issue:
   https://issues.apache.org/jira/browse/MATH-932

> o.a.c.m. fraction -> all except Big FractionField to commons.

+1

> BigFractionField to TLP

+1

> o.a.c.m. genetics -> TLP

+1
But should be dropped there (IMO).  See discussions on the ML and on
JIRA.  [A few months ago, someone proposed to enhance it (as a GSOC
assignment).]

> o.a.c.m. geometry -> TLP

+1

> o.a.c.m. linear -> TLP

+1

> o.a.c.m. ml -> TLP

+1
I thought it could be a component, but perhaps to sophisticated (?).

> o.a.c.m. ode -> TLP

+1

> o.a.c.m. optim and optimization -> combine into one TLP component

Just TLP.
[It's already a problem to get _one_ TLP. :-}]

> o.a.c.m. random and distribution -> combine into one TLP component

"o.a.c.m.random" in 3.x is to be deprecated.

o.a.c.m.rng ("develop" branch) -> Commons component.

"o.a.c.m.random" in "develop" branch is WIP (TBD).

o.a.c.m.distribution -> Commons component (that depends on the "RNG"
component).

> o.a.c.m. primes -> commons

+1

> o.a.c.m. special -> dormant

A lot of work has been put into this (thanks to Sébastien Brisard) in
order to improve the accuracy.
It's used by several classes in "o.a.c.m.distribution"
Should be a Commons component

> o.a.c.m. stat.frequency -> perhaps belongs with random and
> distribution?
> o.a.c.m. stat.clustering -> dormant
> o.a.c.m. stat.correlation -> array based methods to commons, Matrix
> methods
> to TLP
> o.a.c.m. stat.descriptive, inferences, interval, ranking, regression
> ->
> Matrix methods to TLP. Array methods to commons. Statistics objects
> should
> be divided between both after a refactoring.

Not starting a discussion about this package...
[Basic flaws in the design were identified.]

> o.a.c.m.transform -> commons

+1
[Maybe, included in the "complex" component.]

> o.a.c.m.util -> commons, perhaps refactored so the classes are in
> more
> informative package names

+1

But note that this was supposed to be an "internal" package.
In particular, utilities superseded by the JDK should be dropped.


Gilles

[1] Thanks to Christian Grobmeier for the hint.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Jörg Schaible-5
Hi Eric and Gilles,

Gilles wrote:

[snip]


> Here is a list of usage count of CM packages (Github search[1]):
>   o.a.c.math3.analysis      2
>   o.a.c.math3.distribution 39
>   o.a.c.math3.exception     3
>   o.a.c.math3.linear       10
>   o.a.c.math3.primes        1
>   o.a.c.math3.random       25
>   o.a.c.math3.special       2
>   o.a.c.math3.stat         29
>   o.a.c.math3.util         25
>
> [Does someone know how to make this list more representative by adding
> data from other sources?]


Maybe with https://www.openhub.net/p/Apache-Commons-Math

[snip]


>> o.a.c.m. exception -> commons
>
> -1
> Each component use its own exception set.
> No hope to get a consensus on this subject. :-}
>
> It would not be a good idea for a "core" package to have an exception
> library as its sole dependency.
> Also, for core packages, the exception machinery of CM is not
> necessary.
> [Personally, I don't think that it is ever necessary.]


You could use the generalized exceptions in lang that support also a value
map. If a dependency to lang is possible.

[snip]


>> o.a.c.m. optim and optimization -> combine into one TLP component
>
> Just TLP.
> [It's already a problem to get _one_ TLP. :-}]


I am quite sure Eric meant it as a single component in one Math TLP.

[snip]

If we go down this road, it is still not clear how this transition can be
done smoothly for the existing users of CM. Once Math is an own TLP, the old
CM is finally dormant in commons.

IMHO the best way would be a release of 3.x with all packages deprecated
that have been extracted into own Commons components. Thoughts?

- Jörg



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

jochen-2
In reply to this post by Gilles Sadowski
> On Mon, 20 Jun 2016 11:31:23 +0200, Eric Barnhill wrote:
>>
>> Here's a proposed draft for how o.a.c.m might be split into a commons
>> component, and more sophisticated spin-out components, around which we
>> might develop a new Math TLP or incubator project in which components of
>> the project that find backers continue on, and those that do not are
>> frozen
>> at their current state for now but not discarded (henceforth I just call
>> this TLP).

Thanks for that input, Eric. However, please note the following:

We are currently discussing, where CM will be located in the future.
We do so with the explicit intention (at least on my side), that CM
can discuss, and decide such things after a possible relocation on its
own.

Or, to put it different: Discussion of technical details is a bit
premature. Lets do the move first.

Jochen




--
The next time you hear: "Don't reinvent the wheel!"

http://www.keystonedevelopment.co.uk/wp-content/uploads/2014/10/evolution-of-the-wheel-300x85.jpg

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Jörg Schaible-5
Hi Jochen,

Jochen Wiedmann wrote:

>> On Mon, 20 Jun 2016 11:31:23 +0200, Eric Barnhill wrote:
>>>
>>> Here's a proposed draft for how o.a.c.m might be split into a commons
>>> component, and more sophisticated spin-out components, around which we
>>> might develop a new Math TLP or incubator project in which components of
>>> the project that find backers continue on, and those that do not are
>>> frozen
>>> at their current state for now but not discarded (henceforth I just call
>>> this TLP).
>
> Thanks for that input, Eric. However, please note the following:
>
> We are currently discussing, where CM will be located in the future.
> We do so with the explicit intention (at least on my side), that CM
> can discuss, and decide such things after a possible relocation on its
> own.
>
> Or, to put it different: Discussion of technical details is a bit
> premature. Lets do the move first.

That depends. If some packages of the current CM should stay as own
component in Commons, these packages have to be identified.

Cheers,
Jörg


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

jochen-2
On Tue, Jun 21, 2016 at 9:12 AM, Jörg Schaible
<[hidden email]> wrote:

> That depends. If some packages of the current CM should stay as own
> component in Commons, these packages have to be identified.

Whoever would support such a lunacy? Either CM moves entirely, or not at all.

Jochen


--
The next time you hear: "Don't reinvent the wheel!"

http://www.keystonedevelopment.co.uk/wp-content/uploads/2014/10/evolution-of-the-wheel-300x85.jpg

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Eric Barnhill
I think I made a respectable case for such a split in my thread titled
apache,commons,math . Perhaps you could identify some points in that post
that you disagree with? That's what others did.

On Tue, Jun 21, 2016 at 9:30 AM, Jochen Wiedmann <[hidden email]>
wrote:

>
>
> Whoever would support such a lunacy? Either CM moves entirely, or not at
> all.
>
> Jochen
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

jochen-2
On Tue, Jun 21, 2016 at 9:43 AM, Eric Barnhill <[hidden email]> wrote:
> I think I made a respectable case for such a split in my thread titled
> apache,commons,math . Perhaps you could identify some points in that post
> that you disagree with? That's what others did.

As I wrote: I am trying to concentrate efforts into the relocation (or
move), rather than into technical details. Makes sense, IMO.

Jochen


--
The next time you hear: "Don't reinvent the wheel!"

http://www.keystonedevelopment.co.uk/wp-content/uploads/2014/10/evolution-of-the-wheel-300x85.jpg

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Jörg Schaible-5
In reply to this post by jochen-2
Hi Jochen,

Jochen Wiedmann wrote:

> On Tue, Jun 21, 2016 at 9:12 AM, Jörg Schaible
> <[hidden email]> wrote:
>
>> That depends. If some packages of the current CM should stay as own
>> component in Commons, these packages have to be identified.
>
> Whoever would support such a lunacy? Either CM moves entirely, or not at
> all.

If the new Commons Components have been identified, we can have a vote. Then
we'll see what the majority want. All I can say now is that we have currenty
no consensus about anything. Some of the stuff in CM is certainly common
enough to build a valid Commons component.

Cheers,
Jörg



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Ralph Goers

> On Jun 21, 2016, at 12:58 AM, Jörg Schaible <[hidden email]> wrote:
>
> Hi Jochen,
>
> Jochen Wiedmann wrote:
>
>> On Tue, Jun 21, 2016 at 9:12 AM, Jörg Schaible
>> <[hidden email]> wrote:
>>
>>> That depends. If some packages of the current CM should stay as own
>>> component in Commons, these packages have to be identified.
>>
>> Whoever would support such a lunacy? Either CM moves entirely, or not at
>> all.
>
> If the new Commons Components have been identified, we can have a vote. Then
> we'll see what the majority want. All I can say now is that we have currenty
> no consensus about anything. Some of the stuff in CM is certainly common
> enough to build a valid Commons component.
>

Maybe. That could depend on whether there is anyone at Commons that would want to participate in the component. Another option is to follow the pattern used by httpclient. I believe they took the last version of commons httpclient and left a pointer to it in their new web site. But what followed as a TLP was completely different and wasn’t binary compatible. The community seemed to be completely fine with that.

Ralph



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

jochen-2
On Tue, Jun 21, 2016 at 2:54 PM, Ralph Goers <[hidden email]> wrote:

> Maybe. That could depend on whether there is anyone at Commons that would want to participate in the component. Another option is to follow the pattern used by httpclient. I believe they took the last version of commons httpclient and left a pointer to it in their new web site. But what followed as a TLP was completely different and wasn’t binary compatible. The community seemed to be completely fine with that.

Quite right. But I definitely cannot imagine anyone desiring three
development lines (Hipparchus, a fork outside Commons, and a component
inside).

Jochen



--
The next time you hear: "Don't reinvent the wheel!"

http://www.keystonedevelopment.co.uk/wp-content/uploads/2014/10/evolution-of-the-wheel-300x85.jpg

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Eric Barnhill
In reply to this post by Gilles Sadowski
On Tue, Jun 21, 2016 at 12:45 AM, Gilles <[hidden email]>
wrote:


> Mostly yes, but some utilities are used by packages that would
> be good components (e.g. "o.a.c.m.distribution").  For example,
> an efficient and robust root solver ("BrentSolver").
>

The analysis library is admirably integrated. For example, the Solvers take
UnivariateFunctions for input, but so do the interpolators.

If we think that there are key commons-remit functions that require use of
the analysis library "under the hood", than the whole thing should probably
stay in.

Analysis is not an exotic or highly specialised library, so I don't think
keeping it undermines the ability to set up an incubator/TLP with more
specialised components.

On the other hand, Analysis also transposes quite easily to the TLP as a
self-contained unit.

o.a.c.m. complex.ComplexField, Quaternion, RootsOfUnity -> TLP
>>
>
> Not sure about "Quaternion"; could be in the "Complex" component.
>

Quaternion calls FastMath and Precision, so if we are contemplating an
"alt-math" TLP component below, we have to consider how this impacts
commons-remit functions.

o.a.c.m. filter -> dormant
>>
>
> +0
> There is an open issue:
>   https://issues.apache.org/jira/browse/MATH-932
>

Ah. Perhaps this person could be approached as to whether he wants to
maintain this component in a new TLP?


> o.a.c.m. stat.frequency -> perhaps belongs with random and distribution?
>> o.a.c.m. stat.clustering -> dormant
>> o.a.c.m. stat.correlation -> array based methods to commons, Matrix
>> methods
>> to TLP
>> o.a.c.m. stat.descriptive, inferences, interval, ranking, regression ->
>> Matrix methods to TLP. Array methods to commons. Statistics objects should
>> be divided between both after a refactoring.
>>
>
> Not starting a discussion about this package...
> [Basic flaws in the design were identified.]
>

I think we can agree, to get started, to leave the stats package as it is
and keep it in commons, and worry about refactoring it later.

So, in this hypothetical arrangement we're discussing, I think the one
question is really whether the analysis package stays in commons wholesale.
I have to say, I would be interested to help try to maintain it. I would
learn a lot of useful math.

If there is an altmath TLP component with FastMath and Precision, do
BigReal and BigFraction go there also?

Eric
Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Gilles Sadowski
In reply to this post by Jörg Schaible-5
Hello.

On Tue, 21 Jun 2016 09:58:40 +0200, Jörg Schaible wrote:

> Hi Jochen,
>
> Jochen Wiedmann wrote:
>
>> On Tue, Jun 21, 2016 at 9:12 AM, Jörg Schaible
>> <[hidden email]> wrote:
>>
>>> That depends. If some packages of the current CM should stay as own
>>> component in Commons, these packages have to be identified.
>>
>> Whoever would support such a lunacy? Either CM moves entirely, or
>> not at
>> all.

It's not that clear-cut.
Thank you (and James, and Niall) for keeping the ball moving, at a
point where I was thinking that the game was over.  However, we should
all realize that it's not because that all those codes were developed
within a single repository that they all belong together.

[We got into this dire situation because so much code _depended_
(however
people here want to put it differently) on one or two people.
And it's not a size problem, per se; it's that it covers a very broad
scope, requiring expertise levels that are rarely found in one person,
even less so in a person who'd dedicated (him)herself to maintaining a
Java library.  A TLP is something to attempt but I'm not optimistic
that
we'll get much more traction.]

By having some of the functionality severed from CM, it _increases_ the
likelihood that it will be used and contributed to.
And if this functionality is actually "mature", then it won't have to
be (fakely) upgraded (through changing of package name) just because
some other (non-mature) code would need it (to allow breaking changes).

By way of consequence, such "split off" code will fulfill the Commons'
promise of stability.

In turn, the separation will have positive effects on the prospective
TLP, if just by not having to deal with issues thus become
"out-of-scope".
There may well be interactions between the TLP and Commons whenever the
TLP would choose to depend on a Commons component, but there will be
clear API boundaries.

> If the new Commons Components have been identified, we can have a
> vote. Then
> we'll see what the majority want. All I can say now is that we have
> currenty
> no consensus about anything. Some of the stuff in CM is certainly
> common
> enough to build a valid Commons component.

At last, we agree on this!  [That was my main point since day one (June
5).]

Instead of readily discussing the consequence of that observation, we
fought
about "micro-management" of Commons Math... :-(

I'll start a VOTE thread for each new Commons component candidate.

Gilles

> Cheers,
> Jörg


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Rob Tompkins

> On Jun 21, 2016, at 11:10 AM, Gilles <[hidden email]> wrote:
>
> Hello.
>
> On Tue, 21 Jun 2016 09:58:40 +0200, Jörg Schaible wrote:
>> Hi Jochen,
>>
>> Jochen Wiedmann wrote:
>>
>>> On Tue, Jun 21, 2016 at 9:12 AM, Jörg Schaible
>>> <[hidden email]> wrote:
>>>
>>>> That depends. If some packages of the current CM should stay as own
>>>> component in Commons, these packages have to be identified.
>>>
>>> Whoever would support such a lunacy? Either CM moves entirely, or not at
>>> all.
>
> It's not that clear-cut.
> Thank you (and James, and Niall) for keeping the ball moving, at a
> point where I was thinking that the game was over.  However, we should
> all realize that it's not because that all those codes were developed
> within a single repository that they all belong together.
>
> [We got into this dire situation because so much code _depended_ (however
> people here want to put it differently) on one or two people.
> And it's not a size problem, per se; it's that it covers a very broad
> scope, requiring expertise levels that are rarely found in one person,
> even less so in a person who'd dedicated (him)herself to maintaining a
> Java library.  A TLP is something to attempt but I'm not optimistic that
> we'll get much more traction.]
>
> By having some of the functionality severed from CM, it _increases_ the
> likelihood that it will be used and contributed to.
> And if this functionality is actually "mature", then it won't have to
> be (fakely) upgraded (through changing of package name) just because
> some other (non-mature) code would need it (to allow breaking changes).
>
> By way of consequence, such "split off" code will fulfill the Commons'
> promise of stability.
>
> In turn, the separation will have positive effects on the prospective
> TLP, if just by not having to deal with issues thus become "out-of-scope".
> There may well be interactions between the TLP and Commons whenever the
> TLP would choose to depend on a Commons component, but there will be
> clear API boundaries.
>
>> If the new Commons Components have been identified, we can have a vote. Then
>> we'll see what the majority want. All I can say now is that we have currenty
>> no consensus about anything. Some of the stuff in CM is certainly common
>> enough to build a valid Commons component.
>
> At last, we agree on this!  [That was my main point since day one (June 5).]
>
> Instead of readily discussing the consequence of that observation, we fought
> about "micro-management" of Commons Math... :-(
>
> I'll start a VOTE thread for each new Commons component candidate.

Would it make sense here to simply leave commons-math named commons math (mainly because mathematical naming conventions differ from the conventional vernacular [e.g. an artifact named commons-analysis might be confusing]), and deprecate anything that we plan to take and move to the incubator/TLP?

-Rob

>
> Gilles
>
>> Cheers,
>> Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Jörg Schaible
In reply to this post by jochen-2
Hi Jochen,

Jochen Wiedmann wrote:

> On Tue, Jun 21, 2016 at 2:54 PM, Ralph Goers <[hidden email]>
> wrote:
>
>> Maybe. That could depend on whether there is anyone at Commons that would
>> want to participate in the component. Another option is to follow the
>> pattern used by httpclient. I believe they took the last version of
>> commons httpclient and left a pointer to it in their new web site. But
>> what followed as a TLP was completely different and wasn’t binary
>> compatible. The community seemed to be completely fine with that.
>
> Quite right. But I definitely cannot imagine anyone desiring three
> development lines (Hipparchus, a fork outside Commons, and a component
> inside).

Why three? OK, we cannot control Hipparchus, so we can let it aside. The
proposal of Eric and Gilles just describe a scenario where some code of the
big dumping ground CM that has a wide audience and only slightly tied to
deep mathematical theory is ripped out into individual components of Commons
and the rest is moved into a Math TLP and divided there. So where do you see
two development lines at Apache (at least for the same code)?

Cheers,
Jörg


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

ole ersoy
In reply to this post by jochen-2


On 06/21/2016 08:07 AM, Jochen Wiedmann wrote:
> On Tue, Jun 21, 2016 at 2:54 PM, Ralph Goers <[hidden email]> wrote:
>
>> Maybe. That could depend on whether there is anyone at Commons that would want to participate in the component. Another option is to follow the pattern used by httpclient. I believe they took the last version of commons httpclient and left a pointer to it in their new web site. But what followed as a TLP was completely different and wasn’t binary compatible. The community seemed to be completely fine with that.
> Quite right. But I definitely cannot imagine anyone desiring three
> development lines (Hipparchus, a fork outside Commons, and a component
> inside).
There are many people that desire that:
https://github.com/substack/browserify-handbook#module-philosophy

Having one kitchen per chef allows everyone to see what is being cooked up without stepping on toes.  Just prepare your dish and show it.  If other people like it, then feel free to eat, add a little salt, whatever.  What we desire is to see great code get finished and the best and simplest means / tools for managing that.  Having many parallel lines of development means ideas get prototyped and evaluated faster.

Here's another fork:
https://github.com/firefly-math/firefly-math

Cheers,
Ole


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [MATH] what a commons-TLP split could look like

Gilles Sadowski
In reply to this post by Eric Barnhill
Hello.

On Tue, 21 Jun 2016 15:52:40 +0200, Eric Barnhill wrote:

> On Tue, Jun 21, 2016 at 12:45 AM, Gilles
> <[hidden email]>
> wrote:
>
>
>> Mostly yes, but some utilities are used by packages that would
>> be good components (e.g. "o.a.c.m.distribution").  For example,
>> an efficient and robust root solver ("BrentSolver").
>>
>
> The analysis library is admirably integrated. For example, the
> Solvers take
> UnivariateFunctions for input, but so do the interpolators.

I agree.  I have to, since I did some work there... ;-)

> If we think that there are key commons-remit functions that require
> use of
> the analysis library "under the hood", than the whole thing should
> probably
> stay in.

IMO, the guide to "Does it belong in Commons?" should include rules of
thumbs like
  * limited (even single) scope,
  * stability, and
  * usefulness beyond scientific applications and/or expert users.

> Analysis is not an exotic or highly specialised library, so I don't
> think
> keeping it undermines the ability to set up an incubator/TLP with
> more
> specialised components.

No, it doesn't; but it think that the "o.a.c.m.analysis" package falls
short
of the above "rules" (as you pointed out yourself by citing the variety
of
interpolation routines).

Nevertheless, the single "BrentSolver" code could be copied (yes, I
know
that duplicate code is bad!) into a Commons "math-core" component so
that
we avoid a Commons component depending on the math TLP just because it
needs one root solver.
[The one case I had in mind is the "o.a.c.m.distribution" package.  But
you may be right that it actually belongs to the TLP.  If so, problem
solved.]

> On the other hand, Analysis also transposes quite easily to the TLP
> as a
> self-contained unit.
>
> o.a.c.m. complex.ComplexField, Quaternion, RootsOfUnity -> TLP
>>>
>>
>> Not sure about "Quaternion"; could be in the "Complex" component.
>>
>
> Quaternion calls FastMath and Precision,

So does "Complex".

> so if we are contemplating an
> "alt-math" TLP component below, we have to consider how this impacts
> commons-remit functions.
>
> o.a.c.m. filter -> dormant
>>>
>>
>> +0
>> There is an open issue:
>>   https://issues.apache.org/jira/browse/MATH-932
>>
>
> Ah. Perhaps this person could be approached as to whether he wants to
> maintain this component in a new TLP?

Sure.

  >> o.a.c.m. stat.frequency -> perhaps belongs with random and
distribution?

>>> o.a.c.m. stat.clustering -> dormant
>>> o.a.c.m. stat.correlation -> array based methods to commons, Matrix
>>> methods
>>> to TLP
>>> o.a.c.m. stat.descriptive, inferences, interval, ranking,
>>> regression ->
>>> Matrix methods to TLP. Array methods to commons. Statistics objects
>>> should
>>> be divided between both after a refactoring.
>>>
>>
>> Not starting a discussion about this package...
>> [Basic flaws in the design were identified.]
>>
>
> I think we can agree, to get started, to leave the stats package as
> it is
> and keep it in commons, and worry about refactoring it later.

It is part of Commons Math as it was released in v3.6.1, based on the
"MATH_3_X" branch.

Until someone wants to tackle a refactoring and/or fix the identified
issues (cf. JIRA), it shouldn't be released again.
[Or only in a 3.6.x bug fix release.]

This POV also satisfy Patrick Meyer's requirement: it is _not_ dropped,
but won't become a standalone Commons component (unless someone wants
to
maintain it as such) or a module in the TLP (unless someone wants to
fix the identified issues).

> So, in this hypothetical arrangement we're discussing, I think the
> one
> question is really whether the analysis package stays in commons
> wholesale.

I'd favour it being part of the TLP for the reasons given above.

> I have to say, I would be interested to help try to maintain it. I
> would
> learn a lot of useful math.

Hopefully, you can still do that in the TLP.

> If there is an altmath TLP component with FastMath and Precision, do
> BigReal and BigFraction go there also?

According to the "single scope" rule, I'd say that "fraction" should be
in its own component.  [I'll shortly post my list of new component
candidates.]


Gilles

> Eric


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12