
12345

Oh.... and when something implements Serializable, the serial form must be either documented in the Javadocs or the Javadocs must be quite clear and say that the serial form might change in future releases. Either way, don't forget serialVersionUID and to update it if the serial form ever changes.
Sam Halliday wrote
Absolutely remove from the interfaces... but leave on implementations :) That way all future implementations aren't forced to be compatible to their first release.
Hopefully we'll address transport of instances in 2.1 with Matrix Market IO, which should also encourage more efficient compression.
Luc Maisonobe wrote
Sam Halliday a écrit :
> Luc... couldn't agree more regarding Serializable. Adding the Serializable
> interface instantly means you not only have to be API compatible with future
> releases but also binary Serializable compatible. This is what stung MTJ...
> it means you can't swap internal details of fields.
>
> I strongly recommend everybody read the Bloch chapters on Serialisation
> before ever implementing that interface.
OK. What do you suggest now ? Completely remove Serializable (which
would really bother me as I do use it) or remove it from interfaces and
add it appropriately to some implementation classes ?
Luc
>
>
> sebb22 wrote:
>> On 21/05/2009, Luc Maisonobe <Luc.Maisonobe@free.fr> wrote:
>> Nevertheless, adding serialization needs to be considered carefully,
>> as pointed out here:
>>
>> http://www.javapractices.com/topic/TopicAction.do?Id=45>>
>> Furthermore, readObject() cannot be used with final fields. Bloch
>> (Effective Java) suggests using readResolve() instead, but even this
>> has limitations.
>>
>> As the book points out, merely making a class Serializable is
>> equivalent to adding a public constructor which sets all the
>> nontransient fields without perfoming any validation.
>>
>> If there are any constraints on the fields, these have to be addressed
>> in readObject() and/or readResolve() methods.
>>
>

To unsubscribe, email: devunsubscribe@commons.apache.org
For additional commands, email: devhelp@commons.apache.org


1 on declaring things Serializable, especially if this is done everywhere.
Unless there is extensive testing and careful implementation, it is very,
very misleading to advertise this capability. In addition, there are a
number of formats which are reasonable candidates. Casting a particular
format in stone as the one true serialized format seems like a big mistake.
If you need to implement a serializable object, it is much better to mark
the field as transient and pick the wire format you really want to use from
a selection of alternatives.
On Thu, May 21, 2009 at 5:47 AM, sebb < [hidden email]> wrote:
> Merely implementing Serializable will prevent the warnings, however in
> general it's not trivial to ensure that serialization works correctly.
>

Ted Dunning, CTO
DeepDyve


1 on interfaces extending Serializable. It's poor form because it dilutes the meaning of the interface, and also because it places a nontrivial requirement on *every* implementation. Interfaces should extend Serializable only in the rare case that serialization and deserialization of instances are essential activities, thus inherent in the meaning of the interface. I realize that the various implementations provided by Math probably can be serialized successfully without any extra work, but it is unreasonable to assume that the same will be true of all possible implementations (consider, for instance, a matrix view of a dynamic, nonserializable data source). Also, do not neglect the issue with binary compatibility of the serialized form.
+0 for making the implementations serializable, though if you do, I agree with the recommendation to document the classes as not guaranteeing compatibility of the serialized form. Moreover, consider making all Serializable classes declare explicit serialVersionUID constants (preferrably sensible serial numbers, i.e. 1) to make it easier and clearer for updated classes to maintain serialization compatibility if they wish to do so. The down side of serialVersionUID, of course, is that once you declare it, you have to maintain it.
I don't view compiler warnings as a good justification for making the interfaces extend Serializable. Yes, if the interfaces were Serializable then the compiler would be happier, but your code wouldn't be much safer (vs. only implementations being serializable). These are warnings that you want to have, or it least that you want to explicitly suppress.
John Bollinger
________________________________
From: Ted Dunning < [hidden email]>
To: Commons Developers List < [hidden email]>
Sent: Thursday, May 21, 2009 3:08:27 PM
Subject: Re: [math] Re: commonsmath, matrixtoolkitsjava and consolidation
1 on declaring things Serializable, especially if this is done everywhere.
Unless there is extensive testing and careful implementation, it is very,
very misleading to advertise this capability. In addition, there are a
number of formats which are reasonable candidates. Casting a particular
format in stone as the one true serialized format seems like a big mistake.
If you need to implement a serializable object, it is much better to mark
the field as transient and pick the wire format you really want to use from
a selection of alternatives.
On Thu, May 21, 2009 at 5:47 AM, sebb < [hidden email]> wrote:
> Merely implementing Serializable will prevent the warnings, however in
> general it's not trivial to ensure that serialization works correctly.
>

Ted Dunning, CTO
DeepDyve


That's really cool.
BTW, Can I ask about the plan of data distribution strategies of your
'distributed' package in the future? IMO, it seems, it doesn't sit
well with 'commonmath' project.
If if there is a developer who wants to implement 'distributed', pls
let us know, too. I'm working for the Hama
( http://incubator.apache.org/hama) with ScaLAPACK members.
On Thu, May 14, 2009 at 7:18 PM, Sam Halliday < [hidden email]> wrote:
>
> Dear all,
>
> I am a maintainer of the matrixtoolkitsjava
>
> http://code.google.com/p/matrixtoolkitsjava/>
> which is a comprehensive collection of matrix data structures, linear
> solvers, least squares methods, eigenvalue and singular value
> decompositions.
>
> This note is in regard to the commonsmath library. It is clear that our
> projects dovetail, especially when I look at "linear" in version 2.0 of the
> API. It would be good if we could either complement or consolidate efforts,
> rather than reproduce.
>
> It would be excellent if all the functionality of matrixtoolkitsjava were
> available as part of commonsmath. There is already too much diversity and
> unmaintained maths code out there for Java!
>
> As a start, I'd like to discourage the use of a solid implementation for
> SparseReal{Vector, Matrix}... please prefer an interface approach, allowing
> implementations based on the Templates project:
>
> http://www.netlib.org/templates>
> The reason is that the storage implementation should be related to the type
> of data being stored. For example, there are many wellknown kinds of sparse
> matrix that are well suited to particular kinds of calculations... consider
> multiplying sparse matrices that you know to be diagonal!
>
> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
> thinking about linear algebra and have set up unrivalled standard APIs which
> have been implemented right down to the architecture level. It would be a
> major mistake if commonsmath didn't build on their good work.
>
> I believe commonsmath should move to a netlibjava backend (allowing the
> use of machine optimised BLAS/LAPACK).
>
> http://code.google.com/p/netlibjava/>
> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
> former should be possible with some work within the current API, but I fear
> major API changes would be needed for the latter. I do not want the
> commonsmath API to walk into this trap without having first considered
> future architectures! MTJ has a distributed package, but I am not sure if
> this is something that is completely future proof either.
>
> What say ye'?
>
> 
> Sam
>
> 
> View this message in context: http://www.nabble.com/commonsmath%2Cmatrixtoolkitsjavaandconsolidationtp23537813p23537813.html> Sent from the Commons  Dev mailing list archive at Nabble.com.
>
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>
>

Best Regards, Edward J. Yoon @ NHN, corp.
[hidden email]
http://blog.udanax.org
To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


Time zones mean that I tend to come in late here :(. More answers inline.
 Original Message 
From: "Sam Halliday" < [hidden email]>
To: < [hidden email]>
Sent: Thursday, May 21, 2009 3:13 AM
Subject: Re: [math] Re: commonsmath, matrixtoolkitsjava and consolidation
>
> Bill, I've had a look at some of the recent changes... some comments,
> including some new thoughts:
>
>  changing the return type to be actual classes was only supposed to be
> for
> copy() for 2.0. Doing this on multiply, add, etc is a big mistake... there
> is no guarantee that is the best thing to do and it is likely this will
> change in the future. This stands for all implementations. Great thought
> needs to go into each method, not just by reading the current
> implementation.
>
I sort of like being able to use the optimized methods for inlining, but
don't feel strongly on this. I'll go for whatever the community consensus
is (and I don't see one at the moment). It was a search/replace before and
I can do it again ;).
>  I *strongly* urge you to remove Serializable from everything! Please, we
> did this in MTJ and it turned out to be a major pain. A more appropriate
> approach is to define a class for reading/writing Matrix Market files.
> This
> can be a new feature in 2.1. If you're going to leave it, at least
> document
> that the Serializable form is not guaranteed to remain compatible across
> versions.
>
Like Luc, I'm generallly in favor of Serializable. Since some of the posts
on this thread have suggested problems with the current implementation, I'll
try and run some tests over the (what is here, long) weekend. Again, no
consensus so not doing anything immediately.
>  I discourage the use of the classes named *Impl. They will get very
> confusing when other implementations are added later! Instead, I recommend
> the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
> backed implementation using row ordering). This allows a columnbased or
> 1D
> implementation in the future without names getting very confusing. These
> implementations are hidden from users who just use the MatrixUtils help
>
Have no problem doing a deprecateandcopy. Will do once there is a
consensus on the name.
>  1 for inclusion of methods in the sparse interfaces and shape type
> enum.
> This needs more thought and discussion before inclusion. I am happy enough
> for marker interfaces to be included (as long as it is documented that
> these
> interfaces are subject to update in the next release), but strongly
> against
> including methods in them.
>
In the current SVN, these are empty marker interfaces, since we seem to have
a consensus on that.
>  could you please describe the hashing algorithm that OpenMap is using? I
> have previously written a GNU Trove based sparse matrix implementation
> which
> was a map from the primitive "long" to "double". It looks analagous. I
> broke
> the row/column index into a long for the key using binary operations, but
> the hashcode was tricky as it had to be an integer (not long). A naive
> implementation can lead to collisions for typical or symmetric matrices.
> It
> looks like the current implementation is using an int > double backing
> map!
> Wouldn't it make more sense to use a long?
>
I've just been playing CodeMonkey, so Phil and Luc can do more interesting
stuff, and we can get the 2.0 version released ;). OpenIntToDoubleHashMap
was already there when I got involved with [math], so I just used it. Luc
was the one that initially committed OpenIntToDoulbleHashMap, but I think
that it came from a Jira patch (and, yes, I'm too lazy to look up which
one).
> The following code might be useful to you (don't forget the L marker and
> casts or the compiler will silently cast to an int), there are two options
> for a hashcode calculator... I've not stress tested the second one but it
> might be more appropriate as it gives more weight to lower bits. Remember
> that FF is 256, or a full byte... so a full 64 bit long is 8 bytes or 16
> hex
> characters in Java bitwise.
>
> int hash1(long val) {
> int first = Integer.reverse(key2row(val));
> int second = key2col(val);
> return first + second;
> }
>
> int hash2(long val) {
> int first = key2row(val) & 0x0000FFFF;
> int second = key2col(val) << 16;
> return first + second;
> }
>
> long key(int row, int column) {
> return (((long) row) << 32) + (long) column;
> }
>
> int key2col(long key) {
> return (int) (key & 0x00000000FFFFFFFFL);
> }
>
> int key2row(long key) {
> return (int) (key >>> 32);
> }
>
>
> Bill Barker wrote:
>>
>> I have a slight preference for leaving the markers empty until 3.0, but I
>> can remove them as well. But I can wait to see what the community
>> consensus
>> is before making changes.
>>
>
> 
> View this message in context:
> http://www.nabble.com/commonsmath%2Cmatrixtoolkitsjavaandconsolidationtp23537813p23650705.html> Sent from the Commons  Dev mailing list archive at Nabble.com.
>
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


In favor or not, Serializable shouldn't in in widely used interfaces.
As an example, a Lucene index is a reasonable implementation of a sparse
matrix.
Would you require that I have to figure out how to make it serializable just
because I declare it as a Matrix?
Do you imagine that most developers will do more than just punt in such a
situation if the interface absolutely requires that the object be
serializable?
Leave it to particular implementations to be serializable or not. Please,
please, please don't force it into the contract for all implementations.
On Thu, May 21, 2009 at 8:20 PM, Bill Barker < [hidden email]>wrote:
>  I *strongly* urge you to remove Serializable from everything! Please, we
>> did this in MTJ and it turned out to be a major pain. A more appropriate
>> approach is to define a class for reading/writing Matrix Market files.
>> This
>> can be a new feature in 2.1. If you're going to leave it, at least
>> document
>> that the Serializable form is not guaranteed to remain compatible across
>> versions.
>>
>>
> Like Luc, I'm generallly in favor of Serializable. Since some of the posts
> on this thread have suggested problems with the current implementation, I'll
> try and run some tests over the (what is here, long) weekend. Again, no
> consensus so not doing anything immediately.

Ted Dunning, CTO
DeepDyve


It's a shame you can't use the GNU Trove project, it would be a perfect robust backend for primitive mapping.
Bill Barker wrote
I've just been playing CodeMonkey, so Phil and Luc can do more interesting
stuff, and we can get the 2.0 version released ;). OpenIntToDoubleHashMap
was already there when I got involved with [math], so I just used it. Luc
was the one that initially committed OpenIntToDoulbleHashMap, but I think
that it came from a Jira patch (and, yes, I'm too lazy to look up which
one).


Ted Dunning a écrit :
> In favor or not, Serializable shouldn't in in widely used interfaces.
>
> As an example, a Lucene index is a reasonable implementation of a sparse
> matrix.
>
> Would you require that I have to figure out how to make it serializable just
> because I declare it as a Matrix?
>
> Do you imagine that most developers will do more than just punt in such a
> situation if the interface absolutely requires that the object be
> serializable?
>
> Leave it to particular implementations to be serializable or not. Please,
> please, please don't force it into the contract for all implementations.
So we have reached a consensus: remove Serializable from interfaces and
push it down to implementations only.
Any volunteer to do this rather boring work ?
>
> On Thu, May 21, 2009 at 8:20 PM, Bill Barker < [hidden email]>wrote:
>
>>  I *strongly* urge you to remove Serializable from everything! Please, we
>>> did this in MTJ and it turned out to be a major pain. A more appropriate
>>> approach is to define a class for reading/writing Matrix Market files.
>>> This
>>> can be a new feature in 2.1. If you're going to leave it, at least
>>> document
>>> that the Serializable form is not guaranteed to remain compatible across
>>> versions.
>>>
>>>
>> Like Luc, I'm generallly in favor of Serializable. Since some of the posts
>> on this thread have suggested problems with the current implementation, I'll
>> try and run some tests over the (what is here, long) weekend. Again, no
>> consensus so not doing anything immediately.
>
>
>
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


Luc Maisonobe wrote:
> Ted Dunning a écrit :
>
>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>
>> As an example, a Lucene index is a reasonable implementation of a sparse
>> matrix.
>>
>> Would you require that I have to figure out how to make it serializable just
>> because I declare it as a Matrix?
>>
>> Do you imagine that most developers will do more than just punt in such a
>> situation if the interface absolutely requires that the object be
>> serializable?
>>
>> Leave it to particular implementations to be serializable or not. Please,
>> please, please don't force it into the contract for all implementations.
>>
>
> So we have reached a consensus: remove Serializable from interfaces and
> push it down to implementations only.
>
+1
> Any volunteer to do this rather boring work ?
>
I wish I could say yes, but I am running out of buffer space atm ;)
Phil
>
>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker < [hidden email]>wrote:
>>
>>
>>>  I *strongly* urge you to remove Serializable from everything! Please, we
>>>
>>>> did this in MTJ and it turned out to be a major pain. A more appropriate
>>>> approach is to define a class for reading/writing Matrix Market files.
>>>> This
>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>> document
>>>> that the Serializable form is not guaranteed to remain compatible across
>>>> versions.
>>>>
>>>>
>>>>
>>> Like Luc, I'm generallly in favor of Serializable. Since some of the posts
>>> on this thread have suggested problems with the current implementation, I'll
>>> try and run some tests over the (what is here, long) weekend. Again, no
>>> consensus so not doing anything immediately.
>>>
>>
>>
>>
>
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


Edward, Hama looks fantastic! I fully agree, 'distributed' isn't agreeable with commonsmath. However, there might be parts of it that are relevant to Hama.
We should absolutely ensure that the "ducks are aligned" between commonsmath and Hama. It would be winwin if Hama were able to use commonsmath's linear interfaces. commonsmath will hopefully be moving to use a netlibjava style backend (not user facing), we should ensure that ScaLAPACK is handled in the same way.
I have some ideas of people who might be interested... I've encountered them as maintainer of MTJ, but never worked with them. I'll dig through my email archive and make the intros.
Edward J. Yoon2 wrote
That's really cool.
BTW, Can I ask about the plan of data distribution strategies of your
'distributed' package in the future? IMO, it seems, it doesn't sit
well with 'commonmath' project.
If if there is a developer who wants to implement 'distributed', pls
let us know, too. I'm working for the Hama
( http://incubator.apache.org/hama) with ScaLAPACK members.
On Thu, May 14, 2009 at 7:18 PM, Sam Halliday <sam.halliday@gmail.com> wrote:
>
> Dear all,
>
> I am a maintainer of the matrixtoolkitsjava
>
> http://code.google.com/p/matrixtoolkitsjava/>
> which is a comprehensive collection of matrix data structures, linear
> solvers, least squares methods, eigenvalue and singular value
> decompositions.
>
> This note is in regard to the commonsmath library. It is clear that our
> projects dovetail, especially when I look at "linear" in version 2.0 of the
> API. It would be good if we could either complement or consolidate efforts,
> rather than reproduce.
>
> It would be excellent if all the functionality of matrixtoolkitsjava were
> available as part of commonsmath. There is already too much diversity and
> unmaintained maths code out there for Java!
>
> As a start, I'd like to discourage the use of a solid implementation for
> SparseReal{Vector, Matrix}... please prefer an interface approach, allowing
> implementations based on the Templates project:
>
> http://www.netlib.org/templates>
> The reason is that the storage implementation should be related to the type
> of data being stored. For example, there are many wellknown kinds of sparse
> matrix that are well suited to particular kinds of calculations... consider
> multiplying sparse matrices that you know to be diagonal!
>
> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
> thinking about linear algebra and have set up unrivalled standard APIs which
> have been implemented right down to the architecture level. It would be a
> major mistake if commonsmath didn't build on their good work.
>
> I believe commonsmath should move to a netlibjava backend (allowing the
> use of machine optimised BLAS/LAPACK).
>
> http://code.google.com/p/netlibjava/>
> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
> former should be possible with some work within the current API, but I fear
> major API changes would be needed for the latter. I do not want the
> commonsmath API to walk into this trap without having first considered
> future architectures! MTJ has a distributed package, but I am not sure if
> this is something that is completely future proof either.
>
> What say ye'?
>
> 
> Sam
>
> 
> View this message in context: http://www.nabble.com/commonsmath%2Cmatrixtoolkitsjavaandconsolidationtp23537813p23537813.html> Sent from the Commons  Dev mailing list archive at Nabble.com.
>
>
> 
> To unsubscribe, email: devunsubscribe@commons.apache.org
> For additional commands, email: devhelp@commons.apache.org
>
>

Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
To unsubscribe, email: devunsubscribe@commons.apache.org
For additional commands, email: devhelp@commons.apache.org


Sam Halliday a écrit :
> Edward, Hama looks fantastic! I fully agree, 'distributed' isn't agreeable
> with commonsmath. However, there might be parts of it that are relevant to
> Hama.
Is Hama related to Hadoop ?
>
> We should absolutely ensure that the "ducks are aligned" between
> commonsmath and Hama. It would be winwin if Hama were able to use
> commonsmath's linear interfaces. commonsmath will hopefully be moving to
> use a netlibjava style backend (not user facing), we should ensure that
> ScaLAPACK is handled in the same way.
>
> I have some ideas of people who might be interested... I've encountered them
> as maintainer of MTJ, but never worked with them. I'll dig through my email
> archive and make the intros.
>
>
> Edward J. Yoon2 wrote:
>> That's really cool.
>>
>> BTW, Can I ask about the plan of data distribution strategies of your
>> 'distributed' package in the future? IMO, it seems, it doesn't sit
>> well with 'commonmath' project.
>>
>> If if there is a developer who wants to implement 'distributed', pls
>> let us know, too. I'm working for the Hama
>> ( http://incubator.apache.org/hama) with ScaLAPACK members.
>>
>> On Thu, May 14, 2009 at 7:18 PM, Sam Halliday < [hidden email]>
>> wrote:
>>> Dear all,
>>>
>>> I am a maintainer of the matrixtoolkitsjava
>>>
>>> http://code.google.com/p/matrixtoolkitsjava/>>>
>>> which is a comprehensive collection of matrix data structures, linear
>>> solvers, least squares methods, eigenvalue and singular value
>>> decompositions.
>>>
>>> This note is in regard to the commonsmath library. It is clear that our
>>> projects dovetail, especially when I look at "linear" in version 2.0 of
>>> the
>>> API. It would be good if we could either complement or consolidate
>>> efforts,
>>> rather than reproduce.
>>>
>>> It would be excellent if all the functionality of matrixtoolkitsjava
>>> were
>>> available as part of commonsmath. There is already too much diversity
>>> and
>>> unmaintained maths code out there for Java!
>>>
>>> As a start, I'd like to discourage the use of a solid implementation for
>>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>>> allowing
>>> implementations based on the Templates project:
>>>
>>> http://www.netlib.org/templates>>>
>>> The reason is that the storage implementation should be related to the
>>> type
>>> of data being stored. For example, there are many wellknown kinds of
>>> sparse
>>> matrix that are well suited to particular kinds of calculations...
>>> consider
>>> multiplying sparse matrices that you know to be diagonal!
>>>
>>> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
>>> thinking about linear algebra and have set up unrivalled standard APIs
>>> which
>>> have been implemented right down to the architecture level. It would be a
>>> major mistake if commonsmath didn't build on their good work.
>>>
>>> I believe commonsmath should move to a netlibjava backend (allowing the
>>> use of machine optimised BLAS/LAPACK).
>>>
>>> http://code.google.com/p/netlibjava/>>>
>>> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
>>> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
>>> former should be possible with some work within the current API, but I
>>> fear
>>> major API changes would be needed for the latter. I do not want the
>>> commonsmath API to walk into this trap without having first considered
>>> future architectures! MTJ has a distributed package, but I am not sure if
>>> this is something that is completely future proof either.
>>>
>>> What say ye'?
>>>
>>> 
>>> Sam
>>>
>>> 
>>> View this message in context:
>>> http://www.nabble.com/commonsmath%2Cmatrixtoolkitsjavaandconsolidationtp23537813p23537813.html>>> Sent from the Commons  Dev mailing list archive at Nabble.com.
>>>
>>>
>>> 
>>> To unsubscribe, email: [hidden email]
>>> For additional commands, email: [hidden email]
>>>
>>>
>>
>>
>> 
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> [hidden email]
>> http://blog.udanax.org>>
>> 
>> To unsubscribe, email: [hidden email]
>> For additional commands, email: [hidden email]
>>
>>
>>
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


On 22/05/2009, Phil Steitz < [hidden email]> wrote:
> Luc Maisonobe wrote:
>
> > Ted Dunning a écrit :
> >
> >
> > > In favor or not, Serializable shouldn't in in widely used interfaces.
> > >
> > > As an example, a Lucene index is a reasonable implementation of a sparse
> > > matrix.
> > >
> > > Would you require that I have to figure out how to make it serializable
> just
> > > because I declare it as a Matrix?
> > >
> > > Do you imagine that most developers will do more than just punt in such
> a
> > > situation if the interface absolutely requires that the object be
> > > serializable?
> > >
> > > Leave it to particular implementations to be serializable or not.
> Please,
> > > please, please don't force it into the contract for all implementations.
> > >
> > >
> >
> > So we have reached a consensus: remove Serializable from interfaces and
> > push it down to implementations only.
> >
> >
> +1
>
> > Any volunteer to do this rather boring work ?
> >
> >
I can make a start on it.
I suggest that the implementations are flagged (e.g. with TODOs) until
the serialization implementation has been checked and documented.
WDYT?
> I wish I could say yes, but I am running out of buffer space atm ;)
>
> Phil
>
>
> >
> >
> > > On Thu, May 21, 2009 at 8:20 PM, Bill Barker
> < [hidden email]>wrote:
> > >
> > >
> > >
> > > >  I *strongly* urge you to remove Serializable from everything!
> Please, we
> > > >
> > > >
> > > > > did this in MTJ and it turned out to be a major pain. A more
> appropriate
> > > > > approach is to define a class for reading/writing Matrix Market
> files.
> > > > > This
> > > > > can be a new feature in 2.1. If you're going to leave it, at least
> > > > > document
> > > > > that the Serializable form is not guaranteed to remain compatible
> across
> > > > > versions.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > Like Luc, I'm generallly in favor of Serializable. Since some of the
> posts
> > > > on this thread have suggested problems with the current
> implementation, I'll
> > > > try and run some tests over the (what is here, long) weekend. Again,
> no
> > > > consensus so not doing anything immediately.
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
> 
> > To unsubscribe, email:
> [hidden email]
> > For additional commands, email: [hidden email]
> >
> >
> >
>
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


[I changed the subject to help follow the thread]
Phil Steitz a écrit :
> Luc Maisonobe wrote:
>> Ted Dunning a écrit :
>>
>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>>
>>> As an example, a Lucene index is a reasonable implementation of a sparse
>>> matrix.
>>>
>>> Would you require that I have to figure out how to make it
>>> serializable just
>>> because I declare it as a Matrix?
>>>
>>> Do you imagine that most developers will do more than just punt in
>>> such a
>>> situation if the interface absolutely requires that the object be
>>> serializable?
>>>
>>> Leave it to particular implementations to be serializable or not.
>>> Please,
>>> please, please don't force it into the contract for all implementations.
>>>
>>
>> So we have reached a consensus: remove Serializable from interfaces and
>> push it down to implementations only.
>>
> +1
There is one interface at least for which I ask to retain Externalizable
(not really the same as Serializable, but in the same spirit). It is the
StepInterpolator interface in the ode.sampling package. Externalization
for this interface is a desired and important feature used for example
in the ContinuousOutputModel class. The interface is not intended to be
implemented by users, and in fact even the class implementing it are not
directly visible by users: instances are directly built by ODE
integrators (each integrator has its own interpolator).
>> Any volunteer to do this rather boring work ?
>>
> I wish I could say yes, but I am running out of buffer space atm ;)
>
> Phil
>>
>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>>> < [hidden email]>wrote:
>>>
>>>
>>>>  I *strongly* urge you to remove Serializable from everything!
>>>> Please, we
>>>>
>>>>> did this in MTJ and it turned out to be a major pain. A more
>>>>> appropriate
>>>>> approach is to define a class for reading/writing Matrix Market files.
>>>>> This
>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>> document
>>>>> that the Serializable form is not guaranteed to remain compatible
>>>>> across
>>>>> versions.
>>>>>
>>>>>
>>>>>
>>>> Like Luc, I'm generallly in favor of Serializable. Since some of
>>>> the posts
>>>> on this thread have suggested problems with the current
>>>> implementation, I'll
>>>> try and run some tests over the (what is here, long) weekend.
>>>> Again, no
>>>> consensus so not doing anything immediately.
>>>>
>>>
>>>
>>>
>>
>>
>> 
>> To unsubscribe, email: [hidden email]
>> For additional commands, email: [hidden email]
>>
>>
>
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


sebb a écrit :
> On 22/05/2009, Phil Steitz < [hidden email]> wrote:
>> Luc Maisonobe wrote:
>>
>>> Ted Dunning a écrit :
>>>
>>>
>>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>>>
>>>> As an example, a Lucene index is a reasonable implementation of a sparse
>>>> matrix.
>>>>
>>>> Would you require that I have to figure out how to make it serializable
>> just
>>>> because I declare it as a Matrix?
>>>>
>>>> Do you imagine that most developers will do more than just punt in such
>> a
>>>> situation if the interface absolutely requires that the object be
>>>> serializable?
>>>>
>>>> Leave it to particular implementations to be serializable or not.
>> Please,
>>>> please, please don't force it into the contract for all implementations.
>>>>
>>>>
>>> So we have reached a consensus: remove Serializable from interfaces and
>>> push it down to implementations only.
>>>
>>>
>> +1
>>
>>> Any volunteer to do this rather boring work ?
>>>
>>>
>
> I can make a start on it.
>
> I suggest that the implementations are flagged (e.g. with TODOs) until
> the serialization implementation has been checked and documented.
>
> WDYT?
It sounds good. Many thanks.
Luc
>
>> I wish I could say yes, but I am running out of buffer space atm ;)
>>
>> Phil
>>
>>
>>>
>>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>> < [hidden email]>wrote:
>>>>
>>>>
>>>>>  I *strongly* urge you to remove Serializable from everything!
>> Please, we
>>>>>
>>>>>> did this in MTJ and it turned out to be a major pain. A more
>> appropriate
>>>>>> approach is to define a class for reading/writing Matrix Market
>> files.
>>>>>> This
>>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>>> document
>>>>>> that the Serializable form is not guaranteed to remain compatible
>> across
>>>>>> versions.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> Like Luc, I'm generallly in favor of Serializable. Since some of the
>> posts
>>>>> on this thread have suggested problems with the current
>> implementation, I'll
>>>>> try and run some tests over the (what is here, long) weekend. Again,
>> no
>>>>> consensus so not doing anything immediately.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>> 
>>> To unsubscribe, email:
>> [hidden email]
>>> For additional commands, email: [hidden email]
>>>
>>>
>>>
>>
>> 
>> To unsubscribe, email: [hidden email]
>> For additional commands, email: [hidden email]
>>
>>
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


On 22/05/2009, Luc Maisonobe < [hidden email]> wrote:
> sebb a écrit :
>
> > On 22/05/2009, Phil Steitz < [hidden email]> wrote:
> >> Luc Maisonobe wrote:
> >>
> >>> Ted Dunning a écrit :
> >>>
> >>>
> >>>> In favor or not, Serializable shouldn't in in widely used interfaces.
> >>>>
> >>>> As an example, a Lucene index is a reasonable implementation of a sparse
> >>>> matrix.
> >>>>
> >>>> Would you require that I have to figure out how to make it serializable
> >> just
> >>>> because I declare it as a Matrix?
> >>>>
> >>>> Do you imagine that most developers will do more than just punt in such
> >> a
> >>>> situation if the interface absolutely requires that the object be
> >>>> serializable?
> >>>>
> >>>> Leave it to particular implementations to be serializable or not.
> >> Please,
> >>>> please, please don't force it into the contract for all implementations.
> >>>>
> >>>>
> >>> So we have reached a consensus: remove Serializable from interfaces and
> >>> push it down to implementations only.
> >>>
> >>>
> >> +1
> >>
>
> >>> Any volunteer to do this rather boring work ?
> >>>
> >>>
> >
>
> > I can make a start on it.
> >
> > I suggest that the implementations are flagged (e.g. with TODOs) until
> > the serialization implementation has been checked and documented.
> >
> > WDYT?
>
> It sounds good. Many thanks.
>
>
Just started to look at this, and the first interface I checked is
MultivariateMatrixFunction.
This does not have any implementations, except for anonymous classes
in some Test classes.
In this case, presumably only the interface needs to be updated?
Would it be useful to record in the Javadoc is the interface used to
have Serializable?
[As well as in the release notes, of course]
I think this might be useful, but if noone else does, then there's no
point doing it.

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


sebb a écrit :
> On 22/05/2009, Luc Maisonobe < [hidden email]> wrote:
>> sebb a écrit :
>>
>>> On 22/05/2009, Phil Steitz < [hidden email]> wrote:
>> >> Luc Maisonobe wrote:
>> >>
>> >>> Ted Dunning a écrit :
>> >>>
>> >>>
>> >>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>> >>>>
>> >>>> As an example, a Lucene index is a reasonable implementation of a sparse
>> >>>> matrix.
>> >>>>
>> >>>> Would you require that I have to figure out how to make it serializable
>> >> just
>> >>>> because I declare it as a Matrix?
>> >>>>
>> >>>> Do you imagine that most developers will do more than just punt in such
>> >> a
>> >>>> situation if the interface absolutely requires that the object be
>> >>>> serializable?
>> >>>>
>> >>>> Leave it to particular implementations to be serializable or not.
>> >> Please,
>> >>>> please, please don't force it into the contract for all implementations.
>> >>>>
>> >>>>
>> >>> So we have reached a consensus: remove Serializable from interfaces and
>> >>> push it down to implementations only.
>> >>>
>> >>>
>> >> +1
>> >>
>>
>>>>> Any volunteer to do this rather boring work ?
>> >>>
>> >>>
>> >
>>
>>> I can make a start on it.
>> >
>> > I suggest that the implementations are flagged (e.g. with TODOs) until
>> > the serialization implementation has been checked and documented.
>> >
>> > WDYT?
>>
>> It sounds good. Many thanks.
>>
>>
>
> Just started to look at this, and the first interface I checked is
> MultivariateMatrixFunction.
>
> This does not have any implementations, except for anonymous classes
> in some Test classes.
The interface is intended to be implemented by users of the optimization
package.
>
> In this case, presumably only the interface needs to be updated?
Yes.
>
> Would it be useful to record in the Javadoc is the interface used to
> have Serializable?
> [As well as in the release notes, of course]
In this case, I don't think so. It is a new interface not yet published.
>
> I think this might be useful, but if noone else does, then there's no
> point doing it.
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


sebb a écrit :
> On 22/05/2009, Luc Maisonobe < [hidden email]> wrote:
>> sebb a écrit :
>>
>>> On 22/05/2009, Phil Steitz < [hidden email]> wrote:
>> >> Luc Maisonobe wrote:
>> >>
>> >>> Ted Dunning a écrit :
>> >>>
>> >>>
>> >>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>> >>>>
>> >>>> As an example, a Lucene index is a reasonable implementation of a sparse
>> >>>> matrix.
>> >>>>
>> >>>> Would you require that I have to figure out how to make it serializable
>> >> just
>> >>>> because I declare it as a Matrix?
>> >>>>
>> >>>> Do you imagine that most developers will do more than just punt in such
>> >> a
>> >>>> situation if the interface absolutely requires that the object be
>> >>>> serializable?
>> >>>>
>> >>>> Leave it to particular implementations to be serializable or not.
>> >> Please,
>> >>>> please, please don't force it into the contract for all implementations.
>> >>>>
>> >>>>
>> >>> So we have reached a consensus: remove Serializable from interfaces and
>> >>> push it down to implementations only.
>> >>>
>> >>>
>> >> +1
>> >>
>>
>>>>> Any volunteer to do this rather boring work ?
>> >>>
>> >>>
>> >
>>
>>> I can make a start on it.
>> >
>> > I suggest that the implementations are flagged (e.g. with TODOs) until
>> > the serialization implementation has been checked and documented.
>> >
>> > WDYT?
>>
>> It sounds good. Many thanks.
>>
>>
>
> Just started to look at this, and the first interface I checked is
> MultivariateMatrixFunction.
>
> This does not have any implementations, except for anonymous classes
> in some Test classes.
The interface is intended to be implemented by users of the optimization
package.
>
> In this case, presumably only the interface needs to be updated?
Yes.
>
> Would it be useful to record in the Javadoc is the interface used to
> have Serializable?
> [As well as in the release notes, of course]
In this case, I don't think so. It is a new interface not yet published.
>
> I think this might be useful, but if noone else does, then there's no
> point doing it.
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


> Is Hama related to Hadoop ?
Yes, it is.
On Fri, May 22, 2009 at 8:05 PM, Luc Maisonobe < [hidden email]> wrote:
> Sam Halliday a écrit :
>> Edward, Hama looks fantastic! I fully agree, 'distributed' isn't agreeable
>> with commonsmath. However, there might be parts of it that are relevant to
>> Hama.
>
> Is Hama related to Hadoop ?
>
>>
>> We should absolutely ensure that the "ducks are aligned" between
>> commonsmath and Hama. It would be winwin if Hama were able to use
>> commonsmath's linear interfaces. commonsmath will hopefully be moving to
>> use a netlibjava style backend (not user facing), we should ensure that
>> ScaLAPACK is handled in the same way.
>>
>> I have some ideas of people who might be interested... I've encountered them
>> as maintainer of MTJ, but never worked with them. I'll dig through my email
>> archive and make the intros.
>>
>>
>> Edward J. Yoon2 wrote:
>>> That's really cool.
>>>
>>> BTW, Can I ask about the plan of data distribution strategies of your
>>> 'distributed' package in the future? IMO, it seems, it doesn't sit
>>> well with 'commonmath' project.
>>>
>>> If if there is a developer who wants to implement 'distributed', pls
>>> let us know, too. I'm working for the Hama
>>> ( http://incubator.apache.org/hama) with ScaLAPACK members.
>>>
>>> On Thu, May 14, 2009 at 7:18 PM, Sam Halliday < [hidden email]>
>>> wrote:
>>>> Dear all,
>>>>
>>>> I am a maintainer of the matrixtoolkitsjava
>>>>
>>>> http://code.google.com/p/matrixtoolkitsjava/>>>>
>>>> which is a comprehensive collection of matrix data structures, linear
>>>> solvers, least squares methods, eigenvalue and singular value
>>>> decompositions.
>>>>
>>>> This note is in regard to the commonsmath library. It is clear that our
>>>> projects dovetail, especially when I look at "linear" in version 2.0 of
>>>> the
>>>> API. It would be good if we could either complement or consolidate
>>>> efforts,
>>>> rather than reproduce.
>>>>
>>>> It would be excellent if all the functionality of matrixtoolkitsjava
>>>> were
>>>> available as part of commonsmath. There is already too much diversity
>>>> and
>>>> unmaintained maths code out there for Java!
>>>>
>>>> As a start, I'd like to discourage the use of a solid implementation for
>>>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>>>> allowing
>>>> implementations based on the Templates project:
>>>>
>>>> http://www.netlib.org/templates>>>>
>>>> The reason is that the storage implementation should be related to the
>>>> type
>>>> of data being stored. For example, there are many wellknown kinds of
>>>> sparse
>>>> matrix that are well suited to particular kinds of calculations...
>>>> consider
>>>> multiplying sparse matrices that you know to be diagonal!
>>>>
>>>> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
>>>> thinking about linear algebra and have set up unrivalled standard APIs
>>>> which
>>>> have been implemented right down to the architecture level. It would be a
>>>> major mistake if commonsmath didn't build on their good work.
>>>>
>>>> I believe commonsmath should move to a netlibjava backend (allowing the
>>>> use of machine optimised BLAS/LAPACK).
>>>>
>>>> http://code.google.com/p/netlibjava/>>>>
>>>> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
>>>> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
>>>> former should be possible with some work within the current API, but I
>>>> fear
>>>> major API changes would be needed for the latter. I do not want the
>>>> commonsmath API to walk into this trap without having first considered
>>>> future architectures! MTJ has a distributed package, but I am not sure if
>>>> this is something that is completely future proof either.
>>>>
>>>> What say ye'?
>>>>
>>>> 
>>>> Sam
>>>>
>>>> 
>>>> View this message in context:
>>>> http://www.nabble.com/commonsmath%2Cmatrixtoolkitsjavaandconsolidationtp23537813p23537813.html>>>> Sent from the Commons  Dev mailing list archive at Nabble.com.
>>>>
>>>>
>>>> 
>>>> To unsubscribe, email: [hidden email]
>>>> For additional commands, email: [hidden email]
>>>>
>>>>
>>>
>>>
>>> 
>>> Best Regards, Edward J. Yoon @ NHN, corp.
>>> [hidden email]
>>> http://blog.udanax.org>>>
>>> 
>>> To unsubscribe, email: [hidden email]
>>> For additional commands, email: [hidden email]
>>>
>>>
>>>
>>
>
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>
>

Best Regards, Edward J. Yoon @ NHN, corp.
[hidden email]
http://blog.udanax.org
To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]


Hama is a proposed distributed matrix implementation system that is based on
using mapreduce to implement basic matrix operations and uses hbase to
store matrices. Getting useful performance out of this substrate for dense
matrix operations is likely to be fairly challenging due to I/O costs. For
sparse operations that exceed memory size, it may be more attractive.
Hama has been around for nearly two years. So far, it appears that there is
an implementation of matrix multiply and add. Performance numbers are
underwhelming for dense matrices. On sample problem of multiplying 5000 x
5000 random matrices, hama achieves a speed on 8 workstations that is about
1/3 of the speed of R running on a laptop.
Performance on sparse matrices may be better.
On Fri, May 22, 2009 at 10:13 AM, Edward J. Yoon < [hidden email]>wrote:
> > Is Hama related to Hadoop ?
>
> Yes, it is.

Ted Dunning, CTO
DeepDyve


 Original Message 
From: "Luc Maisonobe" < [hidden email]>
To: "Commons Developers List" < [hidden email]>
Sent: Friday, May 22, 2009 4:19 AM
Subject: Re: [math] Serialization
> [I changed the subject to help follow the thread]
>
> Phil Steitz a écrit :
>> Luc Maisonobe wrote:
>>> Ted Dunning a écrit :
>>>
>>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>>>
>>>> As an example, a Lucene index is a reasonable implementation of a
>>>> sparse
>>>> matrix.
>>>>
>>>> Would you require that I have to figure out how to make it
>>>> serializable just
>>>> because I declare it as a Matrix?
>>>>
>>>> Do you imagine that most developers will do more than just punt in
>>>> such a
>>>> situation if the interface absolutely requires that the object be
>>>> serializable?
>>>>
>>>> Leave it to particular implementations to be serializable or not.
>>>> Please,
>>>> please, please don't force it into the contract for all
>>>> implementations.
>>>>
>>>
>>> So we have reached a consensus: remove Serializable from interfaces and
>>> push it down to implementations only.
>>>
>> +1
>
> There is one interface at least for which I ask to retain Externalizable
> (not really the same as Serializable, but in the same spirit). It is the
> StepInterpolator interface in the ode.sampling package. Externalization
> for this interface is a desired and important feature used for example
> in the ContinuousOutputModel class. The interface is not intended to be
> implemented by users, and in fact even the class implementing it are not
> directly visible by users: instances are directly built by ODE
> integrators (each integrator has its own interpolator).
>
>>> Any volunteer to do this rather boring work ?
I can take a stab at it (but may have fewer spare cycles than sebb).
>>>
>> I wish I could say yes, but I am running out of buffer space atm ;)
>>
>> Phil
>>>
>>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>>>> < [hidden email]>wrote:
>>>>
>>>>
>>>>>  I *strongly* urge you to remove Serializable from everything!
>>>>> Please, we
>>>>>
>>>>>> did this in MTJ and it turned out to be a major pain. A more
>>>>>> appropriate
>>>>>> approach is to define a class for reading/writing Matrix Market
>>>>>> files.
>>>>>> This
>>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>>> document
>>>>>> that the Serializable form is not guaranteed to remain compatible
>>>>>> across
>>>>>> versions.
>>>>>>
>>>>>>
>>>>>>
>>>>> Like Luc, I'm generallly in favor of Serializable. Since some of
>>>>> the posts
>>>>> on this thread have suggested problems with the current
>>>>> implementation, I'll
>>>>> try and run some tests over the (what is here, long) weekend.
>>>>> Again, no
>>>>> consensus so not doing anything immediately.
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> 
>>> To unsubscribe, email: [hidden email]
>>> For additional commands, email: [hidden email]
>>>
>>>
>>
>>
>> 
>> To unsubscribe, email: [hidden email]
>> For additional commands, email: [hidden email]
>>
>
>
> 
> To unsubscribe, email: [hidden email]
> For additional commands, email: [hidden email]
>
>

To unsubscribe, email: [hidden email]
For additional commands, email: [hidden email]

12345
