[math] No User Guide for new Optimization methods

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[math] No User Guide for new Optimization methods

Nigel R Murray
For those of us who have projects dependent on the deprecated Optimization
methods (using Jacobians for differentiable functions), would someone
consider updating the User Guide to give optimization examples using the
new methods?

Instead of Jacobians, the new methods appear to use DerivativeStructure
objects.  The current version of the user guide, which is located at
http://commons.apache.org/proper/commons-math/userguide/optimization.html,
only describes the deprecated methods.

In the meantime, if some enlightened soul could post a quick migration
example to this mailing list, that would be awesome.  I'm currently using
the LevenbergMarquardtOptimizer with
the DifferentiableMultivariateVectorFunction for the derivative so any
pointers on how to implement similar functionality using the non-deprecated
methods would be most helpful.

I can't be the only one stuck on this issue.

Thanks!

Nigel
Reply | Threaded
Open this post in threaded view
|

Re: [math] No User Guide for new Optimization methods

Luc Maisonobe-2
Hi Nigel,

Le 24/07/2015 21:14, Nigel R Murray a écrit :
> For those of us who have projects dependent on the deprecated Optimization
> methods (using Jacobians for differentiable functions), would someone
> consider updating the User Guide to give optimization examples using the
> new methods?

I'll try to write up something in the next few days.

>
> Instead of Jacobians, the new methods appear to use DerivativeStructure
> objects.  The current version of the user guide, which is located at
> http://commons.apache.org/proper/commons-math/userguide/optimization.html,
> only describes the deprecated methods.
>
> In the meantime, if some enlightened soul could post a quick migration
> example to this mailing list, that would be awesome.  I'm currently using
> the LevenbergMarquardtOptimizer with
> the DifferentiableMultivariateVectorFunction for the derivative so any
> pointers on how to implement similar functionality using the non-deprecated
> methods would be most helpful.
>
> I can't be the only one stuck on this issue.

You are right, this point has been raised a couple of times, we are
clearly behind for this. I will attempt some documentation improvements.

Sorry for that
Luc

>
> Thanks!
>
> Nigel
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] No User Guide for new Optimization methods

Luc Maisonobe-2
In reply to this post by Nigel R Murray
Hi Nigel,

Le 24/07/2015 21:14, Nigel R Murray a écrit :
> For those of us who have projects dependent on the deprecated Optimization
> methods (using Jacobians for differentiable functions), would someone
> consider updating the User Guide to give optimization examples using the
> new methods?

I have updated the user guide by adding a new chapter dedicated to
least squares (see
<http://commons.apache.org/proper/commons-math/userguide/leastsquares.html>).

The Levenberg-Marquardt optimizer belongs to this section.

I did not yet update the remainder of the optimization documentation,
so it still only refers to the deprecated methods for now.

>
> Instead of Jacobians, the new methods appear to use DerivativeStructure
> objects.  The current version of the user guide, which is located at
> http://commons.apache.org/proper/commons-math/userguide/optimization.html,
> only describes the deprecated methods.

The specific least squares optimizers to not mandate use of
DerivativeStructure. They can be used to set up the Jacobians, of
course, but they do not appear in the API. It was considered an error
to put this at API level so when least-squares was stabilized (which is
not yet the case for the other optimizers), we went back to a
traditional pair separating value and derivative in the model funtions.

>
> In the meantime, if some enlightened soul could post a quick migration
> example to this mailing list, that would be awesome.  I'm currently using
> the LevenbergMarquardtOptimizer with
> the DifferentiableMultivariateVectorFunction for the derivative so any
> pointers on how to implement similar functionality using the non-deprecated
> methods would be most helpful.

The API for the least squares should be easy to convert if you start
from the deprecated ones (more simple than starting from the
intermediate API with DerivativeStructure). The main difference is the
introduction of a LeastSquaresProblem interface to hold anything related
to the problem (typically model function, derivatives, start point,
target, ...). The simplest way to create this kind of object is to use
the LeastSquaresBuilder. See the updated userguide in the page
referenced above.

If you already have done some work using DerivativeStructure, you can
still use it (it should simplify getting correct derivatives in
difficult problems), but you will have to extract the values on one
side and the partial derivatives on the other side, using getValue()
and getPartialDerivatives(). If you want some help on this, you can ask
here a more specific questions, I'll try to provide an answer.

best regards,
Luc

>
> I can't be the only one stuck on this issue.
>
> Thanks!
>
> Nigel
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] No User Guide for new Optimization methods

Nigel R Murray
Thank you for the quick turnaround Luc.  I didn't realize that the
DerivativeStructure did not have to be used directly.  That actually makes
everything much simpler (and, in my opinion, was absolutely the right
design choice).  From what you've written so far, I can stick with the
MultivariateMatrixFunction for the Jacobian.

I went ahead and changed my code to mimic your example.  The only
significant issue I ran across was the weight matrix for the
LeastSquaresBuilder.  Before, it was a vector whose elements corresponded
to the data elements.  But the LeastSquaresBuilder requires a square
matrix.  The references to the weight matrix in Section 14 (LeastSquares)
still indicate a vector.  For now I have removed the weights in order to
use the system default.

Thanks again.

Nigel

On Mon, Jul 27, 2015 at 3:58 PM, Luc Maisonobe <[hidden email]> wrote:

> Hi Nigel,
>
> Le 24/07/2015 21:14, Nigel R Murray a écrit :
> > For those of us who have projects dependent on the deprecated
> Optimization
> > methods (using Jacobians for differentiable functions), would someone
> > consider updating the User Guide to give optimization examples using the
> > new methods?
>
> I have updated the user guide by adding a new chapter dedicated to
> least squares (see
> <http://commons.apache.org/proper/commons-math/userguide/leastsquares.html
> >).
>
> The Levenberg-Marquardt optimizer belongs to this section.
>
> I did not yet update the remainder of the optimization documentation,
> so it still only refers to the deprecated methods for now.
>
> >
> > Instead of Jacobians, the new methods appear to use DerivativeStructure
> > objects.  The current version of the user guide, which is located at
> >
> http://commons.apache.org/proper/commons-math/userguide/optimization.html,
> > only describes the deprecated methods.
>
> The specific least squares optimizers to not mandate use of
> DerivativeStructure. They can be used to set up the Jacobians, of
> course, but they do not appear in the API. It was considered an error
> to put this at API level so when least-squares was stabilized (which is
> not yet the case for the other optimizers), we went back to a
> traditional pair separating value and derivative in the model funtions.
>
> >
> > In the meantime, if some enlightened soul could post a quick migration
> > example to this mailing list, that would be awesome.  I'm currently using
> > the LevenbergMarquardtOptimizer with
> > the DifferentiableMultivariateVectorFunction for the derivative so any
> > pointers on how to implement similar functionality using the
> non-deprecated
> > methods would be most helpful.
>
> The API for the least squares should be easy to convert if you start
> from the deprecated ones (more simple than starting from the
> intermediate API with DerivativeStructure). The main difference is the
> introduction of a LeastSquaresProblem interface to hold anything related
> to the problem (typically model function, derivatives, start point,
> target, ...). The simplest way to create this kind of object is to use
> the LeastSquaresBuilder. See the updated userguide in the page
> referenced above.
>
> If you already have done some work using DerivativeStructure, you can
> still use it (it should simplify getting correct derivatives in
> difficult problems), but you will have to extract the values on one
> side and the partial derivatives on the other side, using getValue()
> and getPartialDerivatives(). If you want some help on this, you can ask
> here a more specific questions, I'll try to provide an answer.
>
> best regards,
> Luc
>
> >
> > I can't be the only one stuck on this issue.
> >
> > Thanks!
> >
> > Nigel
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] No User Guide for new Optimization methods

Gilles Sadowski
On Tue, 28 Jul 2015 07:44:35 -0400, Nigel R Murray wrote:

> Thank you for the quick turnaround Luc.  I didn't realize that the
> DerivativeStructure did not have to be used directly.  That actually
> makes
> everything much simpler (and, in my opinion, was absolutely the right
> design choice).  From what you've written so far, I can stick with
> the
> MultivariateMatrixFunction for the Jacobian.
>
> I went ahead and changed my code to mimic your example.  The only
> significant issue I ran across was the weight matrix for the
> LeastSquaresBuilder.  Before, it was a vector whose elements
> corresponded
> to the data elements.  But the LeastSquaresBuilder requires a square
> matrix.  The references to the weight matrix in Section 14
> (LeastSquares)
> still indicate a vector.  For now I have removed the weights in order
> to
> use the system default.

For the particular case of diagonal weights:
   
http://commons.apache.org/proper/commons-math/javadocs/api-3.5/org/apache/commons/math3/linear/DiagonalMatrix.html

Regards,
Gilles

> > [...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]