[math] CMA-ES input sigma

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[math] CMA-ES input sigma

Luc Maisonobe
Hello,

I am trying to use CMA-ES optimizer with simple boundaries.
It seems the inputSigma parameter should be normalized as it is checked
against the [0; 1] range in the checkParameters private method and as
its value defaults to 0.3 if not not set in the initializeCMA private
method.

I would have expected this value to be in the same units as the user
parameters and to be normalized as part of an internal processing step
instead of relying to the user doing this. I think the method need
normalized values internally, as per the encode/decode methods in the
inner class FitnessFunction suggest.

What do you think about it ? Should we keep normalized inputSigma (end
hence improve documentation so people know they have to normalize the
value) or should we accept values in the same units as the other
parameters and use "encode" to do the normalisation ?

As far as I am concerned, I would prefer the second solution, i.e. keep
normalization an internal implementation detail.

Luc


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] CMA-ES input sigma

Bruce A Johnson-2
I had exactly this problem (where it wasn't obvious that the sigma parameter needed to be normalized to [0-1]), a few days ago.  I think your second solution is the more user friendly.

Bruce

On Nov 7, 2011, at 7:58 AM, Luc Maisonobe wrote:

> Hello,
>
> I am trying to use CMA-ES optimizer with simple boundaries.
> It seems the inputSigma parameter should be normalized as it is checked
> against the [0; 1] range in the checkParameters private method and as
> its value defaults to 0.3 if not not set in the initializeCMA private
> method.
>
> I would have expected this value to be in the same units as the user
> parameters and to be normalized as part of an internal processing step
> instead of relying to the user doing this. I think the method need
> normalized values internally, as per the encode/decode methods in the
> inner class FitnessFunction suggest.
>
> What do you think about it ? Should we keep normalized inputSigma (end
> hence improve documentation so people know they have to normalize the
> value) or should we accept values in the same units as the other
> parameters and use "encode" to do the normalisation ?
>
> As far as I am concerned, I would prefer the second solution, i.e. keep
> normalization an internal implementation detail.
>
> Luc
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] CMA-ES input sigma

Gilles Sadowski
In reply to this post by Luc Maisonobe
Hi Luc.

> I am trying to use CMA-ES optimizer with simple boundaries.
> It seems the inputSigma parameter should be normalized as it is checked
> against the [0; 1] range in the checkParameters private method and as
> its value defaults to 0.3 if not not set in the initializeCMA private
> method.
>
> I would have expected this value to be in the same units as the user
> parameters and to be normalized as part of an internal processing step
> instead of relying to the user doing this. I think the method need
> normalized values internally, as per the encode/decode methods in the
> inner class FitnessFunction suggest.
>
> What do you think about it ? Should we keep normalized inputSigma (end
> hence improve documentation so people know they have to normalize the
> value) or should we accept values in the same units as the other
> parameters and use "encode" to do the normalisation ?
>
> As far as I am concerned, I would prefer the second solution, i.e. keep
> normalization an internal implementation detail.

I like implementation details.


Best regards,
Gilles

P.S. Please don't forget that the "CMAESOptimizer" is not yet upgraded to
     use the new "optimize" API (for simple bounds); I intended to change
     that by next week.

P.P.S. If, by any chance, you could use your current work in order to expand
       the code coverage of the unit tests for "BOBYQAOptimizer", that would
       be most useful! [And this optimizer's API is ready for use.]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] CMA-ES input sigma

Luc Maisonobe
Le 07/11/2011 14:24, Gilles Sadowski a écrit :

> Hi Luc.
>
>> I am trying to use CMA-ES optimizer with simple boundaries.
>> It seems the inputSigma parameter should be normalized as it is checked
>> against the [0; 1] range in the checkParameters private method and as
>> its value defaults to 0.3 if not not set in the initializeCMA private
>> method.
>>
>> I would have expected this value to be in the same units as the user
>> parameters and to be normalized as part of an internal processing step
>> instead of relying to the user doing this. I think the method need
>> normalized values internally, as per the encode/decode methods in the
>> inner class FitnessFunction suggest.
>>
>> What do you think about it ? Should we keep normalized inputSigma (end
>> hence improve documentation so people know they have to normalize the
>> value) or should we accept values in the same units as the other
>> parameters and use "encode" to do the normalisation ?
>>
>> As far as I am concerned, I would prefer the second solution, i.e. keep
>> normalization an internal implementation detail.
>
> I like implementation details.

OK, I'll do that.

>
>
> Best regards,
> Gilles
>
> P.S. Please don't forget that the "CMAESOptimizer" is not yet upgraded to
>      use the new "optimize" API (for simple bounds); I intended to change
>      that by next week.

Ah, thanks. This explain some strange behaviour I get. I will let you
adapt this, for now I will simply set the boundaries both in the
constructor and in the call to optimize. I guess the constructot will be
changed later so the boundaries are set only in optimize, is this right ?


>
> P.P.S. If, by any chance, you could use your current work in order to expand
>        the code coverage of the unit tests for "BOBYQAOptimizer", that would
>        be most useful! [And this optimizer's API is ready for use.]

I'll try to do it. However the models I optimize are quite large and
depend on an external library (Orekit, of course), so this will need
much rework, so I'm afraid I will do it only if I encounter problems
that need some debugging.

Luc

>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] CMA-ES input sigma

Luc Maisonobe
Le 07/11/2011 14:42, Luc Maisonobe a écrit :

> Le 07/11/2011 14:24, Gilles Sadowski a écrit :
>> Hi Luc.
>>
>>> I am trying to use CMA-ES optimizer with simple boundaries.
>>> It seems the inputSigma parameter should be normalized as it is checked
>>> against the [0; 1] range in the checkParameters private method and as
>>> its value defaults to 0.3 if not not set in the initializeCMA private
>>> method.
>>>
>>> I would have expected this value to be in the same units as the user
>>> parameters and to be normalized as part of an internal processing step
>>> instead of relying to the user doing this. I think the method need
>>> normalized values internally, as per the encode/decode methods in the
>>> inner class FitnessFunction suggest.
>>>
>>> What do you think about it ? Should we keep normalized inputSigma (end
>>> hence improve documentation so people know they have to normalize the
>>> value) or should we accept values in the same units as the other
>>> parameters and use "encode" to do the normalisation ?
>>>
>>> As far as I am concerned, I would prefer the second solution, i.e. keep
>>> normalization an internal implementation detail.
>>
>> I like implementation details.
>
> OK, I'll do that.

Done in revision 1198741. I have created and resolved Jira issue 702 to
track the problem.

Luc

>
>>
>>
>> Best regards,
>> Gilles
>>
>> P.S. Please don't forget that the "CMAESOptimizer" is not yet upgraded to
>>      use the new "optimize" API (for simple bounds); I intended to change
>>      that by next week.
>
> Ah, thanks. This explain some strange behaviour I get. I will let you
> adapt this, for now I will simply set the boundaries both in the
> constructor and in the call to optimize. I guess the constructot will be
> changed later so the boundaries are set only in optimize, is this right ?
>
>
>>
>> P.P.S. If, by any chance, you could use your current work in order to expand
>>        the code coverage of the unit tests for "BOBYQAOptimizer", that would
>>        be most useful! [And this optimizer's API is ready for use.]
>
> I'll try to do it. However the models I optimize are quite large and
> depend on an external library (Orekit, of course), so this will need
> much rework, so I'm afraid I will do it only if I encounter problems
> that need some debugging.
>
> Luc
>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] CMA-ES input sigma

Gilles Sadowski
In reply to this post by Luc Maisonobe
> [...]
> >
> > P.S. Please don't forget that the "CMAESOptimizer" is not yet upgraded to
> >      use the new "optimize" API (for simple bounds); I intended to change
> >      that by next week.
>
> Ah, thanks. This explain some strange behaviour I get. I will let you
> adapt this, for now I will simply set the boundaries both in the
> constructor and in the call to optimize. I guess the constructot will be
> changed later so the boundaries are set only in optimize, is this right ?

Exactly.


Gilles

> [...]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

AW: [math] CMA-ES input sigma

Dr. Dietmar Wolz
Will have a look when Gilles finished the change next week.
Dietmar

-----Ursprüngliche Nachricht-----
Von: Gilles Sadowski [mailto:[hidden email]]
Gesendet: Montag, 7. November 2011 15:22
An: [hidden email]
Betreff: Re: [math] CMA-ES input sigma

> [...]
> >
> > P.S. Please don't forget that the "CMAESOptimizer" is not yet upgraded
to
> >      use the new "optimize" API (for simple bounds); I intended to
change
> >      that by next week.
>
> Ah, thanks. This explain some strange behaviour I get. I will let you
> adapt this, for now I will simply set the boundaries both in the
> constructor and in the call to optimize. I guess the constructot will
> be changed later so the boundaries are set only in optimize, is this right
?

Exactly.


Gilles

> [...]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]