How to use NonLinearConjugateGradientOptimizer

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

How to use NonLinearConjugateGradientOptimizer

spx15
Hi,

I have been struggling for several days to figure out how to use
NonLinearConjugateGradientOptimizer.
My goal is to optimize a multivariate unconstrained function (for example
f(x,y,z) = x^2 + y^2 + z^2) with conjugate gradient method. I have already
managed to optimize my function with SimplexOptimizer and want to try
Gradient Descent now.

For now, my code is (inspired by all the examples I could found):

import java.util.Arrays;

import org.apache.commons.math3.analysis.MultivariateFunction;
import org.apache.commons.math3.analysis.MultivariateVectorFunction;
import
org.apache.commons.math3.analysis.differentiation.DerivativeStructure;
import org.apache.commons.math3.analysis.differentiation.GradientFunction;
import
org.apache.commons.math3.analysis.differentiation.MultivariateDifferentiableFunction;
import org.apache.commons.math3.optim.InitialGuess;
import org.apache.commons.math3.optim.MaxEval;
import org.apache.commons.math3.optim.PointValuePair;
import org.apache.commons.math3.optim.SimpleValueChecker;
import org.apache.commons.math3.optim.nonlinear.scalar.GoalType;
import
org.apache.commons.math3.optim.nonlinear.scalar.GradientMultivariateOptimizer;
import org.apache.commons.math3.optim.nonlinear.scalar.ObjectiveFunction;
import
org.apache.commons.math3.optim.nonlinear.scalar.ObjectiveFunctionGradient;
import
org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer;
import
org.apache.commons.math3.optim.nonlinear.scalar.noderiv.NelderMeadSimplex;
import
org.apache.commons.math3.optim.nonlinear.scalar.noderiv.SimplexOptimizer;

public class ApacheTest {
       
    public static void main(String[] args) {
   
        SimplexOptimizer simplexOptimizer = new SimplexOptimizer(1e-10,
1e-30);
       
        NonLinearConjugateGradientOptimizer gradientOptimizer = new
NonLinearConjugateGradientOptimizer(NonLinearConjugateGradientOptimizer.Formula.POLAK_RIBIERE,
new SimpleValueChecker(1e-13, 1e-13));
       
        final MyFunction f = new MyFunction();

        simplexOptimizerMethod(simplexOptimizer, f);
        conjugateGradientMethod(gradientOptimizer, f);
    }

        private static void
conjugateGradientMethod(NonLinearConjugateGradientOptimizer optimizer,
MyFunction f) {
               
                ObjectiveFunction objF = new ObjectiveFunction(f);
               
                final PointValuePair optimum =
                                optimizer.optimize(new MaxEval(1000), objF, GoalType.MINIMIZE, new
InitialGuess(new double[]{ 10, 10, 10 }));

                System.out.println(Arrays.toString(optimum.getPoint()) + " : "
                    + optimum.getSecond());
               
        }

        private static void simplexOptimizerMethod(SimplexOptimizer optimizer,
final MyFunction f) {
                final PointValuePair optimum =
            optimizer.optimize(
                new MaxEval(10000),
                new ObjectiveFunction(f),
                GoalType.MINIMIZE,
                new InitialGuess(new double[]{ 100, 100, 60 }),
                new NelderMeadSimplex(new double[]{ 0.2, 0.2, 0.2 }));

        System.out.println(Arrays.toString(optimum.getPoint()) + " : "
            + optimum.getSecond());
        }

    private static class MyFunction implements MultivariateFunction {
     
        public double value(double[] variables) {
       
            final double x = variables[0];
            final double y = variables[1];
            final double z = variables[2];
           
            return x*x + y*y + z*z;
           
        }
       
    }
}

When I execute the code I get the following error:

Exception in thread "main" java.lang.NullPointerException
        at
org.apache.commons.math3.optim.nonlinear.scalar.GradientMultivariateOptimizer.computeObjectiveGradient(GradientMultivariateOptimizer.java:53)
        at
org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.doOptimize(NonLinearConjugateGradientOptimizer.java:254)
        at
org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.doOptimize(NonLinearConjugateGradientOptimizer.java:46)
        at
org.apache.commons.math3.optim.BaseOptimizer.optimize(BaseOptimizer.java:153)
        at
org.apache.commons.math3.optim.BaseMultivariateOptimizer.optimize(BaseMultivariateOptimizer.java:65)
        at
org.apache.commons.math3.optim.nonlinear.scalar.MultivariateOptimizer.optimize(MultivariateOptimizer.java:63)
        at
org.apache.commons.math3.optim.nonlinear.scalar.GradientMultivariateOptimizer.optimize(GradientMultivariateOptimizer.java:73)
        at
org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.optimize(NonLinearConjugateGradientOptimizer.java:244)
        at
com.sri.ai.expresso.apache.ApacheTest.conjugateGradientMethod(ApacheTest.java:96)
        at com.sri.ai.expresso.apache.ApacheTest.main(ApacheTest.java:87)


I think that maybe the problem is a missing argument of type
ObjectiveFunctionGradient in the optimize method:

final PointValuePair optimum =
                                optimizer.optimize(new MaxEval(1000), objF, GoalType.MINIMIZE, new
InitialGuess(new double[]{ 10, 10, 10 }));

However, I don't know how to build it.

Does anyone has an idea, or an example which looks like what I am looking
for?

Thank you very much



--
Sent from: http://apache-commons.680414.n4.nabble.com/Commons-User-f735979.html

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to use NonLinearConjugateGradientOptimizer

Gilles Sadowski
On Tue, 8 May 2018 11:44:17 -0700 (MST), spx15 wrote:

> Hi,
>
> I have been struggling for several days to figure out how to use
> NonLinearConjugateGradientOptimizer.
> My goal is to optimize a multivariate unconstrained function (for
> example
> f(x,y,z) = x^2 + y^2 + z^2) with conjugate gradient method. I have
> already
> managed to optimize my function with SimplexOptimizer and want to try
> Gradient Descent now.
>
> For now, my code is (inspired by all the examples I could found):
>
> import java.util.Arrays;
>
> import org.apache.commons.math3.analysis.MultivariateFunction;
> import org.apache.commons.math3.analysis.MultivariateVectorFunction;
> import
>
> org.apache.commons.math3.analysis.differentiation.DerivativeStructure;
> import
> org.apache.commons.math3.analysis.differentiation.GradientFunction;
> import
>
> org.apache.commons.math3.analysis.differentiation.MultivariateDifferentiableFunction;
> import org.apache.commons.math3.optim.InitialGuess;
> import org.apache.commons.math3.optim.MaxEval;
> import org.apache.commons.math3.optim.PointValuePair;
> import org.apache.commons.math3.optim.SimpleValueChecker;
> import org.apache.commons.math3.optim.nonlinear.scalar.GoalType;
> import
>
> org.apache.commons.math3.optim.nonlinear.scalar.GradientMultivariateOptimizer;
> import
> org.apache.commons.math3.optim.nonlinear.scalar.ObjectiveFunction;
> import
>
> org.apache.commons.math3.optim.nonlinear.scalar.ObjectiveFunctionGradient;
> import
>
> org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer;
> import
>
> org.apache.commons.math3.optim.nonlinear.scalar.noderiv.NelderMeadSimplex;
> import
>
> org.apache.commons.math3.optim.nonlinear.scalar.noderiv.SimplexOptimizer;
>
> public class ApacheTest {
>
>     public static void main(String[] args) {
>
>         SimplexOptimizer simplexOptimizer = new
> SimplexOptimizer(1e-10,
> 1e-30);
>
>         NonLinearConjugateGradientOptimizer gradientOptimizer = new
>
> NonLinearConjugateGradientOptimizer(NonLinearConjugateGradientOptimizer.Formula.POLAK_RIBIERE,
> new SimpleValueChecker(1e-13, 1e-13));
>
>         final MyFunction f = new MyFunction();
>
>         simplexOptimizerMethod(simplexOptimizer, f);
>         conjugateGradientMethod(gradientOptimizer, f);
>     }
>
> private static void
> conjugateGradientMethod(NonLinearConjugateGradientOptimizer
> optimizer,
> MyFunction f) {
>
> ObjectiveFunction objF = new ObjectiveFunction(f);
>
> final PointValuePair optimum =
> optimizer.optimize(new MaxEval(1000), objF, GoalType.MINIMIZE,
> new
> InitialGuess(new double[]{ 10, 10, 10 }));
>
>        System.out.println(Arrays.toString(optimum.getPoint()) + " :
> "
>            + optimum.getSecond());
>
> }
>
> private static void simplexOptimizerMethod(SimplexOptimizer
> optimizer,
> final MyFunction f) {
> final PointValuePair optimum =
>             optimizer.optimize(
>                 new MaxEval(10000),
>                 new ObjectiveFunction(f),
>                 GoalType.MINIMIZE,
>                 new InitialGuess(new double[]{ 100, 100, 60 }),
>                 new NelderMeadSimplex(new double[]{ 0.2, 0.2, 0.2
> }));
>
>         System.out.println(Arrays.toString(optimum.getPoint()) + " :
> "
>             + optimum.getSecond());
> }
>
>     private static class MyFunction implements MultivariateFunction {
>
>         public double value(double[] variables) {
>
>             final double x = variables[0];
>             final double y = variables[1];
>             final double z = variables[2];
>
>             return x*x + y*y + z*z;
>
>         }
>
>     }
> }
>
> When I execute the code I get the following error:
>
> Exception in thread "main" java.lang.NullPointerException
> at
>
> org.apache.commons.math3.optim.nonlinear.scalar.GradientMultivariateOptimizer.computeObjectiveGradient(GradientMultivariateOptimizer.java:53)
> at
>
> org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.doOptimize(NonLinearConjugateGradientOptimizer.java:254)
> at
>
> org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.doOptimize(NonLinearConjugateGradientOptimizer.java:46)
> at
>
> org.apache.commons.math3.optim.BaseOptimizer.optimize(BaseOptimizer.java:153)
> at
>
> org.apache.commons.math3.optim.BaseMultivariateOptimizer.optimize(BaseMultivariateOptimizer.java:65)
> at
>
> org.apache.commons.math3.optim.nonlinear.scalar.MultivariateOptimizer.optimize(MultivariateOptimizer.java:63)
> at
>
> org.apache.commons.math3.optim.nonlinear.scalar.GradientMultivariateOptimizer.optimize(GradientMultivariateOptimizer.java:73)
> at
>
> org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.optimize(NonLinearConjugateGradientOptimizer.java:244)
> at
>
> com.sri.ai.expresso.apache.ApacheTest.conjugateGradientMethod(ApacheTest.java:96)
> at com.sri.ai.expresso.apache.ApacheTest.main(ApacheTest.java:87)
>
>
> I think that maybe the problem is a missing argument of type
> ObjectiveFunctionGradient in the optimize method:
>
> final PointValuePair optimum =
> optimizer.optimize(new MaxEval(1000), objF, GoalType.MINIMIZE,
> new
> InitialGuess(new double[]{ 10, 10, 10 }));
>
> However, I don't know how to build it.
>
> Does anyone has an idea, or an example which looks like what I am
> looking
> for?
>
> Thank you very much
>

See "parseOptimizationData" method here:
   
http://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/optim/nonlinear/scalar/gradient/NonLinearConjugateGradientOptimizer.html

It is called by "optimize" and the list of arguments to the latter must
contain one argument of type "ObjectiveFunctionGradient".

The unit tests classes show examples:
   
https://git1-us-west.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/test/java/org/apache/commons/math4/optim/nonlinear/scalar/gradient/NonLinearConjugateGradientOptimizerTest.java;h=df10e649cdf6d10f0f9b1c89a3a433aa23aedcd9;hb=HEAD


HTH,
Gilles



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]