# [math] How to use LevenbergMarquardtOptimizer for finding the optimal dampening factor for exponential smoothing

5 messages
Open this post in threaded view
|

## [math] How to use LevenbergMarquardtOptimizer for finding the optimal dampening factor for exponential smoothing

 I am trying to write a Java program for generating a forecast using exponential smoothing as described here: https://www.itl.nist.gov/div898/handbook/pmc/section4/pmc431.htm. As described at the linked document, exponential smoothing uses a dampening factor "alpha". The document further goes on to say that an optimal value for "alpha" can be found using the "Marquardt procedure", which I take as referring to the Levenberg-Marquardt algorithm. From the linked document, it seems that the problem of finding the optimal "alpha" is treated as a least-squares problem and fed into the optimizer, with an initial guess for "alpha". After extensive web search I could not find any ready example of using the Levenberg-Marquardt algorithm to find "alpha" for this kind of a problem, with any programming language. So, I dug into the Javadocs and test cases for the class *LevenbergMarquardtOptimizer* to see if I could come up with a solution of my own. My program is given an array of values, say *[9, 8, 9, 12, 10, 12, 11, 7, 13, 9, 11, 10]*, and an initial guess for "alpha", say *0.2*. I have been able to determine that this information needs to be converted into a *LeastSquaresProblem*, for which I have done the following so far:    1. Set the input array as the *target*;    2. Set the starting point *start *as the initial value of alpha (*{ 0.2    }*);    3. Set *weight* to *[1, 1 - alpha, (1 - alpha)^2, ...]*; and    4. Set the optimization function of the model to return smooth values    for each of the input values. I am now unsure how the Jacobian should be calculated. I would like to know if I have approached the problem correctly so far, and how to calculate the Jacobian. I have not been able to find any material on the web or printed form that describes the procedure for finding the Jacobian for a problem like this. Any help or pointers will be greatly appreciated.
Open this post in threaded view
|

## Re: [math] How to use LevenbergMarquardtOptimizer for finding the optimal dampening factor for exponential smoothing

 Dear Manish, There are a few issues with your approach: first: as far as I understand, the LevenbergMarquardtOptimizer is   designed to optimize more than one parameter. In the present case   alpha is the sole parameter to be optimized. There is no Jacobian for   the one-dimensional case. In the web-side that you mentioned, they recommend to use a proper   starting value for the smoothed sequence as additional unknown   parameter. In your case it would be something around 9. With a second   parameter to be optimized, you could formally use the   LevenbergMarquardtOptimizer. second: you alpha-value should be restrained to stay in the range of   0: > I am trying to write a Java program for generating a forecast using > exponential smoothing as described here: > https://www.itl.nist.gov/div898/handbook/pmc/section4/pmc431.htm. As > described at the linked document, exponential smoothing uses a dampening > factor "alpha". The document further goes on to say that an optimal value > for "alpha" can be found using the "Marquardt procedure", which I take as > referring to the Levenberg-Marquardt algorithm. From the linked document, > it seems that the problem of finding the optimal "alpha" is treated as a > least-squares problem and fed into the optimizer, with an initial guess for > "alpha". > > After extensive web search I could not find any ready example of using the > Levenberg-Marquardt algorithm to find "alpha" for this kind of a problem, > with any programming language. So, I dug into the Javadocs and test cases > for the class *LevenbergMarquardtOptimizer* to see if I could come up with > a solution of my own. My program is given an array of values, say *[9, 8, > 9, 12, 10, 12, 11, 7, 13, 9, 11, 10]*, and an initial guess for "alpha", > say *0.2*. I have been able to determine that this information needs to be > converted into a *LeastSquaresProblem*, for which I have done the following > so far: > > >    1. Set the input array as the *target*; >    2. Set the starting point *start *as the initial value of alpha (*{ 0.2 >    }*); >    3. Set *weight* to *[1, 1 - alpha, (1 - alpha)^2, ...]*; and >    4. Set the optimization function of the model to return smooth values >    for each of the input values. > > I am now unsure how the Jacobian should be calculated. I would like to know > if I have approached the problem correctly so far, and how to calculate the > Jacobian. I have not been able to find any material on the web or printed > form that describes the procedure for finding the Jacobian for a problem > like this. > > Any help or pointers will be greatly appreciated. > --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]
Open this post in threaded view
|