[math] Multithreaded performances

classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[math] Multithreaded performances

Romain Manni-Bucau
Hi,

ATM sirona (a java monitoring library in incubator) relies a lot on
Summary stats object from [math3] but it needed a lock to ensure
consistency. I know there is a synchronized version but this one
scales less then the locked one.

My question is quite simple then: will [math] add an implementation
with thread safety guarantee and good performances? I think for
instance to the LongAdder of Doug Lea which could be used as a good
base.

Romain Manni-Bucau
Twitter: @rmannibucau
Blog: http://rmannibucau.wordpress.com/
LinkedIn: http://fr.linkedin.com/in/rmannibucau
Github: https://github.com/rmannibucau

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Ted Dunning
My experience is that the only way to get really high performance with
counter-like objects is to have one per thread and combine them on read.




On Mon, Nov 4, 2013 at 8:49 AM, Romain Manni-Bucau <[hidden email]>wrote:

> Hi,
>
> ATM sirona (a java monitoring library in incubator) relies a lot on
> Summary stats object from [math3] but it needed a lock to ensure
> consistency. I know there is a synchronized version but this one
> scales less then the locked one.
>
> My question is quite simple then: will [math] add an implementation
> with thread safety guarantee and good performances? I think for
> instance to the LongAdder of Doug Lea which could be used as a good
> base.
>
> Romain Manni-Bucau
> Twitter: @rmannibucau
> Blog: http://rmannibucau.wordpress.com/
> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> Github: https://github.com/rmannibucau
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Romain Manni-Bucau
Thats more or less what does LongAdder in a more clever way.
Le 4 nov. 2013 18:15, "Ted Dunning" <[hidden email]> a écrit :

> My experience is that the only way to get really high performance with
> counter-like objects is to have one per thread and combine them on read.
>
>
>
>
> On Mon, Nov 4, 2013 at 8:49 AM, Romain Manni-Bucau <[hidden email]
> >wrote:
>
> > Hi,
> >
> > ATM sirona (a java monitoring library in incubator) relies a lot on
> > Summary stats object from [math3] but it needed a lock to ensure
> > consistency. I know there is a synchronized version but this one
> > scales less then the locked one.
> >
> > My question is quite simple then: will [math] add an implementation
> > with thread safety guarantee and good performances? I think for
> > instance to the LongAdder of Doug Lea which could be used as a good
> > base.
> >
> > Romain Manni-Bucau
> > Twitter: @rmannibucau
> > Blog: http://rmannibucau.wordpress.com/
> > LinkedIn: http://fr.linkedin.com/in/rmannibucau
> > Github: https://github.com/rmannibucau
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Phil Steitz
In reply to this post by Romain Manni-Bucau
On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:

> Hi,
>
> ATM sirona (a java monitoring library in incubator) relies a lot on
> Summary stats object from [math3] but it needed a lock to ensure
> consistency. I know there is a synchronized version but this one
> scales less then the locked one.
>
> My question is quite simple then: will [math] add an implementation
> with thread safety guarantee and good performances? I think for
> instance to the LongAdder of Doug Lea which could be used as a good
> base.

The short answer is yes, patches welcome.

Ted makes a good point, though; and there is already some support
for aggregation in the stats classes in [math] (i.e., you can
aggregate the results of per-thread stats by using, e.g.
AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
this to more stats.

Phil

>
> Romain Manni-Bucau
> Twitter: @rmannibucau
> Blog: http://rmannibucau.wordpress.com/
> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> Github: https://github.com/rmannibucau
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Romain Manni-Bucau
But aggregation needs to lock so not a real solution. Lock is fine on real
cases but not in simple/light ones. ThreadLocal leaks...so a trade off
should be found
Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit :

> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
> > Hi,
> >
> > ATM sirona (a java monitoring library in incubator) relies a lot on
> > Summary stats object from [math3] but it needed a lock to ensure
> > consistency. I know there is a synchronized version but this one
> > scales less then the locked one.
> >
> > My question is quite simple then: will [math] add an implementation
> > with thread safety guarantee and good performances? I think for
> > instance to the LongAdder of Doug Lea which could be used as a good
> > base.
>
> The short answer is yes, patches welcome.
>
> Ted makes a good point, though; and there is already some support
> for aggregation in the stats classes in [math] (i.e., you can
> aggregate the results of per-thread stats by using, e.g.
> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
> this to more stats.
>
> Phil
>
> >
> > Romain Manni-Bucau
> > Twitter: @rmannibucau
> > Blog: http://rmannibucau.wordpress.com/
> > LinkedIn: http://fr.linkedin.com/in/rmannibucau
> > Github: https://github.com/rmannibucau
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Phil Steitz
On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
> But aggregation needs to lock so not a real solution. Lock is fine on real
> cases but not in simple/light ones. ThreadLocal leaks...so a trade off
> should be found

Depends on the use case.  If the use case is

0) launch a bunch of threads and let them gather stats individually
1) aggregate results

Then the static aggregate method in AggregateSummaryStatistics that
takes a collection as input will work with no locking required.

Phil

> Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit :
>
>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
>>> Hi,
>>>
>>> ATM sirona (a java monitoring library in incubator) relies a lot on
>>> Summary stats object from [math3] but it needed a lock to ensure
>>> consistency. I know there is a synchronized version but this one
>>> scales less then the locked one.
>>>
>>> My question is quite simple then: will [math] add an implementation
>>> with thread safety guarantee and good performances? I think for
>>> instance to the LongAdder of Doug Lea which could be used as a good
>>> base.
>> The short answer is yes, patches welcome.
>>
>> Ted makes a good point, though; and there is already some support
>> for aggregation in the stats classes in [math] (i.e., you can
>> aggregate the results of per-thread stats by using, e.g.
>> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
>> this to more stats.
>>
>> Phil
>>
>>> Romain Manni-Bucau
>>> Twitter: @rmannibucau
>>> Blog: http://rmannibucau.wordpress.com/
>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>> Github: https://github.com/rmannibucau
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Romain Manni-Bucau
In sirona we collect (aggregate) data each N ms and we can still use stats
during aggregation (worse case surely)
Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :

> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
> > But aggregation needs to lock so not a real solution. Lock is fine on
> real
> > cases but not in simple/light ones. ThreadLocal leaks...so a trade off
> > should be found
>
> Depends on the use case.  If the use case is
>
> 0) launch a bunch of threads and let them gather stats individually
> 1) aggregate results
>
> Then the static aggregate method in AggregateSummaryStatistics that
> takes a collection as input will work with no locking required.
>
> Phil
> > Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit :
> >
> >> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
> >>> Hi,
> >>>
> >>> ATM sirona (a java monitoring library in incubator) relies a lot on
> >>> Summary stats object from [math3] but it needed a lock to ensure
> >>> consistency. I know there is a synchronized version but this one
> >>> scales less then the locked one.
> >>>
> >>> My question is quite simple then: will [math] add an implementation
> >>> with thread safety guarantee and good performances? I think for
> >>> instance to the LongAdder of Doug Lea which could be used as a good
> >>> base.
> >> The short answer is yes, patches welcome.
> >>
> >> Ted makes a good point, though; and there is already some support
> >> for aggregation in the stats classes in [math] (i.e., you can
> >> aggregate the results of per-thread stats by using, e.g.
> >> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
> >> this to more stats.
> >>
> >> Phil
> >>
> >>> Romain Manni-Bucau
> >>> Twitter: @rmannibucau
> >>> Blog: http://rmannibucau.wordpress.com/
> >>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> >>> Github: https://github.com/rmannibucau
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: [hidden email]
> >>> For additional commands, e-mail: [hidden email]
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Phil Steitz
On 11/4/13 12:58 PM, Romain Manni-Bucau wrote:
> In sirona we collect (aggregate) data each N ms and we can still use stats
> during aggregation (worse case surely)

Can you explain more clearly exactly what you are doing?  What needs
to be aggregated when?  How are the threads managed?  

Phil

> Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :
>
>> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
>>> But aggregation needs to lock so not a real solution. Lock is fine on
>> real
>>> cases but not in simple/light ones. ThreadLocal leaks...so a trade off
>>> should be found
>> Depends on the use case.  If the use case is
>>
>> 0) launch a bunch of threads and let them gather stats individually
>> 1) aggregate results
>>
>> Then the static aggregate method in AggregateSummaryStatistics that
>> takes a collection as input will work with no locking required.
>>
>> Phil
>>> Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit :
>>>
>>>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
>>>>> Hi,
>>>>>
>>>>> ATM sirona (a java monitoring library in incubator) relies a lot on
>>>>> Summary stats object from [math3] but it needed a lock to ensure
>>>>> consistency. I know there is a synchronized version but this one
>>>>> scales less then the locked one.
>>>>>
>>>>> My question is quite simple then: will [math] add an implementation
>>>>> with thread safety guarantee and good performances? I think for
>>>>> instance to the LongAdder of Doug Lea which could be used as a good
>>>>> base.
>>>> The short answer is yes, patches welcome.
>>>>
>>>> Ted makes a good point, though; and there is already some support
>>>> for aggregation in the stats classes in [math] (i.e., you can
>>>> aggregate the results of per-thread stats by using, e.g.
>>>> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
>>>> this to more stats.
>>>>
>>>> Phil
>>>>
>>>>> Romain Manni-Bucau
>>>>> Twitter: @rmannibucau
>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>> Github: https://github.com/rmannibucau
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Ted Dunning
In reply to this post by Romain Manni-Bucau
I still think that what you need is a thread-safe copy rather than a
thread-safe mutate.  Even if you force every thread to do the copy, the
aggregation still still wins on complexity/correctness/performance ideas.


On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau
<[hidden email]>wrote:

> In sirona we collect (aggregate) data each N ms and we can still use stats
> during aggregation (worse case surely)
> Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :
>
> > On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
> > > But aggregation needs to lock so not a real solution. Lock is fine on
> > real
> > > cases but not in simple/light ones. ThreadLocal leaks...so a trade off
> > > should be found
> >
> > Depends on the use case.  If the use case is
> >
> > 0) launch a bunch of threads and let them gather stats individually
> > 1) aggregate results
> >
> > Then the static aggregate method in AggregateSummaryStatistics that
> > takes a collection as input will work with no locking required.
> >
> > Phil
> > > Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit :
> > >
> > >> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
> > >>> Hi,
> > >>>
> > >>> ATM sirona (a java monitoring library in incubator) relies a lot on
> > >>> Summary stats object from [math3] but it needed a lock to ensure
> > >>> consistency. I know there is a synchronized version but this one
> > >>> scales less then the locked one.
> > >>>
> > >>> My question is quite simple then: will [math] add an implementation
> > >>> with thread safety guarantee and good performances? I think for
> > >>> instance to the LongAdder of Doug Lea which could be used as a good
> > >>> base.
> > >> The short answer is yes, patches welcome.
> > >>
> > >> Ted makes a good point, though; and there is already some support
> > >> for aggregation in the stats classes in [math] (i.e., you can
> > >> aggregate the results of per-thread stats by using, e.g.
> > >> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
> > >> this to more stats.
> > >>
> > >> Phil
> > >>
> > >>> Romain Manni-Bucau
> > >>> Twitter: @rmannibucau
> > >>> Blog: http://rmannibucau.wordpress.com/
> > >>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> > >>> Github: https://github.com/rmannibucau
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: [hidden email]
> > >>> For additional commands, e-mail: [hidden email]
> > >>>
> > >>>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: [hidden email]
> > >> For additional commands, e-mail: [hidden email]
> > >>
> > >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Romain Manni-Bucau
A counter has a stat object, then methods are intercepted and duration is
added to the stat object (1 by method). Value is copied each 4s.
Le 4 nov. 2013 23:23, "Ted Dunning" <[hidden email]> a écrit :

> I still think that what you need is a thread-safe copy rather than a
> thread-safe mutate.  Even if you force every thread to do the copy, the
> aggregation still still wins on complexity/correctness/performance ideas.
>
>
> On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau
> <[hidden email]>wrote:
>
> > In sirona we collect (aggregate) data each N ms and we can still use
> stats
> > during aggregation (worse case surely)
> > Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :
> >
> > > On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
> > > > But aggregation needs to lock so not a real solution. Lock is fine on
> > > real
> > > > cases but not in simple/light ones. ThreadLocal leaks...so a trade
> off
> > > > should be found
> > >
> > > Depends on the use case.  If the use case is
> > >
> > > 0) launch a bunch of threads and let them gather stats individually
> > > 1) aggregate results
> > >
> > > Then the static aggregate method in AggregateSummaryStatistics that
> > > takes a collection as input will work with no locking required.
> > >
> > > Phil
> > > > Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit
> :
> > > >
> > > >> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
> > > >>> Hi,
> > > >>>
> > > >>> ATM sirona (a java monitoring library in incubator) relies a lot on
> > > >>> Summary stats object from [math3] but it needed a lock to ensure
> > > >>> consistency. I know there is a synchronized version but this one
> > > >>> scales less then the locked one.
> > > >>>
> > > >>> My question is quite simple then: will [math] add an implementation
> > > >>> with thread safety guarantee and good performances? I think for
> > > >>> instance to the LongAdder of Doug Lea which could be used as a good
> > > >>> base.
> > > >> The short answer is yes, patches welcome.
> > > >>
> > > >> Ted makes a good point, though; and there is already some support
> > > >> for aggregation in the stats classes in [math] (i.e., you can
> > > >> aggregate the results of per-thread stats by using, e.g.
> > > >> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
> > > >> this to more stats.
> > > >>
> > > >> Phil
> > > >>
> > > >>> Romain Manni-Bucau
> > > >>> Twitter: @rmannibucau
> > > >>> Blog: http://rmannibucau.wordpress.com/
> > > >>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> > > >>> Github: https://github.com/rmannibucau
> > > >>>
> > > >>>
> ---------------------------------------------------------------------
> > > >>> To unsubscribe, e-mail: [hidden email]
> > > >>> For additional commands, e-mail: [hidden email]
> > > >>>
> > > >>>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: [hidden email]
> > > >> For additional commands, e-mail: [hidden email]
> > > >>
> > > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Phil Steitz
In reply to this post by Ted Dunning
On 11/4/13 2:22 PM, Ted Dunning wrote:
> I still think that what you need is a thread-safe copy rather than a
> thread-safe mutate.

I was just thinking the same thing.  Patches welcome.

Phil

>   Even if you force every thread to do the copy, the
> aggregation still still wins on complexity/correctness/performance ideas.
>
>
> On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau
> <[hidden email]>wrote:
>
>> In sirona we collect (aggregate) data each N ms and we can still use stats
>> during aggregation (worse case surely)
>> Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :
>>
>>> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
>>>> But aggregation needs to lock so not a real solution. Lock is fine on
>>> real
>>>> cases but not in simple/light ones. ThreadLocal leaks...so a trade off
>>>> should be found
>>> Depends on the use case.  If the use case is
>>>
>>> 0) launch a bunch of threads and let them gather stats individually
>>> 1) aggregate results
>>>
>>> Then the static aggregate method in AggregateSummaryStatistics that
>>> takes a collection as input will work with no locking required.
>>>
>>> Phil
>>>> Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit :
>>>>
>>>>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
>>>>>> Hi,
>>>>>>
>>>>>> ATM sirona (a java monitoring library in incubator) relies a lot on
>>>>>> Summary stats object from [math3] but it needed a lock to ensure
>>>>>> consistency. I know there is a synchronized version but this one
>>>>>> scales less then the locked one.
>>>>>>
>>>>>> My question is quite simple then: will [math] add an implementation
>>>>>> with thread safety guarantee and good performances? I think for
>>>>>> instance to the LongAdder of Doug Lea which could be used as a good
>>>>>> base.
>>>>> The short answer is yes, patches welcome.
>>>>>
>>>>> Ted makes a good point, though; and there is already some support
>>>>> for aggregation in the stats classes in [math] (i.e., you can
>>>>> aggregate the results of per-thread stats by using, e.g.
>>>>> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
>>>>> this to more stats.
>>>>>
>>>>> Phil
>>>>>
>>>>>> Romain Manni-Bucau
>>>>>> Twitter: @rmannibucau
>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>> Github: https://github.com/rmannibucau
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Romain Manni-Bucau
The copy will lock too. And it doesnt solve leak issue of the one instance
by thread solution, no?
Le 4 nov. 2013 23:27, "Phil Steitz" <[hidden email]> a écrit :

> On 11/4/13 2:22 PM, Ted Dunning wrote:
> > I still think that what you need is a thread-safe copy rather than a
> > thread-safe mutate.
>
> I was just thinking the same thing.  Patches welcome.
>
> Phil
> >   Even if you force every thread to do the copy, the
> > aggregation still still wins on complexity/correctness/performance ideas.
> >
> >
> > On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau
> > <[hidden email]>wrote:
> >
> >> In sirona we collect (aggregate) data each N ms and we can still use
> stats
> >> during aggregation (worse case surely)
> >> Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :
> >>
> >>> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
> >>>> But aggregation needs to lock so not a real solution. Lock is fine on
> >>> real
> >>>> cases but not in simple/light ones. ThreadLocal leaks...so a trade off
> >>>> should be found
> >>> Depends on the use case.  If the use case is
> >>>
> >>> 0) launch a bunch of threads and let them gather stats individually
> >>> 1) aggregate results
> >>>
> >>> Then the static aggregate method in AggregateSummaryStatistics that
> >>> takes a collection as input will work with no locking required.
> >>>
> >>> Phil
> >>>> Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit :
> >>>>
> >>>>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> ATM sirona (a java monitoring library in incubator) relies a lot on
> >>>>>> Summary stats object from [math3] but it needed a lock to ensure
> >>>>>> consistency. I know there is a synchronized version but this one
> >>>>>> scales less then the locked one.
> >>>>>>
> >>>>>> My question is quite simple then: will [math] add an implementation
> >>>>>> with thread safety guarantee and good performances? I think for
> >>>>>> instance to the LongAdder of Doug Lea which could be used as a good
> >>>>>> base.
> >>>>> The short answer is yes, patches welcome.
> >>>>>
> >>>>> Ted makes a good point, though; and there is already some support
> >>>>> for aggregation in the stats classes in [math] (i.e., you can
> >>>>> aggregate the results of per-thread stats by using, e.g.
> >>>>> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
> >>>>> this to more stats.
> >>>>>
> >>>>> Phil
> >>>>>
> >>>>>> Romain Manni-Bucau
> >>>>>> Twitter: @rmannibucau
> >>>>>> Blog: http://rmannibucau.wordpress.com/
> >>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> >>>>>> Github: https://github.com/rmannibucau
> >>>>>>
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: [hidden email]
> >>>>>> For additional commands, e-mail: [hidden email]
> >>>>>>
> >>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: [hidden email]
> >>>>> For additional commands, e-mail: [hidden email]
> >>>>>
> >>>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: [hidden email]
> >>> For additional commands, e-mail: [hidden email]
> >>>
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Phil Steitz
On 11/4/13 2:31 PM, Romain Manni-Bucau wrote:
> The copy will lock too.

Right.  That is why I asked exactly how things work.  If you can't
lock during aggregation, we need something different.

> And it doesnt solve leak issue of the one instance
> by thread solution, no?

Correct, again depends on the setup how big a problem that is / what
can be done to manage it.

Phil

> Le 4 nov. 2013 23:27, "Phil Steitz" <[hidden email]> a écrit :
>
>> On 11/4/13 2:22 PM, Ted Dunning wrote:
>>> I still think that what you need is a thread-safe copy rather than a
>>> thread-safe mutate.
>> I was just thinking the same thing.  Patches welcome.
>>
>> Phil
>>>   Even if you force every thread to do the copy, the
>>> aggregation still still wins on complexity/correctness/performance ideas.
>>>
>>>
>>> On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau
>>> <[hidden email]>wrote:
>>>
>>>> In sirona we collect (aggregate) data each N ms and we can still use
>> stats
>>>> during aggregation (worse case surely)
>>>> Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :
>>>>
>>>>> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
>>>>>> But aggregation needs to lock so not a real solution. Lock is fine on
>>>>> real
>>>>>> cases but not in simple/light ones. ThreadLocal leaks...so a trade off
>>>>>> should be found
>>>>> Depends on the use case.  If the use case is
>>>>>
>>>>> 0) launch a bunch of threads and let them gather stats individually
>>>>> 1) aggregate results
>>>>>
>>>>> Then the static aggregate method in AggregateSummaryStatistics that
>>>>> takes a collection as input will work with no locking required.
>>>>>
>>>>> Phil
>>>>>> Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a écrit :
>>>>>>
>>>>>>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> ATM sirona (a java monitoring library in incubator) relies a lot on
>>>>>>>> Summary stats object from [math3] but it needed a lock to ensure
>>>>>>>> consistency. I know there is a synchronized version but this one
>>>>>>>> scales less then the locked one.
>>>>>>>>
>>>>>>>> My question is quite simple then: will [math] add an implementation
>>>>>>>> with thread safety guarantee and good performances? I think for
>>>>>>>> instance to the LongAdder of Doug Lea which could be used as a good
>>>>>>>> base.
>>>>>>> The short answer is yes, patches welcome.
>>>>>>>
>>>>>>> Ted makes a good point, though; and there is already some support
>>>>>>> for aggregation in the stats classes in [math] (i.e., you can
>>>>>>> aggregate the results of per-thread stats by using, e.g.
>>>>>>> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
>>>>>>> this to more stats.
>>>>>>>
>>>>>>> Phil
>>>>>>>
>>>>>>>> Romain Manni-Bucau
>>>>>>>> Twitter: @rmannibucau
>>>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>>>> Github: https://github.com/rmannibucau
>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>
>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Ted Dunning
The copy doesn't have to lock if you build the right data structure.

The thread leak problem can be more serious.




On Mon, Nov 4, 2013 at 2:47 PM, Phil Steitz <[hidden email]> wrote:

> On 11/4/13 2:31 PM, Romain Manni-Bucau wrote:
> > The copy will lock too.
>
> Right.  That is why I asked exactly how things work.  If you can't
> lock during aggregation, we need something different.
>
> > And it doesnt solve leak issue of the one instance
> > by thread solution, no?
>
> Correct, again depends on the setup how big a problem that is / what
> can be done to manage it.
>
> Phil
> > Le 4 nov. 2013 23:27, "Phil Steitz" <[hidden email]> a écrit :
> >
> >> On 11/4/13 2:22 PM, Ted Dunning wrote:
> >>> I still think that what you need is a thread-safe copy rather than a
> >>> thread-safe mutate.
> >> I was just thinking the same thing.  Patches welcome.
> >>
> >> Phil
> >>>   Even if you force every thread to do the copy, the
> >>> aggregation still still wins on complexity/correctness/performance
> ideas.
> >>>
> >>>
> >>> On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau
> >>> <[hidden email]>wrote:
> >>>
> >>>> In sirona we collect (aggregate) data each N ms and we can still use
> >> stats
> >>>> during aggregation (worse case surely)
> >>>> Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :
> >>>>
> >>>>> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
> >>>>>> But aggregation needs to lock so not a real solution. Lock is fine
> on
> >>>>> real
> >>>>>> cases but not in simple/light ones. ThreadLocal leaks...so a trade
> off
> >>>>>> should be found
> >>>>> Depends on the use case.  If the use case is
> >>>>>
> >>>>> 0) launch a bunch of threads and let them gather stats individually
> >>>>> 1) aggregate results
> >>>>>
> >>>>> Then the static aggregate method in AggregateSummaryStatistics that
> >>>>> takes a collection as input will work with no locking required.
> >>>>>
> >>>>> Phil
> >>>>>> Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a
> écrit :
> >>>>>>
> >>>>>>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> ATM sirona (a java monitoring library in incubator) relies a lot
> on
> >>>>>>>> Summary stats object from [math3] but it needed a lock to ensure
> >>>>>>>> consistency. I know there is a synchronized version but this one
> >>>>>>>> scales less then the locked one.
> >>>>>>>>
> >>>>>>>> My question is quite simple then: will [math] add an
> implementation
> >>>>>>>> with thread safety guarantee and good performances? I think for
> >>>>>>>> instance to the LongAdder of Doug Lea which could be used as a
> good
> >>>>>>>> base.
> >>>>>>> The short answer is yes, patches welcome.
> >>>>>>>
> >>>>>>> Ted makes a good point, though; and there is already some support
> >>>>>>> for aggregation in the stats classes in [math] (i.e., you can
> >>>>>>> aggregate the results of per-thread stats by using, e.g.
> >>>>>>> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
> >>>>>>> this to more stats.
> >>>>>>>
> >>>>>>> Phil
> >>>>>>>
> >>>>>>>> Romain Manni-Bucau
> >>>>>>>> Twitter: @rmannibucau
> >>>>>>>> Blog: http://rmannibucau.wordpress.com/
> >>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> >>>>>>>> Github: https://github.com/rmannibucau
> >>>>>>>>
> >>>>>>>>
> >> ---------------------------------------------------------------------
> >>>>>>>> To unsubscribe, e-mail: [hidden email]
> >>>>>>>> For additional commands, e-mail: [hidden email]
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: [hidden email]
> >>>>>>> For additional commands, e-mail: [hidden email]
> >>>>>>>
> >>>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: [hidden email]
> >>>>> For additional commands, e-mail: [hidden email]
> >>>>>
> >>>>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Bernd Eckenfels
Am 05.11.2013, 00:44 Uhr, schrieb Ted Dunning <[hidden email]>:
> The thread leak problem can be more serious.

What Thread Leak problem? TLS is cited to have that leak since ages, I am  
not sure I ever have seen one triggered. So, who has details on that? Even  
the JCL uses (more) thread local objects. For example in JDK8 the  
BigInteger has a thread local SecureRandom instance.

One has to be carefull in regards to keeping classloaders alive, but that  
can be worked around with an expire, weak references or simply by only  
storing system classes (for example long[]!)

Greetings
Bernd

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Phil Steitz
In reply to this post by Ted Dunning
On 11/4/13 3:44 PM, Ted Dunning wrote:
> The copy doesn't have to lock if you build the right data structure.

The individual stats objects need to update multiple quantities
atomically when new values come in.  Consistency in the copy
requires that you suppress updates while the copy is in progress
unless you implement some kind of update queue internally.   What
exactly do you mean by "the right data structure?"

This discussion is bringing me back to the view that we had before
we did anything to support synchronization - clients are better
situated to manage the supporting infrastructure.  As long as we
support aggregation, clients should be able to do what they need to do.

If the use case here, for example, is that lots of client threads
need to contribute to aggregate stats and you can't pause any of
them ever to aggregate, there is no free lunch, you are going to
have to queue somewhere.  The primitive solution of queuing on the
sync lock for a single stats object is heavy and bad.  Trying to
build more lightweight queueing into the stat object itself is
interesting, but I wonder if there is other stuff from these threads
that should also be queued, or in general there is always going to
be stuff about the application's execution environment unknown to us
that make whatever we do less than optimal.

All that said, patches welcome for lighter weight locking or
explicit update queue management inside the stats objects.

Phil

>
> The thread leak problem can be more serious.
>
>
>
>
> On Mon, Nov 4, 2013 at 2:47 PM, Phil Steitz <[hidden email]> wrote:
>
>> On 11/4/13 2:31 PM, Romain Manni-Bucau wrote:
>>> The copy will lock too.
>> Right.  That is why I asked exactly how things work.  If you can't
>> lock during aggregation, we need something different.
>>
>>> And it doesnt solve leak issue of the one instance
>>> by thread solution, no?
>> Correct, again depends on the setup how big a problem that is / what
>> can be done to manage it.
>>
>> Phil
>>> Le 4 nov. 2013 23:27, "Phil Steitz" <[hidden email]> a écrit :
>>>
>>>> On 11/4/13 2:22 PM, Ted Dunning wrote:
>>>>> I still think that what you need is a thread-safe copy rather than a
>>>>> thread-safe mutate.
>>>> I was just thinking the same thing.  Patches welcome.
>>>>
>>>> Phil
>>>>>   Even if you force every thread to do the copy, the
>>>>> aggregation still still wins on complexity/correctness/performance
>> ideas.
>>>>>
>>>>> On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau
>>>>> <[hidden email]>wrote:
>>>>>
>>>>>> In sirona we collect (aggregate) data each N ms and we can still use
>>>> stats
>>>>>> during aggregation (worse case surely)
>>>>>> Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a écrit :
>>>>>>
>>>>>>> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
>>>>>>>> But aggregation needs to lock so not a real solution. Lock is fine
>> on
>>>>>>> real
>>>>>>>> cases but not in simple/light ones. ThreadLocal leaks...so a trade
>> off
>>>>>>>> should be found
>>>>>>> Depends on the use case.  If the use case is
>>>>>>>
>>>>>>> 0) launch a bunch of threads and let them gather stats individually
>>>>>>> 1) aggregate results
>>>>>>>
>>>>>>> Then the static aggregate method in AggregateSummaryStatistics that
>>>>>>> takes a collection as input will work with no locking required.
>>>>>>>
>>>>>>> Phil
>>>>>>>> Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a
>> écrit :
>>>>>>>>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> ATM sirona (a java monitoring library in incubator) relies a lot
>> on
>>>>>>>>>> Summary stats object from [math3] but it needed a lock to ensure
>>>>>>>>>> consistency. I know there is a synchronized version but this one
>>>>>>>>>> scales less then the locked one.
>>>>>>>>>>
>>>>>>>>>> My question is quite simple then: will [math] add an
>> implementation
>>>>>>>>>> with thread safety guarantee and good performances? I think for
>>>>>>>>>> instance to the LongAdder of Doug Lea which could be used as a
>> good
>>>>>>>>>> base.
>>>>>>>>> The short answer is yes, patches welcome.
>>>>>>>>>
>>>>>>>>> Ted makes a good point, though; and there is already some support
>>>>>>>>> for aggregation in the stats classes in [math] (i.e., you can
>>>>>>>>> aggregate the results of per-thread stats by using, e.g.
>>>>>>>>> AggregateSummaryStatistics#aggregate).  See MATH-1016 re extending
>>>>>>>>> this to more stats.
>>>>>>>>>
>>>>>>>>> Phil
>>>>>>>>>
>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>> Twitter: @rmannibucau
>>>>>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>>>>>> Github: https://github.com/rmannibucau
>>>>>>>>>>
>>>>>>>>>>
>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>>>
>>>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>>
>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>
>>>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Romain Manni-Bucau
Well before patching a consistent solution needs to be found.

Sorry to go back to LongAdder but the idea is to add instances but not as
much as thread. Just what is needed. It means you decrease the number of
instance to aggregate so the lock time.

The queue solution sounds worse since a queue is either empty or full ;)
Le 5 nov. 2013 05:23, "Phil Steitz" <[hidden email]> a écrit :

> On 11/4/13 3:44 PM, Ted Dunning wrote:
> > The copy doesn't have to lock if you build the right data structure.
>
> The individual stats objects need to update multiple quantities
> atomically when new values come in.  Consistency in the copy
> requires that you suppress updates while the copy is in progress
> unless you implement some kind of update queue internally.   What
> exactly do you mean by "the right data structure?"
>
> This discussion is bringing me back to the view that we had before
> we did anything to support synchronization - clients are better
> situated to manage the supporting infrastructure.  As long as we
> support aggregation, clients should be able to do what they need to do.
>
> If the use case here, for example, is that lots of client threads
> need to contribute to aggregate stats and you can't pause any of
> them ever to aggregate, there is no free lunch, you are going to
> have to queue somewhere.  The primitive solution of queuing on the
> sync lock for a single stats object is heavy and bad.  Trying to
> build more lightweight queueing into the stat object itself is
> interesting, but I wonder if there is other stuff from these threads
> that should also be queued, or in general there is always going to
> be stuff about the application's execution environment unknown to us
> that make whatever we do less than optimal.
>
> All that said, patches welcome for lighter weight locking or
> explicit update queue management inside the stats objects.
>
> Phil
> >
> > The thread leak problem can be more serious.
> >
> >
> >
> >
> > On Mon, Nov 4, 2013 at 2:47 PM, Phil Steitz <[hidden email]>
> wrote:
> >
> >> On 11/4/13 2:31 PM, Romain Manni-Bucau wrote:
> >>> The copy will lock too.
> >> Right.  That is why I asked exactly how things work.  If you can't
> >> lock during aggregation, we need something different.
> >>
> >>> And it doesnt solve leak issue of the one instance
> >>> by thread solution, no?
> >> Correct, again depends on the setup how big a problem that is / what
> >> can be done to manage it.
> >>
> >> Phil
> >>> Le 4 nov. 2013 23:27, "Phil Steitz" <[hidden email]> a écrit :
> >>>
> >>>> On 11/4/13 2:22 PM, Ted Dunning wrote:
> >>>>> I still think that what you need is a thread-safe copy rather than a
> >>>>> thread-safe mutate.
> >>>> I was just thinking the same thing.  Patches welcome.
> >>>>
> >>>> Phil
> >>>>>   Even if you force every thread to do the copy, the
> >>>>> aggregation still still wins on complexity/correctness/performance
> >> ideas.
> >>>>>
> >>>>> On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau
> >>>>> <[hidden email]>wrote:
> >>>>>
> >>>>>> In sirona we collect (aggregate) data each N ms and we can still use
> >>>> stats
> >>>>>> during aggregation (worse case surely)
> >>>>>> Le 4 nov. 2013 21:48, "Phil Steitz" <[hidden email]> a
> écrit :
> >>>>>>
> >>>>>>> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote:
> >>>>>>>> But aggregation needs to lock so not a real solution. Lock is fine
> >> on
> >>>>>>> real
> >>>>>>>> cases but not in simple/light ones. ThreadLocal leaks...so a trade
> >> off
> >>>>>>>> should be found
> >>>>>>> Depends on the use case.  If the use case is
> >>>>>>>
> >>>>>>> 0) launch a bunch of threads and let them gather stats individually
> >>>>>>> 1) aggregate results
> >>>>>>>
> >>>>>>> Then the static aggregate method in AggregateSummaryStatistics that
> >>>>>>> takes a collection as input will work with no locking required.
> >>>>>>>
> >>>>>>> Phil
> >>>>>>>> Le 4 nov. 2013 18:42, "Phil Steitz" <[hidden email]> a
> >> écrit :
> >>>>>>>>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote:
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> ATM sirona (a java monitoring library in incubator) relies a lot
> >> on
> >>>>>>>>>> Summary stats object from [math3] but it needed a lock to ensure
> >>>>>>>>>> consistency. I know there is a synchronized version but this one
> >>>>>>>>>> scales less then the locked one.
> >>>>>>>>>>
> >>>>>>>>>> My question is quite simple then: will [math] add an
> >> implementation
> >>>>>>>>>> with thread safety guarantee and good performances? I think for
> >>>>>>>>>> instance to the LongAdder of Doug Lea which could be used as a
> >> good
> >>>>>>>>>> base.
> >>>>>>>>> The short answer is yes, patches welcome.
> >>>>>>>>>
> >>>>>>>>> Ted makes a good point, though; and there is already some support
> >>>>>>>>> for aggregation in the stats classes in [math] (i.e., you can
> >>>>>>>>> aggregate the results of per-thread stats by using, e.g.
> >>>>>>>>> AggregateSummaryStatistics#aggregate).  See MATH-1016 re
> extending
> >>>>>>>>> this to more stats.
> >>>>>>>>>
> >>>>>>>>> Phil
> >>>>>>>>>
> >>>>>>>>>> Romain Manni-Bucau
> >>>>>>>>>> Twitter: @rmannibucau
> >>>>>>>>>> Blog: http://rmannibucau.wordpress.com/
> >>>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> >>>>>>>>>> Github: https://github.com/rmannibucau
> >>>>>>>>>>
> >>>>>>>>>>
> >>>> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail: [hidden email]
> >>>>>>>>>> For additional commands, e-mail: [hidden email]
> >>>>>>>>>>
> >>>>>>>>>>
> >> ---------------------------------------------------------------------
> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
> >>>>>>>>> For additional commands, e-mail: [hidden email]
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: [hidden email]
> >>>>>>> For additional commands, e-mail: [hidden email]
> >>>>>>>
> >>>>>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: [hidden email]
> >>>> For additional commands, e-mail: [hidden email]
> >>>>
> >>>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Ted Dunning
In reply to this post by Phil Steitz
On Mon, Nov 4, 2013 at 8:23 PM, Phil Steitz <[hidden email]> wrote:

> On 11/4/13 3:44 PM, Ted Dunning wrote:
> > The copy doesn't have to lock if you build the right data structure.
>
> The individual stats objects need to update multiple quantities
> atomically when new values come in.  Consistency in the copy
> requires that you suppress updates while the copy is in progress
> unless you implement some kind of update queue internally.   What
> exactly do you mean by "the right data structure?"
>

I was talking about lockless data structures in general.

Are you sure that real transactions are a requirement here?
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Romain Manni-Bucau
You cant stop the app cause you take a snapshot of the monitoring metrics
so yes
Le 5 nov. 2013 06:46, "Ted Dunning" <[hidden email]> a écrit :

> On Mon, Nov 4, 2013 at 8:23 PM, Phil Steitz <[hidden email]> wrote:
>
> > On 11/4/13 3:44 PM, Ted Dunning wrote:
> > > The copy doesn't have to lock if you build the right data structure.
> >
> > The individual stats objects need to update multiple quantities
> > atomically when new values come in.  Consistency in the copy
> > requires that you suppress updates while the copy is in progress
> > unless you implement some kind of update queue internally.   What
> > exactly do you mean by "the right data structure?"
> >
>
> I was talking about lockless data structures in general.
>
> Are you sure that real transactions are a requirement here?
>
Reply | Threaded
Open this post in threaded view
|

Re: [math] Multithreaded performances

Ted Dunning
That isn't what I meant.

Do you really think that more than one metric has to update (increment,
say) at precisely the same time?


On Mon, Nov 4, 2013 at 9:49 PM, Romain Manni-Bucau <[hidden email]>wrote:

> You cant stop the app cause you take a snapshot of the monitoring metrics
> so yes
> Le 5 nov. 2013 06:46, "Ted Dunning" <[hidden email]> a écrit :
>
> > On Mon, Nov 4, 2013 at 8:23 PM, Phil Steitz <[hidden email]>
> wrote:
> >
> > > On 11/4/13 3:44 PM, Ted Dunning wrote:
> > > > The copy doesn't have to lock if you build the right data structure.
> > >
> > > The individual stats objects need to update multiple quantities
> > > atomically when new values come in.  Consistency in the copy
> > > requires that you suppress updates while the copy is in progress
> > > unless you implement some kind of update queue internally.   What
> > > exactly do you mean by "the right data structure?"
> > >
> >
> > I was talking about lockless data structures in general.
> >
> > Are you sure that real transactions are a requirement here?
> >
>
12