[logging] lon4j only implementation to solve out of memory errors

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[logging] lon4j only implementation to solve out of memory errors

Paul Austin-3
I am using several of the other commons components and hibernate that
use the commons-logging API to abstract from the underlying log4j or
Java Logging APIs.

Unfortunately due to a bug in JDK 1.4 which does not release references
to class definitions correctly when commons-logging is containing in a
J2EE web application WAR file and the application  is redeployed. The
only reliable solution is to deploy log4j and the commons-logging and
commons-logging-optional (1.0.5-alpha) jar files as shared libraries for
the entire J2EE server. That approach has worked well in my development
environment.

The application is however going to be deployed in a shared server that
I have little control over and it is unlikely that I will be able to
deploy these libraries as shared components as all the other
applications have local log4j config files so it would require a lot of
testing and tweaking of other applications to get everything to work
correctly.

One potential solution I can think for this is to create a clone of
commons-logging that just provides the same interfaces but is statically
bound to log4j so that it would not have to do all the lookups and
caching of class references that goes on right now. This would be a
similar thing to slf4j.org (UGLI) but with the existing commons logging
interface so it can be dropped in as a replacement for commons-logging
for other API's that already use it.

Has anyone looked at this or know of any other solutions to the out of
memory issues.

Cheers,
Paul
Reply | Threaded
Open this post in threaded view
|

Re: [logging] lon4j only implementation to solve out of memory errors

Simon Kitching
Hi Paul,

On Mon, 2005-07-25 at 12:25 -0700, Paul Austin wrote:

> I am using several of the other commons components and hibernate that
> use the commons-logging API to abstract from the underlying log4j or
> Java Logging APIs.
>
> Unfortunately due to a bug in JDK 1.4 which does not release references
> to class definitions correctly when commons-logging is containing in a
> J2EE web application WAR file and the application  is redeployed. The
> only reliable solution is to deploy log4j and the commons-logging and
> commons-logging-optional (1.0.5-alpha) jar files as shared libraries for
> the entire J2EE server. That approach has worked well in my development
> environment.

I'm not sure what "bug in JDK 1.4" you're referring to. Do you have any
further info on this?

If commons-logging is deployed via a shared classloader and the servlet
engine is not commons-logging-aware then components such as webapps need
to ensure that LogFactory.release is called when they are undeployed in
order to avoid exactly the situation you describe. Note that Tomcat is
"commons-logging-aware" and does this automatically. This isn't a "bug",
it's a known cleanup step. The commons-logging-optional.jar does
*sometimes* remove the need for the call to LogFactory.release but not
always which is why I prefer to use LogFactory.release instead of or as
well as the optional jar.

I don't know of any situations where such a memory leak is *caused* by
having commons-logging in the component's WEB-INF/lib. More information
would be welcome.

What container (eg tomcat, jboss) are you using?

>
> The application is however going to be deployed in a shared server that
> I have little control over and it is unlikely that I will be able to
> deploy these libraries as shared components as all the other
> applications have local log4j config files so it would require a lot of
> testing and tweaking of other applications to get everything to work
> correctly.

Yep, that's the problem of putting libs in a "shared" classloader. It's
an approach I don't like at all for exactly that reason.

>
> One potential solution I can think for this is to create a clone of
> commons-logging that just provides the same interfaces but is statically
> bound to log4j so that it would not have to do all the lookups and
> caching of class references that goes on right now. This would be a
> similar thing to slf4j.org (UGLI) but with the existing commons logging
> interface so it can be dropped in as a replacement for commons-logging
> for other API's that already use it.

Yep, adopting some of the ideas from UGLI is something that has been
looked at. There are some definite benefits, but also some disadvantages
to this approach. Some prototyping of this has been done but nothing
that is likely to be released this year.


> Has anyone looked at this or know of any other solutions to the out of
> memory issues.

If you can provide more information I would be willing to help you look
into this. I don't see any reason why the current commons-logging
wouldn't work for you provided things are set up correctly.

Regards,

Simon


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [logging] lon4j only implementation to solve out of memory errors

Paul Austin-3
Hi Simon,

I am running my application on Tomcat 5.5 under Sun JDK 1.4.2_07-b05 on
Windows XP. For the production environment it will be Oracle Application
Server 9.0.4.1 under JDK 1.4.2 on Solaris 2.8 (I haven't had access to
test on that server yet).

The original discussion where I started to find out about the problem
was on the hibernate forums.

http://forum.hibernate.org/viewtopic.php?t=935948&highlight=permgen

One of the things I tried was adding a context listener which had the
LogFactory.release method called. There are also 3 filters which use
springframework/hibernate which use libraries that may use
commons-logging. I'm guessing that perhaps the destroy method on the
filters is called after I call LogFactory.release thus some new log
instances are created thus undoing the work in the release method.

I'll try to get a stripped down copy of the application that still has
the issue put together to help solve this problem. Won't be this week as
I have to deliver the application.

Cheers,
Paul

On Tue, 2005-07-26 at 16:58 +1200, Simon Kitching wrote:

> Hi Paul,
>
> On Mon, 2005-07-25 at 12:25 -0700, Paul Austin wrote:
> > I am using several of the other commons components and hibernate that
> > use the commons-logging API to abstract from the underlying log4j or
> > Java Logging APIs.
> >
> > Unfortunately due to a bug in JDK 1.4 which does not release references
> > to class definitions correctly when commons-logging is containing in a
> > J2EE web application WAR file and the application  is redeployed. The
> > only reliable solution is to deploy log4j and the commons-logging and
> > commons-logging-optional (1.0.5-alpha) jar files as shared libraries for
> > the entire J2EE server. That approach has worked well in my development
> > environment.
>
> I'm not sure what "bug in JDK 1.4" you're referring to. Do you have any
> further info on this?
>
> If commons-logging is deployed via a shared classloader and the servlet
> engine is not commons-logging-aware then components such as webapps need
> to ensure that LogFactory.release is called when they are undeployed in
> order to avoid exactly the situation you describe. Note that Tomcat is
> "commons-logging-aware" and does this automatically. This isn't a "bug",
> it's a known cleanup step. The commons-logging-optional.jar does
> *sometimes* remove the need for the call to LogFactory.release but not
> always which is why I prefer to use LogFactory.release instead of or as
> well as the optional jar.
>
> I don't know of any situations where such a memory leak is *caused* by
> having commons-logging in the component's WEB-INF/lib. More information
> would be welcome.
>
> What container (eg tomcat, jboss) are you using?
>
> >
> > The application is however going to be deployed in a shared server that
> > I have little control over and it is unlikely that I will be able to
> > deploy these libraries as shared components as all the other
> > applications have local log4j config files so it would require a lot of
> > testing and tweaking of other applications to get everything to work
> > correctly.
>
> Yep, that's the problem of putting libs in a "shared" classloader. It's
> an approach I don't like at all for exactly that reason.
>
> >
> > One potential solution I can think for this is to create a clone of
> > commons-logging that just provides the same interfaces but is statically
> > bound to log4j so that it would not have to do all the lookups and
> > caching of class references that goes on right now. This would be a
> > similar thing to slf4j.org (UGLI) but with the existing commons logging
> > interface so it can be dropped in as a replacement for commons-logging
> > for other API's that already use it.
>
> Yep, adopting some of the ideas from UGLI is something that has been
> looked at. There are some definite benefits, but also some disadvantages
> to this approach. Some prototyping of this has been done but nothing
> that is likely to be released this year.
>
>
> > Has anyone looked at this or know of any other solutions to the out of
> > memory issues.
>
> If you can provide more information I would be willing to help you look
> into this. I don't see any reason why the current commons-logging
> wouldn't work for you provided things are set up correctly.
>
> Regards,
>
> Simon
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
Reply | Threaded
Open this post in threaded view
|

Re: [logging] lon4j only implementation to solve out of memory errors

Simon Kitching
On Tue, 2005-07-26 at 08:09 -0700, Paul Austin wrote:

> Hi Simon,
>
> I am running my application on Tomcat 5.5 under Sun JDK 1.4.2_07-b05
> on Windows XP. For the production environment it will be Oracle
> Application Server 9.0.4.1 under JDK 1.4.2 on Solaris 2.8 (I haven't
> had access to test on that server yet).
>
> The original discussion where I started to find out about the problem
> was on the hibernate forums.
>
> http://forum.hibernate.org/viewtopic.php?t=935948&highlight=permgen

Hmm. I wish people would stop trying to push libraries that are
component-specific up into the container's lib directories. Component
libs belong in the component. SLAP. Component libs belong in the
component SLAP. Repeat as necessary. Note that this doesn't apply to
Tomcat's inclusion of JCL in the container - JCL is *part* of tomcat,
and JCL explicitly calls LogFactory.release when unloading webapps so
the problem doesn't occur there.

Unfortunately Sun's standard libs provide no way to implement a clean
workaround when people insist on deploying JCL in the container. The
components in a container are meant to be "scoped" by their context
classloader - but there is *absolutely no way* to attach arbitrary data
to the classloader so that when the classloader is dropped that data is
also recycled. Well, except by ensuring the data is stored as static
fields on a class loaded by that context classloader. But people insist
on not doing that and pushing the JCL libs up into the container's lib
directories. SLAP.

The closest approximation to a solution is to have a map keyed by
context classloader but that requires people to then explicitly release
the map entry when the webapp is unloaded.


> One of the things I tried was adding a context listener which had the
> LogFactory.release method called. There are also 3 filters which use
> springframework/hibernate which use libraries that may use
> commons-logging. I'm guessing that perhaps the destroy method on the
> filters is called after I call LogFactory.release thus some new log
> instances are created thus undoing the work in the release method.

Hmm.. that might be possible.

>
> I'll try to get a stripped down copy of the application that still has
> the issue put together to help solve this problem. Won't be this week
> as I have to deliver the application.

There is one other thing worth trying. It has occurred to me that if
there are multiple copies of LogFactory lying around then maybe in some
cases release needs to be called on each copy not just one.

You could try putting this in the ServletContextListener's destroy
method:
    Class[] releaseSig = new Class[] {ClassLoader.class};
    ClassLoader c = Thread.currentThread().getContextClassLoader();
    Object[] releaseParams = new Object[1];
    releaseParams[0] = c;
    while (c != null) {
        Class lf = c.loadClass(LogFactory.class.getName());
        if (lf != null) {
            try {
                Method m = lf.getMethod("release", releaseSig);
                m.invoke(releaseParams);
            } catch(Exception e) {
                // ignore
            }
        }
        c = c.getParent();
    }

Note that the above code hasn't been tested (it might not even compile)

Regards,

Simon


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [logging] lon4j only implementation to solve out of memory errors

Simon Kitching
On Wed, 2005-07-27 at 20:19 +1200, Simon Kitching wrote:

> On Tue, 2005-07-26 at 08:09 -0700, Paul Austin wrote:
> > Hi Simon,
> >
> > I am running my application on Tomcat 5.5 under Sun JDK 1.4.2_07-b05
> > on Windows XP. For the production environment it will be Oracle
> > Application Server 9.0.4.1 under JDK 1.4.2 on Solaris 2.8 (I haven't
> > had access to test on that server yet).
> >
> > The original discussion where I started to find out about the problem
> > was on the hibernate forums.
> >
> > http://forum.hibernate.org/viewtopic.php?t=935948&highlight=permgen
>
> Hmm. I wish people would stop trying to push libraries that are
> component-specific up into the container's lib directories. Component
> libs belong in the component. SLAP. Component libs belong in the
> component SLAP. Repeat as necessary. Note that this doesn't apply to
> Tomcat's inclusion of JCL in the container - JCL is *part* of tomcat,
> and JCL explicitly calls LogFactory.release when unloading webapps so
> the problem doesn't occur there.

I guess Hibernate is a little bit of a special case here: it could be
regarded as part of the container, or an extension to it. But in that
case, if it introduces a dependency on commons-logging then it should be
responsible for ensuring commons-logging is managed correctly - and that
means ensuring LogFactory.release is called when a webapp is unloaded
like tomcat does.

I'll follow up with the Hibernate folks on this.

Regards,

Simon



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [logging] lon4j only implementation to solve out of memory errors

Paul Austin-3
In reply to this post by Paul Austin-3
Hi Simon,

I think adding the following ContextListener solved the problem.

http://static.springframework.org/spring/docs/1.1.5/api/org/springframework/web/util/IntrospectorCleanupListener.html

Paul

On Tue, 2005-07-26 at 08:09 -0700, Paul Austin wrote:

> Hi Simon,
>
> I am running my application on Tomcat 5.5 under Sun JDK 1.4.2_07-b05 on
> Windows XP. For the production environment it will be Oracle Application
> Server 9.0.4.1 under JDK 1.4.2 on Solaris 2.8 (I haven't had access to
> test on that server yet).
>
> The original discussion where I started to find out about the problem
> was on the hibernate forums.
>
> http://forum.hibernate.org/viewtopic.php?t=935948&highlight=permgen
>
> One of the things I tried was adding a context listener which had the
> LogFactory.release method called. There are also 3 filters which use
> springframework/hibernate which use libraries that may use
> commons-logging. I'm guessing that perhaps the destroy method on the
> filters is called after I call LogFactory.release thus some new log
> instances are created thus undoing the work in the release method.
>
> I'll try to get a stripped down copy of the application that still has
> the issue put together to help solve this problem. Won't be this week as
> I have to deliver the application.
>
> Cheers,
> Paul
>
> On Tue, 2005-07-26 at 16:58 +1200, Simon Kitching wrote:
>
> > Hi Paul,
> >
> > On Mon, 2005-07-25 at 12:25 -0700, Paul Austin wrote:
> > > I am using several of the other commons components and hibernate that
> > > use the commons-logging API to abstract from the underlying log4j or
> > > Java Logging APIs.
> > >
> > > Unfortunately due to a bug in JDK 1.4 which does not release references
> > > to class definitions correctly when commons-logging is containing in a
> > > J2EE web application WAR file and the application  is redeployed. The
> > > only reliable solution is to deploy log4j and the commons-logging and
> > > commons-logging-optional (1.0.5-alpha) jar files as shared libraries for
> > > the entire J2EE server. That approach has worked well in my development
> > > environment.
> >
> > I'm not sure what "bug in JDK 1.4" you're referring to. Do you have any
> > further info on this?
> >
> > If commons-logging is deployed via a shared classloader and the servlet
> > engine is not commons-logging-aware then components such as webapps need
> > to ensure that LogFactory.release is called when they are undeployed in
> > order to avoid exactly the situation you describe. Note that Tomcat is
> > "commons-logging-aware" and does this automatically. This isn't a "bug",
> > it's a known cleanup step. The commons-logging-optional.jar does
> > *sometimes* remove the need for the call to LogFactory.release but not
> > always which is why I prefer to use LogFactory.release instead of or as
> > well as the optional jar.
> >
> > I don't know of any situations where such a memory leak is *caused* by
> > having commons-logging in the component's WEB-INF/lib. More information
> > would be welcome.
> >
> > What container (eg tomcat, jboss) are you using?
> >
> > >
> > > The application is however going to be deployed in a shared server that
> > > I have little control over and it is unlikely that I will be able to
> > > deploy these libraries as shared components as all the other
> > > applications have local log4j config files so it would require a lot of
> > > testing and tweaking of other applications to get everything to work
> > > correctly.
> >
> > Yep, that's the problem of putting libs in a "shared" classloader. It's
> > an approach I don't like at all for exactly that reason.
> >
> > >
> > > One potential solution I can think for this is to create a clone of
> > > commons-logging that just provides the same interfaces but is statically
> > > bound to log4j so that it would not have to do all the lookups and
> > > caching of class references that goes on right now. This would be a
> > > similar thing to slf4j.org (UGLI) but with the existing commons logging
> > > interface so it can be dropped in as a replacement for commons-logging
> > > for other API's that already use it.
> >
> > Yep, adopting some of the ideas from UGLI is something that has been
> > looked at. There are some definite benefits, but also some disadvantages
> > to this approach. Some prototyping of this has been done but nothing
> > that is likely to be released this year.
> >
> >
> > > Has anyone looked at this or know of any other solutions to the out of
> > > memory issues.
> >
> > If you can provide more information I would be willing to help you look
> > into this. I don't see any reason why the current commons-logging
> > wouldn't work for you provided things are set up correctly.
> >
> > Regards,
> >
> > Simon
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
Reply | Threaded
Open this post in threaded view
|

Re: [logging] lon4j only implementation to solve out of memory errors

Simon Kitching
[AARGH- I hate top-posting!]

Thanks for the info Paul. The point about flushing the Introspector
cache is a good one. I've added some info to the wiki about this. Note,
however, that commons-logging doesn't use any bean introspection itself.

In the case of Tomcat 5.5, tomcat already explicitly flushes the
introspector caches on webapp undeploy so code running in that container
shouldn't be hit by this JDK problem.

I don't know much about spring; it might well need this to be done
manually.

Regards,

Simon

On Fri, 2005-08-05 at 11:36 -0700, Paul Austin wrote:

> Hi Simon,
>
> I think adding the following ContextListener solved the problem.
>
> http://static.springframework.org/spring/docs/1.1.5/api/org/springframework/web/util/IntrospectorCleanupListener.html
>
> Paul
>
> On Tue, 2005-07-26 at 08:09 -0700, Paul Austin wrote:
>
> > Hi Simon,
> >
> > I am running my application on Tomcat 5.5 under Sun JDK 1.4.2_07-b05 on
> > Windows XP. For the production environment it will be Oracle Application
> > Server 9.0.4.1 under JDK 1.4.2 on Solaris 2.8 (I haven't had access to
> > test on that server yet).
> >
> > The original discussion where I started to find out about the problem
> > was on the hibernate forums.
> >
> > http://forum.hibernate.org/viewtopic.php?t=935948&highlight=permgen
> >
> > One of the things I tried was adding a context listener which had the
> > LogFactory.release method called. There are also 3 filters which use
> > springframework/hibernate which use libraries that may use
> > commons-logging. I'm guessing that perhaps the destroy method on the
> > filters is called after I call LogFactory.release thus some new log
> > instances are created thus undoing the work in the release method.
> >
> > I'll try to get a stripped down copy of the application that still has
> > the issue put together to help solve this problem. Won't be this week as
> > I have to deliver the application.
> >
> > Cheers,
> > Paul
> >
> > On Tue, 2005-07-26 at 16:58 +1200, Simon Kitching wrote:
> >
> > > Hi Paul,
> > >
> > > On Mon, 2005-07-25 at 12:25 -0700, Paul Austin wrote:
> > > > I am using several of the other commons components and hibernate that
> > > > use the commons-logging API to abstract from the underlying log4j or
> > > > Java Logging APIs.
> > > >
> > > > Unfortunately due to a bug in JDK 1.4 which does not release references
> > > > to class definitions correctly when commons-logging is containing in a
> > > > J2EE web application WAR file and the application  is redeployed. The
> > > > only reliable solution is to deploy log4j and the commons-logging and
> > > > commons-logging-optional (1.0.5-alpha) jar files as shared libraries for
> > > > the entire J2EE server. That approach has worked well in my development
> > > > environment.
> > >
> > > I'm not sure what "bug in JDK 1.4" you're referring to. Do you have any
> > > further info on this?
> > >
> > > If commons-logging is deployed via a shared classloader and the servlet
> > > engine is not commons-logging-aware then components such as webapps need
> > > to ensure that LogFactory.release is called when they are undeployed in
> > > order to avoid exactly the situation you describe. Note that Tomcat is
> > > "commons-logging-aware" and does this automatically. This isn't a "bug",
> > > it's a known cleanup step. The commons-logging-optional.jar does
> > > *sometimes* remove the need for the call to LogFactory.release but not
> > > always which is why I prefer to use LogFactory.release instead of or as
> > > well as the optional jar.
> > >
> > > I don't know of any situations where such a memory leak is *caused* by
> > > having commons-logging in the component's WEB-INF/lib. More information
> > > would be welcome.
> > >
> > > What container (eg tomcat, jboss) are you using?
> > >
> > > >
> > > > The application is however going to be deployed in a shared server that
> > > > I have little control over and it is unlikely that I will be able to
> > > > deploy these libraries as shared components as all the other
> > > > applications have local log4j config files so it would require a lot of
> > > > testing and tweaking of other applications to get everything to work
> > > > correctly.
> > >
> > > Yep, that's the problem of putting libs in a "shared" classloader. It's
> > > an approach I don't like at all for exactly that reason.
> > >
> > > >
> > > > One potential solution I can think for this is to create a clone of
> > > > commons-logging that just provides the same interfaces but is statically
> > > > bound to log4j so that it would not have to do all the lookups and
> > > > caching of class references that goes on right now. This would be a
> > > > similar thing to slf4j.org (UGLI) but with the existing commons logging
> > > > interface so it can be dropped in as a replacement for commons-logging
> > > > for other API's that already use it.
> > >
> > > Yep, adopting some of the ideas from UGLI is something that has been
> > > looked at. There are some definite benefits, but also some disadvantages
> > > to this approach. Some prototyping of this has been done but nothing
> > > that is likely to be released this year.
> > >
> > >
> > > > Has anyone looked at this or know of any other solutions to the out of
> > > > memory issues.
> > >
> > > If you can provide more information I would be willing to help you look
> > > into this. I don't see any reason why the current commons-logging
> > > wouldn't work for you provided things are set up correctly.
> > >
> > > Regards,
> > >
> > > Simon
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [logging] lon4j only implementation to solve out of memory errors

Jacob Kjome
At 11:03 AM 8/6/2005 +1200, you wrote:
 >[AARGH- I hate top-posting!]
 >
 >Thanks for the info Paul. The point about flushing the Introspector
 >cache is a good one. I've added some info to the wiki about this. Note,
 >however, that commons-logging doesn't use any bean introspection itself.
 >
 >In the case of Tomcat 5.5, tomcat already explicitly flushes the
 >introspector caches on webapp undeploy so code running in that container
 >shouldn't be hit by this JDK problem.
 >

Yes, in recent versions, and only after it was figured out that it wasn't
Log4j causing thread death issues, even though it looked like it. See:
http://issues.apache.org/bugzilla/show_bug.cgi?id=27371#c27

Even then, it doesn't seem to fix the issue 100% of the time
http://issues.apache.org/bugzilla/show_bug.cgi?id=26372

Seems like there ought to be a right way to do this to fix things
100%.  Seems like a core JVM issue.  Maybe Sun should think about
addressing this since (or at least pointing out how to avoid it once and
for all) application servers seem to be the primary use-case for Java these
days.  They ought to work reliably!

Jake

 >I don't know much about spring; it might well need this to be done
 >manually.
 >
 >Regards,
 >
 >Simon
 >
 >On Fri, 2005-08-05 at 11:36 -0700, Paul Austin wrote:
 >> Hi Simon,
 >>
 >> I think adding the following ContextListener solved the problem.
 >>
 >>
 >http://static.springframework.org/spring/docs/1.1.5/api/org/springframework/
 >web/util/IntrospectorCleanupListener.html
 >>
 >> Paul
 >>
 >> On Tue, 2005-07-26 at 08:09 -0700, Paul Austin wrote:
 >>
 >> > Hi Simon,
 >> >
 >> > I am running my application on Tomcat 5.5 under Sun JDK 1.4.2_07-b05 on
 >> > Windows XP. For the production environment it will be Oracle Application
 >> > Server 9.0.4.1 under JDK 1.4.2 on Solaris 2.8 (I haven't had access to
 >> > test on that server yet).
 >> >
 >> > The original discussion where I started to find out about the problem
 >> > was on the hibernate forums.
 >> >
 >> > http://forum.hibernate.org/viewtopic.php?t=935948&highlight=permgen
 >> >
 >> > One of the things I tried was adding a context listener which had the
 >> > LogFactory.release method called. There are also 3 filters which use
 >> > springframework/hibernate which use libraries that may use
 >> > commons-logging. I'm guessing that perhaps the destroy method on the
 >> > filters is called after I call LogFactory.release thus some new log
 >> > instances are created thus undoing the work in the release method.
 >> >
 >> > I'll try to get a stripped down copy of the application that still has
 >> > the issue put together to help solve this problem. Won't be this week as
 >> > I have to deliver the application.
 >> >
 >> > Cheers,
 >> > Paul
 >> >
 >> > On Tue, 2005-07-26 at 16:58 +1200, Simon Kitching wrote:
 >> >
 >> > > Hi Paul,
 >> > >
 >> > > On Mon, 2005-07-25 at 12:25 -0700, Paul Austin wrote:
 >> > > > I am using several of the other commons components and hibernate that
 >> > > > use the commons-logging API to abstract from the underlying log4j or
 >> > > > Java Logging APIs.
 >> > > >
 >> > > > Unfortunately due to a bug in JDK 1.4 which does not release
references
 >> > > > to class definitions correctly when commons-logging is containing
in a
 >> > > > J2EE web application WAR file and the application  is redeployed. The
 >> > > > only reliable solution is to deploy log4j and the commons-logging and
 >> > > > commons-logging-optional (1.0.5-alpha) jar files as shared libraries
 >for
 >> > > > the entire J2EE server. That approach has worked well in my
development
 >> > > > environment.
 >> > >
 >> > > I'm not sure what "bug in JDK 1.4" you're referring to. Do you have any
 >> > > further info on this?
 >> > >
 >> > > If commons-logging is deployed via a shared classloader and the servlet
 >> > > engine is not commons-logging-aware then components such as webapps
need
 >> > > to ensure that LogFactory.release is called when they are undeployed in
 >> > > order to avoid exactly the situation you describe. Note that Tomcat is
 >> > > "commons-logging-aware" and does this automatically. This isn't a
"bug",
 >> > > it's a known cleanup step. The commons-logging-optional.jar does
 >> > > *sometimes* remove the need for the call to LogFactory.release but not
 >> > > always which is why I prefer to use LogFactory.release instead of or as
 >> > > well as the optional jar.
 >> > >
 >> > > I don't know of any situations where such a memory leak is *caused* by
 >> > > having commons-logging in the component's WEB-INF/lib. More information
 >> > > would be welcome.
 >> > >
 >> > > What container (eg tomcat, jboss) are you using?
 >> > >
 >> > > >
 >> > > > The application is however going to be deployed in a shared
server that
 >> > > > I have little control over and it is unlikely that I will be able to
 >> > > > deploy these libraries as shared components as all the other
 >> > > > applications have local log4j config files so it would require a
lot of
 >> > > > testing and tweaking of other applications to get everything to work
 >> > > > correctly.
 >> > >
 >> > > Yep, that's the problem of putting libs in a "shared" classloader. It's
 >> > > an approach I don't like at all for exactly that reason.
 >> > >
 >> > > >
 >> > > > One potential solution I can think for this is to create a clone of
 >> > > > commons-logging that just provides the same interfaces but is
 >statically
 >> > > > bound to log4j so that it would not have to do all the lookups and
 >> > > > caching of class references that goes on right now. This would be a
 >> > > > similar thing to slf4j.org (UGLI) but with the existing commons
logging
 >> > > > interface so it can be dropped in as a replacement for
commons-logging
 >> > > > for other API's that already use it.
 >> > >
 >> > > Yep, adopting some of the ideas from UGLI is something that has been
 >> > > looked at. There are some definite benefits, but also some
disadvantages
 >> > > to this approach. Some prototyping of this has been done but nothing
 >> > > that is likely to be released this year.
 >> > >
 >> > >
 >> > > > Has anyone looked at this or know of any other solutions to the
out of
 >> > > > memory issues.
 >> > >
 >> > > If you can provide more information I would be willing to help you look
 >> > > into this. I don't see any reason why the current commons-logging
 >> > > wouldn't work for you provided things are set up correctly.
 >> > >
 >> > > Regards,
 >> > >
 >> > > Simon
 >> > >
 >> > >
 >> > > ---------------------------------------------------------------------
 >> > > To unsubscribe, e-mail: [hidden email]
 >> > > For additional commands, e-mail: [hidden email]
 >> > >
 >
 >
 >---------------------------------------------------------------------
 >To unsubscribe, e-mail: [hidden email]
 >For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [logging] lon4j only implementation to solve out of memory errors

Simon Kitching
In reply to this post by Simon Kitching
On Wed, 2005-08-10 at 19:25 +1200, Simon Kitching wrote:

> Hi Paul,
>
> Try the following URL, select "view" on the latest file and search for
> Introspector. Then try walking back through the versions until you find
> a version that doesn't have the fix. It looks like 1.46 is the relevant
> patch, so Version 5.0.28 doesn't have it, but 5.5.8 does.
>
> http://cvs.apache.org/viewcvs.cgi/jakarta-tomcat-catalina/catalina/src/share/org/apache/catalina/loader/WebappClassLoader.java?only_with_tag=
>
> Cheers,
>
> Simon
>
> On Mon, 2005-08-08 at 10:35 -0700, Paul Austin wrote:
> > Hi Simon,
> >
> > You mention that Tomcat 5.5 flushes both the Introspector cache and
> > does the Commons Logging cleanup to. Which version of 5.5 was this
> > introduced as I have been running with Tomcat 5.5 and had the problems
> > until I used the solution I mentioned below.
> >
> > Paul
> >
> > On Sat, 2005-08-06 at 11:03 +1200, Simon Kitching wrote:
> > > [AARGH- I hate top-posting!]
> > >
> > > Thanks for the info Paul. The point about flushing the Introspector
> > > cache is a good one. I've added some info to the wiki about this. Note,
> > > however, that commons-logging doesn't use any bean introspection itself.
> > >
> > > In the case of Tomcat 5.5, tomcat already explicitly flushes the
> > > introspector caches on webapp undeploy so code running in that container
> > > shouldn't be hit by this JDK problem.
> > >
> > > I don't know much about spring; it might well need this to be done
> > > manually.
> > >
> > > Regards,
> > >
> > > Simon
> > >
> > > On Fri, 2005-08-05 at 11:36 -0700, Paul Austin wrote:
> > > > Hi Simon,
> > > >
> > > > I think adding the following ContextListener solved the problem.
> > > >
> > > > http://static.springframework.org/spring/docs/1.1.5/api/org/springframework/web/util/IntrospectorCleanupListener.html
> > > >
> > > > Paul
> > > >
> > > > On Tue, 2005-07-26 at 08:09 -0700, Paul Austin wrote:
> > > >
> > > > > Hi Simon,
> > > > >
> > > > > I am running my application on Tomcat 5.5 under Sun JDK 1.4.2_07-b05 on
> > > > > Windows XP. For the production environment it will be Oracle Application
> > > > > Server 9.0.4.1 under JDK 1.4.2 on Solaris 2.8 (I haven't had access to
> > > > > test on that server yet).
> > > > >
> > > > > The original discussion where I started to find out about the problem
> > > > > was on the hibernate forums.
> > > > >
> > > > > http://forum.hibernate.org/viewtopic.php?t=935948&highlight=permgen
> > > > >
> > > > > One of the things I tried was adding a context listener which had the
> > > > > LogFactory.release method called. There are also 3 filters which use
> > > > > springframework/hibernate which use libraries that may use
> > > > > commons-logging. I'm guessing that perhaps the destroy method on the
> > > > > filters is called after I call LogFactory.release thus some new log
> > > > > instances are created thus undoing the work in the release method.
> > > > >
> > > > > I'll try to get a stripped down copy of the application that still has
> > > > > the issue put together to help solve this problem. Won't be this week as
> > > > > I have to deliver the application.
> > > > >
> > > > > Cheers,
> > > > > Paul
> > > > >
> > > > > On Tue, 2005-07-26 at 16:58 +1200, Simon Kitching wrote:
> > > > >
> > > > > > Hi Paul,
> > > > > >
> > > > > > On Mon, 2005-07-25 at 12:25 -0700, Paul Austin wrote:
> > > > > > > I am using several of the other commons components and hibernate that
> > > > > > > use the commons-logging API to abstract from the underlying log4j or
> > > > > > > Java Logging APIs.
> > > > > > >
> > > > > > > Unfortunately due to a bug in JDK 1.4 which does not release references
> > > > > > > to class definitions correctly when commons-logging is containing in a
> > > > > > > J2EE web application WAR file and the application  is redeployed. The
> > > > > > > only reliable solution is to deploy log4j and the commons-logging and
> > > > > > > commons-logging-optional (1.0.5-alpha) jar files as shared libraries for
> > > > > > > the entire J2EE server. That approach has worked well in my development
> > > > > > > environment.
> > > > > >
> > > > > > I'm not sure what "bug in JDK 1.4" you're referring to. Do you have any
> > > > > > further info on this?
> > > > > >
> > > > > > If commons-logging is deployed via a shared classloader and the servlet
> > > > > > engine is not commons-logging-aware then components such as webapps need
> > > > > > to ensure that LogFactory.release is called when they are undeployed in
> > > > > > order to avoid exactly the situation you describe. Note that Tomcat is
> > > > > > "commons-logging-aware" and does this automatically. This isn't a "bug",
> > > > > > it's a known cleanup step. The commons-logging-optional.jar does
> > > > > > *sometimes* remove the need for the call to LogFactory.release but not
> > > > > > always which is why I prefer to use LogFactory.release instead of or as
> > > > > > well as the optional jar.
> > > > > >
> > > > > > I don't know of any situations where such a memory leak is *caused* by
> > > > > > having commons-logging in the component's WEB-INF/lib. More information
> > > > > > would be welcome.
> > > > > >
> > > > > > What container (eg tomcat, jboss) are you using?
> > > > > >
> > > > > > >
> > > > > > > The application is however going to be deployed in a shared server that
> > > > > > > I have little control over and it is unlikely that I will be able to
> > > > > > > deploy these libraries as shared components as all the other
> > > > > > > applications have local log4j config files so it would require a lot of
> > > > > > > testing and tweaking of other applications to get everything to work
> > > > > > > correctly.
> > > > > >
> > > > > > Yep, that's the problem of putting libs in a "shared" classloader. It's
> > > > > > an approach I don't like at all for exactly that reason.
> > > > > >
> > > > > > >
> > > > > > > One potential solution I can think for this is to create a clone of
> > > > > > > commons-logging that just provides the same interfaces but is statically
> > > > > > > bound to log4j so that it would not have to do all the lookups and
> > > > > > > caching of class references that goes on right now. This would be a
> > > > > > > similar thing to slf4j.org (UGLI) but with the existing commons logging
> > > > > > > interface so it can be dropped in as a replacement for commons-logging
> > > > > > > for other API's that already use it.
> > > > > >
> > > > > > Yep, adopting some of the ideas from UGLI is something that has been
> > > > > > looked at. There are some definite benefits, but also some disadvantages
> > > > > > to this approach. Some prototyping of this has been done but nothing
> > > > > > that is likely to be released this year.
> > > > > >
> > > > > >
> > > > > > > Has anyone looked at this or know of any other solutions to the out of
> > > > > > > memory issues.
> > > > > >
> > > > > > If you can provide more information I would be willing to help you look
> > > > > > into this. I don't see any reason why the current commons-logging
> > > > > > wouldn't work for you provided things are set up correctly.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Simon
> > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: [hidden email]
> > > > > > For additional commands, e-mail: [hidden email]
> > > > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]