[FileUpload] files greater than 2 GB

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[FileUpload] files greater than 2 GB

elihusmails
Is there a limitation in tomcat that limits files to 2 GB?  I would like to
write a servlet that will be accepting files larger than 2 GB using the 64
bit JDK.

TIA
--
..Cheers
Mark
Reply | Threaded
Open this post in threaded view
|

Re: [FileUpload] files greater than 2 GB

Martin Cooper-3
The limitation is in the Servlet spec: The getContentLength method returns
an 'int'.

--
Martin Cooper


On 6/5/07, Mark Webb <[hidden email]> wrote:

>
> Is there a limitation in tomcat that limits files to 2 GB?  I would like
> to
> write a servlet that will be accepting files larger than 2 GB using the 64
> bit JDK.
>
> TIA
> --
> ..Cheers
> Mark
>
Reply | Threaded
Open this post in threaded view
|

Re: [FileUpload] files greater than 2 GB

serff
That doesn't stop you from uploading a file larger than 2GB though.  You
must use fileupload 1.2.  1.1 relies on the content length being set if
I remember right.  But I use FileUpload 1.2 and JBoss (which uses
tomcat) and we upload 10GB files today over https.

Andrew
Martin Cooper wrote:

> The limitation is in the Servlet spec: The getContentLength method
> returns
> an 'int'.
>
> --
> Martin Cooper
>
>
> On 6/5/07, Mark Webb <[hidden email]> wrote:
>>
>> Is there a limitation in tomcat that limits files to 2 GB?  I would like
>> to
>> write a servlet that will be accepting files larger than 2 GB using
>> the 64
>> bit JDK.
>>
>> TIA
>> --
>> ..Cheers
>> Mark
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [FileUpload] files greater than 2 GB

elihusmails
are you using a 64 bit JVM in order to get the 10GB file?

On 6/5/07, Andrew <[hidden email]> wrote:

>
> That doesn't stop you from uploading a file larger than 2GB though.  You
> must use fileupload 1.2.  1.1 relies on the content length being set if
> I remember right.  But I use FileUpload 1.2 and JBoss (which uses
> tomcat) and we upload 10GB files today over https.
>
> Andrew
> Martin Cooper wrote:
> > The limitation is in the Servlet spec: The getContentLength method
> > returns
> > an 'int'.
> >
> > --
> > Martin Cooper
> >
> >
> > On 6/5/07, Mark Webb <[hidden email]> wrote:
> >>
> >> Is there a limitation in tomcat that limits files to 2 GB?  I would
> like
> >> to
> >> write a servlet that will be accepting files larger than 2 GB using
> >> the 64
> >> bit JDK.
> >>
> >> TIA
> >> --
> >> ..Cheers
> >> Mark
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


--
..Cheers
Mark
Reply | Threaded
Open this post in threaded view
|

Re: [FileUpload] files greater than 2 GB

J Lum
I'm using 1.2 with Weblogic and cannot upload > 2G files.  It seems to get to the last read and then I get the exception below.  Has anyone had success with Weblogic?

java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at weblogic.servlet.internal.PostInputStream.read(PostInputStream.java:136)
        at weblogic.servlet.internal.PostInputStream.read(PostInputStream.java:156)
        at weblogic.servlet.internal.ServletInputStreamImpl.read(ServletInputStreamImpl.java:180)
        at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:959)
        at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:887)
        at java.io.InputStream.read(InputStream.java:89)
        at TestServlet.writeStream(TestServlet.java:86)
        at TestServlet.doPost(TestServlet.java:39)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)
        at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:465)
        at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:348)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6981)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
        at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3892)
        at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2766)
        at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)

Mark Webb-4 wrote
are you using a 64 bit JVM in order to get the 10GB file?

On 6/5/07, Andrew <lists@serff.net> wrote:
>
> That doesn't stop you from uploading a file larger than 2GB though.  You
> must use fileupload 1.2.  1.1 relies on the content length being set if
> I remember right.  But I use FileUpload 1.2 and JBoss (which uses
> tomcat) and we upload 10GB files today over https.
>
> Andrew
> Martin Cooper wrote:
> > The limitation is in the Servlet spec: The getContentLength method
> > returns
> > an 'int'.
> >
> > --
> > Martin Cooper
> >
> >
> > On 6/5/07, Mark Webb <elihusmails@gmail.com> wrote:
> >>
> >> Is there a limitation in tomcat that limits files to 2 GB?  I would
> like
> >> to
> >> write a servlet that will be accepting files larger than 2 GB using
> >> the 64
> >> bit JDK.
> >>
> >> TIA
> >> --
> >> ..Cheers
> >> Mark
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>


--
..Cheers
Mark
Reply | Threaded
Open this post in threaded view
|

Re: [FileUpload] files greater than 2 GB

serff
In reply to this post by elihusmails
The server is using the 64 bit VM but the client is not.  I guess I
don't know why you would need the 64 bit VM because you aren't loading
the entire file into memory.  Also, make sure you are using the
streaming API in 1.2 (otherwise you will be keeping it all in memory).
The problem you have is if you are trying to keep progress.  You can't
keep progress in an int cause it will over flow.

Andrew
Mark Webb wrote:

> are you using a 64 bit JVM in order to get the 10GB file?
>
> On 6/5/07, Andrew <[hidden email]> wrote:
>>
>> That doesn't stop you from uploading a file larger than 2GB though.  You
>> must use fileupload 1.2.  1.1 relies on the content length being set if
>> I remember right.  But I use FileUpload 1.2 and JBoss (which uses
>> tomcat) and we upload 10GB files today over https.
>>
>> Andrew
>> Martin Cooper wrote:
>> > The limitation is in the Servlet spec: The getContentLength method
>> > returns
>> > an 'int'.
>> >
>> > --
>> > Martin Cooper
>> >
>> >
>> > On 6/5/07, Mark Webb <[hidden email]> wrote:
>> >>
>> >> Is there a limitation in tomcat that limits files to 2 GB?  I would
>> like
>> >> to
>> >> write a servlet that will be accepting files larger than 2 GB using
>> >> the 64
>> >> bit JDK.
>> >>
>> >> TIA
>> >> --
>> >> ..Cheers
>> >> Mark
>> >>
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: [FileUpload] files greater than 2 GB

Aaron Freeman-3
In reply to this post by serff
You are able to transfer files larger than 2GB using IE and FireFox as a
client, with FileUpload 1.2?

Aaron

> -----Original Message-----
> From: Andrew [mailto:[hidden email]]
> Sent: Tuesday, June 05, 2007 2:39 PM
> To: Jakarta Commons Users List
> Subject: Re: [FileUpload] files greater than 2 GB
>
> That doesn't stop you from uploading a file larger than 2GB
> though.  You must use fileupload 1.2.  1.1 relies on the
> content length being set if I remember right.  But I use
> FileUpload 1.2 and JBoss (which uses
> tomcat) and we upload 10GB files today over https.
>
> Andrew
> Martin Cooper wrote:
> > The limitation is in the Servlet spec: The getContentLength method
> > returns an 'int'.
> >
> > --
> > Martin Cooper
> >
> >
> > On 6/5/07, Mark Webb <[hidden email]> wrote:
> >>
> >> Is there a limitation in tomcat that limits files to 2 GB?
>  I would
> >> like to write a servlet that will be accepting files
> larger than 2 GB
> >> using the 64 bit JDK.
> >>
> >> TIA
> >> --
> >> ..Cheers
> >> Mark
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [FileUpload] files greater than 2 GB

serff
No, I have a custom Java client that uploads.  Sorry I didn't make that
clear.  It uses the Apache Commons HTTP library  to do the upload.  I
think I've posted other emails with example code to this list as well if
you're interested.

I can't try it with straight browser because we have other things
required in the call to upload.  So I can't be much help there...could
definitely be client bound...

Andrew
Aaron Freeman wrote:

> You are able to transfer files larger than 2GB using IE and FireFox as a
> client, with FileUpload 1.2?
>
> Aaron
>
>  
>> -----Original Message-----
>> From: Andrew [mailto:[hidden email]]
>> Sent: Tuesday, June 05, 2007 2:39 PM
>> To: Jakarta Commons Users List
>> Subject: Re: [FileUpload] files greater than 2 GB
>>
>> That doesn't stop you from uploading a file larger than 2GB
>> though.  You must use fileupload 1.2.  1.1 relies on the
>> content length being set if I remember right.  But I use
>> FileUpload 1.2 and JBoss (which uses
>> tomcat) and we upload 10GB files today over https.
>>
>> Andrew
>> Martin Cooper wrote:
>>    
>>> The limitation is in the Servlet spec: The getContentLength method
>>> returns an 'int'.
>>>
>>> --
>>> Martin Cooper
>>>
>>>
>>> On 6/5/07, Mark Webb <[hidden email]> wrote:
>>>      
>>>> Is there a limitation in tomcat that limits files to 2 GB?
>>>>        
>>  I would
>>    
>>>> like to write a servlet that will be accepting files
>>>>        
>> larger than 2 GB
>>    
>>>> using the 64 bit JDK.
>>>>
>>>> TIA
>>>> --
>>>> ..Cheers
>>>> Mark
>>>>
>>>>        
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>>    
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>  

Reply | Threaded
Open this post in threaded view
|

Re: [FileUpload] files greater than 2 GB

David Wall
Just to be sure, did you set  maxPostSize="0" in your Connector in
conf/server.xml?  Don't have the details handy, but I think that option
might be needed.

David


Here's an example in mine:

<Connector port="8080" maxHttpHeaderSize="8192" maxPostSize="0"
               maxThreads="150" minSpareThreads="20" maxSpareThreads="50"
               enableLookups="false" redirectPort="443" acceptCount="100"
               connectionTimeout="20000" disableUploadTimeout="true" />

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: [FileUpload] files greater than 2 GB

Aaron Freeman-2
In reply to this post by serff
Yes, that was going to freak me out. :)

If he is using a standard browser as a client he is probably bumping up
against browser limitations.

-a

> -----Original Message-----
> From: Andrew [mailto:[hidden email]]
> Sent: Wednesday, June 06, 2007 4:05 PM
> To: Jakarta Commons Users List
> Subject: Re: [FileUpload] files greater than 2 GB
>
> No, I have a custom Java client that uploads.  Sorry I didn't
> make that clear.  It uses the Apache Commons HTTP library  to
> do the upload.  I think I've posted other emails with example
> code to this list as well if you're interested.
>
> I can't try it with straight browser because we have other
> things required in the call to upload.  So I can't be much
> help there...could definitely be client bound...
>
> Andrew
> Aaron Freeman wrote:
> > You are able to transfer files larger than 2GB using IE and
> FireFox as
> > a client, with FileUpload 1.2?
> >
> > Aaron
> >
> >  
> >> -----Original Message-----
> >> From: Andrew [mailto:[hidden email]]
> >> Sent: Tuesday, June 05, 2007 2:39 PM
> >> To: Jakarta Commons Users List
> >> Subject: Re: [FileUpload] files greater than 2 GB
> >>
> >> That doesn't stop you from uploading a file larger than
> 2GB though.  
> >> You must use fileupload 1.2.  1.1 relies on the content
> length being
> >> set if I remember right.  But I use FileUpload 1.2 and
> JBoss (which
> >> uses
> >> tomcat) and we upload 10GB files today over https.
> >>
> >> Andrew
> >> Martin Cooper wrote:
> >>    
> >>> The limitation is in the Servlet spec: The
> getContentLength method
> >>> returns an 'int'.
> >>>
> >>> --
> >>> Martin Cooper
> >>>
> >>>
> >>> On 6/5/07, Mark Webb <[hidden email]> wrote:
> >>>      
> >>>> Is there a limitation in tomcat that limits files to 2 GB?
> >>>>        
> >>  I would
> >>    
> >>>> like to write a servlet that will be accepting files
> >>>>        
> >> larger than 2 GB
> >>    
> >>>> using the 64 bit JDK.
> >>>>
> >>>> TIA
> >>>> --
> >>>> ..Cheers
> >>>> Mark
> >>>>
> >>>>        
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail:
> [hidden email]
> >>
> >>
> >>    
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail:
> [hidden email]
> >
> >  
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]