[jira] Created: (SANDBOX-170) Creating TAR archive of an 8 GB file causes IOException

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SANDBOX-170) Creating TAR archive of an 8 GB file causes IOException

ASF GitHub Bot (Jira)
Creating TAR archive of an 8 GB file causes IOException
-------------------------------------------------------

                 Key: SANDBOX-170
                 URL: http://issues.apache.org/jira/browse/SANDBOX-170
             Project: Commons Sandbox
          Issue Type: Bug
          Components: Compress
    Affects Versions: Nightly Builds
         Environment: Win xp pro sp2; JDK 1.5.0_07
            Reporter: Sam Smith


Yesterday I reported a bug with extraction of TAR archives:
        https://issues.apache.org/jira/browse/SANDBOX-168

Today I found that when I tried to creat a TAR archive of a file that is exactly 8 GB, that is,
        (8L * 1024L * 1024L * 1024L) - 1
that it failed with the following strange error:

Exception in thread "main" java.io.IOException: request to write '32768' bytes exceeds size in header of '-1' bytes
        at org.apache.commons.compress.archivers.tar.TarOutputStream.write(TarOutputStream.java:341)
        at bb.io.StreamUtil.transfer(StreamUtil.java:312)
        at bb.io.StreamUtil.transfer(StreamUtil.java:290)
        at bb.io.TarUtil.writeFileData(TarUtil.java:404)
        at bb.io.TarUtil.archive(TarUtil.java:382)
        at bb.io.TarUtil.archive(TarUtil.java:334)
        at bb.io.TarUtil$Test.test_archive_extract_fileLengthLimit(TarUtil.java:610)
        at bb.io.TarUtil$Test.main(TarUtil.java:542)

The code that I used to try and create the TAR archive was the same described in that previous bug report, so I will not repeat it here.

I was doing this in an attempt to discern if the org.apache.commons.compress TAR code has any file size limits, since classic TAR is limited to 8 BG files but org.apache.commons.compress supports GNU TAR long path names, so maybe it has no file size limits either.  (There appears to be no documentation about this in the TAR code.)

What is strange is that the code first DID apparently succeed in TARing a 10 GB file, albeit, when I tried to see if a third party piece of software (7-zip) would extract it, that program objected saying that it did not support the TAR format type encpuntered in the file.

Since I need to use a file size limit that is consistent with other programs, I then tried to TAR a file of exactly 8 GB in size.  This is when I encountered the above error.


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SANDBOX-170) Creating TAR archive of an 8 GB file causes IOException

ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/SANDBOX-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12490427 ]

Jeff Crump commented on SANDBOX-170:
------------------------------------

I was faced with a similar issue today.   While the entry size is a long, the write() method takes an int -- and if you get too big, you overflow.  Here is the exception I got when trying to write 9GB entry

java.io.IOException request to write '8192' bytes exceeds size in header of '1390354048' bytes
    at org.apache.commons.compress.tar.TarOutputStream.write(TarOutputStream.java:-3)
    at java.nio.channels.Channels$WritableByteChannelImpl.write(Channels.java:-3)
    at org.archivas.io.TransferSupport.writeBuffer(TransferSupport.java:-3)
    at org.archivas.io.TransferSupport.writeChunk(TransferSupport.java:-4)

It looks like the Ant folks have taken care of this already?
http://www.cenqua.com/fisheye/demo/browse/ant/src/main/org/apache/tools/tar/TarOutputStream.java

Why is the m_size field a long, but the write() API take an int?



> Creating TAR archive of an 8 GB file causes IOException
> -------------------------------------------------------
>
>                 Key: SANDBOX-170
>                 URL: https://issues.apache.org/jira/browse/SANDBOX-170
>             Project: Commons Sandbox
>          Issue Type: Bug
>          Components: Compress
>    Affects Versions: Nightly Builds
>         Environment: Win xp pro sp2; JDK 1.5.0_07
>            Reporter: Sam Smith
>
> Yesterday I reported a bug with extraction of TAR archives:
> https://issues.apache.org/jira/browse/SANDBOX-168
> Today I found that when I tried to creat a TAR archive of a file that is exactly 8 GB, that is,
> (8L * 1024L * 1024L * 1024L) - 1
> that it failed with the following strange error:
> Exception in thread "main" java.io.IOException: request to write '32768' bytes exceeds size in header of '-1' bytes
>         at org.apache.commons.compress.archivers.tar.TarOutputStream.write(TarOutputStream.java:341)
>         at bb.io.StreamUtil.transfer(StreamUtil.java:312)
>         at bb.io.StreamUtil.transfer(StreamUtil.java:290)
>         at bb.io.TarUtil.writeFileData(TarUtil.java:404)
>         at bb.io.TarUtil.archive(TarUtil.java:382)
>         at bb.io.TarUtil.archive(TarUtil.java:334)
>         at bb.io.TarUtil$Test.test_archive_extract_fileLengthLimit(TarUtil.java:610)
>         at bb.io.TarUtil$Test.main(TarUtil.java:542)
> The code that I used to try and create the TAR archive was the same described in that previous bug report, so I will not repeat it here.
> I was doing this in an attempt to discern if the org.apache.commons.compress TAR code has any file size limits, since classic TAR is limited to 8 BG files but org.apache.commons.compress supports GNU TAR long path names, so maybe it has no file size limits either.  (There appears to be no documentation about this in the TAR code.)
> What is strange is that the code first DID apparently succeed in TARing a 10 GB file, albeit, when I tried to see if a third party piece of software (7-zip) would extract it, that program objected saying that it did not support the TAR format type encpuntered in the file.
> Since I need to use a file size limit that is consistent with other programs, I then tried to TAR a file of exactly 8 GB in size.  This is when I encountered the above error.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]