[jira] [Created] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
Base32 would decode some invalid Base32 encoded string into arbitrary value
---------------------------------------------------------------------------

                 Key: CODEC-134
                 URL: https://issues.apache.org/jira/browse/CODEC-134
             Project: Commons Codec
          Issue Type: Bug
    Affects Versions: 1.6
         Environment: All
            Reporter: Hanson Char


Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".

Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221729#comment-13221729 ]

Hanson Char commented on CODEC-134:
-----------------------------------

I suspect similar vulnerability exists in the Bsee64 codec, and if so should be fixed as well.
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

Re: [jira] [Created] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

Gary Gregory-4
In reply to this post by ASF GitHub Bot (Jira)
Can you provide a patch please?

Gary

On Mar 3, 2012, at 18:16, "Hanson Char (Created) (JIRA)" <[hidden email]> wrote:

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
>
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221888#comment-13221888 ]

Gary D. Gregory commented on CODEC-134:
---------------------------------------

Can you please provide a patch?
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221889#comment-13221889 ]

Gary D. Gregory commented on CODEC-134:
---------------------------------------

Note that we might elect to file this in the garbage-in-garbage-out category.
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221939#comment-13221939 ]

Hanson Char commented on CODEC-134:
-----------------------------------

Hi Gary, I think I have a rather nice and elegant patch for this; but if you don't mind waiting, I'd like to get some clearance from my current employer.  Thanks.
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222062#comment-13222062 ]

Hanson Char commented on CODEC-134:
-----------------------------------

diff --git a/src/main/java/org/apache/commons/codec/binary/Base32.java b/src/main/java/org/apache/commons/codec/binary/Base32.java
index a9da10f..9cd293b 100644
--- a/src/main/java/org/apache/commons/codec/binary/Base32.java
+++ b/src/main/java/org/apache/commons/codec/binary/Base32.java
@@ -274,7 +274,7 @@ public class Base32 extends BaseNCodec {
      * @param inPos
      *            Position to start reading data from.
      * @param inAvail
-     *            Amount of bytes available from input for encoding.
+     *            Amount of bytes available from input for decoding.
      *
      * Output is written to {@link #buffer} as 8-bit octets, using {@link #pos} as the buffer position
      */
@@ -320,30 +320,30 @@ public class Base32 extends BaseNCodec {
             //  we ignore partial bytes, i.e. only multiples of 8 count
             switch (modulus) {
                 case 2 : // 10 bits, drop 2 and output one byte
-                    buffer[pos++] = (byte) ((bitWorkArea >> 2) & MASK_8BITS);
+                    buffer[pos++] = (byte) (dropBits(2) & MASK_8BITS);
                     break;
                 case 3 : // 15 bits, drop 7 and output 1 byte
-                    buffer[pos++] = (byte) ((bitWorkArea >> 7) & MASK_8BITS);
+                    buffer[pos++] = (byte) (dropBits(7) & MASK_8BITS);
                     break;
                 case 4 : // 20 bits = 2*8 + 4
-                    bitWorkArea = bitWorkArea >> 4; // drop 4 bits
+                    bitWorkArea = dropBits(4); // drop 4 bits
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
                 case 5 : // 25bits = 3*8 + 1
-                    bitWorkArea = bitWorkArea >> 1;
+                    bitWorkArea = dropBits(1);
                     buffer[pos++] = (byte) ((bitWorkArea >> 16) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
                 case 6 : // 30bits = 3*8 + 6
-                    bitWorkArea = bitWorkArea >> 6;
+                    bitWorkArea = dropBits(6);
                     buffer[pos++] = (byte) ((bitWorkArea >> 16) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
                 case 7 : // 35 = 4*8 +3
-                    bitWorkArea = bitWorkArea >> 3;
+                    bitWorkArea = dropBits(3);
                     buffer[pos++] = (byte) ((bitWorkArea >> 24) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea >> 16) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
@@ -352,6 +352,28 @@ public class Base32 extends BaseNCodec {
             }
         }
     }
+    
+    /**
+     * <p>
+     * Drops the specified number of least significant bits from the
+     * {@link #bitWorkArea}.
+     * </p>
+     *
+     * @param numBitsToDrop
+     *            number of least significant bits to drop
+     *
+     * @return the value of {@link #bitWorkArea} after dropping the
+     *            specified number of least significant bits
+     *
+     * @throws IllegalArgumentException
+     *            if the bits being dropped contain any non-zero value
+     */
+    private long dropBits(int numBitsToDrop) {
+        if ((bitWorkArea & numBitsToDrop) != 0) {
+            throw new IllegalArgumentException("Last encoded character (before the paddings if any) is a valid base 32 alphabet but not a possible value");
+        }
+        return bitWorkArea >> numBitsToDrop;
+    }
 
     /**
      * <p>
diff --git a/src/main/java/org/apache/commons/codec/binary/Base64.java b/src/main/java/org/apache/commons/codec/binary/Base64.java
index 1ee0eba..4261f88 100644
--- a/src/main/java/org/apache/commons/codec/binary/Base64.java
+++ b/src/main/java/org/apache/commons/codec/binary/Base64.java
@@ -410,7 +410,7 @@ public class Base64 extends BaseNCodec {
      * @param inPos
      *            Position to start reading data from.
      * @param inAvail
-     *            Amount of bytes available from input for encoding.
+     *            Amount of bytes available from input for decoding.
      */
     @Override
     void decode(byte[] in, int inPos, int inAvail) {
@@ -455,17 +455,39 @@ public class Base64 extends BaseNCodec {
            //   case 1: // 6 bits - ignore entirely
            //       break;
                 case 2 : // 12 bits = 8 + 4
-                    bitWorkArea = bitWorkArea >> 4; // dump the extra 4 bits
+                    bitWorkArea = dropBits(4); // drop the extra 4 bits
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
                 case 3 : // 18 bits = 8 + 8 + 2
-                    bitWorkArea = bitWorkArea >> 2; // dump 2 bits
+                    bitWorkArea = dropBits(2); // drop 2 bits
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
             }
         }
     }
+    
+    /**
+     * <p>
+     * Drops the specified number of least significant bits from the
+     * {@link #bitWorkArea}.
+     * </p>
+     *
+     * @param numBitsToDrop
+     *            number of least significant bits to drop
+     *
+     * @return the value of {@link #bitWorkArea} after dropping the
+     *            specified number of least significant bits
+     *
+     * @throws IllegalArgumentException
+     *            if the bits being dropped contain any non-zero value
+     */
+    private int dropBits(int numBitsToDrop) {
+        if ((bitWorkArea & numBitsToDrop) != 0) {
+            throw new IllegalArgumentException("Last encoded character (before the paddings if any) is a valid base 64 alphabet but not a possible value");
+        }
+        return bitWorkArea >> numBitsToDrop;
+    }
 
     /**
      * Tests a given byte array to see if it contains only valid characters within the Base64 alphabet. Currently the

               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hanson Char updated CODEC-134:
------------------------------

    Attachment: patch.txt

Sorry, should have attached the patch as a file.
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Issue Comment Edited] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222063#comment-13222063 ]

Hanson Char edited comment on CODEC-134 at 3/4/12 11:47 PM:
------------------------------------------------------------

patch.txt attached
               
      was (Author: hchar):
    Sorry, should have attached the patch as a file.
                 

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hanson Char updated CODEC-134:
------------------------------

    Comment: was deleted

(was: diff --git a/src/main/java/org/apache/commons/codec/binary/Base32.java b/src/main/java/org/apache/commons/codec/binary/Base32.java
index a9da10f..9cd293b 100644
--- a/src/main/java/org/apache/commons/codec/binary/Base32.java
+++ b/src/main/java/org/apache/commons/codec/binary/Base32.java
@@ -274,7 +274,7 @@ public class Base32 extends BaseNCodec {
      * @param inPos
      *            Position to start reading data from.
      * @param inAvail
-     *            Amount of bytes available from input for encoding.
+     *            Amount of bytes available from input for decoding.
      *
      * Output is written to {@link #buffer} as 8-bit octets, using {@link #pos} as the buffer position
      */
@@ -320,30 +320,30 @@ public class Base32 extends BaseNCodec {
             //  we ignore partial bytes, i.e. only multiples of 8 count
             switch (modulus) {
                 case 2 : // 10 bits, drop 2 and output one byte
-                    buffer[pos++] = (byte) ((bitWorkArea >> 2) & MASK_8BITS);
+                    buffer[pos++] = (byte) (dropBits(2) & MASK_8BITS);
                     break;
                 case 3 : // 15 bits, drop 7 and output 1 byte
-                    buffer[pos++] = (byte) ((bitWorkArea >> 7) & MASK_8BITS);
+                    buffer[pos++] = (byte) (dropBits(7) & MASK_8BITS);
                     break;
                 case 4 : // 20 bits = 2*8 + 4
-                    bitWorkArea = bitWorkArea >> 4; // drop 4 bits
+                    bitWorkArea = dropBits(4); // drop 4 bits
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
                 case 5 : // 25bits = 3*8 + 1
-                    bitWorkArea = bitWorkArea >> 1;
+                    bitWorkArea = dropBits(1);
                     buffer[pos++] = (byte) ((bitWorkArea >> 16) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
                 case 6 : // 30bits = 3*8 + 6
-                    bitWorkArea = bitWorkArea >> 6;
+                    bitWorkArea = dropBits(6);
                     buffer[pos++] = (byte) ((bitWorkArea >> 16) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
                 case 7 : // 35 = 4*8 +3
-                    bitWorkArea = bitWorkArea >> 3;
+                    bitWorkArea = dropBits(3);
                     buffer[pos++] = (byte) ((bitWorkArea >> 24) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea >> 16) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
@@ -352,6 +352,28 @@ public class Base32 extends BaseNCodec {
             }
         }
     }
+    
+    /**
+     * <p>
+     * Drops the specified number of least significant bits from the
+     * {@link #bitWorkArea}.
+     * </p>
+     *
+     * @param numBitsToDrop
+     *            number of least significant bits to drop
+     *
+     * @return the value of {@link #bitWorkArea} after dropping the
+     *            specified number of least significant bits
+     *
+     * @throws IllegalArgumentException
+     *            if the bits being dropped contain any non-zero value
+     */
+    private long dropBits(int numBitsToDrop) {
+        if ((bitWorkArea & numBitsToDrop) != 0) {
+            throw new IllegalArgumentException("Last encoded character (before the paddings if any) is a valid base 32 alphabet but not a possible value");
+        }
+        return bitWorkArea >> numBitsToDrop;
+    }
 
     /**
      * <p>
diff --git a/src/main/java/org/apache/commons/codec/binary/Base64.java b/src/main/java/org/apache/commons/codec/binary/Base64.java
index 1ee0eba..4261f88 100644
--- a/src/main/java/org/apache/commons/codec/binary/Base64.java
+++ b/src/main/java/org/apache/commons/codec/binary/Base64.java
@@ -410,7 +410,7 @@ public class Base64 extends BaseNCodec {
      * @param inPos
      *            Position to start reading data from.
      * @param inAvail
-     *            Amount of bytes available from input for encoding.
+     *            Amount of bytes available from input for decoding.
      */
     @Override
     void decode(byte[] in, int inPos, int inAvail) {
@@ -455,17 +455,39 @@ public class Base64 extends BaseNCodec {
            //   case 1: // 6 bits - ignore entirely
            //       break;
                 case 2 : // 12 bits = 8 + 4
-                    bitWorkArea = bitWorkArea >> 4; // dump the extra 4 bits
+                    bitWorkArea = dropBits(4); // drop the extra 4 bits
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
                 case 3 : // 18 bits = 8 + 8 + 2
-                    bitWorkArea = bitWorkArea >> 2; // dump 2 bits
+                    bitWorkArea = dropBits(2); // drop 2 bits
                     buffer[pos++] = (byte) ((bitWorkArea >> 8) & MASK_8BITS);
                     buffer[pos++] = (byte) ((bitWorkArea) & MASK_8BITS);
                     break;
             }
         }
     }
+    
+    /**
+     * <p>
+     * Drops the specified number of least significant bits from the
+     * {@link #bitWorkArea}.
+     * </p>
+     *
+     * @param numBitsToDrop
+     *            number of least significant bits to drop
+     *
+     * @return the value of {@link #bitWorkArea} after dropping the
+     *            specified number of least significant bits
+     *
+     * @throws IllegalArgumentException
+     *            if the bits being dropped contain any non-zero value
+     */
+    private int dropBits(int numBitsToDrop) {
+        if ((bitWorkArea & numBitsToDrop) != 0) {
+            throw new IllegalArgumentException("Last encoded character (before the paddings if any) is a valid base 64 alphabet but not a possible value");
+        }
+        return bitWorkArea >> numBitsToDrop;
+    }
 
     /**
      * Tests a given byte array to see if it contains only valid characters within the Base64 alphabet. Currently the
)
   

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222101#comment-13222101 ]

Gary D. Gregory commented on CODEC-134:
---------------------------------------

Hi Hanson,

Thank you for the patch.

Can you provide unit tests with the patch?

Thank you,
Gary
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hanson Char updated CODEC-134:
------------------------------

    Attachment: diff-120304-20.txt
   

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-20.txt, patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222165#comment-13222165 ]

Hanson Char commented on CODEC-134:
-----------------------------------

Hi Gary,

Please find attached diff-120304-20.txt for the unit tests.  Note that I've also modified the value of Base64TestData#CODEC_101_MULTIPLE_OF_3 from "123" to "124", as "123" turns out to be an impossible base64 encoded value with "3" being the last character.

Regards,
Hanson
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-20.txt, patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222167#comment-13222167 ]

Gary D. Gregory commented on CODEC-134:
---------------------------------------

Hi Hanson,

At first glance, changing CODEC_101_MULTIPLE_OF_3 to a value that is NOT a multiple of three does not "smell" right.

Can you find a way to test this without making the constant name "lie"? After all CODEC_101_MULTIPLE_OF_3 must be a multiple of three for a good reason! ;)

There
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-20.txt, patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222172#comment-13222172 ]

Hanson Char commented on CODEC-134:
-----------------------------------

Hi Gary,

I am not sure I understand the meaning of the name "CODEC_101_MULTIPLE_OF_3".  Does it mean a multiple of 3 in terms of the number of bits or bytes ?  If it is bytes, both "123" and "124" contain 3 bytes.  If it is the number of "1" in the UTF-8 bit array, the byte array of "123" contains 10 bits (on) whereas that of "124" contains 9 bits.

In the only two test cases (in Base64InputStreamTest.java) where CODEC_101_MULTIPLE_OF_3 is used, CODEC_101_MULTIPLE_OF_3 appears to be used simply as a straight base 64 encoded string that the test case attempt to decode (via a Base64InputStream).  Since the "123" cannot be a possible base 64 encoded value, the two test cases should fail rather than pass.

What am I missing ?

Regards,
Hanson
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-20.txt, patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hanson Char updated CODEC-134:
------------------------------

    Attachment: diff-120304-22.txt
   

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-20.txt, diff-120304-22.txt, patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222179#comment-13222179 ]

Hanson Char commented on CODEC-134:
-----------------------------------

I've just uploaded the latest patch, diff-120304-22.txt, superseding all previous patches.  This patch passed all the unit tests via "mvn test".
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-22.txt, patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hanson Char updated CODEC-134:
------------------------------

    Attachment:     (was: diff-120304-20.txt)
   

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-22.txt, patch.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hanson Char updated CODEC-134:
------------------------------

    Attachment:     (was: patch.txt)
   

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-22.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CODEC-134) Base32 would decode some invalid Base32 encoded string into arbitrary value

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/CODEC-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222629#comment-13222629 ]

Gary D. Gregory commented on CODEC-134:
---------------------------------------

Hello Hanson,

Now that I took a closer look at Base64TestData, I can see that CODEC_101_MULTIPLE_OF_3 is about input length. So fiddling with it should be OK but I do not recommend it because it is specifically maintained for CODEC-101. Better not to confusing things here IMO.

Can you provide a patch using svn diff please, I cannot apply this patch as-in with Eclipse/SVN.

Thank you,
Gary
               

> Base32 would decode some invalid Base32 encoded string into arbitrary value
> ---------------------------------------------------------------------------
>
>                 Key: CODEC-134
>                 URL: https://issues.apache.org/jira/browse/CODEC-134
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.6
>         Environment: All
>            Reporter: Hanson Char
>              Labels: security
>         Attachments: diff-120304-22.txt
>
>
> Example, there is no byte array value that can be encoded into the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4L===", but the existing Base32 implementation would not reject it but decode it into an arbitrary value which if re-encoded again using the same implementation would result in the string "C5CYMIHWQUUZMKUGZHGEOSJSQDE4K===".
> Instead of blindly decoding the invalid string, the Base32 codec should reject it (eg by throwing IlleglArgumentException) to avoid security exploitation (such as tunneling additional information via seemingly valid base 32 strings).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
12