From: Alexander Zangerl <exmh@bin.snafu.priv.at>
Date: Thu, 14 Jun 2012 12:31:40 +0000 (-0500)
Subject: The base64-decoder doesn't work on 64-bit big-endian architectures:
X-Git-Url: http://git.marmaro.de/?a=commitdiff_plain;h=436e6b477fa6dabe40404c84334e148b49682708;p=mmh

The base64-decoder doesn't work on 64-bit big-endian architectures:
mhstore and co. write files of the correct length but which consist
only of \0s.

The culprit is uip/mhparse.c, which contains a pretty ugly base64
decoder (in two places). that thing allocates a "long int" as a
container for the 24 bits that you get from four base-64 chars. the
container is then accessed both as a single entity and as a byte
array, which naturally depends on how wide the long int is and in what
order things end up in there.

The code distinguishes between big- and little-endian systems but
wrongly assumes that sizeof(long int) == 4, which isn't universally
true (quel surprise...).

The attached patch fixes the issue, but in the long run we should
insist on posix and use a64l().
---

diff --git a/uip/mhparse.c b/uip/mhparse.c
index 4c3a194..91cc411 100644
--- a/uip/mhparse.c
+++ b/uip/mhparse.c
@@ -1737,10 +1737,15 @@ openBase64 (CT ct, char **file)
     CE ce;
     MD5_CTX mdContext;
 
+    /* the decoder works on the least-significant three bytes of the bits integer,
+       but their position in memory depend on both endian-ness and size of 
+       long int... for little-endian architectures the size is irrelevant, for
+       big-endian archs it's crucial... ideally we'd adopt posix and use a64l instead
+       of this mess. */
     b  = (unsigned char *) &bits;
-    b1 = &b[endian > 0 ? 1 : 2];
-    b2 = &b[endian > 0 ? 2 : 1];
-    b3 = &b[endian > 0 ? 3 : 0];
+    b1 = &b[endian > 0 ? sizeof(bits)==8?5:1 : 2];
+    b2 = &b[endian > 0 ? sizeof(bits)==8?6:2 : 1];
+    b3 = &b[endian > 0 ? sizeof(bits)==8?7:3 : 0];
 
     ce = ct->c_cefile;
     if (ce->ce_fp) {
@@ -2825,10 +2830,16 @@ readDigest (CT ct, char *cp)
     unsigned char *dp, value, *ep;
     unsigned char *b, *b1, *b2, *b3;
 
-    b  = (unsigned char *) &bits,
-    b1 = &b[endian > 0 ? 1 : 2],
-    b2 = &b[endian > 0 ? 2 : 1],
-    b3 = &b[endian > 0 ? 3 : 0];
+    /* the decoder works on the least-significant three bytes of the bits integer,
+       but their position in memory depend on both endian-ness and size of 
+       long int... for little-endian architectures the size is irrelevant, for
+       big-endian archs it's crucial... ideally we'd adopt posix and use a64l instead
+       of this mess. */
+    b  = (unsigned char *) &bits;
+    b1 = &b[endian > 0 ? sizeof(bits)==8?5:1 : 2];
+    b2 = &b[endian > 0 ? sizeof(bits)==8?6:2 : 1];
+    b3 = &b[endian > 0 ? sizeof(bits)==8?7:3 : 0];
+
     bitno = 18;
     bits = 0L;
     skip = 0;