Programming magic, glory, and juices.

Byte array to hex string and back again

December 1st, 2007


In BitTorrent, info hashs are used when communicating with a tracker. An info hash such as “06042aa75e23b7be9156e981910e4847ec434e19″ is byte-encoded and then url encoded and sent to the tracker via an HTTP request.

http://localhost:2380/scrape?info_hash=%06%04%2A%A7%5E%23%B7%BE%91V%E9%81%91%0EHG%ECCN%19

So the following conversion must be made:

"06042aa75e23b7be9156e981910e4847ec434e19" = 

00000000: 06 04 2A A7 5E 23 B7 BE 91 56 E9 81 91 0E 48 47 [ ..*.^#...V....HG ]
00000010: EC 43 4E 19 00 00 00 00 00 00 00 00 00 00 00 00 [ .CN............. ] = 

%06%04%2A%A7%5E%23%B7%BE%91V%E9%81%91%0EHG%ECCN%19

There are several ways to byte encode and decode a hex string. For example, you could resort to using scanf and printf with “%02x”.. but this can be slow. Using the following preprocessor definitions we can simplify encoding and decoding while achieving the best possible performance. And performance matters, especially when you are going to be encoding and decoding extensively, for example if you were to print out the hex representation of a large memory buffer to the console.

#define BASE16SYM			("0123456789ABCDEF")
#define BASE16VAL			("\x0\x1\x2\x3\x4\x5\x6\x7\x8\x9|||||||\xA\xB\xC\xD\xE\xF")

#define BASE16_ENCODELO(b)		(BASE16SYM[((uint8)(b)) >> 4])
#define BASE16_ENCODEHI(b)		(BASE16SYM[((uint8)(b)) & 0xF])

#define BASE16_DECODELO(b)		(BASE16VAL[Char_Upper(b) - '0'] << 4)
#define BASE16_DECODEHI(b)		(BASE16VAL[Char_Upper(b) - '0'])

Now to decode the hex string, “06042aa75e23b7be9156e981910e4847ec434e19″, to a byte array, we can do the following use the BASE16_DECODE preprocessor definitions as shown in the following example.

TargetPtr = Target;

while (*Source != '\0')
   {
   TargetPtr[0]  = BASE16_DECODELO(Souce[0]);
   TargetPtr[0] |= BASE16_DECODEHI(Souce[1]);

   TargetPtr += 1;
   Source += 2;
   }

*TargetPtr = ‘\0′;

So now we have converted “06042aa75e23b7be9156e981910e4847ec434e19″ to

00000000: 06 04 2A A7 5E 23 B7 BE 91 56 E9 81 91 0E 48 47 [ ..*.^#...V....HG ]
00000010: EC 43 4E 19 00 00 00 00 00 00 00 00 00 00 00 00 [ .CN............. ]

If we wanted to encode the byte array back to a hex string we could use the BASE16_ENCODE preprocessor definitions like so

TargetPtr = Target;

while (*Source != '\0')
   {
   TargetPtr[0] = BASE16_ENCODELO(*Souce);
   TargetPtr[1] = BASE16_ENCODEHI(*Souce);

   TargetPtr += 2;
   Source += 1;
   }

To finish off the exercise we have to url encode the byte encoded array. So for each value that is not printable, that is, “06 04 2A A7 5E 23 B7 BE 91″, “E9 81 91 0E”, and “EC” we convert it to its equivalent hex string value and prepend a percent sign %. We can url encode the info hash using one function.

int32 InfoHash_UrlEncode(char *InfoHash, char *Encoded, int32 MaxEncoded)
{
   char *EncodedPtr, EncodedByte;

   Debug_AssertPointer(InfoHash);
   Debug_AssertPointer(Encoded);

   String_Empty(Encoded);

   EncodedPtr = Encoded;

   while (*InfoHash != '\0')
      {
      EncodedByte  = BASE16_DECODELO(InfoHash[0]);
      EncodedByte |= BASE16_DECODEHI(InfoHash[1]);

      if (Char_IsAlphaNumeric(EncodedByte) == TRUE)
         {
         EncodedPtr[0] = Char_Upper(EncodedByte);
         }
      else
         {
         EncodedPtr[0] = ‘%’;
         EncodedPtr[1] = InfoHash[0];
         EncodedPtr[2] = InfoHash[1];

         EncodedPtr += 2;
         }

      EncodedPtr += 1;
      InfoHash   += 2;
      }

   *EncodedPtr = ‘\0′;

   return TRUE;
}

So calling InfoHash_UrlEncode(”06042aa75e23b7be9156e981910e4847ec434e19″, Encoded, MAXENCODED), we see that Encoded is “%06%04%2A%A7%5E%23%B7%BE%91V%E9%81%91%0EHG%ECCN%19″.

One Response to “Byte array to hex string and back again”
  1. C# Converting hexadecimal string to/from byte array, FAST! Says:

    [...] previously wrote about how to quickly convert a byte array to its hex representation and back again. Now I’ve converted the same source code to C#. For me, String.Format is too slow to do such [...]

Leave a Reply