hpincket

Gzipping UUIDs

UUIDs are generally uncompressable. This is because their values can be anything and multiple UUIDs generaly don't repeat in a payload. So you may be surprised to learn that a JSON list of UUIDs compresses to 56% of their original size. Here's an example:

[
  "572585559c5c4278b6947144d010dad3",
  "b97ec0750f67468d964519e18000ccf6",
  "c1c69970b9bc4c0a9266a3350a9b2ab1",
  "2eb10805db484db7a196f89882d32142",
  "4b3569aaac154195aaa5ea114eff6566",
  "e253f04c1f414b7291e682ea465e8e19"
]

Now we compress it

>>> import uuid
>>> import gzip
>>> jdata = open('data.json').read()
>>> len(jdata)
216
>>> len(gzip.compress(jdata))
159
>>> 159 / 216
0.7361111111111112

Hmm that's not what I said. What if we repeat the procedure with a list of 20 UUIDs?

58% Not bad. We can of course automate this by comparing data for an increasing number of UUIDs.

The data looks linear, so we can model it which gives us the following slopes.

Format	Additional Bytes / N
Json	36.0
Gzip	20.0

And a data compression ratio of 1.8.

Wait, but how?

You may be wondering how UUIDs could compress to nearly half their size. The answer isn't profound, it just depends on an assumption you probably overlooked:

The string representation of a UUID is wasteful. Each UUID is 32 hex digits. There are 16 possible Hex values (2^4 or 4 bits). And since each character requires at minimum 1 byte, we end up using 32 bytes for 16 bytes of information. So when it comes time to compress the IDs it's no wonder we get compression ratio of nearly 2.

Other articles

Debugging concurrent requests, for simple cases

Mon 31 May 2021
By Harrison Pincket

In technical.

This is a simple strategy for confirming that some values are shared between requests. There must be a more 'correct' way to inspect values in concurrent requests, but I was unable to find it. I normally use a debugger to set breakpoints, but there was no way to set a …
read more
Prolog Exercise: A Min and Max predicate in CLPFD
Wed 05 May 2021
By Harrison Pincket

In technical.

Yesterday I implemented a predicate to compute the min and max of a list. Today I'm doing the same, but now with constraints.

The code looks pretty much the same thanks to the handy min and max constraints.
```
:- use_module(library(clpfd)).

min_and_max([V], A, B) :-
    V #= A, V #= B.
min_and_max …
```
read more
Prolog exercise: Min and Max of a list
Wed 05 May 2021
By Harrison Pincket

In technical.

As a fun exercise I implemented a predicate to calculate the minimum and maximum element of a list.
```
min_and_max([A], A, A).
min_and_max([H|R], N, X):-
    min_and_max(R, RN, RX),
    N is min(H, RN),
    X is max(H, RX).
```
Next I wrote a version with last-call optimization.
```
min_and_max_2 …
```
read more
One thing I miss from Mercury

Sun 02 May 2021
By Harrison Pincket

In technical.

It's been a year since I've written any Mercury (the purely logic/functional programming language), but there's one feature I miss. It's nothing fancy, and certainly one of the less interesting features Mercury has to offer.

The Problem

To understand this particular feature let's look at where other languages fail …
read more
My Experience with Xonsh

Fri 25 December 2020
By Harrison Pincket

In technical.

Since January I've been using Xonsh as my main shell on both my work and personal machines. In this post I will describe what I liked and what I didn't. TLDR; I liked that I found myself scripting more under Xonsh, but it can be slow and inconvenient as a …
read more
Constructing tests for legacy code

Fri 16 October 2020
By Harrison Pincket

In technical.

Every company has legacy code sitting somewhere that isn't well understood. Maybe it was written long ago and the original authors have moved on. Here's a high ROI approach to getting that code tested.

The code I have in mind was written in a time when priorities were different, the …
read more
Using Goatcounter on NearlyFreeSpeech.net

Sat 10 October 2020
By Harrison Pincket

In technical.

Here's some quick and dirty notes on setting up GoatCounter on Nearly Free Speech dot net. There are some rather opaque hoops you need to jump through in order to setup the service. I make no claim that this tutorial is perfect, but hopefully it helps someone else who is …
read more
Hunting for a Link

Fri 22 May 2020
By Harrison Pincket

In technical.

There you are, browsing the web, maybe using a popular search engine. You enter your query and scan the results for a title that answers your question. There it is! You see it, the third result. You move your mouse to click.

And what's that? what's happening? You've clicked on …
read more
Python string concatenation

Sat 30 November 2019
By Harrison Pincket

In technical.

I spent a little bit of time looking into Python string concatenation times. I learned that newer versions of Python implement an optimzation for the plus equals operator. In the future I'd like to know what this optimization is. It seemed simple enough to escape by introducing an intermediate variable …
read more

Page 1 / 4 »

Other articles

The Problem

social