The Unique Package

Another new package that was introduced in Go 1.23 is called the Unique Package. I've got the release notes here. And if I jump to the Unique Package, it gives some explanation of it, but it's a little complicated.

The new Unique Package provides facilities for canonicalizing values. It's like interning, if you've heard of that phrase before. I'm not going to read the rest of the description here because I think it's quite confusing.

However, somebody in the Go community on Twitter, Joseph, has wrote this amazing blog post that summarizes everything really well so I'm going to walk through this because I think it does a much better job of explaining how this might be useful to you. So interning is a term originally introduced by Lisp - that's super interesting - and it's the process of storing only one copy of a value in memory and sharing a unique reference to instead of allocating multiple copies and wasting memory. For instance, the Go compiler already performs interning on string constants at compile time.

Instead of allocating multiple identical strings if we allocate a single instance of the string and share a reference to it like below so we have hello world which is greet one and we have greet two which is also hello world and we print the memory addresses of those you can see that they've got exactly the same memory address which is useful - it means that we're not wasting memory. Before Go 1.23, turning runtime values was handled by third-party packages.

So let's take a quick look at this - here's a package that somebody wrote that does interning. If we jump into it, it's not that crazy code. What it's basically doing is creating a pool - interesting - it's making a sync.pool and it makes a map and every time that you try and create a string it checks to see this in the map or not and if it is it returns it otherwise it puts a new one into map.

So it's basically using a hash map and saying if the value already exists just give it to me back otherwise create a new one. This is quite simple to implement which is really interesting actually but you can see this is from nine years ago it was originally created which is cool so it's taken nine years to go from idea that someone was working on to making it into the standard library.

So you can use a unique package by its handle type - there's a globally unique identity for any comparable value meaning two handles compare equal exactly if the two values used to create the handle would have also compared equal. That's kind of confusing but there's a really good example below which is why I wanted to use this blog post. So internally the handle type is backed by a concurrent safe map so it's very similar to the implementation we just saw and could you achieve the same behavior with a map of unique values which is a good question.

It seems like you could do that. So how do you use the unique package in Go 1.23? Let's take a look at this example - if you've got an IPv6 address and you say make me one of these and make me another one it will print out addresses are equal if they're exactly the same which this instance should so let's take this example. Let me just switch this to the Go 1.23 branch and we should be able to paste this whole thing in here and run it.

It looks like there's a tiny mistake in this so we can let the author know in a sec but if we take h1 the value and let's run it again and we can see addresses are equal and it prints out the address too so this is working. Let's jump back over here and as well as proving that it works and showing us how to use it I think this is a really nice example too - we can take a look at the performance of doing such a thing.

So if we scroll down there's some really nice benchmarks here and you can see that running benchmarks we can see regardless of the size of string the nanosecond per operation is consistently low regardless of whether performing comparison over a string 10 copies long or a million whereas a non-canonicalized version relative to the size of the string. So this is really interesting - when would you use this? This is going to be something that's going to be really important for people who are trying to squeeze the most out of their Go program.

So for most people I don't think this will be that useful but when you're at a point where you know memory is limited and all the maybe situations where you want to guarantee you're using the exact instance of something this could be very useful. Recommend not starting with this - this is an optimization, this is something that you'll do when you absolutely need to squeeze more memory performance out of your application.

So while this is interesting I would imagine most people won't need to use this and I would recommend not reaching for it immediately until you are certain that you need to optimize here. I'm just going to end by saying a big thank you to Joseph for creating this. Here's his profile - you should go ahead and give him a follow if you found this interesting. He posts really interesting Go content. Thank you for writing this - I always love reading other people's views especially when they go deep into specific packages and I hope you found this video useful and I'll see you in the next one.