Recently I have been pondering a fact that is a bit unsettling. If you maintain a library, and add an exported method to one of your exported structs, you would expect all your library users to still be compatible with this API change. In terms of semantic versioning, this would be a minor version bump:
Given a version number MAJOR.MINOR.PATCH, increment the:
- MAJOR version when you make incompatible API changes,
- MINOR version when you add functionality in a backwards compatible manner, and
- PATCH version when you make backwards compatible bug fixes.
After all, adding new features is usually backwards compatible, and as long as you don’t change the existing API, simply upgrading to the library (and not making any other change) should have no effect.
This is not true in Go, and in some cases it can lead to real bugs.
Let’s define what we mean by backwards compatibility. The Atlassian REST API policy has a sensible definition:
An API is Backwards Compatible if a program written against one version of that API will continue
to work the same way, without modification, against future versions of the API.
In terms of semver, this means that you can always upgrade a library from say
v5.6.1, without fear of breakage. As long as you stay on the same major
Embedding structs breaks modularity
Image you are the maintainer of a library
car, currently at version
There is a downstream app using it like this, which embeds
Car and serializes it to
Running this will output:
Now you decide to add a
MarshalJSON() method to
Car. Let’s start simple: the JSON marshaler you add will produce the same JSON as before.:
Quiz question: what will the app output, once it upgrades to
If you are like me, you would expect the output to be the same, and embark on an epic bug search to figure out why your program broke by just doing a minor library upgrade. The output the app produces now however is:
Note how the
Weight field is suddenly missing. Since
Car, it now also
json.Marshaler, which is used by
json.Marshal() for serialization.
This is a breaking change, as the app might be using the JSON serialization as input to some database which enforces the previous schema, send it over the wire in a strictly specified protocol, or expose it in an API of its own.
Since the app did not change a single line of code except for upgrading the dependency, we must
conclude that the
car library should actually have been a major version bump to
“But wait!”, you say. “Obviously I would not add a
MarshalJSON method that doesn’t actually change
the JSON output, like it is done in this contrived example. I would only add it if I wanted to
change the default serialization, and then it would obviously be a breaking change!”. This could be
argued, seeing that
json.Marshaler is in the standard library and
MarshalJSON is widely used, so
you would expect breaking changes if you change the default JSON serialization. When I was
confronted with a bug of this sort, the library author did not consider it, understandably, as it
can be a bit unintuitive. But fair enough, mistakes happen.
json.Marshaler is simply an interface, and the same issue can happen with any interface,
even those only defined in the app, which the library author can’t know anything about.
Interface in Go (i.e. duck-typing) breaks modularity
Let’s revisit the same example, but using a different interface. Say the app prefers flying over driving:
myRide could be of a number of different types defined in the app, which all do or do not
You can already imagine where this is going: if you as a library author think it would be a nice feature to make the DeLorean fly, you would add:
Running the app now after upgrading to
v1.2.0 results in a panic:
DeLorean is flying panic: I am a bug! [...]
The app again, by just upgrading the library to the next minor version, shows a breaking change.
Neither did the author of the library
car expect that simply adding a method to a struct could be
a breaking change for a library user, nor did the library user expect that upgrading a dependency
while staying on the same major version would break their program.
In duck-typed languages like Go, adding an exported method to an exported struct should, strictly speaking, always result in a major version bump, as it is always potentially a breaking change for downstream users. This applies to many other duck-typed or dynamic languages, such as Python, JS, Ruby.
This is highly unintuitive, and as far as I can tell, most of the time such additions only result in minor version bumps.
Duck-typing and semantic versioning seem fundamentally incompatible. Semver is all about avoiding dependency hell by defining which versions of a library are compatible. In duck-typing however, any addition can be a breaking change.
In Go, there is an additional pitfall in the semantics of interfaces in combination with embedded structs. Based on this, I recommend to avoid embedding structs in general, especially for imported packages, and to always be explicit about which methods and fields you access. This way, additions to the embedded structs are much less likely to inadvertently change the behavior of your program.
In the above example, this would mean:
For package maintainers, I recommend to bump the major version when adding methods that implement well-known interfaces from the stdlib or from popular non-stdlib packages.