A Beginners Guide to Sensible Abstractions using Golang

Like a whole host of other articles on the subject, this article attempts to showcase a simply, easy to understand structure for a Golang web server that I use for all my projects, with a particular focus on how to implement sensible abstractions.

The article is intended for beginners, and there are much deeper pieces of literature on this topic. The intention of this article is to help those writing their first Go programs avoid the pile of spaghetti that can result from not using interfaces correctly.

Furthermore, I’ll attempt to show not only how one should apply “Clean Architecture” or “Hexagonal Architecture”, but also why these things (which boil down to just sensible abstractions, lets be honest!) can make your life a lot easier when it comes to extending, modifying and testing your code.

I’ll start with some diagrams on what these abstractions look like and how they depend on each other, followed by a smattering of code. Needless to say, these patterns are generic and in no way Go specific. In fact, the book that I base all my modern programs structure upon: “Clean Architecture” by Rob. C. Martin, discusses these principles in the context of Java programs. However this article deals with applying them to a Go program.

Layers of a Server

Imagine we need to write a program in Go that does the following.

Receive a HTTP request asking for a Customers information, and respond with that information.

We can effectively imagine it to be a “customer-gateway”, that perhaps drives the customer portal part of the website. It might one day aggregate some information from several backend services and feed it to the website.

To begin lets assume our application is simple. It has a database with a load of customers information in it. Therefore our initial task is trivial: receive the request, reach into the database to find the customers’ information and respond with it. Easy!

Applying my usual abstractions, I’ll end up with the following packages, which I’ll preemptively describe below:

.
├── cmd
│   └── customer-gateway
├── entities
│   └── customer.go
├── entrypoints
│   └── http
│       └── get_customer.go
├── repositories
│   └── customers
│       ├── get_customer.go
│       └── repository.go
└── usecases
    ├── get_customer.go
    └── interfaces.go

• The entities are our business domain objects. These won’t dependent on anything else, these are going to be returned from the repositories to the usecases (more on that later).
• repositories deal with the boring bits of our program. They could be retrieving things from a database, an external API, and in memory map, it really doesn’t matter. There are two crucial things a repository must do: It must return an entity when called, and it can’t be a leaky abstraction. It must completely contain the data source it communicates with.
• usecases is our business logic. It will be called by the entrypoints to do something interesting. It calls the methods in the repositories to get data, and does something interesting with it that provides value.
• entrypoints are “triggers” for the program. In our case we’ve only got HTTP, but this could also be a CLI, TCP connection etc. It simply kicks off the usecase, and returns data once done.

Dependencies

Our graph of dependencies is shown below. You can think of “dependencies” as anything defined in the “import” section of your packages.

                    +--------------+
                    |              |
                    | entities     |
                    |              |
                    +--------------+
        +-------------^    ^     ^------------+
        |                  |                  |
        |                  |                  |
        |                  |                  |
        |                  |                  |
 +--------------+   +--------------+   +--------------+
 |              |   |              |   |              |
 | entrypoints  |   | repositories |   |  usecases    |
 |              |   |              |   |              |
 +--------------+   +--------------+   +--------------+

Notice that none of our packages dependent on each other apart from the entities. They will all be loosely coupled through the use of interfaces. Lets take a look at some code, and then talk about why we might want this.

Please note that all the following code won’t actually run! It’s mainly to show intent.

Our `repositories/customers` code might look something like this.

repositories/customers/repository.go

repositories/customers/get_customer.go

Nothing particularly groundbreaking here. The most important takeaway here is that the GetCustomer method returns an entity, not its own customerData type. Why? So that our business logic in the usecases package is not tightly coupled to the database. Lets see this in action in the usecase.

Firstly, lets quickly look at the entity customer. Nothing to see here; its a simple struct for now.

entities/customer.go

And now lets take a look at the “Business Logic” or usecase.

usecases/get_customer.go

A few takeaways in this small file, so lets go through it.

1. The CustomerGetter interface provides an abstraction for the customer repository we’ve just seen. Notice that the interfaces are declared in the consumer not the provider. This is of huge importance. Defining the interface in the consumer means the two packages are completely decoupled thanks to Go’s implicit satisfaction of interfaces. I would probably propose that as a general rule: Interfaces should be defined in the consumer, not the provider.
2. The Database repository will be injected as an interface. When the usecase is called, in turn it simply calls the repository, which gets the data from the database and returns the results. Nice and simple. So simple in fact, that we actually don’t have any business logic so far!

The interaction between these three packages is at the heart of this article, and implementing this interface poorly is one of the most common mistakes I see developers new to Go make. Lets go over why this dependency inversion is so important.

Extendability

A requirement has come through for the “number of bookings” that a given customer has made to be shown on our customer portal. To get this information, our customer gateway program must contact a different, internal service that manages bookings. The “Bookings Service” handily provides an API we can call. We can imagine our newly desired architecture to look like this.

Thanks to our original abstraction, integrating with the booking service becomes easy. We can take the following steps.

1. We add some code for retrieving a booking from the bookings API. After adding the new repository, our new file tree looks like this. Note that I’ve not added a separate bookings/repository.go file for brevity.

.
├── cmd
│   └── customer-gateway
├── entities
│   └── customer.go
├── entrypoints
│   └── http
│       └── get_customer.go
├── go.mod
├── repositories
│   ├── bookings
│   │   └── count_customer_bookings.go
│   └── customers
│       ├── get_customer.go
│       └── repository.go
└── usecases
    ├── get_customer.go
    └── interfaces.go

The implementation is drafted below in what is effectively pseudocode.

repositories/bookings/count_customer_bookings.go

Nothing particularly interesting to see here. However, extending our business logic in the `usecases` to combine the two data sources is trivially easy.

usecases/get_customer.go after adding the new data source

We simply define a new interface, pass in something that satisfies that interface and call its one defined method, integrating the results with those of the previous customers call.

Notice that all our changes were additive; we didn’t really have to modify anything. The Customer entity was updated with a new field, which I haven’t shown.

Open to Modification

Our new block diagram looks a bit wonky doesn’t it? Our customer-gateway service has its own database, but also relies on the bookings gateway? In fact what happens if another internal service wants to communicate with the customer-gateway? They’ll have to bypass our frontend-facing authentication! We sit down the engineering team and decide its time to move to the following architecture.

Decoupling the architecture

That’s more like it! Our customer gateway is purely responsible for authentication to the frontend and aggregating results from the two internal only systems. This opens up the customer service for use with other internal systems, just like we used the booking service. What does it mean for our customer gateway however? Thankfully, we only have to touch the customer repository, changing it to retrieve results from an API, rather than a database. Our usecase can remain exactly the same.

This should conceptually make sense. Only the source of our data has changed, our business logic hasn’t. We still gather customer information and serve it to the frontend. Thanks to our handy abstraction, the rest of the program can stay the same.

Testing

Having your business logic tests depend on a database being present is completely undesirable. It makes tests rigid, inflexible and slow. Additionally, your business logic shouldn’t care about a database. Its a trivial concern for it. Your business logic layer shouldn’t care where the data comes from. Thanks to our prior abstractions, we can simply mock the repository layer in our tests, and focus on the business logic. Take a look at the test file below, using gomock for the mocking.

potential draft for usecases/get_customer_test.go

Notice how our tests are completely independent of concerns like the database or API? We can test each layer of the application in isolation.

Conclusion

To conclude, I think there are four main points that this article can be boiled down to.

1. Define interfaces in the consumer, not the provider.

This helps to completely decouple the two. In the example above, imagine the usecase as saying

“I don’t care who you are but I know you have this methods. I’ll call them, and return to me either my own types, or a higher level package”.

By doing this we ensure our business logic is not dependent on lower level concerns like databases and HTTP. If the interface is defined in the provider, then the consumer must still import the provider. However thanks to Go’s implicit interface implementation, defining the interface in the consumer means that the two packages need not import each other at all.

2. Mock the interfaces in tests

Without this, point 1 is completely moot, as your business logic is dependent on a database in the tests. By mocking at every layer, our tests stay flexible.

3. Have lower level abstractions return higher level objects

Just as in our example the repositories returned the entities. If they return their own types, then the decoupling hasn’t succeeded. In fact, looking at the import section of your packages is a good litmus test for how tightly coupled they are.

4. Bring it all together in the func main()

“That’s all well and good” I hear you say, “but where does the rubber meet the road? Where do I actually inject the repositories into the business logic?”

The answer: in the main()! The main function is the messiest place of the program. The config is read, HTTP clients are instantiated, database connections are started, repositories are injected into usecases, usecases are injected into entrypoints etc. All the gory steps of actually starting your program should take place in and around the main. Once all layers of the application are set up, the application can start.

I hope you enjoyed this article and it helps to justify why good abstractions will make your life a lot easier. Having worked on projects before where “abtraction is pointless” and “we won’t swap these dependencies” and the absolute plate of spagetti that results from it, this is a topic that’s quite close to my heart. Its cheap, makes your life easier, and is 1000% easier to go from abstractions to concretions than concretions to abstractions so just do it from the get go would be my advice!