Blog

Go modules – Go Dependency Management

25 Nov, 2018
Xebia Background Header Wave

Dependency management is Go was always problematic. There are several strategies and tools available that help us manage our codebase and dependencies, but no single solution really hit the mark. Go modules (GM) tries to solve this problem once and for all. Lets take a look!

GOPATH Dependency Management

GOPAH dependency management is how we traditionally add dependencies in Go. We download a snapshot of source code to ‘GOPATH’ with the ‘go get’ command and compile our main program. When we upgrade a library with ‘go get -u’, our codebase could break. A reason could be a transitive dependency that is incompatible with a library that our application depends on.
For example, our program depends on ‘google/uuid’, but the AWS SDK for Go also has a dependency on ‘google/uuid’ but a different version. When these libraries are incompatible, our code base is broken. This problem could arise when our program previously worked, but after an upgrade of a library, things are broken. Since we have a single ‘GOPATH’, there is no simple way to fix this problem.

Alternatives Approaches

Alternative approaches to dependency management were tried eg. goven, godep, glide, gb, govendor, dep and the vendor directory. Unfortunately, the results of the strategies vary and what is worse, split the Go community into several dependency management strategies and tools.

Go Modules

Go modules (GM), provided by Go, tries to solve Go dependency management once and for all. GM introducing new concepts like a Module, the go.mod file, semantic version selection and a new set of commands that have been added to the Go tool.

Working outside GOPATH

The most profound change to GM is that modules can exists outside of the GOPATH. With GM you can put your modules anywhere on the file system. On my system I keep all my projects in the ‘~/projects’ directory. With GM I can create a project directory and create a Go module in that directory. With GM, Go will download all dependencies, compile, test and install the module to a static binary.

Hello World Module

Lets create a Hello World application:

$ pwd
cd ~/projects
$ mkdir hello-gm
$ cd hello-gm
# initialize a new module
$ go mod init github.com/dnvriend/hello-gm
go: creating new go.mod: module github.com/dnvriend/hello-gm
$ ls
go.mod

The go mod init command created a single file called go.mod in the root of the directory that contains a single line:

module github.com/dnvriend/hello-gm

Because we have a go.mod file in the root of our project, we have created our first GM called ‘github.com/dnvriend/hello-gm’.

Implementing Main

Lets create a file main.go in the root of the project folder.

package main

import (
    "fmt"
    "rsc.io/quote"
)

func main() {
    fmt.Println(quote.Hello())
}

Lets run it:

$ go run main.go
go: finding rsc.io/quote v1.5.2
go: downloading rsc.io/quote v1.5.2
go: finding rsc.io/sampler v1.3.0
go: finding golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
go: downloading rsc.io/sampler v1.3.0
go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
Hello, world.

Go has downloaded the dependencies and run the example. Lets examine what has happened.

go.mod file

Lets examine what has happened in the directory:

.
├── go.mod
├── go.sum
└── main.go

We have the following files:

  • main.go: contains the hello world example
  • go.mod: defines the module and dependencies and contains enough information for reproducible builds
  • go.sum: a new file. The go.sum contains checksums and is for module validation purposes only

helloworld go.mod file:

module github.com/dnvriend/hello-gm

require rsc.io/quote v1.5.2

Go automatically added a dependency to the rsc.io/quote module. Go examined the go.mod file of quote and detected other dependencies.

rsc.io/quote go.mod file:

module rsc.io/quote

require (
    rsc.io/quote/v3 v3.0.0
    rsc.io/sampler v1.3.0
)

Go downloaded all the dependencies to the Module Cache directory, updated the go.mod file of ‘hello world’ and ran the example. Of course we can manually edit the go.mod file and put a version there that we need.

Listing all dependencies

We can get a list of all dependencies by typing:

$ go list -m  all
github.com/dnvriend/hello-gm
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
rsc.io/quote v1.5.2
rsc.io/sampler v1.3.0

Modules Cache Directory

The cache directory for GM is $GOPATH/pkg/mod. When you remove the directory, Go has to download and build all dependencies again.

Hello Buckets

Lets create a new app called ‘hello-buckets’ that will show all the S3 buckets in our account.

$ cd ~/projects
$ mkdir hello-buckets
$ cd hello-buckets
$ go mod init com.github/dnvriend/hello-buckets

Lets add the necessary dependencies:

$ go get github.com/aws/aws-sdk-go
go: finding github.com/aws/aws-sdk-go v1.15.82
go: downloading github.com/aws/aws-sdk-go v1.15.82
$ go get github.com/olekukonko/tablewriter
go: finding github.com/mattn/go-runewidth v0.0.3
go: downloading github.com/mattn/go-runewidth v0.0.3

Lets examine the hello-buckets module:

go.mod:

$ cat go.mod
module com.github/dnvriend/hello-buckets

require (
    github.com/aws/aws-sdk-go v1.15.82 // indirect
   github.com/mattn/go-runewidth v0.0.3 // indirect
   github.com/olekukonko/tablewriter v0.0.1 // indirect
)

Implementing Main

Lets create a file main.go in the root of the project folder.

package main

import (
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
    "github.com/olekukonko/tablewriter"
    "os"
)

func main() {
    sess, _ := session.NewSession()
    svc := s3.New(sess, aws.NewConfig().WithRegion("eu-west-1"))
    request := s3.ListBucketsInput{}
    resp, _ := svc.ListBuckets(&request)

    table := tablewriter.NewWriter(os.Stdout)
    table.SetHeader([]string{"Name", "CreationDate"})
    for _, bucket := range resp.Buckets {
        date := *bucket.CreationDate
        table.Append([]string{*bucket.Name, date.String()})
    }
    table.Render()
}

Lets run the example:

$ go run main.go
+-------------------------------------------------+-------------------------------+
|                      NAME                       |         CREATIONDATE          |
+-------------------------------------------------+-------------------------------+
| aws-athena-query-results-612483924670-eu-west-1 | 2018-10-01 20:02:38 +0000 UTC |
| aws-athena-query-results-eu-west-1-612483924670 | 2018-10-01 19:58:47 +0000 UTC |
| cf-templates-1vt4xsan6d30f-eu-west-1            | 2018-10-24 10:12:26 +0000 UTC |
| dennisvriend                                    | 2018-08-15 19:47:35 +0000 UTC |
| dennisvriend.com                                | 2018-06-04 18:56:29 +0000 UTC |
| dennisvriend.io                                 | 2018-06-27 17:15:10 +0000 UTC |
| dennisvriend.nl                                 | 2018-06-05 09:30:11 +0000 UTC |
| dnvriend-awslabs                                | 2018-08-15 17:12:37 +0000 UTC |
| dnvriend-data                                   | 2018-07-08 09:58:10 +0000 UTC |
| dnvriend.com                                    | 2018-06-05 09:30:24 +0000 UTC |
| dnvriend.nl                                     | 2018-06-05 09:30:33 +0000 UTC |
| elasticbeanstalk-eu-west-1-612483924670         | 2018-07-02 06:32:18 +0000 UTC |
| www.dennisvriend.com                            | 2018-09-15 17:46:49 +0000 UTC |
| www.dnvriend.nl                                 | 2018-09-15 17:47:05 +0000 UTC |
+-------------------------------------------------+-------------------------------+

Conclusion

Go modules makes Go dependency management really easy. The go.mod file reminds me of other dependency systems like build.sbt of the Scala Build Tool (SBT) where you specify ‘libraryDependencies’ by name and version. Go v1.11 is integrated with the module system. The commands ‘go get’ and ‘go mod’ detect when it is working in a module directory and download dependent modules. Go Modules also allows me to work outside of the GOPATH so I can create modules in project directories like with ‘hello-gm’ and ‘hello-buckets.’

Questions?

Get in touch with us to learn more about the subject and related solutions

Explore related posts