Dependency management is Go was always problematic. There are several strategies and tools available that help us manage our codebase and dependencies, but no single solution really hit the mark. Go modules (GM) tries to solve this problem once and for all. Lets take a look!
GOPATH Dependency Management
GOPAH dependency management is how we traditionally add dependencies in Go. We download a snapshot of source code to ‘GOPATH’ with the ‘go get’ command and compile our main program. When we upgrade a library with ‘go get -u’, our codebase could break. A reason could be a transitive dependency that is incompatible with a library that our application depends on.
For example, our program depends on ‘google/uuid’, but the AWS SDK for Go also has a dependency on ‘google/uuid’ but a different version. When these libraries are incompatible, our code base is broken. This problem could arise when our program previously worked, but after an upgrade of a library, things are broken. Since we have a single ‘GOPATH’, there is no simple way to fix this problem.
Alternatives Approaches
Alternative approaches to dependency management were tried eg. goven, godep, glide, gb, govendor, dep and the vendor directory. Unfortunately, the results of the strategies vary and what is worse, split the Go community into several dependency management strategies and tools.
Go Modules
Go modules (GM), provided by Go, tries to solve Go dependency management once and for all. GM introducing new concepts like a Module, the go.mod file, semantic version selection and a new set of commands that have been added to the Go tool.
Working outside GOPATH
The most profound change to GM is that modules can exists outside of the GOPATH. With GM you can put your modules anywhere on the file system. On my system I keep all my projects in the ‘~/projects’ directory. With GM I can create a project directory and create a Go module in that directory. With GM, Go will download all dependencies, compile, test and install the module to a static binary.
Hello World Module
Lets create a Hello World application:
$ pwd
cd ~/projects
$ mkdir hello-gm
$ cd hello-gm
# initialize a new module
$ go mod init github.com/dnvriend/hello-gm
go: creating new go.mod: module github.com/dnvriend/hello-gm
$ ls
go.mod
The go mod init
command created a single file called go.mod
in the root of the directory that contains a single line:
module github.com/dnvriend/hello-gm
Because we have a go.mod
file in the root of our project, we have created our first GM called ‘github.com/dnvriend/hello-gm’.
Implementing Main
Lets create a file main.go in the root of the project folder.
package main
import (
"fmt"
"rsc.io/quote"
)
func main() {
fmt.Println(quote.Hello())
}
Lets run it:
$ go run main.go
go: finding rsc.io/quote v1.5.2
go: downloading rsc.io/quote v1.5.2
go: finding rsc.io/sampler v1.3.0
go: finding golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
go: downloading rsc.io/sampler v1.3.0
go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
Hello, world.
Go has downloaded the dependencies and run the example. Lets examine what has happened.
go.mod file
Lets examine what has happened in the directory:
.
├── go.mod
├── go.sum
└── main.go
We have the following files:
- main.go: contains the hello world example
- go.mod: defines the module and dependencies and contains enough information for reproducible builds
- go.sum: a new file. The go.sum contains checksums and is for module validation purposes only
helloworld go.mod file:
module github.com/dnvriend/hello-gm
require rsc.io/quote v1.5.2
Go automatically added a dependency to the rsc.io/quote module. Go examined the go.mod file of quote and detected other dependencies.
rsc.io/quote go.mod file:
module rsc.io/quote
require (
rsc.io/quote/v3 v3.0.0
rsc.io/sampler v1.3.0
)
Go downloaded all the dependencies to the Module Cache directory, updated the go.mod
file of ‘hello world’ and ran the example. Of course we can manually edit the go.mod
file and put a version there that we need.
Listing all dependencies
We can get a list of all dependencies by typing:
$ go list -m all
github.com/dnvriend/hello-gm
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
rsc.io/quote v1.5.2
rsc.io/sampler v1.3.0
Modules Cache Directory
The cache directory for GM is $GOPATH/pkg/mod
. When you remove the directory, Go has to download and build all dependencies again.
Hello Buckets
Lets create a new app called ‘hello-buckets’ that will show all the S3 buckets in our account.
$ cd ~/projects
$ mkdir hello-buckets
$ cd hello-buckets
$ go mod init com.github/dnvriend/hello-buckets
Lets add the necessary dependencies:
$ go get github.com/aws/aws-sdk-go
go: finding github.com/aws/aws-sdk-go v1.15.82
go: downloading github.com/aws/aws-sdk-go v1.15.82
$ go get github.com/olekukonko/tablewriter
go: finding github.com/mattn/go-runewidth v0.0.3
go: downloading github.com/mattn/go-runewidth v0.0.3
Lets examine the hello-buckets
module:
go.mod:
$ cat go.mod
module com.github/dnvriend/hello-buckets
require (
github.com/aws/aws-sdk-go v1.15.82 // indirect
github.com/mattn/go-runewidth v0.0.3 // indirect
github.com/olekukonko/tablewriter v0.0.1 // indirect
)
Implementing Main
Lets create a file main.go in the root of the project folder.
package main
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/olekukonko/tablewriter"
"os"
)
func main() {
sess, _ := session.NewSession()
svc := s3.New(sess, aws.NewConfig().WithRegion("eu-west-1"))
request := s3.ListBucketsInput{}
resp, _ := svc.ListBuckets(&request)
table := tablewriter.NewWriter(os.Stdout)
table.SetHeader([]string{"Name", "CreationDate"})
for _, bucket := range resp.Buckets {
date := *bucket.CreationDate
table.Append([]string{*bucket.Name, date.String()})
}
table.Render()
}
Lets run the example:
$ go run main.go
+-------------------------------------------------+-------------------------------+
| NAME | CREATIONDATE |
+-------------------------------------------------+-------------------------------+
| aws-athena-query-results-612483924670-eu-west-1 | 2018-10-01 20:02:38 +0000 UTC |
| aws-athena-query-results-eu-west-1-612483924670 | 2018-10-01 19:58:47 +0000 UTC |
| cf-templates-1vt4xsan6d30f-eu-west-1 | 2018-10-24 10:12:26 +0000 UTC |
| dennisvriend | 2018-08-15 19:47:35 +0000 UTC |
| dennisvriend.com | 2018-06-04 18:56:29 +0000 UTC |
| dennisvriend.io | 2018-06-27 17:15:10 +0000 UTC |
| dennisvriend.nl | 2018-06-05 09:30:11 +0000 UTC |
| dnvriend-awslabs | 2018-08-15 17:12:37 +0000 UTC |
| dnvriend-data | 2018-07-08 09:58:10 +0000 UTC |
| dnvriend.com | 2018-06-05 09:30:24 +0000 UTC |
| dnvriend.nl | 2018-06-05 09:30:33 +0000 UTC |
| elasticbeanstalk-eu-west-1-612483924670 | 2018-07-02 06:32:18 +0000 UTC |
| www.dennisvriend.com | 2018-09-15 17:46:49 +0000 UTC |
| www.dnvriend.nl | 2018-09-15 17:47:05 +0000 UTC |
+-------------------------------------------------+-------------------------------+
Conclusion
Go modules makes Go dependency management really easy. The go.mod
file reminds me of other dependency systems like build.sbt
of the Scala Build Tool (SBT) where you specify ‘libraryDependencies’ by name and version. Go v1.11 is integrated with the module system. The commands ‘go get’ and ‘go mod’ detect when it is working in a module directory and download dependent modules. Go Modules also allows me to work outside of the GOPATH so I can create modules in project directories like with ‘hello-gm’ and ‘hello-buckets.’