Learning Go

In my blog AWS Lambda with Golang we created an AWS Lambda that executed Go code. We created, packaged and ran the Lambda. This time will take a look at the Go Programming Language and some of the features that the Go standard library provides. That way we get a good understanding of what is provided by Go and why the language is so popular.
In this blog I assume that the reader as experience programming in other programming languages like Java or Python, so I will only show small examples without explaining them too much. Lets take a look!

A Popular Language

A lot of well known open source projects are programmed in Go like Docker, Kubernetes, Etcd, Influx DB, Traefik. Go is supported by Amazon Web Services (AWS) and Google Cloud Platform (GCP) to create web applications and in case of AWS, lambda functions.

History

Go is a programming language designed by Google. Version 1 was released in March 2012. At the time of writing, the latest release of Go is v1.11, released in August 2018. Go is a statically typed, compiled language. The compiler generates a static binary that can be run without the need for a virtual machine. Like other managed languages like Java and c-sharp, Go has the benefits of memory safety and garbage collection. Go is created by Robert Griesemer, Rob Pike, and Ken Thompson.

Installing Go

Installing Go is easy on Mac or Linux. On the Mac just type ‘brew install go’. On Linux, depending on the distribution type ‘sudo yum install golang.x86_64’.

There are a lot of resources to learn Go. The Tour of Go provides a online tour of the language. The documentation of Go is one of the best I’ve seen and is very accessible. There are lot of books to learn Go like Go By Example and An introduction to programming in Go. There are plenty of free Youtube videos like GopherCon.

The Go CLI

The Go programming language has a single CLI command called ‘go’. The following commands are important:

go version: prints the version of go and exits
go run: run a program
go build: compile the program to static binary
go fmt: formats the Go source files
go env: print the Go environment variables and exits
go get: download and install source files
go test: run tests

$ go version
go version go1.11.2 darwin/amd64

# get help for a command
$ go help get

$ go env GOPATH
/Users/dennis/go

The Go Workspace

Go uses workspaces to aggregate codebases. A workspace is a directory where the ‘GOPATH’ points to. By default the ‘GODIR’ points to ‘~/go’. By typing ‘go env GOPATH’ you get see what the current workspace is. You can change the default workspace by setting ‘GOPATH’ to a different directory by means of environment variables.
The Go workspace is an aggregation of codebases. This means that the workspace can contain a single project or multiple projects. The result of a build on the workspace is a binary artifact. The binary artifact most often is the aggregation of the code bases from the workspace. This means a single dependency graph that is available in the workspace. Create multiple workspaces when you need to separate code bases from each other ie. a different dependency graph.
For this blog we will be using the default workspace that points to ‘~/go’. A workspace has the following directories:

.
├── bin
└── src
    └── github.com
        └── dnvriend

You put your source files in the ‘src/github.com/dnvriend/blog-examples’. The ‘blog-examples’ directory contains a ‘.git’ directory and is uploaded to github. Binary artifacts are stored in the ‘bin’ directory.

Hello World

We need to start with a Hello World example. Create the file ‘src/github.com/dnvriend/blog-examples/main.go’ and add the following code. To run the example type ‘go run src/github.com/dnvriend/blog-examples/main.go’. To create a static binary type ‘go install src/github.com/dnvriend/blog-examples/main.go’.

package main

import (
    "fmt"
    "log"
)

func main() {
    log.Println("Hello World!") // write to stdlog
   fmt.Println("Hello World!") // write to stdout
   println("Hello World!") // write to stderr
}

Primitive Types

Go supports the following primitive types:

boolean
int, int8, int16, int32, int64
float32, float64
byte
complex64, complex128
string

Lets see how to use these types:

package main

import (
    "fmt"
)

func main() {
    const a bool = true
    const b int = 42
    const c float32 = 3.1415
    const d string = "Dennis"
    const e complex64 = 12 + 5i
    fmt.Println(a, b, c, d, e)
}

There is a shorter notation that only works in functions:

package main

import (
    "fmt"
)

func main() {
    a := true
    b := 42
    c := 3.1415
    d := "Dennis"
    e := 12 + 5i
    fmt.Println(a, b, c, d, e)
}

Functions

Functions use the ‘func’ keyword and work as you would expect.

package main

import (
    "fmt"
)

// a function
func world() string {
    return "World"
}

// higher order function
func hello(f func() string) string {
    // apply the function 'f'
   return "Hello " + f() + "!"
}

func main() {
    // pass the function reference to hello
   fmt.Println(hello(world))
}

Imports

We already have seen imports. Lets import the ‘math’ package.

package main

import (
    "math"
    "fmt"
)

func main() {
    pi := math.Pi
    circum := pi*6
    surface := math.Pow(6, 2) * pi / 4
    fmt.Printf("circum %f surface %f", circum, surface)
}

We can also create our own package and import it. Lets create a directory called ‘formulas’ in our project directory and create the file ‘circle.go’:

package formulas

import "math"

func Circum(d float64) float64 {
    return math.Pi*d
}

func Surface(d float64) float64 {
    return math.Pow(d, 2) * math.Pi / 4 
}

Lets import the package:

package main

import (
    "fmt"
    "github.com/dnvriend/blog-examples/formulas"
)

func main() {
    d := 6.25342
    fmt.Printf("circum %f surface %f", formulas.Circum(d), formulas.Surface(d))
}

Try to change the case of Circum and Surface in ‘circle.go’, what happens?

Exported Identifiers

An identifier may be exported to permit access to it from another package. An identifier is exported if both:

the first character of the identifier’s name is an upper case letter,
the identifier is declared in the package block or it is a field name or method name.

Control Flow

package main

import (
    "fmt"
)

func main() {
    x := 2
    if x == 1 {
        fmt.Println("One")
    } else if x == 2 {
        fmt.Println("Two")
    } else {
        fmt.Println("Something else")
    }

    switch x {
    case 1:
        fmt.Println("One")
    case 2:
        fmt.Println("Two")
    default:
        fmt.Println("Something else...")
    }   
}

Make

The function ‘make’ creates slices, maps, and channels only, and it returns an initialized version of said object. Slices, maps and channels must be initialized before use.

package main

import (
    "fmt"
)

func main() {
    xs := make([]int, 3)
    xs[0] = 1
    xs[1] = 2
    kv := make(map[string]string)
    kv["foo"] = "bar"
    kv["baz"] = "quz"
    fmt.Printf("xs is %d, and ys is %s", xs, kv)
}

Collections

Go supports Array, Slice and Map collections.

package main

import (
    "fmt"
)

func main() {
    // array
   xs := [2]int{1, 2}
    // slice
   ys := []int{1, 2, 3}
    // map
   kv := map[string]string {"foo":"bar", "baz":"quz", "abc":"def"}
    fmt.Printf("xs=%d, ys=%d, zs=%s", xs, ys, kv)
}

Loops

Go can loop, iterate over collections and ranges:

package main

import (
    "fmt"
)

func main() {
    for i := 0; i < 5; i++ {
            fmt.Println("Counter:", i)
    }
    xs := []int{1, 2, 3, 4}
    for i, e := range xs {
            fmt.Printf("i=%d and e=%dn", i, e)
    }
    kv := map[string]string {"foo":"bar", "baz":"quz"}
    for k, v := range(kv) {
        fmt.Printf("k=%s, v=%sn", k, v)
    }
}

Error Handling: Defer, Panic and Recover

Go does not have exception controls like try, catch, finally, but solves handling errors in an interesting way]. Go uses the functions panic and recover to handle errors and the defer keyword to put a function on the stack that will always be called after stack execution.

package main

import (
    "fmt"
)

func main() {
    defer func() {
        if err := recover(); err != nil {
            // handle the error
           fmt.Println("Handling error: ", err)
            // 'throw' ie. 'panic' with a new message
           panic("My Error Message")
        } else {
            fmt.Println("Nothing happened")
        }
    }()
    xs := []int{}
    fmt.Println("xs:", xs[0])
}

Pointers

Go works with pointers, which are primitives that point to memory location. Pointers have a use in Go because whenever you call a method with an object reference, the object is copied. Because Go allows for fast and efficient applications, for values that have a large memory footprint, you can also pass a pointer which does not copy the object.
In the example below, the function ‘addOne’ receives a pointer. To change the underlying value, the pointer must be dereferenced with the ‘*’ operator.

package main

import (
    "fmt"
)

func addOne(x *int) {
    *x = *x + 2
}

func main() {
    x := 1
    addOne(&x)
    fmt.Println("x is", x)
}

Goroutines

Go supports threading which are called ‘goroutines’. Threads are very easy to create, just put the keyword ‘go’ before the name of a function when you call it. The example uses the sync package and uses a WaitGroup to wait for a collection of goroutines to finish. A better way to work with threads are channels.

package main

import (
    "fmt"
    "time"
    "sync"
)

var wg sync.WaitGroup

func say(msg string) {
    for i := 0; i < 3; i++ {
        fmt.Printf("%s World!n", msg)
        time.Sleep(time.Millisecond * 100)       
    }
    wg.Done()
}

func main() {
    wg.Add(1)
    go say("Hello")
    wg.Add(1)
    go say("Goodbye")
    wg.Wait()
}

Channels

Channels are the pipes that connect goroutines. You can send values into channels from one goroutine and receive those values into another goroutine.

package main

import (
    "fmt"
    "time"
)

func pinger(ping chan string, pong chan string) {
    for {
        fmt.Println(<-ping)
        time.Sleep(time.Millisecond * 300)
        pong <- "pong"
    }
}

func ponger(ping chan string, pong chan string) {
    for {
        fmt.Println(<-pong)
        time.Sleep(time.Millisecond * 300)
        ping <- "ping"
    }
}

func main() {
    done := make(chan bool)
    ping := make(chan string, 10)
    pong := make(chan string, 10)
    go pinger(ping, pong)
    go ponger(ping, pong)

    ping <- "ping"
    <-done
    fmt.Println("Done")
}

Logger

Go has support for logging messages. The log package defines the Logger that generates lines of output.

package main

import (
    "log"
)

func main() {
    log.Println("Normal")
    log.Fatal("Fatal")
    // log and then panic
   log.Panic("Panic")
}

Random Nummber Generator

Go provides a pseudo random number generator in the math/rand package.

package main

import (
    "fmt"
    "math/rand"
)

func main() {
    for i := 0; i < 5; i++ {
        x := rand.Intn(100)
        y := rand.Float64() * 5
        fmt.Println(x, y)
    }
}

UUID

Go does not come with a built-in UUID function. Fortunately Google has provided UUID as a library. The documentation is available at godoc. To install the library type:

$ go get github.com/google/uuid

To generate random UUIDs type:

package main

import (
    "fmt"
    "github.com/google/uuid"
)

func main() {
    for i := 0; i< 5; i++ {
        id, _ := uuid.NewRandom()
        fmt.Println(id.String())
    }
}

Writing Files

Go has support for writing files. The package os provides a platform-independent interface to operating system functionality. The function Create creates a file and returns a File that can be used for I/O operations. The WriteString function writes a string to a file. The Write function writes bytes to a file. The package io provides basic I/O primitives that is used by other packages. THe package ioutil implements I/O utility functions like ReadFile reads the whole file and returns all contents as bytes. The function WriteFile writes data as bytes to a file. The package bufio provides buffered I/O capabilities like NewWriter and NewReader that provides buffered I/O capabilities.

Write a string to a file:

package main

import (
    "fmt"
    "os"
)

func check(e error) {
    if e != nil {
        panic(e)
    }
}

func main() {
    f, err := os.Create("/tmp/test.txt")
    check(err)
    defer f.Close()
    n3, err := f.WriteString("Hello World!n")
    check(err)
    fmt.Printf("wrote %d bytesn", n3)
    f.Sync()
}

Write bytes to a file:

package main

import (
    "fmt"
    "io/ioutil"
    "os"
    "bufio"
)

func check(e error) {
    if e != nil {
        panic(e)
    }
}

func main() {
    // write bytes to a file
   data := []byte("Hello There!n")
    err := ioutil.WriteFile("/tmp/test1.dat", data, 0644)
    check(err)

    // write bytes to a file
   data2 := []byte{115, 111, 109, 101, 10}
    file2, err := os.Create("/tmp/test2.dat")
    defer file2.Close()
    check(err)
    n, err := file2.Write(data2)
    fmt.Printf("wrote %d bytesn", n)
    check(err)
    file2.Sync()

    // write bytes to a file more efficiently
   file3, err := os.Create("/tmp/test3.dat")
    defer file3.Close()  
    writer := bufio.NewWriter(file3)
    n2, err := writer.WriteString("bufferedn")
    fmt.Printf("wrote %d bytesn", n2)
    writer.Flush()
}

Reading Files

Go has support for reading files. In the example below we read the files that we have created in the ‘Writing Files’ example.

package main

import (
    "fmt"
    "io/ioutil"
    "os"
    "bufio"
)

func check(e error) {
    if e != nil {
        panic(e)
    }
}

func main() {
    // read the whole file
   data, err := ioutil.ReadFile("/tmp/test1.dat")
    check(err)
    fmt.Print(string(data))

    // read 5 bytes
   f, err := os.Open("/tmp/test1.dat")
    defer f.Close()
    check(err)
    b1 := make([]byte, 5)
    n1, err := f.Read(b1)
    check(err)
    fmt.Printf("%d bytes: %sn", n1, string(b1))

    // start reading from the 6th byte
   o2, err := f.Seek(6, 0)
    check(err)
    b2 := make([]byte, 6)
    n2, err := f.Read(b2)
    check(err)
    fmt.Printf("%d bytes @ %d: %sn", n2, o2, string(b2))

    // set the file pointer to the beginning
   _, err = f.Seek(0, 0)
    check(err)
    r4 := bufio.NewReader(f)
    // read 5 bytes without advancing the reader
    b4, err := r4.Peek(5)
    check(err)
    fmt.Printf("5 bytes: %sn", string(b4))
}

Reading and Writing CSV

Go has support for reading and writing CSV files.

Writing CSV:

package main

import (
    "encoding/csv"
    "os"
    "log"
    "bufio"
)

func main() {
    records := [][]string{
        {"first_name", "last_name", "username"},
        {"Rob", "Pike", "rob"},
        {"Ken", "Thompson", "ken"},
        {"Robert", "Griesemer", "gri"},
    }

    // write a record to console
   f, _ := os.Create("/tmp/names.csv")
    defer f.Close() 
    cw := csv.NewWriter(bufio.NewWriter(f))
    for _, record := range records {
        if err := cw.Write(record); err != nil {
            log.Fatal("error writing record to csv:", err)
        }
    }
    cw.Flush()
}

Reading CSV:

package main

import (
    "encoding/csv"
    "os"
    "bufio"
    "fmt"
)

func main() {

    // write a record to console
   f, _ := os.Open("/tmp/names.csv")
    defer f.Close()
    cr := csv.NewReader(bufio.NewReader(f))
    names, _ := cr.ReadAll()
    for _, row := range names {
        for i, field := range row {
            fmt.Println(i, field)
        }       
    }   
}

Compression

Go has support for compressing and decompressing data. Go supports bzip2, flate, gzip, lzw and zlib. Because AWS uses gzip compression we will use that for the example.

Writing GZIP:

package main

import (
    "compress/gzip"
    "os"
)

func check(e error) {
    if e != nil {
        panic(e)
    }
}

func main() {
    f, e1 := os.Create("/tmp/test.gz")
    defer f.Close()
    check(e1)
    gz := gzip.NewWriter(f)
    defer func() {
        gz.Flush()
        gz.Close()
    }()
    _, e2 := gz.Write([]byte("Hello World!n"))
    check(e2)
}

Reading GZIP:

package main

import (
    "bytes"
    "compress/gzip"
    "fmt"
    "io"
    "os"
)

func check(e error) {
    if e != nil {
        panic(e)
    }
}

func main() {
    // var buf bytes.Buffer
   f, e1 := os.Open("/tmp/test.gz")
    defer f.Close()
    check(e1)
    gz, e2 := gzip.NewReader(f)
    check(e2)
    var buf bytes.Buffer
    _, e3 := io.Copy(&buf, gz)
    check(e3)
    fmt.Print(string(buf.Bytes()))
}

Base64 Encoding/Decoding

Go has support for base64 encoding.

package main

import (
    "encoding/base64"
    "fmt"
)

func main() {
    msg := "Hello World!"
    encoded := base64.StdEncoding.EncodeToString([]byte(msg))
    fmt.Println(encoded)

    enc := "SGVsbG8gV29ybGQh"
    decoded, _ := base64.StdEncoding.DecodeString(enc)
    fmt.Println(string(decoded))
}

Hashing

Go has support for hasing algorithms. Go supports crypto/md5, crypto/sha1, crypto/sha256, crypto/sha512.

md5 digest:

package main

import (
    "crypto/md5"
    "fmt"
)

func main() {
    h := md5.New()
    h.Write([]byte("Hello World!n"))
    fmt.Printf("%x", h.Sum(nil))
}

sha-256 digest:

package main

import (
    "crypto/sha256"
    "fmt"
)

func main() {
    h := sha256.New()
    h.Write([]byte("Hello World!n"))
    fmt.Printf("%x", h.Sum(nil))
}

Crypto

Go supports several encryption algorihms like crypto/aes, crypto/des, crypto/x509.

Encrypt AES:

package main

import (
    "crypto/aes"
    "fmt"
    "io"
    "crypto/rand"
    "crypto/cipher"
    "encoding/hex"
)

func main() {
    key, _ := hex.DecodeString("6368616e676520746869732070617373776f726420746f206120736563726574")
    plaintext := []byte("Hello World!")
    block, _ := aes.NewCipher(key)
    nonce := make([]byte, 12)
    io.ReadFull(rand.Reader, nonce)
    aesgcm, _ := cipher.NewGCM(block)
    ciphertext := aesgcm.Seal(nil, nonce, plaintext, nil)
    fmt.Printf("Nonce: %xn", nonce)
    fmt.Printf("Cipher: %xn", ciphertext)
}

Decrypt AES:

package main

import (
    "crypto/aes"
    "fmt"
    "crypto/cipher"
    "encoding/hex"
)

func main() {    
    key, _ := hex.DecodeString("6368616e676520746869732070617373776f726420746f206120736563726574")
    ciphertext, _ := hex.DecodeString("c8eaf641d7a25d32d489a4666d7bfb3fad2c8b482ffebc891076876a")
    nonce, _ := hex.DecodeString("de3964e81c1a32bfc9b37b98")
    block, _ := aes.NewCipher(key)
    aesgcm, _ := cipher.NewGCM(block)
    plaintext, _ := aesgcm.Open(nil, nonce, ciphertext, nil)
    fmt.Printf("%sn", plaintext)
}

Date and Time

Go supports date and time with the time package. Text can also be parsed to time objects.

package main

import (
    "time"
    "fmt"
)

func main() {    
    // now returns a Time
   now := time.Now()
    fmt.Println(now.Format(time.RFC3339))
    fmt.Printf("%d-%02d-%02dT%02d:%02d:%02d-00:00n",
        now.Year(), now.Month(), now.Day(),
        now.Hour(), now.Minute(), now.Second())
    parsed, _ := time.Parse(time.RFC3339, "2018-11-24T19:02:17+01:00")
    fmt.Println(parsed)    
}

JSON Encoding

Go has native support for JSON encoding with the encoding/json package.

package main

import (
    "encoding/json"
    "fmt"
)

type Person struct {
    Name string <code>json:"name"
    Age int json:"age"
}

func main() {
    person := Person { "Dennis", 42 }
    bytes, _ := json.Marshal(person)
    fmt.Println(string(bytes))
    p := Person{}
    json.Unmarshal(bytes, &p)
    fmt.Println(p)
}

HTTP Server

Go provides both a HTTP client and server in the net/http package.

package main

import (
    "io"
    "net/http"
)

func rootHandler(w http.ResponseWriter, req *http.Request) {
    io.WriteString(w, "Hello, world!n")
}

func main() {
    http.HandleFunc("/", rootHandler)
    http.ListenAndServe(":8080", nil)
}

HTTP Client

A full features http client is provided by the net/http package.

package main

import (
    "net/http"
    "fmt"
    "io/ioutil"
)

func main() {
    res, _ := http.Get("https://www.google.nl/robots.txt")
    defer res.Body.Close()
    robots, _ := ioutil.ReadAll(res.Body)
    fmt.Printf("%s", robots)
}

Testing

Go provides a testing package and a test runner. Tests are run by typing ‘go test’ which executes test functions.

package calc

import (
    "github.com/dnvriend/blog-examples/calc"
    "testing"
)

func TestAddOne(t *testing.T) {
    x := calc.AddOne(1)
    if x != 2 {
        t.Fatal("Does not add one")
    }
}

Although Go supports testing code, you still need a library to make testing more developer friendly. The library Testify provides common assertions, mocks and assertion errors when tests fail. Testify can be installed by typing ‘go get github.com/stretchr/testify’.

package calc

import (
    "github.com/dnvriend/blog-examples/calc"
    "github.com/stretchr/testify/assert"
    "testing"
)

func TestAddOne(t *testing.T) {
    assert.Equal(t, calc.AddOne(1), 2, "Does not add one")
}

Conclusion

Go is a full featured programming general programming language to create highly efficient, modern applications. The standard library of Go provides most features modern networked applications need. Go provides HTTP client and server implementations, cryptographic, compression, hashing algorithms and JSON encoding functions. Go supports goroutines and channels to create concurrent applications. Next time we’ll look at how to create CLI applications with Go.

Learning Go – creating simple, reliable and efficient software