This is the second in a series of articles about writing a small reading list app in Go for personal use.

Today we are going to make part of the R of our CRUD app:

• add a model (maps to a db table)
• add a template to display books
• add a route that displays that template
• change our default route to redirect

By the end of this article we will be able to load a page that displays the books in the database. (But we’ll still have to manually add books to the database.)

Define The Model

First, let’s add Gorm to our project:

$ go get gorm.io/gorm
$ go get gorm.io/driver/sqlite

The convenience that Gorm buys us is that we can just declare a struct with some special tags, and Gorm will:

• create a database table for the struct
• support creating, updating, and deleting table rows from the struct instance
• support querying the table and returning struct instances from the resulting rows

It also supports joins between tables and other advanced usage, in addition to giving us the ability to perform raw SQL queries. (This last part is key, because I have yet to use an ORM where I didn’t eventuallly end up hitting a wall and needing raw SQL.)

package main

type Book struct {
   ID     uint
   Title  string
   Author string
}

That is all we need to do to declare a basic model that will map to a table of books with an integer primary key plus title and author strings.

In order to make that table, we can call Gorm’s AutoMigrate function. This will create tables, columns, foreign keys, etc. as needed. It’s ok for our needs now, but a future article will take a deeper look at doing “proper” database migrations. We pass as arguments an instance of each model for which we want a table.

I like to put the call to AutoMigrate in a function that I can also call from my unit tests. This helps to ensure that the tests are exercising the exact same database schema as the code. So add setupDatabase to main.go:

func setupDatabase(db *gorm.DB) error {
   err := db.AutoMigrate(
           &Book{},
   )
   if err != nil {
           return fmt.Errorf("Error migrating database: %s", err)
   }
   return nil
}

Then in our main we can open a database and pass it to the setup function. Note that the database will be created as “aklatan.db” in whatever directory the program is started. This isn’t ideal – we should be able to specify a path either via environment variable or command line argument. I’ll fix this in a future post.

func main() {
   db, err := gorm.Open(sqlite.Open("aklatan.db"), &gorm.Config{})
   if err != nil {
           log.Fatalf("Failed to connect to database: %s", err)
   }
   err = setupDatabase(db)
   if err != nil {
           log.Fatalf("Database setup error: %s", err)
   }
   r := gin.Default()
   setupRouter(r, db) // <<--- note the extra arg here
   err = r.Run(":3000")
   if err != nil {
           log.Fatalf("gin Run error: %s", err)
   }
}

For test code for this model… there’s not really a lot to do. Mostly all we can really do is validate the mechanics of setting up the database. If we add validation or other logic to our model later we will need more meaningful tests. One thing we will need is a way to get a database handle in any of our tests – whether they are model tests or controller tests. We can set that up now, and create a trivial test that both exercises it and checks that our database has the books table. Create models_test.go.

package main

import (
   "testing"

   "gorm.io/driver/sqlite"
   "gorm.io/gorm"
)

func freshDb(t *testing.T, path ...string) *gorm.DB {
   t.Helper()

   var dbUri string

   // Note: path can be specified in an individual test for debugging
   // purposes -- so the db file can be inspected after the test runs.
   // Normally it should be left off so that a truly fresh memory db is
   // used every time.
   if len(path) == 0 {
           dbUri = ":memory:"
   } else {
           dbUri = path[0]
   }

   db, err := gorm.Open(sqlite.Open(dbUri), &gorm.Config{})
   if err != nil {
           t.Fatalf("Error opening memory db: %s", err)
   }
   if err := setupDatabase(db); err != nil {
           t.Fatalf("Error setting up db: %s", err)
   }
   return db
}

// This tests that a fresh database returns no rows (but no error) when
// fetching Books.
func TestBookEmpty(t *testing.T) {
   db := freshDb(t)
   books := []Book{}
   if err := db.Find(&books).Error; err != nil {
           t.Fatalf("Error querying books from fresh db: %s", err)
   }
   if len(books) != 0 {
           t.Errorf("Expected 0 books, got %d", len(books))
   }
}

We can call freshDb from test code to get an in-memory database that is independent of other tests. This makes it so that (1) tests can run in parallel without interfering with each other and (2) database operations in tests run faster because they don’t have to go to disk. The test passes:

% go test -race .
ok      gitlab.com/bstpierre/aklatan    0.053s

Sometimes it is convenient when debugging a failing test to be able to have a persistent copy. For these situations there is an optional path argument that an individual test can pass to get a database file – when this happens to me I just edit the test’s call like freshDb(t, "/tmp/test.db"), run the failing test like go test -run MyTest, and then examine the database file with sqlite3.

The test just gets a db instance and queries it for books. We can verify that it works by commenting out the call to AutoMigrate in setupDatabase, rerunning the test, and seeing it fail because the query returns an error.

Free Bonus: Makefile

It’s tedious to have to remember to rebuild the executable, run the tests with the correct flags, run lint, etc. Or sometimes you forget to do one of those things and then the CI pipeline fails after posting a merge request.

It’s much easier to capture all of the steps in a Makefile and then we can just run make to update everything. Note that the Makefile below doesn’t generate coverage by default – if you want to update coverage, type make all cover.

Finally: this Makefile is fairly generic – I’ve used this on a handful of projects, and all that needs to change is the PROJECT := line. I won’t say that it’s one-size-fits-all, but for a simple project like this it’s likely to meet your needs with little or no tweaking required.

PROJECT := aklatan
ALLGO := $(wildcard *.go */*.go cmd/*/*.go)
ALLHTML := $(wildcard templates/*/*.html)

.DELETE_ON_ERROR:

.PHONY: all
all: lint check $(PROJECT) $(CMDS)

.PHONY: lint
lint: .lint

.lint: $(ALLGO)
   golangci-lint run --timeout 180s
   @touch [email protected]

.coverage:
   @mkdir -p .coverage

.PHONY: check
check: .coverage ./.coverage/$(PROJECT).out

 ./.coverage/$(PROJECT).out: $(ALLGO) $(ALLHTML) Makefile
   go test $(TESTFLAGS) -coverprofile=./.coverage/$(PROJECT).out ./...

.PHONY: cover
# When running manually, capture just the total percentage (and
# beautify it slightly because the tool output is usually too wide).
cover: .coverage ./.coverage/$(PROJECT).html
   @echo "Checking overall code coverage..."
   @go tool cover -func .coverage/$(PROJECT).out | sed -n -e '/^total/s/:.*statements)[^0-9]*/: /p'

./.coverage/$(PROJECT).html: ./.coverage/$(PROJECT).out
   go tool cover -html=./.coverage/$(PROJECT).out -o ./.coverage/$(PROJECT).html

# XXX *.go isn't quite right here -- it will rebuild when tests are
# touched, but it's good enough.
$(PROJECT): $(ALLGO)
   go build .

$(CMDS): $(ALLGO)
   for cmd in $(CMDS); do go build ./cmd/$$cmd; done

# XXX This only works if go-junit-report is installed. It's not part of go.mod
# because I don't want to force a dependency, but it is part of the ci docker
# image.
report.xml: $(ALLGO) Makefile
   go test $(TESTFLAGS) -v ./... 2>&1 | go-junit-report > [email protected]
   go tool cover -func .coverage/$(PROJECT).out

Next week’s post will share a couple of quick tips that I use to tightly integrate this Makefile into my workflow.

Create The View

In templates/books/index.html we can create a template that lists the title and author of each book. This template will accept a variable books that is a slice of Book.

{{ define "books/index.html" }}
{{ template "base/header.html" . }}

<h2>My Books</h2>

<ul>
  {{ range .books }}
  <li>{{ .Title }} -- {{ .Author }}</li>
  {{ end }}
</ul>

{{ template "base/footer.html" . }}
{{ end }}

As you probably noticed, this pulls in two other templates – a header and footer. We will use the header and footer to ensure that all the pages in our app are consistently structured.

The header is very basic – we can add metadata, links to stylesheets, etc. later.

{{ define "base/header.html" }}
<!DOCTYPE html>
<html class="h-100" lang="en">
  <head>
    <title>Aklatan: My Library</title>
  </head>
  <body>

{{ end }}

The footer is similarly basic. Note the placeholders for a copyright notice and a stubbed-out script block.

{{ define "base/footer.html" }}
  <footer>
      <div>Copyright 2022, Me</div>
  </footer>

  {{/* scripts go here
  <script src="/assets/my_script_file.js"></script>
  */}}
  </body>
</html>
{{ end }}

You can rerun make here and restart aklatan, but you won’t see these new templates used in the browser yet because our route handler is still showing the “hello” template.

Change the Controller

We need to add a route for our book list and then add a handler that passes the book list to the template. The book list handler will have to query the database. However, if you recall the defaultHandler we added last week, it doesn’t get any kind of database handle as a parameter. So we need to provide a way for our handlers to access the database.

Storing the handle in a global variable might be a tempting way to do this – except that it’s completely wrong. For all the reasons that global variables are undesirable in general, and for some specific reasons that I’ll point out below.

Instead we’ll add a middleware function that sets a database handle on the context object. In gin, middleware takes the shape of a function that takes a *gin.Context. Let’s look at the code.

// Middleware to connect the database for each request that uses this
// middleware.
func connectDatabase(db *gorm.DB) gin.HandlerFunc {
   return func(c *gin.Context) {
           c.Set("database", db)
   }
}

First, we define a function that returns a middleware function – connectDatabase is NOT the middleware, it’s more like a middleware factory. The function that it returns is a closure around the db variable, and it just sets a key/value pair on the context using database as the key. Handlers can Get("database") to access the database handle.

Next, we’ll define the book list handler.

func bookIndexHandler(c *gin.Context) {
   db := c.Value("database").(*gorm.DB)
   books := []Book{}
   if err := db.Find(&books).Error; err != nil {
           c.AbortWithStatus(http.StatusInternalServerError)
           return
   }
   c.HTML(http.StatusOK, "books/index.html", gin.H{"books": books})
}

This is fairly straightforward: get the handle, query the database, and pass the books to the template. If we get an error, abort with a 500.

Now we can make changes to setupRouter:

• add the middleware
• set up a route for the book list
• set up the default route to redirect to the book list

func setupRouter(r *gin.Engine, db *gorm.DB) {
   r.LoadHTMLGlob("templates/**/*.html")
   r.Use(connectDatabase(db))
   r.GET("/books/", bookIndexHandler)
   r.GET("/", func(c *gin.Context) {
           c.Redirect(http.StatusMovedPermanently, "/books/")
   })
}

This changed the signature of setupRouter – it needs a db pointer now – so we have to change main to pass this in.

func main() {
   db, err := gorm.Open(sqlite.Open("aklatan.db"), &gorm.Config{})
   if err != nil {
           log.Fatalf("Failed to connect to database: %s", err)
   }
   err = setupDatabase(db)
   if err != nil {
           log.Fatalf("Database setup error: %s", err)
   }
   r := gin.Default()
   setupRouter(r, db) // <<--- note the extra arg here
   err = r.Run(":3000")
   if err != nil {
           log.Fatalf("gin Run error: %s", err)
   }
}

And finally, we have to update our test:

• change the name to reflect that we’re testing the book list, not the “hello” handler
• pass a fresh db into setupRouter
• change the expected condition from a string comparison to strings.Contains() – in general this is how a lot of our handler tests will work, by looking for strings in the content of the pages

func TestBookIndex(t *testing.T) {
   t.Parallel()
   w := httptest.NewRecorder()
   ctx, r := gin.CreateTestContext(w)
   setupRouter(r, freshDb(t))
   req, err := http.NewRequestWithContext(ctx, "GET", "/books/", nil)
   if err != nil {
           t.Errorf("got error: %s", err)
   }
   r.ServeHTTP(w, req)
   if http.StatusOK != w.Code {
           t.Fatalf("expected response code %d, got %d", http.StatusOK, w.Code)
   }
   body := w.Body.String()
   expected := "<h2>My Books</h2>"

   if !strings.Contains(body, expected) {
           t.Fatalf("expected response body to contain '%s', got '%s'", expected, body)
   }
}

This test is completely inadequate! But our changes make it so that we can compile and run. We’ll add better tests next week.

We don’t need the “hello” message any more, so you can git rm templates/base/default.html and in main.go you can remove defaultHandler.

Now you can run make to compile and run tests, and ./aklatan to start the app. Browsing to http://localhost:3000/ should redirect to /books/ and you should see the title and footer… but no books yet, because the database is empty. We don’t have any UI for inserting books yet, but we can do it at the command line.

$ sqlite3 aklatan.db
SQLite version 3.34.1 2021-01-20 14:10:07
Enter ".help" for usage hints.
sqlite> insert into books values (1, 'The Go Programming Language', 'Donovan, Kernighan');
sqlite> insert into books values (2, 'Network Programming with Go', 'Woodbeck');

Now reload the browser and you’ll see the two books in the list.

Extra Free Bonus: badges!

Let’s add pipeline and coverage badges to our README. At the top of README.md, add the following lines, replacing $user with your GitLab username and $project with your project name. If your default branch isn’t main you will also need to change that.

![](https://gitlab.com/$user/$project/badges/main/pipeline.svg)
![](https://gitlab.com/$user/$project/badges/main/coverage.svg)

When you merge these changes and the pipeline completes you will see something like the following in your project’s main page:

Next: Better Tests

Next week we’ll look at some testing strategies and set up a more comprehensive test for our handler.

• gin