Using Couchbase Server In A Golang Web Application

Not too long ago I wrote an article regarding how to create a RESTful API using the Go programming language, but in it I only used mock data rather than an actual database. What happens if we want to use a database with Golang? What database, or more importantly, what kind of database should be used? Most APIs transfer data in JSON format, so it might make sense to store data in the same format. This means that a relational database might not make sense. Instead, NoSQL databases fit quite well when it comes to RESTful APIs. A popular NoSQL database that stores data in JSON format is the open source Couchbase Server.

We’re going to take a look at how to include Couchbase Server into our RESTful web application written in the Go programming language.

The RESTful API example I wrote previously is going to be the foundation to this tutorial, so if you haven’t checked it out, I definitely recommend you do. However, you won’t be completely lost if you don’t as I’m going to try to relay all the necessary information.

At this point I’m going to assume you have Golang installed and configured on your machine.

Creating a New Project with Dependencies

We’re going to create a new Golang project for simplicity. Using the Terminal (Mac and Linux) or Command Prompt (Windows), execute the following:

mkdir -p $GOPATH/src/github.com/nraboy/cbproject

Essentially I just created those directories with the command above. If the command doesn’t work on your operating system, create them manually.

Inside our project directory, we’re going to be working with a file called main.go. It will be the only file in our project. Before we start coding, we should probably download any project dependencies.

From the Command Prompt or Terminal, execute the following:

go get github.com/satori/go.uuid
go get github.com/couchbase/gocb
go get github.com/gorilla/mux

The dependencies above include a package for generating UUID values that we’ll use for document ids, the Couchbase Go SDK, and Mux for expanding the routing capabilities of our HTTP server.

Configuring Couchbase Server for the Golang Project

Before we can start coding we need to make sure Couchbase Server is configured and ready for use. We won’t be seeing how to install Couchbase Server. If you are using a Mac, Linux, or Windows machine, you can see one of the available getting started guides for installation. Instead we’ll be configuring a bucket for storing all the NoSQL documents for our application.

The goal here is to create a Couchbase bucket called restful-sample that has appropriate indexes for performing N1QL queries via our application code.

Inside the Couchbase administrative dashboard, choose Data Buckets and then choose Create New Data Bucket. Give the new bucket a name and define a storage capacity.

Next we need to create indexes for this new bucket.

When it comes to creating indexes there are a few ways to do this. If you’re using Couchbase 4.5 or higher you can use the Query Workbench. In the Query Workbench execute the following query:

CREATE PRIMARY INDEX ON `restful-sample` USING GSI;

At least one index is required to use N1QL. More indexes that have deeper complexity than the one I demonstrated will result in faster querying.

If you don’t have Couchbase Server 4.5 or higher installed, a different strategy must be used for creating indexes. You must use the Couchbase Query Shell (CBQ) that ships with Couchbase Server 4.0 and higher. For information on how to access it, you can visit here. The same query used in the Query Workbench should be used in the shell.

At this point we can focus on the actual application.

Designing the RESTful Web Application

With all the setup out of the way we can start coding the application. This application will have five basic endpoints for handling information about people. This is a standard create, retrieve, update, and delete (CRUD) application. Each endpoint will communicate with the NoSQL database differently.

Inside the project’s $GOPATH/src/github.com/nraboy/cbproject/main.go file, add the following code:

package main

import (
    "encoding/json"
    "log"
    "net/http"

    "github.com/couchbase/gocb"
    "github.com/gorilla/mux"
    "github.com/satori/go.uuid"
)

type Person struct {
    ID        string `json:"id,omitempty"`
    Firstname string `json:"firstname,omitempty"`
    Lastname  string `json:"lastname,omitempty"`
    Email     string `json:"email,omitempty"`
}

type N1qlPerson struct {
    Person Person `json:"person"`
}

var bucket *gocb.Bucket

func GetPersonEndpoint(w http.ResponseWriter, req *http.Request) {
    var n1qlParams []interface{}
    query := gocb.NewN1qlQuery("SELECT * FROM `restful-sample` AS person WHERE META(person).id = $1")
    params := mux.Vars(req)
    n1qlParams = append(n1qlParams, params["id"])
    rows, _ := bucket.ExecuteN1qlQuery(query, n1qlParams)
    var row N1qlPerson
    rows.One(&row)
    json.NewEncoder(w).Encode(row.Person)
}

func GetPeopleEndpoint(w http.ResponseWriter, req *http.Request) {
    var person []Person
    query := gocb.NewN1qlQuery("SELECT * FROM `restful-sample` AS person")
    rows, _ := bucket.ExecuteN1qlQuery(query, nil)
    var row N1qlPerson
    for rows.Next(&row) {
        person = append(person, row.Person)
    }
    json.NewEncoder(w).Encode(person)
}

func CreatePersonEndpoint(w http.ResponseWriter, req *http.Request) {
    var person Person
    var n1qlParams []interface{}
    _ = json.NewDecoder(req.Body).Decode(&person)
    query := gocb.NewN1qlQuery("INSERT INTO `restful-sample` (KEY, VALUE) VALUES ($1, {'firstname': $2, 'lastname': $3, 'email': $4})")
    n1qlParams = append(n1qlParams, uuid.NewV4().String())
    n1qlParams = append(n1qlParams, person.Firstname)
    n1qlParams = append(n1qlParams, person.Lastname)
    n1qlParams = append(n1qlParams, person.Email)
    _, err := bucket.ExecuteN1qlQuery(query, n1qlParams)
    if err != nil {
        w.WriteHeader(401)
        w.Write([]byte(err.Error()))
        return
    }
    json.NewEncoder(w).Encode(person)
}

func UpdatePersonEndpoint(w http.ResponseWriter, req *http.Request) {
    var person Person
    var n1qlParams []interface{}
    _ = json.NewDecoder(req.Body).Decode(&person)
    query := gocb.NewN1qlQuery("UPDATE `restful-sample` USE KEYS $1 SET firstname = $2, lastname = $3, email = $4")
    params := mux.Vars(req)
    n1qlParams = append(n1qlParams, params["id"])
    n1qlParams = append(n1qlParams, person.Firstname)
    n1qlParams = append(n1qlParams, person.Lastname)
    n1qlParams = append(n1qlParams, person.Email)
    _, err := bucket.ExecuteN1qlQuery(query, n1qlParams)
    if err != nil {
        w.WriteHeader(401)
        w.Write([]byte(err.Error()))
        return
    }
    json.NewEncoder(w).Encode(person)
}

func DeletePersonEndpoint(w http.ResponseWriter, req *http.Request) {
    var n1qlParams []interface{}
    query := gocb.NewN1qlQuery("DELETE FROM `restful-sample` AS person WHERE META(person).id = $1")
    params := mux.Vars(req)
    n1qlParams = append(n1qlParams, params["id"])
    _, err := bucket.ExecuteN1qlQuery(query, n1qlParams)
    if err != nil {
        w.WriteHeader(401)
        w.Write([]byte(err.Error()))
        return
    }
    json.NewEncoder(w).Encode(&Person{})
}

func main() {
    router := mux.NewRouter()
    cluster, _ := gocb.Connect("couchbase://localhost")
    bucket, _ = cluster.OpenBucket("restful-sample", "")
    router.HandleFunc("/people", GetPeopleEndpoint).Methods("GET")
    router.HandleFunc("/people/{id}", GetPersonEndpoint).Methods("GET")
    router.HandleFunc("/people", CreatePersonEndpoint).Methods("PUT")
    router.HandleFunc("/people/{id}", UpdatePersonEndpoint).Methods("POST")
    router.HandleFunc("/people/{id}", DeletePersonEndpoint).Methods("DELETE")
    log.Fatal(http.ListenAndServe(":12345", router))
}

There is a lot going on in the above code so it would be a good idea to break it down.

The first thing we’re doing is importing all the dependencies that we downloaded earlier in this article. With the dependencies included in the project we can define two structs that will represent the JSON data that we work with. The Person struct will have the essential information about a particular person and it will be saved like this as a JSON document in the database. When querying data, result sets are wrapped in a parent JSON object, thus the need for the N1qlPerson struct. Data will look like this when queried:

{
    "person": {
        "id": "1234",
        "firstname": "Nic",
        "lastname": "Raboy",
        "email": "nic@test.com"
    }
}

Inside the structs you’ll notice the use of the omitempty tag. This means that if any property is empty or null, it will not be included in the JSON result.

Before we jump into the endpoint functions we have the following:

var bucket *gocb.Bucket

This bucket variable is defined outside the main function making it global to the application. This will represent our open Couchbase bucket and be used when querying data.

Let’s skip down to the project’s main function:

func main() {
    router := mux.NewRouter()
    cluster, _ := gocb.Connect("couchbase://localhost")
    bucket, _ = cluster.OpenBucket("restful-sample", "")
    router.HandleFunc("/people", GetPeopleEndpoint).Methods("GET")
    router.HandleFunc("/people/{id}", GetPersonEndpoint).Methods("GET")
    router.HandleFunc("/people", CreatePersonEndpoint).Methods("PUT")
    router.HandleFunc("/people/{id}", UpdatePersonEndpoint).Methods("POST")
    router.HandleFunc("/people/{id}", DeletePersonEndpoint).Methods("DELETE")
    log.Fatal(http.ListenAndServe(":12345", router))
}

In the above main function we define our application router and establish a connection to our Couchbase cluster. In this case the Couchbase cluster is being run locally on my machine. With the Couchbase cluster connection established we can open a particular bucket. The bucket we use in this application will be called restful-sample.

There are five different routes in this application. There is a route for getting all documents, a route for getting a single document, a route for creating a document, a route for updating a document, and finally a route for deleting a document. Notice that each of these routes use GET, PUT, POST, or DELETE. Let’s take a look at each of the endpoint functions that go with each of the routes.

func GetPersonEndpoint(w http.ResponseWriter, req *http.Request) {
    var n1qlParams []interface{}
    query := gocb.NewN1qlQuery("SELECT * FROM `restful-sample` AS person WHERE META(person).id = $1")
    params := mux.Vars(req)
    n1qlParams = append(n1qlParams, params["id"])
    rows, _ := bucket.ExecuteN1qlQuery(query, n1qlParams)
    var row N1qlPerson
    rows.One(&row)
    json.NewEncoder(w).Encode(row.Person)
}

The above GetPersonEndpoint function will get a single person from the database. The document id from the meta information is being compared against the document id parameter that was passed in with the route request. This query is a parameterized query to prevent SQL injection. We are returning JSON data with only the Person struct, not the entire N1qlPerson struct.

func GetPeopleEndpoint(w http.ResponseWriter, req *http.Request) {
    var person []Person
    query := gocb.NewN1qlQuery("SELECT * FROM `restful-sample` AS person")
    rows, _ := bucket.ExecuteN1qlQuery(query, nil)
    var row N1qlPerson
    for rows.Next(&row) {
        person = append(person, row.Person)
    }
    json.NewEncoder(w).Encode(person)
}

The above GetPeopleEndpoint function is similar to the previous, but this time we are expecting a slice of results rather than a single result.

The create, update, and delete functions follow pretty much the same strategy, making things very easy to maintain and understand.

At this point the applicant can be run. When it comes to testing you’ll have to use cURL, Postman, or similar since the web browser cannot typically test PUT, POST, or DELETE endpoints out of the box.

Conclusion

You just saw how to take our previous RESTful API Golang example to the next level by including Couchbase Server as our NoSQL document database. When using Couchbase, we are able to run SQL-like queries called N1QL queries to work with any data in the database. Although we don’t have to use these N1QL queries, it makes things very easy and reduces a lot of potential parsing code.

A video version of this article can be seen below.

Nic Raboy

Nic Raboy is an advocate of modern web and mobile development technologies. He has experience in C#, JavaScript, Golang and a variety of frameworks such as Angular, NativeScript, and Unity. Nic writes about his development experiences related to making web and mobile development easier to understand.