Home brewn service discovery with Go

Introduction

I have a bunch of daemons running on my raspberry pi such as redis and postgres. I want to connect whatever project I am working on locally on my mac to them as dependencies but I have to use the IP of the raspberry pi. It is not a huge deal because my raspberry pi has a static IP but it would be nice if I could use hostname like redis.xyz or postgres.xyz and have it use my raspberry pi automatically. I could update my /etc/hosts file but where is the fun in that?

Thankfully that is possible. Being able to use a human friendly hostname to map to an IP is simply DNS. I can create a DNS server that exposes a write API to map hostnames to IP, which are then used for the read DNS queries.

Dependencies

There are two main parts to this app. The DNS server for the reads and a REST endpoint for the writes. The DNS server will be handled by Miekg’s DNS packagegithub.com/miekg/dns and the rest endpoint by github.com/labstack/echo

Show me the code

State

We will hold our hostname to IP mappings in a map[string]string - this could be in memory, in an embedded k/v store such as badgerDB, or plain old sqlite.

DNS Query

The signature for the method that will handle the DNS queries will be func handleDnsRequest(w dns.ResponseWriter, r *dns.Msg), similar to a regular net/http handler. I am only dealing with A records so this service only responds to A record requests. To parse the query that came in we can use the following function

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
func parseQuery(m *dns.Msg) {
	for _, q := range m.Question {
		switch q.Qtype {
		case dns.TypeA: // <-- A record queries
			log.Printf("Query for %s\n", q.Name)
			ip := records[q.Name] // <--- this is our state
			if ip != "" {
				rr, err := dns.NewRR(fmt.Sprintf("%s A %s", q.Name, ip))
				if err == nil {
					m.Answer = append(m.Answer, rr)
				}
			}
		}
	}
}

And our handler that invokes the parse will be

1
2
3
4
5
6
7
8
9
10
11
12
func handleDnsRequest(w dns.ResponseWriter, r *dns.Msg) {
	m := new(dns.Msg)
	m.SetReply(r)
	m.Compress = false

	switch r.Opcode {
	case dns.OpcodeQuery:
		parseQuery(m)
	}

	w.WriteMsg(m)
}

To wire the server up our main will be

1
2
3
4
5
6
7
8
9
10
11
// attach request handler func
dns.HandleFunc("xyz.", handleDnsRequest) // <-- handle .xyz domains
// start server
port := 5353
server := &dns.Server{Addr: ":" + strconv.Itoa(port), Net: "udp"}
log.Printf("Starting at %d\n", port)
err := server.ListenAndServe()
defer server.Shutdown()
if err != nil {
    log.Fatalf("Failed to start server: %s\n ", err.Error())
}

We can use dig against the server to see if it is working dig @localhost -p 5353 foo.xyz

The full gist can be found here

Write endpoint

To allow entries to get into our state we can expose a write rest endpoint that will take in a hostname and an IP to register it against.

1
2
POST /register
Request Body: {"origin": "redis.xyz", "target": "192.168.1.1"}

For that we can leverage Echo a simple and performant http router.

1
2
3
4
5
6
7
8
9
10
11
12
struct RegisterRequest {
    Origin  string  `json:"origin"`
    Target  string  `json:"target"`
}

e := echo.New()
e.POST("/register", func(c echo.Context) error {
    req := &RegisterRequest{}
    c.Bind(req)
    records[req.Origin] = req.Target
    return c.String(http.StatusOK, "registered")
})

Now we can hit the register whatever domains we want to register against whatever IPs and point our local machine to use this as its dns server and we are good to go.

Conclusion

This is what service discovery is. Provide a way to register and unregister endpoints and allow for reads via DNS. That is what consul does at the end of the day. DNS is also convinient compared to an application level service lookup because it works at the system level - so things like nginx and curl also ‘just work’.