A poor man's Load Balancer

I had a very typical setup with Caddy (or nginx if you prefer) with two reverse proxies:

api.mydomain.co {
  reverse_proxy localhost:8080
}
web.mydomain.co {
  reverse_proxy localhost:3000
}

The process running on 8080 has about 100 endpoints our mobile apps hit. And the process running on 3000 is the whole web app for our site. Caddy was making https urls like:

https://api.mydomain.co/foo

https://web.mydomain.co/bar

pass through to the right port running just normal http not https.

Caddy was handling the SSL certs and the complexity of running on 443 (and also 80 to foward anything from 80 to 443) and it was doing a fine job. There was just one problem. When I did a release of api or web there was about 3 seconds of downtime and Caddy would return 509 errors until the processs can back up and was listening on 8080 or 3000 again. The VM was fine CPU and memory wise, there was no need for a second VM.

Might not seem like a lot (3 seconds) but with lots of people hitting your API 24/7 that’s a bunch of 509 errors every single time we deploy. There’s the obvious solution: get a real load balancer and run two VMs but that will cost like 2x maybe 3x more.

Instead I decided to keep just that one VM. I run two copies of web and api on it now on ports 8080 and 8081, and 3000 and 3001 respectively. And I replaced Caddy with my own little golang program that could be told send 100% of the api traffic to 8081, okay now send 100% to 8080, ok back to 8081, etc. This means I can deploy the web or api that’s getting 0% traffic, wait for that to finish, and then tell my little program it’s time to switch!

func Serve() {
  domains := []string{"web.mydomain.co", "api.mydomain.co"}

  cfg := simplecert.Default
  cfg.Domains = domains
  cfg.CacheDir = "/certs"
  cfg.SSLEmail = "email@gmail.com"
  certReloader, err := simplecert.Init(cfg, nil)

  go http.ListenAndServe(":80", http.HandlerFunc(simplecert.Redirect))

  tlsconf := tlsconfig.NewServerTLSConfig(tlsconfig.TLSModeServerStrict)
  tlsconf.GetCertificate = certReloader.GetCertificateFunc()
}

  handler := http.HandlerFunc(handleRequest)

  s := &http.Server{
    Addr:      ":443",
    Handler:   handler,
    TLSConfig: tlsconf,
  }

  s.ListenAndServeTLS("", "")

So, this is everything you need to run https for all the domains in the array domains. The handleRequest function is simply:

func handleRequest(writer http.ResponseWriter, request *http.Request) {
  host := request.Host
  if strings.Contains(host, "api") {
    ReverseProxyApi.ServeHTTP(writer, request)
  } else if strings.Contains(host, "web") {
    ReverseProxyWeb.ServeHTTP(writer, request)
  }
}

But what are those ReverseProxyApi and ReverseProxyWeb things?

I have these global vars:

var ReverseProxyApi *httputil.ReverseProxy
var ReverseProxyWeb *httputil.ReverseProxy

var ApiPort int = 8080
var WebPort int = 3000

and I create them at start like:

urlForProxy, _ := url.Parse(fmt.Sprintf("http://localhost:%d", port))
httputil.NewSingleHostReverseProxy(urlForProxy)

This is great, now I’m doing everything Caddy was doing. But for the real magic we need to be able to tell it, hey, 8080 that’s old news, make your ReverseProxyApi all over again but on 8081.

So back in handleRequest I added a little more logic like:

  path := request.URL.Path

 if strings.HasPrefix(path, "/AF066DDA-8754-4A16-A5FD-94625A5FF2EE/") { // use your own secret guid!
    tokens := strings.Split(path, "/")
    last := tokens[len(tokens)-1]

    if last == "api" {
      writer.Write([]byte(fmt.Sprintf("%d", ApiPort)))
      return
    } else if last == "web" {
      writer.Write([]byte(fmt.Sprintf("%d", WebPort)))
      return
    }

    if last == "8080" {
      if ApiPort == 8080 {
        ApiPort++
      } else {
        ApiPort--
      }
      ReverseProxyApi = makeReverseProxy(ApiPort)
    } else if last == "3000" {
      if WebPort == 3000 {
        WebPort++
      } else {
        WebPort--
      }
      ReverseProxyWeb = makeReverseProxy(WebPort)
    }
    return
  }

So now I can curl that special url:

https://api.mydomain.co/AF066DDA-8754-4A16-A5FD-94625A5FF2EE/api

and it will tell me the current port for api!

https://api.mydomain.co/AF066DDA-8754-4A16-A5FD-94625A5FF2EE/web

and it will tell me the current port for web!

AND https://api.mydomain.co/AF066DDA-8754-4A16-A5FD-94625A5FF2EE/8080

will bump the ApiPort to 8081 if it’s at 8080 and return it to 8080 if it’s at 8081 already.

Same with https://api.mydomain.co/AF066DDA-8754-4A16-A5FD-94625A5FF2EE/3000

No more 3 seconds of downtime when we deploy! I simply deploy to the port not in use. Then tell the system to switch over to that new port. Rinse and repeat!