Add option to bind to a Unix socket instead of a TCP port (#703)

* feat: add ability to listen on unix socket/named pipe

Add a -socket option that configures the server to listen on a Unix-domain socket or Windows named pipe instead of a TCP port.  This allows webhook to be used behind a reverse proxy on multi-tenant shared hosting without the need to choose (and the permission to bind to) a free port number.

On Windows, -socket is expected to be a named pipe such as \\.\pipe\webhook, and the code uses https://github.com/microsoft/go-winio to bind the listening socket.  On other platforms, -socket is the path to a Unix domain socket such as /tmp/webhook.sock, or an abstract socket name starting with @, bound using the regular net.Listen function with the "network" parameter set to "unix".

Note: this pushes our minimum Go version up to 1.21 as that is what go-winio requires, but that is already the minimum version against which we are testing in the CI matrix.

* tests: add test for the -socket option

Refactored webhook_test so that the test HTTP requests are made using an explicitly-provided http.Client, so we can run at least one test with the server bound to a socket instead of a port number, using an http.Client whose transport has been configured with a suitable Unix-domain or Windows named pipe dialer function.

* tests: use GOROOT to find go command

This should ensure that, even if a developer or CI server has multiple versions of go installed, the version used to build the tools under test will be the same version that is running the test harness.

* fix: clean up Unix socket file before exit

If webhook is restarted with the same settings but the socket file has not been deleted, webhook will be unable to bind and will exit with an error.

* docs: add -socket option to documentation

* docs: add a note about reverse proxies

- README mentions the idea of using webhook behind a reverse proxy, including with the -socket flag
- added a note in Hook-Rules that the ip-whitelist rule type does not work as expected behind a reverse proxy, and you should configure IP restrictions at the proxy level instead

.github/workflows/build.yml

@@ -2,6 +2,13 @@ name: build
 on: [push, pull_request]
 jobs:
   build:
+    env:
+      # The special value "local" tells Go to use the bundled Go
+      # version rather than trying to fetch one according to a
+      # `toolchain` value in `go.mod`. This ensures that we're
+      # really running the Go version in the CI matrix rather than
+      # one that the Go command has upgraded to automatically.
+      GOTOOLCHAIN: local
     strategy:
       matrix:
         go-version: [1.21.x, 1.22.x]

README.md

@@ -109,6 +109,11 @@ In either case, the given file part will be parsed as JSON and added to the `pay
 
 TLS version and cipher suite selection flags are available from the command line. To list available cipher suites, use the `-list-cipher-suites` flag.  The `-tls-min-version` flag can be used with `-list-cipher-suites`.
 
+## Running behind a reverse proxy
+[webhook][w] may be run behind a "reverse proxy" - another web-facing server such as [Apache httpd](https://httpd.apache.org) or [Nginx](https://nginx.org) that accepts requests from clients and forwards them on to [webhook][h].  You can have [webhook][w] listen on a regular TCP port or on a Unix domain socket (with the `-socket` flag), then configure your proxy to send requests for a specific host name or sub-path over that port or socket to [webhook][w].
+
+Note that when running in this mode the [`ip-whitelist`](docs/Hook-Rules.md#match-whitelisted-ip-range) trigger rule will not work as expected, since it will be checking the address of the _proxy_, not the _client_.  Client IP restrictions will need to be enforced within the proxy, before it decides whether to forward the request to [webhook][w].
+
 ## CORS Headers
 If you want to set CORS headers, you can use the `-header name=value` flag while starting [webhook][w] to set the appropriate CORS headers that will be returned with each response.
 

docs/Hook-Rules.md

@@ -269,6 +269,8 @@ The IP can be IPv4- or IPv6-formatted, using [CIDR notation](https://en.wikipedi
 }
 ```
 
+Note this does not work if webhook is running behind a reverse proxy, as the "client IP" will either not be available at all (if webhook is using a Unix socket or named pipe) or it will be the address of the _proxy_, not of the real client.  You will probably need to enforce client IP restrictions in the reverse proxy itself, before forwarding the requests to webhook.
+
 ### Match scalr-signature
 
 The trigger rule checks the scalr signature and also checks that the request was signed less than 5 minutes before it was received. 

docs/Webhook-Parameters.md

@@ -14,7 +14,7 @@ Usage of webhook:
   -hotreload
         watch hooks file for changes and reload them automatically
   -http-methods string
-        globally restrict allowed HTTP methods; separate methods with comma
+        set default allowed HTTP methods (ie. "POST"); separate methods with comma
   -ip string
         ip the webhook should serve hooks on (default "0.0.0.0")
   -key string
@@ -23,6 +23,8 @@ Usage of webhook:
         list available TLS cipher suites
   -logfile string
         send log output to a file; implicitly enables verbose logging
+  -max-multipart-mem int
+        maximum memory in bytes for parsing multipart form data before disk caching (default 1048576)
   -nopanic
         do not panic if hooks cannot be loaded when webhook is not running in verbose mode
   -pidfile string
@@ -35,6 +37,8 @@ Usage of webhook:
         set group ID after opening listening port; must be used with setuid
   -setuid int
         set user ID after opening listening port; must be used with setgid
+  -socket string
+        path to a Unix socket (e.g. /tmp/webhook.sock) or Windows named pipe (e.g. \\.\pipe\webhook) to use instead of listening on an ip and port; if specified, the ip and port options are ignored
   -template
         parse hooks file as a Go template
   -tls-min-version string

go.mod

@@ -1,8 +1,11 @@
 module github.com/adnanh/webhook
 
-go 1.17
+go 1.21
+
+toolchain go1.22.0
 
 require (
+	github.com/Microsoft/go-winio v0.6.2
 	github.com/clbanning/mxj/v2 v2.7.0
 	github.com/dustin/go-humanize v1.0.1
 	github.com/fsnotify/fsnotify v1.7.0

go.sum

@@ -1,3 +1,5 @@
+github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
+github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
 github.com/clbanning/mxj/v2 v2.7.0 h1:WA/La7UGCanFe5NpHF0Q3DNtnCsVoxbPKuyBNHWRyME=
 github.com/clbanning/mxj/v2 v2.7.0/go.mod h1:hNiWqW14h+kc+MdF9C6/YoRfjEJoR3ou6tn/Qo+ve2s=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
@@ -19,7 +21,6 @@ github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORN
 github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
-golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
 golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=

platform_unix.go

@@ -0,0 +1,22 @@
+//go:build !windows
+// +build !windows
+
+package main
+
+import (
+	"flag"
+	"fmt"
+	"net"
+)
+
+func platformFlags() {
+	flag.StringVar(&socket, "socket", "", "path to a Unix socket (e.g. /tmp/webhook.sock) to use instead of listening on an ip and port; if specified, the ip and port options are ignored")
+}
+
+func trySocketListener() (net.Listener, error) {
+	if socket != "" {
+		addr = fmt.Sprintf("{unix:%s}", socket)
+		return net.Listen("unix", socket)
+	}
+	return nil, nil
+}

platform_windows.go

@@ -0,0 +1,23 @@
+//go:build windows
+// +build windows
+
+package main
+
+import (
+	"flag"
+	"fmt"
+	"github.com/Microsoft/go-winio"
+	"net"
+)
+
+func platformFlags() {
+	flag.StringVar(&socket, "socket", "", "path to a Windows named pipe (e.g. \\\\.\\pipe\\webhook) to use instead of listening on an ip and port; if specified, the ip and port options are ignored")
+}
+
+func trySocketListener() (net.Listener, error) {
+	if socket != "" {
+		addr = fmt.Sprintf("{pipe:%s}", socket)
+		return winio.ListenPipe(socket, nil)
+	}
+	return nil, nil
+}

signals.go

@@ -1,3 +1,4 @@
+//go:build !windows
 // +build !windows
 
 package main
@@ -6,6 +7,7 @@ import (
 	"log"
 	"os"
 	"os/signal"
+	"strings"
 	"syscall"
 )
 
@@ -43,6 +45,15 @@ func watchForSignals() {
 					log.Print(err)
 				}
 			}
+			if socket != "" && !strings.HasPrefix(socket, "@") {
+				// we've been listening on a named Unix socket, delete it
+				// before we exit so subsequent runs can re-bind the same
+				// socket path
+				err := os.Remove(socket)
+				if err != nil {
+					log.Printf("Failed to remove socket file %s: %v", socket, err)
+				}
+			}
 			os.Exit(0)
 
 		default:

testutils.go

@@ -0,0 +1,30 @@
+//go:build !windows
+// +build !windows
+
+package main
+
+import (
+	"context"
+	"io/ioutil"
+	"net"
+	"net/http"
+	"os"
+	"path"
+)
+
+func prepareTestSocket(_ string) (socketPath string, transport *http.Transport, cleanup func(), err error) {
+	tmp, err := ioutil.TempDir("", "webhook-socket-")
+	if err != nil {
+		return "", nil, nil, err
+	}
+	cleanup = func() { os.RemoveAll(tmp) }
+	socketPath = path.Join(tmp, "webhook.sock")
+	socketDialer := &net.Dialer{}
+	transport = &http.Transport{
+		DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
+			return socketDialer.DialContext(ctx, "unix", socketPath)
+		},
+	}
+
+	return socketPath, transport, cleanup, nil
+}

testutils_windows.go

@@ -0,0 +1,22 @@
+//go:build windows
+// +build windows
+
+package main
+
+import (
+	"context"
+	"github.com/Microsoft/go-winio"
+	"net"
+	"net/http"
+)
+
+func prepareTestSocket(hookTmpl string) (socketPath string, transport *http.Transport, cleanup func(), err error) {
+	socketPath = "\\\\.\\pipe\\webhook-" + hookTmpl
+	transport = &http.Transport{
+		DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
+			return winio.DialPipeContext(ctx, socketPath)
+		},
+	}
+
+	return socketPath, transport, nil, nil
+}