Warning: Attempt to read property "comment_content" on null in /usr/share/nginx/www/wp-content/plugins/smart-syntax/includes/functions.php on line 17

Sharing the Docker UNIX Socket with Unprivileged Containers: Redux

A few weeks ago I published a blog post discussing methods for sharing the Docker UNIX socket with unprivileged containers; for example when the Docker daemon has user namespaces enabled.

Brian Krebs made some useful comments about my assertion that a socat proxy in a privileged container, un-EXPOSEd to the host network, was one reasonable solution. Specifically, Brian showed how the container is most definitely accessible to other users on the host, given some rudimentary Linux skills for poking around to find the IP of the container in /proc file contents. Therefore, as I noted in my comment response it makes sense to show a more bulletproof example where we continue to provide Docker API access, but with limitations to prove that we could more securely share access without giving away the farm, so to speak.

Of course, my assumption so far is that commenter Brian Krebs is really the “krebsonsecurity.com” author, and not just some acne-challenged 17-year old named “Brian Krebs” from Surprise, Arizona who enjoys using his dad’s old laptop after school, on which he has hand-built and installed Gentoo Linux. We will continue with this assumption as it is much better for my street cred that the “real” Brian Krebs is commenting on my blog! [My lawyers asked me to include that “this blog post is no way infers or avows that the Gentoo Linux distribution, hereafter referred to as “Gentoo,” is merely a toy for pimply faced 17-year old basement hackers“]

So, with that aside, let’s take a look at a more complete solution for sharing the Docker API UNIX socket with unprivileged containers.  The general design I mentioned in a response to Brian includes using the new authorization plugin feature of the Docker engine joined to a more robust forwarder between HTTP -> UNIX socket that inserts a header to mark the API request as coming from an unprivileged user.

Authorization Plugin

The helpful guys at Twistlock contributed an authorization plugin framework to Docker, and it was merged into the 1.10 engine release back in February 2016. Additionally they have open sourced a policy-based plugin using this framework as an example for others to extend and use. For a simple implementation of the design idea mentioned above, I forked their plugin broker and added a second example implementation that uses an injected request header as the filter for approved actions for the Docker daemon API.

To test my new authorization plugin, I need to restart the Docker daemon and tell it I have an authz plugin. You can build the binary using the Makefile in the project above and run it on your Docker host as follows:

 $ sudo ./authz-broker --authz-handler=header \
                       --auditor=header

This will register the plugin in the appropriate location for Docker plugins (/run/docker/plugins), and then you may start the Docker daemon pointing to this plugin:

 $ sudo docker daemon --userns-remap=default \
        --authorization-plugin=authz-broker

I can watch the request/response streams from running Docker commands in the output of my authz-broker process and see that when using normal socket access (not via the forwarder I’m about to set up) I see it uses the codepath in my authorization code that allows any/all API access:

{"allow":     true,
 "err":       "",
 "fields.msg":"action 'docker_info' allowed; all privileges OK",
 "level":     "info",
 "method":    "GET",
 "msg":       "Request",
 "time":      "2016-04-20T10:43:49-04:00",
 "uri":       "/v1.23/info",
 "user":      ""}

Now that I’ve proven my authorization plugin is actually inserted into the API request flow, let’s set up the forwarder that will insert our special header.

Nginx-based Proxy

To make things simpler I’ve moved away from socat in this more advanced example because it turns out it is just dead simple to use a small nginx proxy to do this extra header insertion. As an added level of detail, remember that socat is at its essence just a packet copier, and we now have work we want to do at the HTTP layer. So, sure, I’m wimpy to escape to the ease of HTTP layer tools, but it really makes this example much simpler.

I’ve created a GitHub project with a Dockerfile which starts from the DockerHub hosted nginx image (I’m using the alpine-based one for a nice small image), and adds an extremely simple proxy/forwarder. Note that I have added the nginx user to the docker group, which I know on my host is gid 999; without this addition, the nginx proxy will not have read/write access to the mounted docker UNIX socket. This configuration enables functionality that is an exact copy of our socat worker from the prior post. The only additional work it is doing is adding our special header as it proxies the traffic along:

#server config
server {
    listen 2375;

    location / {
        proxy_pass http://unix:/var/run/docker.sock:;
        proxy_set_header X-Docker-Unprivileged true;
    }
}

We can then run this forwarder as we did our socat container in the last post: as a privileged container with the Docker socket mounted:

$ docker run --privileged --userns=host \
    -v /var/run/docker.sock:/var/run/docker.sock \
    unprivsockfwd

Now we can test our forwarder from an unprivileged container and see if our requests for anything other than ‘read’ APIs are blocked:

$ docker run -ti --rm --link unprivsockfwd:unpriv ubuntu bash
root@b8748db90c09:/# curl http://unpriv:2375/version
{"Version":"1.11.0","ApiVersion":"1.23","GitCommit":"4dc5990","GoVersion":"go1.5.4","Os":"linux","Arch":"amd64","KernelVersion":"4.2.0-35-generic","BuildTime":"2016-04-13T18:38:59.968579007+00:00"}
root@b8748db90c09:/# curl -X POST http://unpriv:2375/containers/create
authorization denied by plugin authz-broker: action 'container_create' not allowed due to unprivileged API access header
root@b8748db90c09:/#

It looks like our authorization plugin is doing what we asked it to do! For any simple HTTP GET API request we get the valid response from the Docker API, but if we try HTTP POST requests (like the container create API), we receive the disallowed response from our authz plugin given we only have unprivileged API access. Using this simple starting point, you could create a much richer set of API filtering depending on your specific allowance for unprivileged access to the Docker API. For example, given you receive the entire request detail, you could create a filter which allows only non-privileged containers to be created which request no volume mounts from the host.

Hopefully this provides a more secure method by which the Docker UNIX socket can be shared with unprivileged containers. I’ll be waiting with bated breath for responses from either of the Brian Krebs. :)

You may also like...

11 Responses

  1. Joachim says:

    I gave this a try however the code at https://github.com/estesp/authz doesn’t compile for me:

    $> make
    golint authz/basic.go || exit; golint authz/basic_test.go || exit; golint authz/doc.go || exit; golint authz/header.go || exit; golint broker/main.go || exit; golint core/doc.go || exit; golint core/interfaces.go || exit; golint core/route_parser.go || exit; golint core/route_parser_test.go || exit; golint core/server.go || exit; golint core/types.go || exit;
    authz/header.go:19:1: comment on exported function NewHeaderAuthZAuthorizer should be of the form “NewHeaderAuthZAuthorizer …”
    gofmt -w authz/basic.go authz/basic_test.go authz/doc.go authz/header.go broker/main.go core/doc.go core/interfaces.go core/route_parser.go core/route_parser_test.go core/server.go core/types.go
    go vet ./core/.; go vet ./broker/.; go vet ./authz/.;
    CGO_ENABLED=0 go build -o authz-broker -a -installsuffix cgo ./broker/main.go
    # command-line-arguments
    broker/main.go:50: undefined: authz.NewHeaderAuthZAuthorizer
    broker/main.go:59: undefined: authz.NewHeaderAuditor
    broker/main.go:59: undefined: authz.HeaderAuditorSettings
    Makefile:25: recipe for target ‘binary’ failed
    make: *** [binary] Error 2

    unfortunately my golang-foo is not good enough to solve this.

    the original implementation from twistlock compiles (so I’m fairly certain it isn’t my ENV considering I am able to compile plenty of other go packages without issues)

    it is a shame because it seems to be the only current implementation that properly routes with nginx.

    thanks for pointers

  2. estesp says:

    I believe this may be due to how/where you have cloned my authz fork of the original twistlock project. Because I did not update the import paths from the original “github.com/twistlock/authz” base, you would need to place this in a GOPATH under that same path:

    # assuming you have GOPATH as $HOME/go
    $ mkdir -p $HOME/go/src/github.com/twistlock
    $ cd $HOME/go/src/github.com/twistlock && git clone https://github.com/estesp/authz
    $ cd authz
    # actually.. before we build, we need to add the Godeps workspace to the $GOPATH
    $ export GOPATH=$GOPATH:`pwd`/Godeps/_workspace
    $ make binary

    That *should* properly build the binary for you. Let me know if you still get any failures!

  3. Joachim says:

    thanks that was it …, I managed to fix it with these changes.

    git diff broker/main.go
    diff –git a/broker/main.go b/broker/main.go
    index 3ea82b0..fa0370d 100644
    — a/broker/main.go
    +++ b/broker/main.go
    @@ -7,8 +7,8 @@ import (

    “github.com/Sirupsen/logrus”
    “github.com/codegangsta/cli”
    – “github.com/twistlock/authz/authz”
    – “github.com/twistlock/authz/core”
    + “github.com/estesp/authz/authz”
    + “github.com/estesp/authz/core”
    )

    const (

    8<———–8<———–8<———–8<———–8<———–

    in my setup I had to change the Makefile since the go get vet throws an error:

    git diff Makefile
    diff –git a/Makefile b/Makefile
    index c8ae65d..126ce83 100755
    — a/Makefile
    +++ b/Makefile
    @@ -12,7 +12,6 @@ fmt:
    gofmt -w $(SRCS)

    vet:
    – @-go get -v golang.org/x/tools/cmd/vet
    $(foreach pkg,$(PKGS),go vet $(pkg);)

    lint:

    8<————-8———————8——————–

    joachim@ejbca01:~/src/github.com/estesp/authz$ go version
    go version go1.7.4 linux/amd64
    joachim@ejbca01:~/src/github.com/estesp/authz$ go env
    GOARCH="amd64"
    GOBIN=""
    GOEXE=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOOS="linux"
    GOPATH="/home/joachim"
    GORACE=""
    GOROOT="/usr/local/go"
    GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
    CC="gcc"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build837981585=/tmp/go-build -gno-record-gcc-switches"
    CXX="g++"
    CGO_ENABLED="1"

    —-

    this setup produced a working authz-broker binary for me.

    thanks!!

  4. Joachim says:

    another silly question:
    I cloned
    https://github.com/estesp/Dockerfiles/blob/master/unprivsockfwd/Dockerfile

    and did:
    $> docker build -t unprivsocketfwd .

    docker build -t unprivsockfwd .
    Sending build context to Docker daemon 4.096 kB
    Step 1 : FROM nginx:alpine
    —> f1fae62370b4
    Step 2 : MAINTAINER Phil Estes
    —> Using cache
    —> 53dbaa737926
    Step 3 : RUN addgroup -g 999 docker
    —> Running in 22bad5614403
    addgroup: gid ‘999’ in use
    The command ‘/bin/sh -c addgroup -g 999 docker’ returned a non-zero code: 1

    the docker gid on my host is also 999 …

    any ideas? thanks!

  5. estesp says:

    Wow, interesting! Looks like the image nginx:alpine now has an entry “999” in /etc/group for a group named “ping”. One workaround is to add this “RUN” line prior to the addgroup:

    RUN grep 999 /etc/group | awk -F: ‘ { print $1 } ‘ | xargs delgroup 2>/dev/null || :

    The funny “|| :” at the end just makes sure it doesn’t cause docker build to error out if the 999 group doesn’t exist for any reason.

  6. Joachim says:

    thanks that fixed it (was unaware of the ||: trick :))

  7. Joachim says:

    minor change: awk -F: ‘ { print $1 } ‘ needs single quotes instead of subshell execution backticks

  8. Dirk says:

    curl http://unpriv:2375

    502 Bad Gateway
    Hi – got 502 from nginx

    502 Bad Gateway
    nginx/1.17.3

  9. Dirk says:

    ok – got it

    log:
    connect() to unix:/var/run/docker.sock failed (13: Permission denied) while connecting to upstream, client: 172.17.0.3, server: , request: “GET / HTTP/1.1”, upstream: “http://unix:/var/run/docker.sock:/”, host: “unpriv:2375”

    need to set privs for nginx inside unprivsockfwd
    chown 101:101 /var/run/docker.sock

  1. April 20, 2016

    […] Note: Please read my follow-up blog post on this topic based on the comments regarding open access to all Docker APIs via the methods discussed […]

  2. February 25, 2017

    […] additional blog post from me on the complexity of access to the Docker daemon UNIX socket from a user namespaced […]

Leave a Reply

Your email address will not be published. Required fields are marked *