docker-reveal

Docker

Why docker?

Docker components

$ docker --version
Docker version 26.0.0, build 2ae903e

$ docker info
Client: Docker Engine - Community
 Version:    26.0.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.13.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.25.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 51
  Running: 49
  Paused: 0
  Stopped: 2
 Images: 49
 Server Version: 26.0.0
 Storage Driver: overlay2
  Backing Filesystem: xfs...

Commands shown:

docker run <image>
docker run -it <image> <command>
docker run -d <image> <command>
docker attach
# Ctrl+P,Q to detach
docker stop <container>
docker kill <container>
docker images
docker ps -a
docker pull <image>
docker pull <image:tag>

Images and containers

Images

Commands shown:

docker pull <image>
docker pull -a <image>
docker images
docker history <image>
docker rmi <image>

Exercise 1 - Hello world

  1. Run the hello-world image in a new container
  2. Remove the container
  3. Remove the image
  4. Also try it the other way around!

Containers

Commands shown:

docker run --name <name> <image> <command>
docker run -it <image> <command>
docker run -d <image> <command>
docker run --rm <image>
docker exec -it <container> <command>
docker top <container>
docker attach <container>

docker start <container>
docker stop <container>
docker restart <container>

docker ps
docker ps -a
docker rm <container>

docker create --name <container> <image>
docker tag <image> <tag>
docker inspect <container>
docker port <container>
docker logs [-f] <container>

docker commit <container> <image>[:<tag>]
docker save -o <file> <image>
tar -tf <file>
docker rmi <image>
docker load -i <file>

docker run -p <public port>:<internal port> <image>
docker run -v <local path>:<container path> <image>

Exercise 2 - Run ping in the background

  1. Pull the alpine image
  2. List all images
  3. Run the alpine image with ping 8.8.8.8 command
  4. List running containers
  5. Show/follow the output from the running ping
  6. Stop container
  7. Start container
  8. Step into the container, list all processes

Exercise 3 - Create a new image manually

  1. Start an ubuntu container and step into it
  2. Install curl and exit
  3. List all containers
  4. Commit the changed container creating a new image
  5. List all images
  6. Name the new image curl_example:1.0
  7. Run the new image with curl https://www.google.com

Creating images using Dockerfile

ENTRYPOINT statement

Best practices

  1. Containers should be ephemeral
  2. Use a .dockerignore file
  3. Use small base images (e.g. Alpine)
  4. Reuse base images across your organization
  5. Use tagged base images
  6. Use tagged app images
  7. Group common operations into a single layer
  8. Avoid installing unnecessary packages or keeping temporary files
  9. Clean up after yourself in the same RUN statement
  10. Run only one process per container (try to avoid supervisord and the like)
  11. Minimize the number of layers

  12. Sort multi-line arguments and indent 4 spaces:

    RUN apt-get update && apt-get install --yes \
    cvs \
    git \
    mercurial \
    subversion
  13. FROM: Use current official repositories

  14. RUN: Split long or complex RUN statements across multiple lines separated

    RUN command-1 && \
    command-2 && \
    command-3
  15. Avoid distribution updates à la RUN apt-get upgrade

  16. Use the JSON array format for CMD to prevent an additional shell as PID 1

    CMD ["executable", "param1", "param2", "..."]
CMD ["apache2", "-DFOREGROUND"]
CMD ["perl", "-de0"]
CMD ["python"]
CMD ["php", "-a"]
  17. Use ENTRYPOINT only when required
  18. EXPOSE the usual ports for your applications
  19. Prefer COPY over ADD

  20. Leverage the build cache, disable if necessary: docker build --no-cache=true -t <image>[:<tag>] .

    # Will use cache unless requirements.txt change.
COPY requirements.txt /tmp/
RUN pip install --requirement /tmp/requirements.txt
# Will use cache unless any file changes.
COPY . /tmp/

  21. Do not use ADD to download files, although it’s possible. Use RUN with curl, unzip/... and rm instead to keep images small:

    # Bad - 3 layers.
ADD http://example.com/big.tar.xz /usr/src/things/
RUN tar -xJf /usr/src/things/big.tar.xz -C /usr/src/things
RUN make -C /usr/src/things all

# Good - 1 layer.
RUN mkdir -p /usr/src/things \
    && curl -SL http://example.com/big.tar.xz \
      | tar -xJC /usr/src/things \
    && make -C /usr/src/things all
  22. Use gosu when required to run as non-root
  23. Have integration tests
  24. Develop the Dockerfile in a running container, REPL-style
  25. Include a HEALTHCHECK in the Dockerfile or run containers with --health-{cmd,interval,timeout,retries}
  26. Define directories that will contain persistent data with VOLUMEs
  27. Use multi-stage builds if you have SDK requirements for the build that you do not need for production

Commands shown:

docker build .
docker build --tag <image>:<tag> .
docker build --build-arg <arg>=<value>
docker run --env <var>=<value>
docker run -p <external port>:<internal port> <image>
docker run -p <external port>:<internal port>/udp <image>
docker run -p <ip address>:<external port>:<internal port> <image>
docker run -P <image>
docker port <container>

Exercise 4 - Create a new image using a Dockerfile

  1. Repeat the steps from Exercise 3 above using a Dockerfile

Exercise 5 - Create an app image using a Dockerfile

  1. Create a Dockerfile for https://github.com/agross/docker-hello-app
  2. Build docker image
  3. Run container from image
  4. Open web site at http://localhost:8080

Exercise 6 - Define additional environment variables

  1. Stop container from Exercise 5
  2. Run container from image, with a custom environment variable THE_ANSWER=42
  3. Open web site at http://localhost:8080 and see if THE_ANSWER is there

Exercise 7 - Overlay files from the image

  1. Stop container from Exercise 5

  2. Create a new file index.jade in the current directory:

    html
  body Content from the host
  3. Start a new container, but overlay the /app/views/ directory with the directory that contains the index.jade file above
  4. Refresh browser and check if “Content from the host” is displayed
  5. Enter the running container and change the contents of /app/views/index.jade (e.g. using echo)
  6. Refresh browser
  7. Check the contents of index.jade on your host

Registries and repos

docker login [<server>]
docker push <image>

Exercise 8 - Push and pull with Docker Hub

  1. Create an account at hub.docker.com
  2. Create a new repo named hello under your user account on Docker Hub
  3. Tag the image created in Exercise 5 as <hub user name>/hello
  4. Log in to Docker Hub using the Docker Client
  5. Push the tagged image
  6. Check https://hub.docker.com/r/<hub user name>/hello/tags/ for your uploaded image
  7. Locally remove the tagged image
  8. Run the image just pushed to Docker Hub

Volume mounts

docker volume ls
docker volume create <name>

docker run -d <image>
docker run -d -v <name>:<mount point> <image> <command>

Exercise 9 - Write output to volume

  1. Create persistent volume named pings
  2. Run alpine with sh -c 'ping 8.8.8.8 > /data/ping.txt' with /data being mounted to the pings volume
  3. Run another alpine container that mounts pings and inspect pings.txt contents using tail -f

Container networking

Exercise 10 - Container networking

  1. Create a network named hello
  2. Run two instances of agross/hello on the hello network, but --name them differently (one and two) and to not publish ports
  3. Inspect one and two, look for network settings
  4. Inspect the hello network
  5. Step into either one or two and try to ping the other

  6. Run nginx on the hello network

    docker run --rm -d --name nginx --network hello -p 80:80 nginx
  7. Browse http://localhost to verify that nginx is working
  8. Stop nginx

  9. Write a nginx config file (hello.conf) that uses one and two as upstreams:

    upstream hello {
  server one:8080;
  server two:8080;
}

server {
  listen 80;
  location / {
    proxy_pass http://hello;
  }
}

  10. Restart nginx, this time with the conf above bind-mounted to /etc/nginx/conf.d/default.conf

    docker run -d --name nginx --network hello -p 80:80 -v $PWD/hello.conf:/etc/nginx/conf.d/default.conf nginx
  11. Browse http://localhost again and refresh a few times

Multi-container apps

Exercise 11 - Start WordPress

  1. Download wordpress and mariadb
  2. Create a new network for both apps

  3. Run a MariaDB container on the network from step 2 and inject some environment variables:

    MARIADB_ROOT_PASSWORD=secret
MARIADB_DATABASE=wordpress
MARIADB_USER=wordpress
MARIADB_PASSWORD=wordpress

    Is there a better way than multiple --env parameters? Hint: Use a file.

  4. Run WordPress on the network from step 2 and tell it where it can find the database server:

    WORDPRESS_DB_HOST=<mariadb container name>:3306
WORDPRESS_DB_USER=wordpress
WORDPRESS_DB_PASSWORD=wordpress
  5. Browse http://localhost and create a WordPress site. Did you forget to publish ports? ;-)
  6. Restart the WordPress container to see if your installation was persisted
  7. Congratulations, you just emulated docker-compose!

docker compose use-cases

  1. Development environments:
    • Running web apps in an isolated environment is crucial
    • The compose file allows to document service dependencies
    • Multi-page “developer getting started guides” can be avoided
  2. Automated testing environments
    • Create & destroy isolated testing environments easily
    • Concurrent builds do not interfere
  3. Production
    • All config items are in one place
    • Passwords are near-meaningless
    • Services do not interfere

Features

  1. docker compose creates a per-composition network by default, named after the current directory (unless docker compose -p <name> ... is specified).
  2. When docker compose up runs it finds any containers from previous runs and reuses the volumes from the old container (i.e. data is restored).
  3. When a service restarts and nothing has changed, docker compose reuses existing containers because it caches the configuration bits that were used to create a container.

  4. Variables in the docker-compose.yaml file can be used to customize the composition for different environments.

    web:
  ports:
    - "${EXTERNAL_PORT}:5000"

  5. Override settings for different environments, e.g. production.yml containing changes specific for production:

    docker compose -f docker-compose.yml -f production.yml up -d
  6. You can scale services with docker-compose scale <service>=<instances>

Exercise 12 - WordPress using docker compose

Since WordPress cannot run without a database connection we need to ensure that the database is ready to accept connections before WordPress starts. But Docker does not really care about startup order and service readiness. There are several solutions to this problem. Some involve using external tools like:

Using these external tools require you to change a container’s ENTRYPOINT or CMD (depending on how the image defines those). This change involves defining the dependency using the external tool and also telling the external tool what it means to start e.g. WordPress.

Docker’s builtin method, which is only available to docker-compose.yaml files using version: 2 (e.g 2.x), is to define a HEALTHCHECK-based dependency. Here the dependent container (database) must define healthiness and the depending container (WordPress) can then define its dependency to be satisfied if the dependent is healthy.

 services:
  db:
    image: mysql

    # MySQL does not come with a HEALTHCHECK, so we need to define our own.
    healthcheck:
      # This check tests weather MySQL is ready to accept connections.
      test: ["CMD", "mysql", "--user", "root", "--password=secret", "--execute", "SELECT 1;"]
      # Allow 15 seconds for MySQL initialization before running the first check.
      start_period: 15s

  app:
    image: wordpress

    # Start the WordPress container after the database is healthy.
    depends_on:
      db:
        condition: service_healthy

You may use either method in the next exercise.

  1. Create a directory my-wordpress and enter it
  2. Create a new docker-compose.yaml

  3. Paste the following:

    services:
  mariadb:
    image: mariadb
    environment:
      MARIADB_ROOT_PASSWORD: secret
    volumes:
      - ./wp/db/conf:/etc/mysql/conf.d:ro
      - ./wp/db/data:/var/lib/mysql

    # Required because of "condition: service_healthy" below.
    healthcheck:
      test: ["CMD", "mariadb", "-uroot", "-psecret", "-e", "SELECT 1;"]
      start_period: 5s
      interval: 5s

  wordpress:
    image: wordpress:php7.0

    # Define "db" alias for the mariadb service.
    links:
      - mariadb:db

    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_PASSWORD: secret
    ports:
      - 80:80
    restart: always
    volumes:
      - ./wp/data:/var/www/html/wp-content

    # Define startup order.
    depends_on:
      mariadb:
        condition: service_healthy
      smtp:
        condition: service_started

  smtp:
    image: mwader/postfix-relay
    restart: always
    environment:
      POSTFIX_myhostname: example.com
  4. Run docker compose up
  5. Create a new WordPress site
  6. Inspect running docker containers. What do you see?
  7. Inspect docker networks. What do you see?

Exercise 13 - Upgrade WordPress to use PHP 7.1

  1. Have the composition from Exercise 11 running
  2. Change docker-compose.yaml such that the php7.1 tag is used for wordpress

  3. Rebuild and restart the WordPress container:

    docker compose up --no-deps -d wordpress

Exercise 14 - Maintain your environment

  1. Scale mariadb to 2 instances
  2. Step into mariadb using docker compose exec. In which instance did you step?
  3. Restart the composition. How many mariadb instances are there?
  4. Pause and unpause the application
  5. Tear down the application and remove all containers, networks, etc.

Exercise 15 - Debug your environment

  1. Have a look at the logs for the composition and for a single service
  2. Step into the WordPress container and kill all Apache processes with kill $(pgrep -f apache). Is the container restarted? Why?
  3. Retrieve the public port for WordPress

Exercise 16 - Switch to volume

  1. Stop the service from Exercise 11 (docker compose stop)

  2. Add a new top-level section:

    volumes:
  wp-db-conf:
  wp-db-data:
  wp-data:
  3. Change all volumes: to use volume mounts instead of bind mounts. E.g. - ./wp/mariadb/data:/var/lib/mysql becomes - wp-db-data:/var/lib/mysql
  4. Start the service again
  5. Browse http://localhost. Is the WordPress site still available? Why? What should have been done in addition to changing the configuration?

Exercise 17 - Locally build composition

  1. Assume agross/hello needs a database
  2. Can you think of a docker-compose.yaml file that builds a composition of agross/hello’s source code at https://github.com/agross/docker-hello and e.g. MariaDB?