v1.0, 15th May 2014

Introduction

Evolution of IT

Evolution of IT

The Challenge of Multiple Stacks and Multiple Hardware Environments

Challenge

Analogy: Shipping Pre-1960

Analogy

Solution to Shipping Challenge Was a Standard Container

Standard Containers

The Solution to Software Shipping is Also a Standard Container System

Docker Container

Basic Docker Functions

Docker Functions

Containers vs. Traditional VMs

Docker Vs VMs

Modfiying and Updating Containers

Modifying and updating with containers

Docker Usecases

  • PaaS

  • Application deployment

  • Sandboxing

  • CI and CD

  • Desktop Deployment

  • etc

Linux containers - Underlying Technology

Docker Execution Driver
Docker layers

Docker Components

Image

It is a template which is used to launch containers.

Container

It is like a directory. A Docker container holds everything that is needed for an application to run.

Index

It manages user accounts, permissions, search, tagging, and all that nice stuff that’s in the public web interface.

Registry

It stores and serves up the actual image assets, and it delegates authentication to the index.

Repository

A registry holds a collection of named repositories, which themselves are a collection of images tracked by GUIDs.

Installation

Fedora

$ sudo yum -y install docker-io
$ sudo systemctl start docker
$ sudo systemctl enable docker

RHEL

$ sudo yum -y install docker-io
$ sudo service docker start
$ sudo chkconfig docker on

Ubuntu

$ sudo apt-get update
$ sudo apt-get install docker.io
$ sudo ln -sf /usr/bin/docker.io /usr/local/bin/docker

Check your Docker Install

$ docker info
Containers: 6
Images: 15
Storage Driver: devicemapper
 Pool Name: docker-253:0-15251-pool
 Data file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata file: /var/lib/docker/devicemapper/devicemapper/metadata
 Data Space Used: 1623.6 Mb
 Data Space Total: 102400.0 Mb
 Metadata Space Used: 2.1 Mb
 Metadata Space Total: 2048.0 Mb
Execution Driver: native-0.1
Kernel Version: 3.14.2-200.fc20.x86_64

Docker basic commands

search in public docker index

$ docker search fedora
NAME                                DESCRIPTION                                     STARS     OFFICIAL   TRUSTED
mattdm/fedora                       A basic Fedora image corresponding roughly...   50
fedora                              (Semi) Official Fedora base image.              39
mattdm/fedora-small                 A small Fedora image on which to build. Co...   8
goldmann/wildfly                    A WildFly application server running on a ...   3                    [OK]
tutum/fedora-20                     Fedora 20 image with SSH access. For the r...   1                    [OK]
search in private docker index

TODO

pull an image

$ docker pull fedora
Pulling repository fedora
b7de3133ff98: Download complete
5cc9e91966f7: Download complete
511136ea3c5a: Download complete
ef52fb1fe610: Download complete
Tip During rootconf workshop
  • Use private repo

  • Download the image and import

listing the images

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
fedora              rawhide             5cc9e91966f7        2 days ago          372.7 MB
fedora              20                  b7de3133ff98        2 weeks ago         372.7 MB
fedora              heisenbug           b7de3133ff98        2 weeks ago         372.7 MB
fedora              latest              b7de3133ff98        2 weeks ago         372.7 MB

Start a container from an image

$ docker run -i -t fedora /bin/bash
bash-4.2#

Listing the containers

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
336c2bd436a3        fedora:20           /bin/bash           34 seconds ago      Up 32 seconds                           thirsty_darwin
Tip listing all the containers (running and stopped)
$ docker ps -a

Saving a container state

$ docker run -i -t fedora /bin/bash
bash-4.2# echo "Rootconf workshop 2014" > /etc/motd

On a differet terminal

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
d8243f5ef7        fedora:latest       /bin/bash           3 minutes ago       Up 3 minutes                            stoic_shockley
$ docker commit -a "Neependra Khare" -m "Rootconf 2014" d8243f5ef7  rootconf:motd
3c960f8457229d1fefcf5ac89e4265b6aff321c61bced32a8de8053f46e60cab

Listing the images in tree format

root@dhcp-0-125 docker]# docker images -t
Warning: '-t' is deprecated, it will be removed soon. See usage.
└─511136ea3c5a Virtual Size: 0 B
  └─ef52fb1fe610 Virtual Size: 0 B
    ├─5cc9e91966f7 Virtual Size: 372.7 MB Tags: fedora:rawhide
    └─b7de3133ff98 Virtual Size: 372.7 MB Tags: fedora:20, fedora:heisenbug, fedora:latest
      └─3c960f845722 Virtual Size: 372.7 MB Tags: rootconf:motd
Tip To display them graphically
$ docker images --viz | dot -Tpng -o /tmp/docker.png
Docker images
Tip By looking into docker metadata
$ cat /var/lib/docker/repositories-devicemapper  | python -mjson.tool
{
    "Repositories": {
        "fedora": {
            "20": "b7de3133ff989df914ae9382a1e8bb6771aeb7b07c5d7eeb8ee266b1ccff5709",
            "heisenbug": "b7de3133ff989df914ae9382a1e8bb6771aeb7b07c5d7eeb8ee266b1ccff5709",
            "latest": "b7de3133ff989df914ae9382a1e8bb6771aeb7b07c5d7eeb8ee266b1ccff5709",
            "rawhide": "5cc9e91966f7b9c84c44682cb76886d95f2d5c5e7e689770410af3f27471641a"
        },
        "rootconf": {
            "motd": "3c960f8457229d1fefcf5ac89e4265b6aff321c61bced32a8de8053f46e60cab"
        }
    }
}

Looking details for an layer

$ cat  /var/lib/docker/graph/3c960f8457229d1fefcf5ac89e4265b6aff321c61bced32a8de8053f46e60cab/json | python -m json.tool
{
    "Size": 37,
    "architecture": "amd64",
    "author": "Neependra Khare",
    "comment": "Rootconf 2014",
    "config": {
        "AttachStderr": false,
        "AttachStdin": false,
        "AttachStdout": false,
        "Cmd": [
            "/bin/sh",
            "-c",
            "echo welcome rootconf2014 docker workshop  > /etc/motd"
        ],
        "CpuShares": 0,
        "Domainname": "",
        "Entrypoint": null,
        "Env": [
            "HOME=/",
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "ExposedPorts": null,
        "Hostname": "",
        "Image": "",
        "Memory": 0,
        "MemorySwap": 0,
        "NetworkDisabled": false,
        "OnBuild": null,
        "OpenStdin": true,
        "PortSpecs": null,
        "StdinOnce": false,
        "Tty": true,
        "User": "",
        "Volumes": null,
        "WorkingDir": ""
    },
    "container": "d8243f5ef7aada4f8ec8e3b803eb274b2bcd9655c8b22aa701bd6b38f35738eb",
    "container_config": {
        "AttachStderr": false,
        "AttachStdin": false,
        "AttachStdout": false,
        "Cmd": [
            "/bin/sh",
            "-c",
            "echo welcome rootconf2014 docker workshop  > /etc/motd"
        ],
        "CpuShares": 0,
        "Domainname": "",
        "Entrypoint": null,
        "Env": [
            "HOME=/",
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "ExposedPorts": null,
        "Hostname": "d8243f5ef7aa",
        "Image": "fedora",
        "Memory": 0,
        "MemorySwap": 0,
        "NetworkDisabled": false,
        "OnBuild": null,
        "OpenStdin": true,
        "PortSpecs": null,
        "StdinOnce": false,
        "Tty": true,
        "User": "",
        "Volumes": null,
        "WorkingDir": ""
    },
    "created": "2014-05-12T07:22:48.406627204Z",
    "docker_version": "0.11.1",
    "id": "3c960f8457229d1fefcf5ac89e4265b6aff321c61bced32a8de8053f46e60cab",
    "os": "linux",
    "parent": "b7de3133ff989df914ae9382a1e8bb6771aeb7b07c5d7eeb8ee266b1ccff5709"
}

Attaching to a container

$ ID=$(sudo docker run -d fedora /bin/sh -c "while true; do echo Rootconf14 ; sleep 1; done")
$ docker attach $ID

Stopping a conatiner

$ docker stop $ID

Starting a container

$ docker start $ID

Restart the container

$ docker restart $ID

Looking at the logs of container

$ docker logs $ID

Killing a container

$ docker stop $ID
$ docker rm $ID

Delete all stopped containers

$ docker rm `docker ps -a -q`

Privileged access inside container

$ sudo docker run -t -i fedora /bin/bash
bash-4.2# docker run -t -i fedora /bin/bash
bash-4.2# mount -t tmpfs none /mnt
mount: permission denied
bash-4.2# exit
$ docker run --privileged -t -i fedora /bin/bash
bash-4.2# mount -t tmpfs none /mnt

Return low level information about container/images like IP address, port mapping

$ docker inspect $ID
$ docker inspect --format='{{.NetworkSettings.IPAddress}}'  $ID

Copy files/folders from the containers filesystem to the host path.

$ docker cp $ID:/etc/motd /tmp/

Run a command inside a directory while starting a container

$ sudo docker run -t -i -w /etc fedora ls

Note It the path does not exist it would get created

Exposing a port while starting a container

$ ID=$(docker run --expose=22,80 -d -i -t rootconf:sshd /bin/bash)
$ IP=$(docker inspect --format='{{.NetworkSettings.IPAddress}}' $ID)
$ telnet $IP 22
$ docker attach $ID
bash-4.2# useradd rootconf
bash-4.2# passwd rootconf
bash-4.2# mkdir /var/run/sshd
bash-4.2# ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N ''
bash-4.2# /usr/sbin/sshd -D
From host machine
$ telnet $IP 22
$ ssh rootconf@$IP

List the changed files and directories in a container’s filesystem

$  docker diff $ID
C /home
A /home/rootconf
A /home/rootconf/.bash_logout
A /home/rootconf/.bash_profile
A /home/rootconf/.bashrc
C /run
A /run/sshd
A /run/sshd.pid
C /run/utmp
C /var/spool/mail
A /var/spool/mail/rootconf
C /var/log/btmp
C /var/log/lastlog
C /var/log/wtmp
C /etc
C /etc/gshadow
C /etc/ssh
A /etc/ssh/ssh_host_rsa_key
A /etc/ssh/ssh_host_rsa_key.pub
C /etc/group-
C /etc/shadow
C /etc/group
C /etc/passwd-
C /etc/gshadow-
C /etc/shadow-
C /etc/passwd
  • A Add

  • D Delete

  • C Change

Build an image

$ mkdir /tmp/rootconf; cd /tmp/rootconf
$ echo "FROM fedora"  >> Dockerfile
$ echo "MAINTAINER Neependra" >> Dockerfile
$ docker build -t rootconf/fedora .
$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
rootconf/fedora     latest              b1a11ec1191d        12 seconds ago      372.7 MB
fedora              rawhide             5cc9e91966f7        2 days ago          372.7 MB
fedora              20                  b7de3133ff98        2 weeks ago         372.7 MB

NOTE:The transfer of context from the local machine to the Docker daemon is what the docker client means when you see the "Uploading context" message.

Tip Install fedora-dockerfiles on Fedora 20 and look docker files example under /usr/share/fedora-dockerfiles/
DockerFile Instructions
  • FROM <image> | <image>:<tag>

    Set the base image
  • MAINTAINER <name>

    Set the author
  • RUN <cmd> | ["executable", "param1", "param2"]

    Executes any commands in a new layer on top of the current image and commit the results
  • CMD ["executable","param1","param2"] | ["param1","param2"] | command param1 param2

    Provides defaults for an executing container
  • EXPOSE <port> [<port> …]

    Open up specified network ports at runtime
  • ENV <key> <value>

    This sets the environment variable <key> to the value <value>
  • ADD <src> <dest>

    Copy new files from source and add them to the container's filesystem at path
  • ENTYRPOINT ["executable", "param1", "param2"] | command param1 param2

    Helps to configure a container that you can run as an executable.
  • VOLUME ["/data"]

    Creates a mount point with the specified name and mark it as holding externally mounted volumes from native host or other containers.
  • USER

    Sets the username or UID to use when running the image.
  • WORKDIR

    Sets the working directory
  • ONBUILD [INSTRUCTION]

    Adds to the image a "trigger" instruction to be executed at a later time, when the image is used as the base for another build.

Change the CMD within container and committing it

# docker commit -run='{"Cmd": ["ls","/etc"]}' $ID fedoranew
# docker run -t -i fedoranew

History of the image

$ docker history rootconf/fedora
 IMAGE               CREATED             CREATED BY                                      SIZE
b1a11ec1191d        52 minutes ago      /bin/sh -c #(nop) MAINTAINER Neependra          0 B
b7de3133ff98        2 weeks ago         /bin/sh -c #(nop) ADD file:a117c9572f1c87cbe6   372.7 MB
ef52fb1fe610        2 weeks ago         /bin/sh -c #(nop) MAINTAINER Lokesh Mandvekar   0 B
511136ea3c5a        11 months ago                                                       0 B

Remove an image

$ docker rmi rootconf/fedora

Insert a file to an image

$ ID=$(docker images -q  | head -n1)
$ docker insert $ID https://raw.githubusercontent.com/nkhare/btrfsFun/master/README.md /tmp/btrfs_example.txt
$ docker run -t <ID> cat /tmp/btrfs_example.txt

Export the contents of a filesystem as a tar archive to STDOUT

$ ID=$(sudo docker run -d -i -t fedora /bin/bash)
$ docker export $ID > /tmp/image.tar

Importing a filesystem

$ cat /tmp/image.tar | docker import  - newimage
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
newimage            latest              03eda5a7643e        11 seconds ago      372.7 MB
rootconf            sshd                c5240f9cb1f1        15 hours ago        560.6 MB
Tip Filesystem can be imported through URL. $ docker import <URL>

Wait until a container stops

$ ID=$(sudo docker run -d fedora /bin/sh -c " echo Rootconf14 ; sleep 10")
$ docker wait $ID
Note It returns the exit code

Share Directories via Volumes

Create the named container with volumes

$ docker run -d -i -t  -v /var/volume1 -v /var/volume2 --name DATA fedora bash

Mount the volumes on another container

$ docker run -t -i -rm --volumes-from DATA --name client1 fedora bash

Mount a host directory on a conatiner

$ docker run  -i -t -v /var/logs:/logs_from_host:ro fedora bash
bash-4.2# ls logs_from_host/

Get events from the docker server

$ docker events

Port redirection

Bind a port to host interface

  • Bind TCP port 8080 of the container to TCP port 80 on 127.0.0.1 of the host machine.

$ docker run -d -i -t -p 127.0.0.1:8080:8080 fedora bash
  • Bind TCP port 8080 of the container to a dynamically allocated TCP port on 127.0.0.1 of the host machine.

$ docker run -d -i -t -p 127.0.0.1::8080 fedora bash
  • Bind TCP port 8080 of the container to TCP port 80 on all available interfaces of the host machine.

$ docker run -d -i -t -p 80:8080 fedora bash
  • Bind TCP port 8080 of the container to a dynamically allocated TCP port on all available interfaces of the host machine.

$ docker run -d -i -t -p 8080 fedora bash

Lookup for mapping for a given container

$ ID=$(docker run -d -i -t -p 127.0.0.1:8080:80 fedora bash)
$ docker port $ID 8080

Linking containers

$ docker run -d -i -t --expose 80 --name server fedora bash
$ docker run -i -t  --name client --link server:linked_server fedora bash
bash-4.2# env
HOSTNAME=1ca41086bd7e
TERM=xterm
LINKED_SERVER_NAME=/client/linked_server
LINKED_SERVER_PORT_80_TCP_PORT=80
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LINKED_SERVER_PORT_80_TCP_ADDR=172.17.0.4
LINKED_SERVER_PORT_80_TCP_PROTO=tcp
LINKED_SERVER_PORT=tcp://172.17.0.4:80
SHLVL=1
HOME=/
LINKED_SERVER_PORT_80_TCP=tcp://172.17.0.4:80
_=/usr/bin/env

Excercise : Try to access from different container

Accessing docker through python bindings

$ pip install docker-py
$ cat example.py
import docker
import sys
# part of https://github.com/humblec/dockit.git
DOCK_SOCK = "unix://var/run/docker.sock"
DOCK_VERSION = "1.9"
# Program to Connect to docker deamon and talk
    try:
        conn = docker.Client(base_url=DOCK_SOCK, version=DOCK_VERSION, timeout=30)
        if conn:
            print "Successfully Connected to the docker deamon \n "
            print "Available methods with connection object %s \n" %(dir(conn))
            print "Retrieve docker information \n"
            print conn.info()
        else:
            print "Connection Failed "
    except Exception as e:
            print e
            sys.exit(1)

Putting CPU and memory constraint for a container

$ ID=$(docker run -d -m=500m -c=2 -t -i fedora /bin/bash)
$ docker inspect $ID | grep -ie memory -e cpu
"Memory": 524288000,
"MemorySwap": 0,
"CpuShares": 2,

Setting up the private registry

$ yum -y install docker-io docker-registry
$ service docker start
$ service docker-registry start

Configure following files

/etc/docker-registry.yml
/etc/sysconfig/docker-registry

Run a container and commit it locally

$ ID=$(docker run -d -m=500m -c=2 -t -i fedora /bin/bash)
$ docker commit $ID localhost.localdomain:5000/fedoralocal
[root@dhcp-0-125 docker]# docker images
REPOSITORY                               TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
localhost.localdomain:5000/fedoralocal   latest              22a2be17c984        12 seconds ago      372.7 MB

References