Clocker

The Docker Cloud Maker

View On GitHub

Introduction

This tutorial is focused on deploying a production ready Docker Swarm.

Pre-requisites

This tutorial assumes you have completed the getting started section of this website and have installed the Apache Brooklyn CLI.

Overview

The production ready swarm cluster is comprised of the following components:

A load-balanced cluster of swarm managers

Swarm managers control a swarm’s nodes and dictate the node on which containers are deployed. We interact directly with the swarm manager cluster’s load balancer as if it were a single docker node. The load-balancer will redirect traffic to a healthy manager when a manager fails. The replacer policy will detect the failure and replace the failed manager.

A cluster of swarm nodes

These nodes are where docker containers are deployed to. The cluster has an AutoScalerPolicy and will scale up due to high CPU usage.

etcd Cluster

Used as a discovery backend for the swarm cluster.

CA Server

This is used to provide TLS certificates for the swarm cluster. This component is designed to be easily replaced. It is strongly recommended that this component is replaced with a production grade CA server of your choice.

Instructions

Setup a cloud location

Firstly, we need to setup a location to deploy the Swarm cluster to. We recommend the following settings:

Please note that we recommend the official Centos 7 images. Images from other providers may be less functional or incompatible.

The following catalog items should enable you to quickly get started on some popular clouds. Download the .bom file of the relevant cloud, add your credentials, and then run:

br add-catalog <CLOUD-PROVIDER>-example-location.bom
brooklyn.catalog:
  id: aws-central-centos7
  name: "AWS Frankfurt CentOS 7"
  itemType: location
  item:
    type: jclouds:aws-ec2
    brooklyn.config:
      region: eu-central-1
      identity: <IDENTITY>
      credential: <CREDENTIAL>
      minRam: 2000

# Make sure you've accepted the TOC for the image before using it. To do so
# go to https://aws.amazon.com/marketplace/pp/B00O7WM7QW and try to start
# an instance with the image. In the process the UI will ask you to accept
# the TOC. There is no need to actually launch the instance.
# If you have not accepted the TOC you'll get 401 responses from EC2'a API.
#
# To find the AMIs for different regions go to (login required):
# https://aws.amazon.com/marketplace/fulfillment?productId=b7ee8a69-ee97-4a49-9e68-afaee216db2e
# and click on "Manual Launch". There you'll see a list of regions and the corresponding image IDs.
      imageId: eu-central-1/ami-9bf712f4

# Provision a maximum of 3 machines in parallel to avoid hitting the
# maximum allowed request limit rate.
      maxConcurrentMachineCreations: 3

      loginUser: centos

Download aws-example-location.bom

brooklyn.catalog:
  id: sl-lon-centos7
  name: "Softlayer London CentOS 7"
  itemType: location
  item:
    type: jclouds:softlayer
    brooklyn.config:
      region: lon02
      identity: <IDENTITY>
      credential: <CREDENTIAL>

      minRam: 2000
      imageId: CENTOS_7_64

Download sl-example-location.bom

brooklyn.catalog:
  id: azure
  name: "Azure North Europe"
  itemType: location
  item:
    type: jclouds:azurecompute
    brooklyn.config:
      identity: <IDENTITY>
      credential: <CREDENTIAL>
      endpoint: <AZURE ENDPOINT>
      vmNameMaxLength: 45
      jclouds.azurecompute.operation.timeout: 120000
      
      # this line disables an invalid openlogic repo
      # sudo yum-config-manager --disable openlogic
      setup.script: data:text/plain;base64,c3VkbyB5dW0tY29uZmlnLW1hbmFnZXIgLS1kaXNhYmxlIG9wZW5sb2dpYw==
      imageId: 5112500ae3b842c8b9c604889f8753c3__OpenLogic-CentOS-72-20160303/North Europe

      regionId: North Europe
      hardwareId: BASIC_A2
      loginUser: user
      templateOptions:
        overrideAuthenticateSudo: true

Download azure-example-location.bom

brooklyn.catalog:
  id: gce-europe-centos7
  name: "Google Compute Engine Europe Centos 7"
  itemType: location
  item:
    type: jclouds:google-compute-engine
    brooklyn.config:
      imageNameRegex: centos-7.*
      region: europe-west1-b
      minRam: 2000
      identity: <IDENTITY>

      # Use a pre-created everything open network to avoid quota limitations
      templateOptions:
        network: <NETWORK>

      credential: <CREDENTIAL>

Download gce-example-location.bom

brooklyn.catalog:
  id: ibm-bluebox-sng-centos7
  name: "IBM BlueBox Singapore CentOS 7"
  itemType: location
  item:
    type: jclouds:openstack-nova
    brooklyn.config:
      endpoint: <ENDPOINT>
      identity:  <IDENTITY>
      credential: <CREDENTIAL>
      jclouds.keystone.credential-type: passwordCredentials

      generate.hostname: true

    # You need to make sure you have a image with name "Centos 7.0"
      imageNameRegex: CentOS 7
      loginUser: centos
      minRam: 2000

# By default open stack will use an existing network
# We recommend creating one and specifying below
      templateOptions:
        networks:
         - "<NETWORK ID>"

# There are a couple of known issues with auto configuring security groups
# on BlueBox using jclouds.  We recommend configuring a security group manually
# that allows all internal communication between VMs and inbound traffic on
# 22, 8080, and 32768-65534 (for swarm) or 30000-32767 (for kubernetes)
# You will also need to set either kubernetes.sharedsecuritygroup.create or
# swarm.sharedsecuritygroup.create to false when you deploy the swarm or kubernetes

      securityGroups: <SECURITY GROUP ID>

Download bb-example-location.bom

Deploy a Swarm Cluster

After the location is setup, it is time to deploy a Docker Swarm.

From your AMP Install, head to the AMP Welcome page. In the quick deploy section select “Docker Swarm with Discovery and CA” and select the location that that we setup in the previous step. You can also change some configuration options such as the minimum and maximum number of nodes. Once you are happy with the configuration press Deploy and your Swarm cluster will be created.

From your Brooklyn Install, head to the Home tab. Click on “Add application” and select “Docker Swarm with Discovery and CA”, then click on “Next”. Select the location that that we setup in the previous step. You can also change some configuration options such as the minimum and maximum number of nodes. Once you are happy with the configuration, press “Deploy” and your Swarm cluster will be created.

To interact with the Swarm cluster, we first need to get certificates from the CA server. To do so, the following script can be used:

#!/usr/bin/env bash

# Utility script for developers to get a certificate from Swarm ca-server
# How to use: 
#     getcert.sh $HOME/.certs http://10.20.30.40:8080 
# (replace the address above with the IP of your CA server. This can be retrieved 
# from the `main.uri` sensor on the CA entity)

CERT_DIR=$1
CA=$2

set -e

mkdir -p ${CERT_DIR}
curl -L ${CA}/cacert/ca.pem --output ${CERT_DIR}/ca.pem
openssl genrsa -out ${CERT_DIR}/key.pem 2048
openssl req  -new -key ${CERT_DIR}/key.pem -days 1825 -out ${CERT_DIR}/csr.pem -subj "/CN=$(hostname)"
curl -X POST --data-binary @${CERT_DIR}/csr.pem ${CA}/sign > ${CERT_DIR}/cert.pem

To communicate with the cluster, you must communicate directly with the Swarm master. To do so, first retrieve the Swarm master URI and port. This can be found by checking for the “host.name” and “swarm.port” sensor. After, ensure you have the Docker CLI installed then set up the following environment variables:

export DOCKER_HOST=tcp://<Swarm Master URI & port>
export DOCKER_TLS_VERIFY=true
export DOCKER_CERT_PATH=<CERT_DIR>

You will now be able to run Docker commands against the Swarm cluster.

What’s next?

Jump into the documentation to learn more about Docker Swarm support in Clocker and have an in-depth overview.