Create Ansible Playbook on GitHub to Build Mesos Clusters

Optional – Setup Git

If you haven’t already setup Git on your local system follow these steps:

  1. Download and install the latest version of Git for your platform.
  2. Launch the appropriate terminal or command line application.
  3. Set the name used for Git commits.
    git config --global user.name "YOUR NAME"
  4. Set the email address used for Git commits. This should match what you used when signing up for GitHub.
    git config --global user.email "YOUR EMAIL ADDRESS"
  5. Configure Git to use the new “simple” style for pushing changes to a repository.
    git config --global push.default simple

Create a new repository on GitHub

  1. In the upper-right corner of any page, click +, and then click New repository.
  2. Enter a name for your new repository. I am calling it “ansible-mesos-cluster” for this project.
  3. Add a description of your repository. For example, “Ansible playbook to build multi-node Apache Mesos clusters.”
  4. Choose between creating a public or private repository. I’ll use public.
  5. As long as you haven’t run git init locally you can select “Initialize this repository with a README.” This allows you to git clone the repository immediately.
  6. Optionally, click “Add a license” and select one. I chose “MIT License” because it let’s people do anything they want with your code as long as they don’t hold you liable and provide attribution.
  7. Click Create repository.

Checkout the new GitHub repository

Clone the HTTPS URL of your new repository. For example:

git clone https://github.com/frankhinek/ansible-mesos-cluster.git  

Move into the directory and create a gitignore file to prevent committing files that don’t need to be in the GitHub repository. For example,

cd ansible-mesos-cluster/  
touch .gitignore  

Enter the following contents into the .gitignore:

.DS_Store
ansible.cfg  
hosts  
*.retry

Basic Ansible Playbook setup

In the root of your project create a hosts file and build out the host groups. We use the zoo_id variable to set the ID of the ZooKeeper so each host knows its place in the cluster.

touch hosts  

The content of the hosts file should look something like this, substituting the IP addressing for your environment:

[mesos_masters]
10.1.200.11 zoo_id=1  
10.1.200.12 zoo_id=2  
10.1.200.13 zoo_id=3

[mesos_slaves]
10.1.200.40  
10.1.200.41

[marathon]
10.1.200.30  

In the root of your project create a site.yml file. This is the master playbook that defines the entire cluster infrastructure.

touch site.yml

The content of the site.yml file should look something like this:

---
# This playbook deploys the entire Mesos cluster infrastructure.
# RUN: ansible-playbook --ask-sudo-pass -i hosts site.yml

- name: deploy and configure the mesos masters
  hosts: mesos_masters
  sudo: True

  roles:
    - { role: mesos, mesos_install_mode: "master", tags: ["mesos-master"] }

- name: deploy and configure the mesos slaves
  hosts: mesos_slaves
  sudo: True

  roles:
    - { role: mesos, mesos_install_mode: "slave", tags: ["mesos-slave"] }

Build Out Group Variables

Create a directory to store the variables used by the host groups in this playbook. Also add a blank file to store the variable definitions.

mkdir group_vars  
touch group_vars/all  

The content of the all file should look something like this:

---
# Variables here are applicable to all host groups

mesos_version: 0.20.0-1.0.ubuntu1404  
mesos_local_address: "{{ansible_eth0.ipv4.address}}"  
mesos_cluster_name: "Cluster01"  
mesos_quorum_count: "2"  
zookeeper_client_port: "2181"  
zookeeper_leader_port: "2888"  
zookeeper_election_port: "3888"  
zookeeper_url: "zk://{{ groups.mesos_masters | join(':' + zookeeper_client_port + ',') }}:{{ zookeeper_client_port }}/mesos"  

Building Out Role Scaffolding

Create a directory to store the roles used by this playbook. Use the ansible-galaxy command line tool to initialize the directory structure each role:

mkdir roles  
cd roles/  
ansible-galaxy init mesos  

Push Changes to GitHub Repository

Add all the files to Git locally:

git add .  

Commit your changes:

git commit -m "Initial commit."  

Push the changes to the GitHub repository. You will be prompted to enter your GitHub username and password.

git push  

Add the Mesos Role Handlers

Modify the roles/mesos/handlers/main.yml file to match:

---
# handlers file for mesos
- name: Start mesos-master
  shell: start mesos-master
  sudo: yes

- name: Stop mesos-master
  shell: stop mesos-master
  sudo: yes

- name: Start mesos-slave
  shell: start mesos-slave
  sudo: yes

- name: Stop mesos-slave
  shell: stop mesos-slave
  sudo: yes

- name: Restart zookeeper
  shell: restart zookeeper
  sudo: yes

- name: Stop zookeeper
  shell: stop zookeeper
  sudo: yes

Add the Mesos Role Tasks

Modify the roles/mesos/tasks/main.yml file to match:

---
# tasks file for mesos

# Common tasks for all Mesos nodes
- name: Add key for Mesosphere repository
  apt_key: url=http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF state=present
  sudo: yes

- name: Determine Linux distribution distributor
  shell: lsb_release -is | tr '[:upper:]' '[:lower:]'
  register: release_distributor

- name: Determine Linux distribution codename
  command: lsb_release -cs
  register: release_codename

- name: Add Mesosphere repository to sources list
  copy:
    content: "deb http://repos.mesosphere.io/{{release_distributor.stdout}} {{release_codename.stdout}} main"
    dest: /etc/apt/sources.list.d/mesosphere.list
    mode: 0644
  sudo: yes

# Tasks for Master, Slave, and ZooKeeper nodes
- name: Install mesos package
  apt: pkg={{item}} state=present update_cache=yes
  with_items:
    - mesos={{ mesos_pkg_version }}
  sudo: yes
  when: mesos_install_mode == "master" or mesos_install_mode == "slave"

- name: Set ZooKeeper URL # used for leader election amongst masters
  copy:
    content: "{{zookeeper_url}}"
    dest: /etc/mesos/zk
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master" or mesos_install_mode == "slave"

# Tasks for Master nodes
- name: Disable the Mesos Slave service
  copy:
    content: "manual"
    dest: /etc/init/mesos-slave.override
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

- name: Set Mesos Master hostname
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-master/hostname
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

- name: Set Mesos Master ip
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-master/ip
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

- name: Set Mesos Master Cluster name
  copy:
    content: "{{mesos_cluster_name}}"
    dest: /etc/mesos-master/cluster
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

- name: Set Mesos Master quorum count
  copy:
    content: "{{mesos_quorum_count}}"
    dest: /etc/mesos-master/quorum
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

# Tasks for Slave nodes
- name: Disable the Mesos Master service
  copy:
    content: "manual"
    dest: /etc/init/mesos-master.override
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "slave"

- name: Disable the ZooKeeper service
  copy:
    content: "manual"
    dest: /etc/init/zookeeper.override
    mode: 0644
  sudo: yes
  notify:
    - Stop zookeeper
  when: mesos_install_mode == "slave"

- name: Set Mesos Slave hostname
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-slave/hostname
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "slave"

- name: Set Mesos Slave ip
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-slave/ip
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "slave"

- name: Set Mesos Slave ip
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-slave/ip
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "slave"

- name: Set Mesos Slave isolation
  copy:
    content: "cgroups/cpu,cgroups/mem"
    dest: /etc/mesos-slave/isolation
    mode: 0644
  sudo: yes
  notify:
    - Start mesos-slave
  when: mesos_install_mode == "slave"

# Tasks for ZooKeeper nodes only
- name: Create zookeeper config file
  template: src=zoo.cfg.j2 dest=/etc/zookeeper/conf/zoo.cfg
  sudo: yes
  when: mesos_install_mode == "master"

- name: Create zookeeper myid file
  template: src=zoo_id.j2 dest=/etc/zookeeper/conf/myid
  sudo: yes
  notify:
    - Restart zookeeper
    - Start mesos-master
  when: mesos_install_mode == "master"

Create ZooKeeper Configuration Templates

Add blank files to store the ZooKeeper configuration file templates.

touch roles/mesos/templates/zoo.cfg.j2  
touch roles/mesos/templates/zoo_id.j2  

The content of the zoo.cfg.j2 file should look like:

tickTime=2000  
dataDir=/var/lib/zookeeper/  
clientPort={{ zookeeper_client_port }}  
initLimit=5  
syncLimit=2  
{% for host in groups['mesos_masters'] %}
server.{{ hostvars[host].zoo_id }}={{ host }}:{{ zookeeper_leader_port }}:{{ zookeeper_election_port }}  
{% endfor %}

The content of the zoo_id.j2 file should look like:

{{ zoo_id }}

Push Changes to GitHub Repository

Add all the files to Git locally:

git add .  

Commit your changes:

git commit -m "Initial commit."  

Push the changes to the GitHub repository. You will be prompted to enter your GitHub username and password.

git push  

Pull Changes from GitHub

Here is an example of pulling down the repository to another host that you may use to build a Mesos cluster. It also indicates how to pull changes to the repository down to the host.

On the new system change to the working directory you want to clone the GitHub repository to.

git clone https://github.com/frankhinek/ansible-mesos-cluster.git  
cd ansible-mesos-cluster/  

You can now modify the hosts file to match the desired cluster node count and IP addressing scheme. Then run the Ansible playbook to build out your cluster. If you or someone else modifies the GitHub repository you can use the steps below pull the modifications down to your repository.

Make the entire working directory match the latest master branch commit and fetch a copy of the current branch and immediately merge it into the local copy.

git checkout  
git pull  

Summary and Next Steps

In this post we setup a repository on GitHub to store an Ansible playbook that builds Mesos clusters, cloned it to a local working directory, added the Ansible code and templates, pushed the changes up to the repository, and talked about how to pull them down on any system to execute the playbook. A role to deploy and configure Marathon is still needed, but rather than drag this post out I’ll leave that exercise to you. If you need some hints you can check out my repository.

The full Ansible Playbook with additional roles and capabilities is stored on my ansible-mesos-cluster GitHub repository. Feel free to explore the additional code I’ve added, clone the repository to test building a Mesos cluster in your lab, or fork to make your own changes.