Incapture Technologies

Inside the Cloud

Incapture Technologies Blog

 

Rapture and Docker

Published:
December 16, 2014
Author:

Docker is a container virtualization platform that “lets you quickly assemble applications from components and eliminates the friction that can come when shipping code” (see www.docker.com).

I thought it would be interesting to see what it would take to completely deploy a sample Rapture environment using Docker. It was straightforward to pull together a number of containers for this purpose and have a complete Rapture environment running on Docker in about 30 minutes – with the boot up time of the environment measured in seconds.

host ~> docker ps
CONTAINER ID        IMAGE                               COMMAND                CREATED             STATUS              PORTS                                               NAMES
c13338c5c5c8        incapture/raptureapiserver:latest   "/bin/bash /rapture/   About an hour ago   Up About an hour    0.0.0.0:49181->8665/tcp                             lonely_ardinghelli
96a4817c7da6        incapture/rabbitmq:latest           "rabbitmq-start"       20 hours ago        Up 20 hours         0.0.0.0:49175->15672/tcp, 0.0.0.0:49176->5672/tcp   rabbitmq
bc7fe0f0ed87        incapture/mongo:latest              "/bin/sh -c usr/bin/   21 hours ago        Up 21 hours         0.0.0.0:49154->27017/tcp                            mongo
host s004 ~>

Defining the lower level images

Docker introduces the concept of an image. We create containers from images. For my sample Rapture environment I used MongoDB as the database system and RabbitMQ as the messaging system. A quick internet search gave me some good recipes for those images – here expressed in their Dockerfiles (the configuration file used to create an image).

For mongodb:

FROM ubuntu:latest
MAINTAINER Alan Moore alan.moore@incapturetechnologies.com
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | tee /etc/apt/sources.list.d/10gen.list
RUN apt-get update
RUN apt-get install -y mongodb-org
RUN mkdir -p /data/db
EXPOSE 27017
ENTRYPOINT usr/bin/mongod

And for rabbitmq (a little bit more tricky due to the key signing of the install)

FROM dockerfile/ubuntu
MAINTAINER Alan Moore alan.moore@incapturetechnologies.com
ADD bin/rabbitmq-start /usr/local/bin/

# Install RabbitMQ.
RUN \
  wget -qO - https://www.rabbitmq.com/rabbitmq-signing-key-public.asc | apt-key add - && \
  echo "deb http://www.rabbitmq.com/debian/ testing main" > /etc/apt/sources.list.d/rabbitmq.list && \
  apt-get update && \
  DEBIAN_FRONTEND=noninteractive apt-get install -y rabbitmq-server && \
  rm -rf /var/lib/apt/lists/* && \
  rabbitmq-plugins enable rabbitmq_management && \
  echo "[{rabbit, [{loopback_users, []}]}]." > /etc/rabbitmq/rabbitmq.config && \
  chmod +x /usr/local/bin/rabbitmq-start

# Define environment variables.
ENV RABBITMQ_LOG_BASE /data/log
ENV RABBITMQ_MNESIA_BASE /data/mnesia

# Define mount points.
VOLUME ["/data/log", "/data/mnesia"]

# Define working directory.
WORKDIR /data

# Define default command.
CMD ["rabbitmq-start"]

# Expose ports.
EXPOSE 5672
EXPOSE 15672

(This was taken almost directly from https://github.com/dockerfile/rabbitmq)

With those files in place I could create the images using these commands (run in the folder that contained the appropriate Dockerfile)

docker built -t incapture/mongo .
docker build -t incapture/rabbitmq .

After these commands completed I had a set of local images that I could use to start MongoDB and RabbitMQ.

Starting the lower level containers

With those images created I could quickly start a container for the two applications. I gave them very specific names in my example rather than using docker’s automatic naming facility:

docker run  -d incapture/mongo --name=mongo
docker run  -d incapture/rabbitmq --name=rabbitmq

After running these commands I have two local containers running on my host – one for MongoDB and one for RabbitMQ.

Linking

Before I created my Rapture image I had to consider how Rapture was going to “find” the two services I had started previously. The connection information for MongoDB and RabbitMQ is stored in a set of configuration files that are either on the resource (class) path of Rapture or in a very specific location on the filesystem – the location driven by an environment variable. Normally this configuration file has the name or ip address of the underlying service – but in this case I didn’t want to put a specific name in the configuration file – I would rather have a more general name and use docker’s linking (on add-host) facility to bind the service I had started to that general name. In this case I used the general names “rabbit” and “mongo”. We’ll see how that is referenced in a moment.

The RaptureAPIServer image

Rapture has a pre-packaged application available that wraps the core platform along with a servlet based binding for the API and a web front end for general operational management. It’s called “RaptureAPIServer”. For this example I am going to take that codebase and create an image from it.

To do this I copied a built (and installable) version of RaptureAPIServer. Once built a RaptureAPIServer has three subfolders – a bin folder for the start scripts and the website, a lib folder for the java libraries and an etc folder for the configuration. I created a local folder for my image creation and copied in the built application. Structurally it looks like this:

Screen Shot 2014-12-16 at 1.22.15 PM

Next I added two additional configuration files for the bindings to the services. I placed these in the etc/rapture/config folder:

RaptureMONGODB.cfg

default=mongodb://rapture:rapture@mongo/RaptureMongoDB

RaptureRABBITMQ.cfg

default=amqp://guest:guest@rabbit:5672/%2f

Note that these refer to a general server “mongo” for MongoDB and “rabbit” for RabbitMQ.

Finally I created the DockerFile for RaptureAPIServer. It is reproduced below:

FROM ubuntu:latest
MAINTAINER Alan Moore <alan.moore@incapturetechnologies.com>
RUN apt-get update && apt-get install -y default-jdk
COPY app/RaptureAPIServer /rapture
ENV RAPTURE_CONFIG_HOME /rapture/etc/rapture/config
WORKDIR /rapture/bin
EXPOSE 8665
ENTRYPOINT [ "/bin/bash", "/rapture/bin/RaptureAPIServer" ]

The docker file basically installs the java jdk (as Rapture needs java to run) and copies the contents of the Rapture application into a folder called “/rapture”. It sets up the environment variable RAPTURE_CONFIG_HOME (where Rapture can optionally look for configuration files), sets the working directory appropriately, exposes the port used by the API and web site and invokes the start script for Rapture.

We build the image in the standard way:

docker build -t incapture/raptureapiserver .

Running Rapture

Finally we can start a Rapture container. The only special thing to do in this case was to link the new container with the previously started services:

docker run -d --link mongo:mongo --link rabbitmq:rabbit -P incapture/raptureapiserver

Rapture starts perfectly well – initializing the MongoDB environment for first use.

21:29:31,716  INFO [main] (ConfigFileReader.java:72) - Found default for RaptureLOGGER.cfg in classpath. URL: jar:file:/rapture/lib/RaptureAppConfig-1.1.29.99999999999999.jar!/rapture/config/defaults/RaptureLOGGER.cfg
21:29:31,720  INFO [main] (ConfigFileReader.java:80) - RAPTURE_CONFIG_HOME is set to /rapture/etc/rapture/config, retrieve config from there for RaptureLOGGER.cfg
21:29:31,723  INFO [main] (ConfigFileReader.java:84) - No global config found for RaptureLOGGER.cfg.
21:29:31,723  INFO [main] (ConfigFileReader.java:86) - No app-specific config found for RaptureLOGGER.cfg.
21:29:31,798  INFO [main] (RaptureAppService.java:22) <> [] - Starting RaptureAPIServer
21:29:31,819  INFO [main] (RaptureAppService.java:23) <> [] - ==================================
21:29:31,821  WARN [main] (RaptureMBeanServer.java:72) <> [] - No JMX port defined
21:29:31,827  INFO [main] (AddinLoader.java:56) <> [] - Loading addins
21:29:31,828  INFO [main] (ConfigFileReader.java:72) <> [] - No default found for RaptureLOCAL.cfg.
21:29:31,829  INFO [main] (ConfigFileReader.java:80) <> [] - RAPTURE_CONFIG_HOME is set to /rapture/etc/rapture/config, retrieve config from there for RaptureLOCAL.cfg
21:29:31,829  INFO [main] (ConfigFileReader.java:84) <> [] - Found global config for RaptureLOCAL.cfg in RAPTURE_CONFIG_HOME. URL: /rapture/etc/rapture/config/RaptureLOCAL.cfg
21:29:31,829  INFO [main] (ConfigFileReader.java:86) <> [] - No app-specific config found for RaptureLOCAL.cfg.
21:29:31,834  INFO [main] (AddinLoader.java:48) <> [] - Using /rapture/lib/RaptureAPI-1.1.29.99999999999999.jar from path of class library as the base folder
21:29:31,835  INFO [main] (AddinLoader.java:64) <> [] - Addins will be loaded from /rapture/addins
21:29:31,835  INFO [main] (RaptureWebServer.java:41) <> [] - Configured as a web server
21:29:31,864  INFO [main] (ConfigLoader.java:38) <> [] - Loading Rapture Config
21:29:31,870  INFO [main] (ConfigFileReader.java:72) <> [] - Found default for rapture.cfg in classpath. URL: jar:file:/rapture/lib/RaptureAppConfig-1.1.29.99999999999999.jar!/rapture/config/defaults/rapture.cfg
21:29:31,870  INFO [main] (ConfigFileReader.java:80) <> [] - RAPTURE_CONFIG_HOME is set to /rapture/etc/rapture/config, retrieve config from there for rapture.cfg
21:29:31,871  INFO [main] (ConfigFileReader.java:84) <> [] - Found global config for rapture.cfg in RAPTURE_CONFIG_HOME. URL: /rapture/etc/rapture/config/rapture.cfg
21:29:31,871  INFO [main] (ConfigFileReader.java:86) <> [] - No app-specific config found for rapture.cfg.
21:29:31,975  INFO [main] (ConfigLoader.java:84) <> [] - Successfully loaded config file
21:29:31,976  INFO [main] (SysRepoCache.java:31) <> [] - Bootstrap config is REP {} USING MONGODB { prefix="rapture.bootstrap" }
21:29:32,076  INFO [main] (ConfigFileReader.java:72) <> [] - Found default for RaptureMONGODB.cfg in classpath. URL: jar:file:/rapture/lib/MongoDb-1.1.29.99999999999999.jar!/rapture/config/defaults/RaptureMONGODB.cfg
21:29:32,076  INFO [main] (ConfigFileReader.java:80) <> [] - RAPTURE_CONFIG_HOME is set to /rapture/etc/rapture/config, retrieve config from there for RaptureMONGODB.cfg
21:29:32,077  INFO [main] (ConfigFileReader.java:84) <> [] - Found global config for RaptureMONGODB.cfg in RAPTURE_CONFIG_HOME. URL: /rapture/etc/rapture/config/RaptureMONGODB.cfg
21:29:32,078  INFO [main] (ConfigFileReader.java:86) <> [] - No app-specific config found for RaptureMONGODB.cfg.
21:29:32,080  INFO [main] (MongoDBFactory.java:58) <> [] - Host is mongodb://rapture:rapture@mongo/RaptureMongoDB
21:29:32,095  INFO [main] (MongoDBFactory.java:62) <> [] - Username is rapture
21:29:32,095  INFO [main] (MongoDBFactory.java:63) <> [] - Host is [mongo]
21:29:32,096  INFO [main] (MongoDBFactory.java:64) <> [] - DBName is RaptureMongoDB
21:29:32,096  INFO [main] (MongoDBFactory.java:65) <> [] - Collection is null
21:29:32,877  INFO [main] (AuditLogCache.java:87) <> [] - Creating audit log provider for kernel with config LOG {} using LOG4J {}
21:29:32,880  INFO [main] (AuditLogFactory.java:52) <> [] - Creating audit log from config - LOG {} using LOG4J {}
21:29:32,892  INFO [main] (ConfigFileReader.java:72) <> [] - No default found for RaptureRUNNER.cfg.
21:29:32,892  INFO [main] (ConfigFileReader.java:80) <> [] - RAPTURE_CONFIG_HOME is set to /rapture/etc/rapture/config, retrieve config from there for RaptureRUNNER.cfg
21:29:32,893  INFO [main] (ConfigFileReader.java:84) <> [] - No global config found for RaptureRUNNER.cfg.
21:29:32,893  INFO [main] (ConfigFileReader.java:86) <> [] - No app-specific config found for RaptureRUNNER.cfg.
21:29:32,894  WARN [main] (ConfigFileReader.java:103) <> [] - No config files found for RaptureRUNNER.cfg
21:29:32,894  INFO [main] (ConfigFileReader.java:112) <> [] - Unable to find overlay file for RaptureRUNNER.cfg
21:29:32,901  INFO [main] (Log4jAudit.java:127) <> [] - Tue Dec 16 21:29:32 UTC 2014 (user=raptureApi)  [kernel] kernel: Instance started
21:29:32,901  INFO [main] (Kernel.java:865) <> [] - AppStyle is webapp
21:29:32,901  INFO [main] (RaptureWebServer.java:51) <> [] - Configuring server on port 8665
21:29:32,907  INFO [main] (RaptureWebServer.java:55) <> [] - Starting WebServer for Rapture

And the web site (available through the host port mapping) is available also:

Screen Shot 2014-12-16 at 1.30.58 PM

Next steps

This was just a quick demonstration on how easy it is to create a Rapture environment within a set of Docker containers. In the real world more work would be involved in automating the bindings and handling how to bind containers that are running on different hosts – but there is nothing inherently difficult about how to do that. In fact a higher level Rapture environment and workflows could manage the topology of these satellite environments!

As before – if you’d like more information about Incapture or Rapture please drop me a line personally or to our general email address info@incapturetechnologies.com and we will get back to you for a more in depth discussion.


Subscribe for updates