code-small.jpg

Cake Team Blogs

Deploying Scala & Akka based applications to Amazon Cloud

Posted by Anirvan Chakraborty

28/01/14 13:56

For the last couple of years @cakesolutions have been heavily involved with projects built on the "Typesafe Reactive Platform". And most of our clients have been using Amazon Elastic Compute Cloud (EC2) as the target deployment environment for their applications. We have tried several different approaches in deploying Scala, Akka & Spray based applications to Amazon EC2 with varying degrees of success. In this post I'm going to describe an approach which has worked pretty well to quickly get our clients' reactive applications to cloud.


IO/Spray boot file

Before I get to deploying, let me quickly remind you that in our reactive applications we favour the approach of making a runnable jar using IO/Spray boot file:

trait Web {
    this: Api with CoreActors with Core =>
    IO(Http)(system) ! Http.Bind(rootService, "0.0.0.0", port = 8080)
}

The above code would provide the web server (spray-can) for the REST api in Api, using the actor system defined in Core. (Note: To learn more about Scala, Akka & Spray based applications have a look at @honzam399's Akka and Spray activator.

Amazon Command Line Interface (CLI) tools

To proceed further you would need to install the following Amazon CLI tools:

(Note: In a future post I intend to summarize the installation procedures of the various CLI tools available for Amazon)

Amazon Machine Image (AMI) based deployment approach

We have based our Amazon deployment approach on heavy usage of AMIs. As we have seen that handling instance life-cycle using custom AMIs guarantees better performance from the Amazon instances (less instance failures, but I don't have statistics to support that!). Moreover it's easier to launch and start an instance by indicating the choice of AMIs. It has also proved immensely useful when we went on to using automatic scaling for instances through the Amazon elastic load balancer (ELB).

For deploying a reactive application, we build a customized base AMI from a plain vanilla AMI off the Amazon market place by installing the required components on top of it. We then save this customized AMI so that it can the be used directly to create and run application specific instances.

Create customized AMI for the application

For the custom application specific AMI we would be using ami-ce7b6fba as the base AMI. This is a plain Ubuntu Server 12.04 Long Term Support available freely on the Amazon AWS market place. You can choose other Ubuntu base images from the AMI locator.

  1. Expose the various parameters that would be used in subsequent commands on your preferred shell.
    $ export SUBNET_ID=(subnet ID where the application instance would be created)
    $ export REGION=eu-west-1
    $ export AVAILABILITY_ZONE=eu-west-1a
    $ export SECURITY_GROUP_NAME=(default or a customized Amazon security group)
    $ export SECURITY_GROUP_ID=(security group ID)
    $ export KEY=(security key to use for secure access)
    $ export INSTANCE_TYPE="m1.large"
    
  2. Launch an instance from the base AMI and configure it using the ami-bootstrap.sh user data file.
    $ ec2-run-instances ami-ce7b6fba -g $SECURITY_GROUP_ID -k $KEY -f ami-bootstrap.sh --instance-type $INSTANCE_TYPE --availability-zone $AVAILABILITY_ZONE
    

    This would start an EC2 instance and would then perform the customization specified on the user data file. From the console output take a note of the instance id of the form i-xxxxxxxx which will be used in the next few steps.

    I have described the contents of the ami-bootstrap.sh on a following section which highlights the customisation performed on the vanilla base image.

  3. Create an AMI image from the recently launched and configured instance.
    $ ec2-create-image i-xxxxxxxx -n "APP" -d "Base image for the application" --region $REGION
    

    Here i-xxxxxxxx is the id of the previously configured instance. This will create a base ami (ami-xxxxxxxx) that we'll refer to as ${base-ami-app} from here on.

You can now shut down the running instances as we have successfully created the customized base AMI for the application which we will be using later on.

Bootstrap script for base AMI

  • Update and configure app-get
  • Install git
  • Install Java 7 JRE
  • Install SBT
  • Configure secure communication with GitHub
  • Checkout and build the application

The user data file ami-bootstrap.sh below describes the custom installation and configuration steps.

    #!/bin/bash
	#This script installs java, sbt and the application
	#Run this script on a new EC2 instance as the user-data script, which is run by `root` on machine start-up.
	set -e -x

	#Config
	export DEBIAN_FRONTEND=noninteractive

	# Update `apt-get` and install required packages
	apt-get update
	apt-get install python-software-properties
	apt-get --no-install-recommends -y install build-essential
	apt-get --no-install-recommends -y install libxslt-dev
	apt-get --no-install-recommends --fix-missing -y install git
	apt-get install unzip -y

	echo 'PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/gems/1.8/bin"' >> /etc/environment
	echo 'DEBIAN_FRONTEND=noninteractive' >> /etc/environment

	apt-get --no-install-recommends -y -f install
	mkdir -p /tmp
	cd /tmp

	apt-get update

	#Setup Java7 JRE
	mkdir -p /usr/lib/jvm
	cd /usr/lib/jvm
	#To get around Oracle's license agreement (which can't be accepted from command line!)
	wget --no-check-certificate --no-cookies --header "Cookie: gpw_e24=test;" http://download.oracle.com/otn-pub/java/jdk/7u51-b13/jdk-7u51-linux-x64.tar.gz
	tar -xzvf jdk-7u51-linux-x64.tar.gz
	rm -rf jdk-7u51-linux-x64.tar.gz
	ln -s jdk1.7.0_51/ default-java
	update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/default-java/bin/java" 1
	update-alternatives --config java

	#Create the application specific user who would own the entire application
	useradd -m -g users -s /bin/bash -u 1001 appowner

	#Enable `appowner` user to securely login to the box
	su - appowner -c "mkdir -p /home/appowner/.ssh"
	cp -f /home/ubuntu/.ssh/authorized_keys /home/appowner/.ssh/
	chown appowner /home/appowner/.ssh/authorized_keys

	#Allow write permission on `/usr/local` for everyone. This would be needed by `appowner` to install other packages
	chmod -R a+w /usr/local

	#Create custom application space for `appowner`
	mkdir -p /mnt/appowner/application-name
	chown -R appowner /mnt/appowner

	#Setup Private key for authenticating against GitHub
	#Note: Setup and configure deploy keys in GitHub for application specific repositories
	#Note: Remember to add GitHub to the known hosts list for smooth secure communication

	# Install SBT
	wget -q apt.typesafe.com/repo-deb-build-0002.deb
	dpkg -i repo-deb-build-0002.deb
	apt-get update
	apt-get --no-install-recommends -y -f install
	apt-get update
	sleep 5
	wget http://repo.scala-sbt.org/scalasbt/sbt-native-packages/org/scala-sbt/sbt/0.12.0/sbt.deb
	dpkg -i sbt.deb
	sleep 5
	rm *.deb

	# Set JDK for `appowner` user
	su - appowner -c "echo 'export PATH="$PATH:/usr/lib/jvm/default-java/bin"' >> ~/.profile"

	#Checkout 'master' from the GitHub repo `application` and compile project
	su - appowner -c "ln -s /mnt/appowner/application-name app-server"
	su - appowner -c "cd /home/appowner && \
	git clone git@github.com:your-github-user/application-name.git /mnt/appowner/application-name/app-server"
	su - appowner -c "cd /home/appowner/app-server && \
	sbt clean compile"

Start application

The steps described here would guide you to first start an EC2 instance specific to your application and then to start your application:

  1. Expose the various parameters that would be used in subsequent commands on your preferred shell.
    	$ export SUBNET_ID=(subnet ID where the application instance would be created)
    	$ export REGION=eu-west-1
    	$ export AVAILABILITY_ZONE=eu-west-1a
    	$ export SECURITY_GROUP_NAME=(default or a customized Amazon security group)
    	$ export SECURITY_GROUP_ID=(security group ID)
    	$ export KEY=(security key to use for secure access)
    	$ export INSTANCE_TYPE="m1.large"
    
  2. Run an instance from the customized AMI
    $ ec2-run-instances ${base-ami-app} -n 1 -g $SECURITY-GROUP -k $KEY --instance-type m1.large --availability-zone $AVAILABILITY_ZONE
    
  3. Once startup completes, open an ssh session into the running instance
    $ ssh -i ${key file} appowner@ec2-XXX-XXX-XXX-XXX.eu-west-1.compute.amazonaws.com
    
  4. Run your reactive application
    $ cd app-server
    	& nohup target/start com.yourcompany.application.main.Server 2>&1 >> /var/log/application-name/app.log &
    

Now you have a running EC2 instance with you application ready and running. The application log is also redirected to /var/log/application-name/app.log.

Hurrah!

Hopefully I have been able to show that getting your reactive application to Amazon cloud is a fairly easy task. In our experience we have seen that getting a proof of concept of a reactive application deployed to Amazon cloud remarkably reduces time to market. And perhaps more importantly reduces the feedback loop and increases buy in from stakeholders.

On my upcoming posts I intend to shed some light on our approach of taking a reactive application to production. I hope to be able to also show you some best practices that we adopt on our continuous integration and continuous delivery approach. So stay tuned!

 

Topics: Scala, Akka

Posts by Topic

see all

Subscribe to Email Updates