Serverless Battleship – Innovation Day at Xebia

on
Nov 19, 2018
in
AWS

At Xebia we know that having fun is the best way to innovate! To keep our consultants inspired and up-to-date we organize innovation days. At innovation days we experiment, test and tinker with the latest technologies. Last innovation day, Friday 16th November 2018, our cloud consultants, Thijs de Vries, Martijn van de Grift, Kevin Kessels and Dennis Vriend battled it out with a cross cloud version of Battleship, who will win? Lets find out!

Battleship

The cloud version of battleship is simple. The consultants get to choose their cloud, their architecture, and their application stack. We start at 09.00 and we create our battleships – read ‘cloud native applications’. We meet in the cloud at 12.00 to battle it out for 15 minutes.
The following rules apply:

Technical:

Architecture:

  • DNS for service discovery,
  • The ‘bullets’ have a JSON format,
  • The bullets have a request/response pattern with ‘ping’ for the request and ‘pong’ for the response, with an UUID as correlation ID and a name to identify the sender,
  • The applications log the ‘shot’ and ‘hit’ in log files for log processing to determine the winner

The teams

Team A consists of Kevin Kessels and Dennis Vriend. Team B consists of Thijs de Vries and Martijn van de Grift. These fine group of cloud consultants are experts in cloud computing in both Amazon Web Services (AWS) and the Google Cloud Platform (GCP) and are specialized in cloud scale architectures, serverless compute and wizards with Python, Go, Scala and Java. This will be an epic battle!

The battleships

In order to do battle, the consultants have to create a battleship of epic cloudscale proportions, which means creating a web application that can hold up to a lot of request. The request is an HTTP POST with ‘ping’ and a HTTP response with ‘pong.’ Create an architecture that can overload the other battleship.

Lets meet the battleships

The battleships are all destroyer-class battleships which means that they are state of the art, and pack a lot of punch! The secret plans are available here, but don’t tell anyone.

Thijs-destroyer

Captain Thijs de Vries has created a battleship that has one hundred guns. These guns are implemented by means of AWS Step Functions. When the destroyer starts shooting, each gun – lambda – will shoot for fifteen minutes on a target of choice, until its sunk.
The battleship with all hundred guns mounted, looks like this:
Serverless Battleship - Innovation Day at Xebia -  Thijs Gun Architecture
When the guns fire, the status looks like this:
Serverless Battleship - Innovation Day at Xebia -  Thijs Gun Status
The gun is implemented with Python and CloudFormation. Lets look at the gun first. The gun uses http.client to aim at an enemy destroyer and shoot with 100 guns at once, a bullet – a JSON message – at the enemy. Because the gun is data driven, all statistics are published to a kinesis stream for real time data analytics.

import http.client
import json
import uuid
import time
import boto3
import os

client = boto3.client("kinesis")

def millis():
    return int(round(time.time() * 1000))

def handler(event, context):
    results = []
    pings = 0
    HOSTNAME = os.environ['HOSTNAME']
    PATH = os.environ['PATH']

    while context.get_remaining_time_in_millis() > 200:
        pings += 1
        start = millis()
        conn = http.client.HTTPSConnection(HOSTNAME)
        headers = {'Content-type': 'application/json', "User-Agent": "Lambda-step-function-gun"}
        foo = {'timestamp': millis(), 'uuid': "thijs" + str(uuid.uuid4()) }
        json_data = json.dumps(foo)
        conn.request('POST', PATH, json_data, headers)
        response = conn.getresponse()
        stop = millis()
        response = response.read().decode()
        data = json.dumps({"elaps": stop - start, "response": response }).encode('utf-8')
        results.append({ "PartitionKey": str(context.log_stream_name), "Data": data })
        if len(results) > 10:
            response = client.put_records(
                Records=results,
                StreamName='ping-war'
            )
            response = []
    return { "done": "true", "pings": pings }

if __name__ == "__main__":
    print(handler({},{}))

Of course the destroyer could also get hit, so there is a ‘pong’ process that listens for requests and returns a ‘pong’. This part is implemented with AWS Chalice, AWS Lambda and AWS API Gateway.

from chalice import Chalice

app = Chalice(app_name='pong')


@app.route('/',methods=['GET', 'POST'])
def index():
    return { 'pong': app.current_request.json_body }

The secret build plans of Captain Thijs de Vries can be downloaded here.

Martijn-destroyer

Captain Martijn vd. Grift has created a battleship with a single big gun. It is implemented in Google Cloud (GCP) and runs in Google App Engine (GAE). The captain motivates that his battleship will win because Google App Engine (GAE) offers a very easy solution to deploy (almost) any kind of application in a production grade environment. GAE applications are auto scaled by default, support CI/CD via the gcloud sdk and even provides an HTTPS endpoint, while also offering scale to zero.
The battleship has been implemented with Flask and Python:

from flask import Flask, request, jsonify
import requests
import uuid
import logging
import time
import json

app = Flask(__name__)

def uuid_generator():
    return str(uuid.uuid4())

def timestamp_generator():
    return int(round(time.time() * 1000))

target_url = 'https://nqyg1t2wwh.execute-api.eu-west-1.amazonaws.com/api'

@app.route('/')
def hello_world():
    return 'Hello World!'

@app.route('/shoot', methods=['GET', 'POST'])
def shoot():
    payload = {'timestamp': timestamp_generator(), 'uuid': uuid_generator(), 'name': 'martijn', 'state': 'ping'}
    requests.post(target_url, json=payload)
    return 'Shooting payload: ' + str(json.dumps(payload))

@app.route('/hit', methods=['POST'])
def hit():
    content = request.get_json()
    print('Got hot by: ' + str(content))
    json_data = {'timestamp': timestamp_generator(), 'uuid': uuid_generator(), 'name': 'martijn', 'state': 'pong'}
    response = app.response_class(
        response=json.dumps(json_data),
        status=200,
        mimetype='application/json'
    )
    return response

if __name__ == '__main__':
    app.run()

The battleship looks very dangerous indeed and provides the following endpoints: GET /: Alive probe endpoint GET, POST /shoot: Shoot to a target url POST /hit: Hit endpoint for the opponent.
The secret build plans of Captain Martijn vd. Grift can be downloaded here.

Dnvriend-destroyer

Captain Dennis Vriend has created a destroyer in GCP using both GAE and Google Cloud Functions (GCF). The destroyer-class battleship consists of one big gun that shoots at other clouds. The big gun is located in GCP and is mounted on GCF. The battleship itself is implemented in GAE. The big-gun is very dangerous! For protection is has an enable feature. The gun can be enabled with the red button. The captain motivates the design that the big-gun must be scalable so it spawns itself when the load increases. When there are multiple targets each gun can aim at a different target making for a good weapon against enemies. Because the big-gun is very dangerous, for protection is has an enable feature. The gun can be enabled with the ‘red button’ which enables the gun.
The battleship is implemented in Flask:

# [START gae_python37_app]
import requests
from flask import Flask

app = Flask(__name__)

enabled = False
shots_fired = 0

@app.route('/')
def hello():
    return 'Hello World!'


@app.route('/red-button', methods=['GET'])
def red_button():
    global enabled
    enabled = True
    print('Enabling the big gun')
    return 'Enables the big-gun...'


@app.route('/shoot', methods=['GET'])
def shoot():
    global shots_fired
    print('Received order to shoot!')
    if enabled:
        for x in range(0, 1000):
            shots_fired += 1
            print(f'[{shots_fired}]: bang bang!')
            requests.post('https://europe-west1-speeltuin-dennis-vriend.cloudfunctions.net/dnvriend_destroyer', json={'shoot': True})
    else:
        print('Not enabled, not shooting...')
    return 'Someone pulled the trigger'

if __name__ == '__main__':
    app.run(host='127.0.0.1', port=8080, debug=True)
# [END gae_python37_app]

The bug gun looks like this:

import json
import random
import time
import uuid

import requests
from flask import jsonify

targets = {
    'thijs': 'https://nqyg1t2wwh.execute-api.eu-west-1.amazonaws.com/api',
    'martijn': 'https://speeltuin-martijn-vd-grift.appspot.com/hit'
}


def get_bullet(uuid: str, timestamp: int) -> dict:
    return {
        "timestamp": timestamp,
        "uuid": uuid,
        "name": "dennisvriend",
        "state": "ping"
    }


def get_uuid() -> str:
    return str(uuid.uuid4())


def get_timestamp():
    return int(round(time.time() * 1000))


def shoot_at(target: str) -> None:
    uuid = get_uuid()
    timestamp = get_timestamp()
    print(f'[SHOOT_AT:{target}]:dennisvriend,{uuid},{timestamp}')
    requests.post(targets.get(target), json=get_bullet(get_uuid(), get_timestamp()))


def shoot():
    "shoot at a random target"
    if bool(random.getrandbits(1)):
        shoot_at('thijs')
    else:
        shoot_at('martijn')


def dnvriend_destroyer(request):
    try:
        got_hit_with_bullet = {}
        if request.get_json().get('shoot'):
            shoot()
        else:
            got_hit_with_bullet = request.get_json()
            got_hit_with_bullet.update({'state': 'pong'})
            received_payload = json.dumps(got_hit_with_bullet)
            timestamp = get_timestamp()
            uuid = got_hit_with_bullet['uuid']
            name = got_hit_with_bullet['name']
            print(f'[GOT_HIT_BY:{name}]:{name},{uuid},{timestamp},{received_payload}')
        return jsonify(got_hit_with_bullet), 200
    except Exception as e:
        print(f'Error: e')
        return jsonify({
            'error': str(e),
            'usage': 'please post a message with a valid body'
        }), 500

The secret build plans of Captain Dennis Vriend can be downloaded here.

Kevin-destroyer

The battleship of Kevin has some delay due to delivery problems from questionable weapon dealers from the far east. Tbd.

Conclusion

The consultants had a lot of fun and learned a lot about looking at each other designs. AWS and GCP both provide a perfect platform for both experimenting and playing with the technologies but also for production workloads. Next time we’ll look at who has won the battle using some big data analytics technologies.

Share this article: Tweet this post / Post on LinkedIn