At some point in your application’s lifecycle, there might come a time when you need to start scaling your data storage. If you are storing media files or other blobs that have no relations between them, you can easily add storage capacity to solve the problem. For (semi-)structured data in a database however, scaling is a whole different story. Simply adding database instances is not enough. You will need to reconsider the usage patterns and decide what solution solves the problem you have. Read more...

Python is very well suited for writing AWS Lambda functions. For local development, you might want to be able to run the whole range of supported Python runtimes on your Mac. Unfortunately, setting this up is not as straightforward as you might think.

Read more...

Deployment Manager is an infrastructure deployment service that automates the creation and management of Google Cloud Platform resources. A lot of GCP resources can be created using the available GDM types. In essence a GDM type is an abstraction on top of the API of a GCP service. However some types are missing, so in this blog I want to show you how to add those yourself. GDM supports jinja2 and Python templates, for this blog we will use python.

Read more...

When you deploy your application on compute instances on-premise or in the cloud, you have to make an educated guess about the utilisation of the resources you provision. If you have an older version of the same application running with proper monitoring, you can base your guesstimate on the current usage of compute nodes. But when this is the first time your application goes to production you are out of luck. Read more...

Google Cloud provides a Docker image registry which you can use to store your private docker images. Although it is a separate Google service, there are no specific IAM roles associated with it, which allow you to control access to your registry. Instead, you use the Google Cloud Storage roles and permissions on the bucket that is implicitly created for you. In this blog we show you how you create the container registry storage bucket in your project and grant access to it.

Read more...

There are many ways to use ‘secrets’ in ECS and ECS Fargate. Most of the time they are exposed using Environment variables, because a few years ago this was the only easy option. Today we have two improved options. You could add a tool to your docker container which retrieves and decrypts your secrets before parsing them to the application. The other one is to embed the retrieval and decryption in the application. Read more...

In previous posts, we showed you how to deploy a private key pair to allow you to login to an EC2 instance. Since september 2018, the AWS Session Manager supports logging into any instance, directly from the command line without SSH. In this blog we will show you how to configure this using CloudFormation.

Read more...

Mounting an persistent EBS volume at boot time is no trivial task. Since the introduction of the Nitro based instance types, EBS volumes no longer appear under the device name specified during the attach command. Instead, the device name will be determined by the kernel and depend upon the order in which it is attached. As a result, switching EC2 instance type or volumes, may cause an incorrect volume to be mounted or not be mounted at all. In this blog we present the details for mounting an EBS volume on any EC2 instance type.

Read more...

Auth0 can be configured as the identity provider for AWS, allowing your Auth0 users to directly log in to the AWS Console. Unfortunately, the functionality to obtain temporary access keys for calling AWS APIs and resources has been deprecated since June 8, 2017. With this command line utility, you can obtain AWS access keys through Auth0.

Read more...

AWS Lambda provides support for quite a few languages out of the box, but oddly enough there are no functional languages in the list. Our esteemed colleague Arjan Molenaar of Xebia has added support for Elixir, a very readable functional programming language that runs on the Erlang VM. Read the full blog on https://xebia.com/blog/building-an-elixir-runtime-for-aws-lambda/.