Recently AWS launched a new service — Amazon DocumentDB designed to give us the performance, scalability, and availability we need when operating MongoDB workloads at scale. In this story, we will implement a stack that consists of these resources:
- S3 storage to provide source code to AWS Lambda.
- AWS Lambda to run our microservice.
- API Gateway to interact with microservice.
- Route 53 to access API Gateway by our domain.
- Amazon DocumentDB to provide MongoDB database.
- EC2 instance to access database locally via SSH.
- VPC so that EC2 instance and AWS Lambda can access Amazon DocumentDB.
Our final goal is to have running Amazon DocumentDB that we can access from local computer and AWS Lambda. Also, we want to make Lambda available at some URL. In the end, we should have a repository that looks like this.
My goal at this article is to provide you information about how those resources work together and how to launch them in a few minutes without stress. You can clone the GitHub repository or go along and describe all resources on the way. If you only need to run Amazon DocumentDB or don’t have a domain you can delete terraform files that contain resources you won’t need.
Variables and Main
First things first, before we start writing any resources we need to set environment variables(AWS credentials).
$ export AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID>
$ export AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY>
Next, let’s create the first terraform file — vars.tf. Here we specify all variables we need to create resources on AWS.
The first variable is the region. And we need to choose from those regions that support Amazon DocumentDB. At the moment of writing, there are only a few of them. Also, we need to take into consideration the fact that prices differ depending on the region.
We can specify whatever name we want — it will be used to name resources.
We need to provide a docdb_password that will be used to connect to MongoDB.
If we need a more powerful instance, we can change the value of the docdb_instance_class variable.
The last three variables needed to serve Lambda via API Gateway on the domain we own. If we don’t have one — we leave those variables undefined.
We can specify variables by editing a file or by setting environment variables like this.
$ export TF_VAR_name=<NAME_FOR_RESOURCES>
$ export TF_VAR_docdb_password=<PASSWORD_FOR_AMAZON_DOCUMENTDB>
$ export TF_VAR_certificate_arn=<YOUR_DOMAIN_SERTIFICATE_ARN>
$ export TF_VAR_zone_id=<YOUR_DOMAIN_ZONE_ID>
$ export TF_VAR_main_domain=<YOUR_DOMAIN>
File main.tf will only contain provider with region taken from variables.
Amazon DocumentDB can be accessed only from resources in the same VPC. Therefore we start by creating a VPC and security group in vpc.tf file.
We will describe a few resources in docdb.tf to run database cluster. In our example cluster will contain only one instance and we disable TLS to make it simpler to connect to the database.
In this example, AWS Lambda will run NodeJS microservice. To use another language one should update two parameters in aws_lambda_function resource — handler and runtime.
To create and run AWS Lambda, we need to have source code in S3 bucket first. But because we don’t have any microservice yet, we create folder lambda and inside of it folder src that contains a file named lambda.js.
By using terraform archive_file and the aws_s3_bucket_object, we can zip the lambda folder into lambda.zip and deploy it to S3 bucket. To access MongoDB in our microservice, we provide connection string as an environment variable to AWS Lambda.
To access Lambda via HTTPS, we will describe API Gateway resources at api_gateway.tf.
If we have a domain in AWS and we specified environment variables, we can create Route 53 record and serve API Gateway at given domain name. At the start of each resource at api_gateway_plus_domain.tf we specify count — if a domain value is not specified count will be equal to zero and resource will not be created.
We only can connect to Amazon DocumentDB when we are at the same VPC. Because of this, we need to create EC2 instance in the same VPC, and by connecting to it via ssh, we can interact with MongoDB.
Our instance will use Ubuntu as an OS. To install MongoDB shell, we will use provisioner that will run listed commands in the newly created instance. Also, we will create a key pair that required to connect to the instance from the local computer. To save the private key in the file, we are using local_file resource.
To access those resources after creation, we need to make some outputs. At outputs.tf we list what data we want to see in the terminal when all resources successfully created.
Now let’s run those two commands to create resources.
$ terraform init
$ terraform apply
Let’s grab URL from outputs, and paste it in the browser.
Now let’s connect to the EC2 instance via SSH.
$ ssh -i tf-<var.name>-ec2 ubuntu@<aws_instance_public_dns>
After we created all resources, the private key appeared in directory with our terraform files, in this example it is tf-geek-api-ec2. Public DNS of instance we can take from outputs.
To connect to MongoDB, we type the command shown below.
$ mongo \
--host <docdb_endpoint> \
--username <docdb_username> \
Both endpoint to database and username we can take from outputs. Password we specified at variables.
To update Lambda with a new version of microservice, we type those commands.
$ export AWS_DEFAULT_REGION=<var.region>
$ aws s3 cp lambda.zip s3://<bucket>/<bucket_key>
$ aws lambda update-function-code --function-name tf-<var.name> --s3-bucket <bucket> --s3-key <bucket_key>
Both endpoint to database and username we can take from outputs. Name and region we specified at variables.
At this story, we created a lot of resources that can be used to create new microservices that will leverage the power of DocumentDB and Lambda.
Reach the next level of focus and productivity with increaser.org.