How to provision resources in AWS using JAVA - AWS CDK
There are many ways to provision resources in AWS.
A good practice is to use an IaC tool that you gain many advantages:
- Version Control — IaC can be version controlled
- CI/CD — After a change in infrastructure, you can use CI/CD tools that to deliver these changes.
- Many Environments — If you need to reply to the same resources in another env, is simple to do this, only run the project with the resources passing the env (depends on how you organize the environments).
- Infrastructure tests — There is the possibility the testing the infrastructure.
AWS Cloud Development Kit
AWS Cloud Development Kit (CDK) accelerates cloud development using common programming languages to model your applications.
Image from AWS Cloud Development Kit (AWS CDK) Developer Guide
The AWS CDK is an open-source software development framework to model and provision cloud application resources through AWS CloudFormation, programmatically with languages like Typescript, Javascript, Go, Python, C#, and Java.
We’re using the Java language in this story because I’m a java fan, so here we go!
Pre-requisite:
- Java 8 +
- NodeJS
- NPM
Installing the CDK
To install de AWS CDK use the npm to do this.
npm install -g aws-cdk
Creating a Project
To create a CDK java project:
mkdir medium-cdk-project && cd medium-cdk-project
cdk init app --language java
after the project is created, we can see the initial structure, some build blocks known as Constructs, which are composed together to form Stacks and Apps.
package com.myorg; import software.amazon.awscdk.App; import software.amazon.awscdk.Environment; import software.amazon.awscdk.StackProps; import java.util.Arrays; public class MediumProjectApp { public static void main(final String[] args) { App app = new App(); new MediumProjectStack(app, "MediumProjectStack", StackProps.builder() .env(Environment.builder() .account("123456789012") .region("us-east-1") .build() ).build() ); app.synth(); } }
package com.myorg; import software.constructs.Construct; import software.amazon.awscdk.Stack; import software.amazon.awscdk.StackProps; // import software.amazon.awscdk.Duration; // import software.amazon.awscdk.services.sqs.Queue; public class MediumProjectStack extends Stack { public MediumProjectStack(final Construct scope, final String id) { this(scope, id, null); } public MediumProjectStack(final Construct scope, final String id, final StackProps props) { super(scope, id, props); // The code that defines your stack goes here // example resource // final Queue queue = Queue.Builder.create(this, "MediumProjectQueue") // .visibilityTimeout(Duration.seconds(300)) // .build(); } }
Concepts
Let’s see the main CDK build blocks.
Constructs
Constructs represent a “cloud component” and encapsulate everything AWS cloud formation needs to create the component.
A thing that a discover when I was writing this story is that a Construct is Constructs are part of the Construct Programming Model (CPM), so, there are other tools that implement this concept as CDK for Terraform (CDKtf), CDK for Kubernetes (CDK8s), and has support languages like Typescript, Python, Java, C# and Go.
Image by https://developer.hashicorp.com/terraform/
Constructs can represent a single AWS resource, for example, a Queue (SQS), a Topic (SNS), a bucket storage (S3)
L1 Construct
Represent all resources available in AWS CloudFormation. These constructs start with the prefix Cfn. For example, CfnDBInstance represents the AWS::RDS::DBInstance
import software.amazon.awscdk.Stack; import software.amazon.awscdk.StackProps; import software.amazon.awscdk.services.rds.CfnDBInstance; import software.constructs.Construct; public class RDSStack extends Stack{ private static final String RDS_ENGINE = "postgres"; private static final String RDS_ENGINE_VERSION = "14"; private static final String RDS_US_EAST_1 = "us-east-1d"; private static final String DB_NAME = "walletManagerDatabase"; private static final String RDS_DB_TYPE = "db.t4g.micro"; public void execute(){ CfnDBInstance.Builder.create(this, "database-stack") .engine(RDS_ENGINE) .engineVersion(RDS_ENGINE_VERSION) .publiclyAccessible(true) .storageEncrypted(false) .allocatedStorage("30") .availabilityZone(RDS_US_EAST_1) .dbName(DB_NAME) .dbInstanceClass(RDS_DB_TYPE) .deleteAutomatedBackups(true) .masterUsername("") .masterUserPassword("") .build(); } }
L2 Construct
Also, represent all resources available in AWS CloudFormation, but at a high level. Offer convenients defaults and reduce the need to know all details about the AWS resource. For example, Topic.Builder.create represent the creation of topic in AWS.
L3 Construct
These constructs are to help you to complete tasks that involve multiple resources in AWS. For example, The LambdaRestApi construct represents an Amazon API Gateway API that’s backed by an AWS Lambda function.
LambdaRestApi api = LambdaRestApi.Builder.create(this, "medium-lambda") .proxy(false) .handler(Function.fromFunctionName(this, "medium-lambda-api", "medium-lambda")) .build(); Resource items = api.getRoot() .addResource("items"); items.addMethod("GET"); // GET /items items.addMethod("POST");// POST /items Resource item = items.addResource("{item}"); item.addMethod("GET");// GET /items/{item}
Stacks
Is the unit of deployment. All AWS resources are defined within a stack.
We can define a lot of stack numbers in we AWS CDK app.
Example:
public class AwsApp { public static void main(final String[] args) { App app = new App(); Account account = new Account(app); String accountId = account.getAccountId(); String region = account.getRegion(); new WalletManagerStack(app, "wallet-manager-aws-resource-stack", StackProps.builder().env(getEnv(accountId, region)) .build()); new WalletTransactionStack(app, "wallet-transaction-aws-resource-stack", StackProps.builder().env(getEnv(accountId, region)) .build()); app.synth(); } }
Also, we can see the number of stacks with the command:
cdk ls -c accountId=00000000000 -c region=us-east-1
Apps
An app is a container for one o more stacks, its serve as stack’s scope.
Let’s see in the practice
public static void main(final String[] args) { App app = new App(); Account account = new Account(app); String accountId = account.getAccountId(); String region = account.getRegion(); executeStacks(app, accountId, region); app.synth(); } private static void executeStacks(App app, String accountId, String region) { new WalletManagerStack(app, "wallet-manager-aws-resource-stack", StackProps.builder().env(getEnv(accountId, region)) .build()); app.synth(); }public static void main(final String[] args) { App app = new App(); Account account = new Account(app); String accountId = account.getAccountId(); String region = account.getRegion(); executeStacks(app, accountId, region); app.synth(); } private static void executeStacks(App app, String accountId, String region) { new WalletManagerStack(app, "wallet-manager-aws-resource-stack", StackProps.builder().env(getEnv(accountId, region)) .build()); app.synth(); }
We instanced a new app, and its serve as stack’s scope. We can deploy any stack with the command:
cdk deploy -c accountId=0000000000 -c region=us-east-1 --wallet-manager-aws-resource-stack
to deploy a stack collection
cdk deploy -c accountId=0000000000 -c region=us-east-1 --all
Deploying in AWS | Hands-on
Let’s create a scenario closer to reality.
We’ve two microservices, wallet-manager, and wallet-transaction
- Wallet Manager is in charge create portfolios save in your database and to publish in a topic
- Wallet Transaction is in charge to consume portfolios sent from Wallet Manager
Architecture Overview
An advantage is to create the AWS resources oriented by tests, so this is a great practice.
Wallet Manager
We can see that in the Architecture Overview image, the Wallet Manager must use some AWS resources that are a topic, and a database. Let’s create them.
Creating topic
We’re to use the Amazon SNS. To create a topic in AWS using AWS CDK is:
Firstly let’s create the test in which we go waiting for the return of the resource created
something like:
// expected { "Type" : "AWS::SNS::Topic", "Properties" : { "TopicName" : "topicName" } }
import org.junit.jupiter.api.Test; import software.amazon.awscdk.App; import software.amazon.awscdk.Environment; import software.amazon.awscdk.StackProps; import software.amazon.awscdk.assertions.Match; import software.amazon.awscdk.assertions.Template; import java.util.Map; class SNSStackTest { @Test void shouldCreateWalletManagerTopic() { StackProps stackProps = StackProps.builder() .env(getEnv("000000000000", "us-east-1")) .build(); SNSStack snsStack = new SNSStack("walletManagerTopic", new App(), stackProps); snsStack.execute(); Template template = Template.fromStack(snsStack); // assert template.hasResourceProperties("AWS::SNS::Topic", Map.of( "TopicName", Match.exact("wallet-manager-portfolio")) ); } public static Environment getEnv(final String accountId, final String region){ return Environment.builder() .account(accountId) .region(region) .build(); } }
After creating the test let’s implement the resource.
package com.walletmanager.sns; import software.amazon.awscdk.Stack; import software.amazon.awscdk.StackProps; import software.amazon.awscdk.services.sns.Topic; import software.constructs.Construct; public class SNSStack extends Stack { public SNSStack(String stackName, Construct construct, StackProps stackProps) { super(construct, stackName, stackProps); } public void execute(){ Topic.Builder.create(this, "wallet-manager-portfolio") .topicName("wallet-manager-portfolio") .build(); } }
Creating RDS database
At the database creation, we gonna get the user and the password through a secret using the AWS secret manager which already has been created
let’s create the test in which we waiting for the result, something like:
// expected { "Type" : "AWS::RDS::DBInstance", "Properties" : { // ... "DBInstanceClass" : String, "DBInstanceIdentifier" : String, "DBName" : String, "Engine" : String, // ... } }
Unit test
import org.junit.jupiter.api.Test; import software.amazon.awscdk.App; import software.amazon.awscdk.Environment; import software.amazon.awscdk.StackProps; import software.amazon.awscdk.assertions.Match; import software.amazon.awscdk.assertions.Template; import java.util.Map; class RDSStackTest { @Test void shouldCreateWalletDB() { StackProps stackProps = StackProps.builder() .env(getEnv("000000000000", "us-east-1")) .build(); App app = new App(); RDSStack rdsStack = new RDSStack("WalletManagerRdsStack", app, stackProps, "walletManagerDBSecret"); rdsStack.execute(); Template template = Template.fromStack(rdsStack); template.hasResourceProperties("AWS::RDS::DBInstance", Map.of( "DBName", Match.exact("walletManagerDatabase"), "EngineVersion", "14") ); } public static Environment getEnv(final String accountId, final String region){ return Environment.builder() .account(accountId) .region(region) .build(); } }
Implementation
import com.fasterxml.jackson.annotation.JsonProperty; import com.fasterxml.jackson.databind.ObjectMapper; import software.amazon.awscdk.Stack; import software.amazon.awscdk.StackProps; import software.amazon.awscdk.services.rds.CfnDBInstance; import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider; import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain; import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider; import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient; import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueRequest; import software.constructs.Construct; public class RDSStack extends Stack { private static final String RDS_ENGINE = "postgres"; private static final String RDS_ENGINE_VERSION = "14"; private static final String RDS_US_EAST_1 = "us-east-1d"; private static final String DB_NAME = "walletManagerDatabase"; private static final String RDS_DB_TYPE = "db.t4g.micro"; private final String secretName; public RDSStack(final String stackName, final Construct scope, final StackProps props, final String secretName) { super(scope, stackName, props); this.secretName = secretName; } public void execute(){ var secretsManagerClient = SecretsManagerClient.builder() .credentialsProvider(getAwsCredentialsProvider()) .build(); var secretValueRequest = GetSecretValueRequest .builder() .secretId(secretName) .build(); var secretValue = secretsManagerClient.getSecretValue(secretValueRequest).secretString(); try{ var objectMapper = new ObjectMapper(); var dbSecretManagerDTO = objectMapper.readValue(secretValue, DBSecretManagerDTO.class); CfnDBInstance.Builder.create(this, "wallet-manager-database-stack") .engine(RDS_ENGINE) .engineVersion(RDS_ENGINE_VERSION) .publiclyAccessible(true) .storageEncrypted(false) .allocatedStorage("30") .availabilityZone(RDS_US_EAST_1) .dbName(DB_NAME) .dbInstanceClass(RDS_DB_TYPE) .deleteAutomatedBackups(true) .masterUsername(dbSecretManagerDTO.username()) .masterUserPassword(dbSecretManagerDTO.password()) .build(); }catch (Exception e){ throw new RuntimeException(e); } } private record DBSecretManagerDTO(@JsonProperty("username") String username, @JsonProperty("password") String password) { } protected AwsCredentialsProvider getAwsCredentialsProvider() { return AwsCredentialsProviderChain.builder() .addCredentialsProvider(DefaultCredentialsProvider.create()) .build(); } }
Running all stacks
import com.walletmanager.rds.RDSStack; import com.walletmanager.sns.SNSStack; import software.amazon.awscdk.Stack; import software.amazon.awscdk.StackProps; import software.constructs.Construct; public class WalletManagerStack extends Stack { public WalletManagerStack(final String stackName, final Construct scope, final StackProps props) { super(scope, stackName, props); SNSStack snsStack = new SNSStack("wallet-manager-sns-stack", scope, props); snsStack.execute(); RDSStack rdsStack = new RDSStack("wallet-rds-resource-stack", scope, props, "walletManagerDBSecret"); rdsStack.execute(); } }
App
import software.amazon.awscdk.App; import software.amazon.awscdk.Environment; import software.amazon.awscdk.StackProps; public class AwsApp { public static void main(final String[] args) { App app = new App(); Account account = new Account(app); String accountId = account.getAccountId(); String region = account.getRegion(); executeStacks(app, accountId, region); app.synth(); } private static void executeStacks(App app, String accountId, String region) { new WalletManagerStack("wallet-manager-resource-stack", app, StackProps.builder().env(getEnv(accountId, region)) .build()); app.synth(); } public static Environment getEnv(final String accountId, final String region){ return Environment.builder() .account(accountId) .region(region) .build(); } }
cdklocal bootstrap -c accountId=000000000000 -c region=us-east-1 --all
SNS log creation
SNS — wallet-manager-portfolio created by AWS CDK
RDS Log creation
RDS Created by AWS CDK
Cloud Formation Stack by AWS CDK
To remove the resources just use the command:
cdk destroy -c accountId=000000000000 -c region=us-east-1 --all
Or use the command to remove a specific stack.
cdk destroy -c accountId=000000000000 -c region=us-east-1 wallet-rds-resource-stack
GitHub Job
https://github.com/andrelucasti/micro-coin-wallet/actions/runs/3749238341/jobs/6367494255
Fully Code
https://github.com/andrelucasti/micro-coin-wallet/tree/main/wallet-manager/wallet-manager-aws-resources
Conclusion
The AWS CDK is a great option for developers because is possible to define resources application using familiar programming languages, an alternative instead of using JSON files or YAML files.
References
AWS CDK Documentation
Stratospheric: From Zero to Production with Spring Boot and AWS