Microservice Communication - AWS SQS | Sending Messages

2o5P...cmGf

20 Jul 2022

Intro

In this article, I'll show you how you can use the Amazon SQS for creating communication between two services.

There is a communication style which is communication by the queues, which is a communication type point to point.

A brief summary about QUEUES

Queues are typically communication point to point. A service that could be called a sender or provider sends a message to a queue and another service that is normally called consumer reads from that queue.

Service A sends a message to the broker and service B consumes the message by the broker

Here we gonna introduce the practice to use AWS SQS — Amazon Simple Queue Service, in which we will go build a queue communication between two microservices as the image below

Context

Amazon Simple Queue Service enables you to decouple and scale microservices between two services. Is fully message queueing.

Example from the service coupled with two modules, module A and module B - img1.0

Service decoupled, keeping module A and changing the module B into a service B with container instances - img1.1

Amazon Simple Queue Service - SQS

There are in SQS two communication types of a message queue:

STANDARD QUEUE: Offer a maximum throughput and the ordering of messages is not guaranteed.

Font: Aws Blog

FIFO QUEUE: guarantee that the messages are processed exactly once in the exact order that was sent.

Font: Aws Blog

Applicability

Ok, we have a little context about AWS and how a queue´s broker works, let’s practice.
There is a big fake project called Coin Wallet but I want to try to do this fake project closer to an actual microservice project, which needs to be scalable, deployable, with a single responsibility, loosely coupled, and owned by a small team.

About Coin Wallet

The Coin Wallet is an application that manages cryptocurrency wallets, the user might manage one or more wallets inside the app storing all coins that are bought in their respective purchase date
In real life, we can store in the wallet, coins/money, so in the Coin Wallet we will go put cryptocurrency, and you can manage your cryptocurrencies.

So we will need a service that be able to:
1- Save purchase order of any coin
2- Save sales order of any coin

Also, we will need a service that be able to:

1- Calculate the wallet total balance
2- Calculate the wallet total balance of each coin

For now … that is it!

Hands-On

General Architecture Overview

Microservice: Wallet Trade

Architecture Overview

Architecture Overview — send messages to SQS queues, purchase order, and sales order

Techstack

In this story I just focus on the producer implementation, if you got curious to see the full implementation, feel free to see my GitHub repo. If you have a question don’t hesitate to send me a message on my social media network.

Wallet Trade Producer

First, we need to create a spring-boot application. Access the spring-boot initializer and generate a project.

We need to add in the pom.xml the AWS SDK.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-aws-dependencies</artifactId>
            <version>2.3.1</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
   <dependency>
      <groupId>io.awspring.cloud</groupId>
      <artifactId>spring-cloud-starter-aws-messaging</artifactId>
   </dependency>
   <dependency>
      <groupId>org.springdoc</groupId>
      <artifactId>springdoc-openapi-ui</artifactId>
      <version>1.6.6</version>
   </dependency>
   <dependency>
      <groupId>software.amazon.awssdk</groupId>
      <artifactId>aws-sdk-java</artifactId>
      <version>${aws.java.sdk.version}</version>
   </dependency>
   <!--parse-->
   <dependency>
      <groupId>com.fasterxml.jackson.datatype</groupId>
      <artifactId>jackson-datatype-jsr310</artifactId>
   </dependency>

<!--spring-boot dependencies-->
  <dependency>
     <groupId>io.awspring.cloud</groupId>
     <artifactId>spring-cloud-starter-aws-messaging</artifactId>
  </dependency>          
          ...
<!--other dependencies-->
          ...
<!-- full code: https://github.com/andrelucasti/wallet-trade -->
<dependencies>

Code Design

Here is a brief summary of the code design of our project

Full code: GitHub repo

Configuration Beans

The first thing that we need to do is configure the application to support sending messages through AWS SDK.

import com.amazonaws.services.sqs.AmazonSQSAsync;
import io.awspring.cloud.messaging.core.QueueMessagingTemplate;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AmazonSQSConfiguration {

    @Bean
    public QueueMessagingTemplate 
     queueMessagingTemplate(AmazonSQSAsync amazonSQSAsync){
        
      return new QueueMessagingTemplate(amazonSQSAsync);
    }
}
// full code: https://github.com/andrelucasti/wallet-trade

Let´s create a new class that will be responsible to receive the queue name and the payload that will be sent to AWS SQS

Producer Implementation

@Service
public class SQSSender {
    
    private final QueueMessagingTemplate queueMessagingTemplate;

    public SQSSender(QueueMessagingTemplate queueMessagingTemplate){
        this.queueMessagingTemplate = queueMessagingTemplate;
    }
    
    public void send(String queueName, String message){
        var payload = MessageBuilder.withPayload(message).build();

        queueMessagingTemplate.send(queueName, payload);
    }
}
// full code: https://github.com/andrelucasti/wallet-trade

Building Infrastructure

Now let’s create the queues in AWS SQS

Building the Dead Letter Queue

The DLQ is basically to store messages during the consumption when happening a problem, by the way, is a great practice for asynchronous communication between services.
Run the command below to create a new dead letter queue — DLQ

aws sqs create-queue --queue-name wallet-calculator-purchase-order-dlq --attributes file://infrastructure/sqs/wallet-calculator-purchase-order-dlq.json

{  
  "VisibilityTimeout": "30",  "MessageRetentionPeriod": "604800"
}

wallet-calculator-purchase-order-dlq.json

Building the Queue

Run the command below to create a new queue that will be responsible to receive the Purchase Order’s payload of the coin wallet

aws sqs create-queue --queue-name wallet-calculator-purchase-order --attributes file://infrastructure/sqs/wallet-calculator-purchase-order.json

{  
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:575308843560:wallet-calculator-purchase-order-dlq\",\"maxReceiveCount\":\"1000\"}",  "VisibilityTimeout": "30",  "MessageRetentionPeriod": "432000"
}

wallet-calculator-purchase-order.json

Listing the queues created

To see all queues created run the command below

aws sqs list-queues

is expected the result like the image

or you can access the amazon console

Sending Messages

Now let’s run the application and send a POST to the endpoint:

localhost:9958/purchase-order

purchase order schema — controller

{
   "walletId*":"string($uuid)",
   "coinSymbol*":"string",
   "coinAmount*":"number($double)",
   "purchaseOrderDate*":"string($date-time)"
}

example

{
  "walletId": "d7541e18-f54b-4202-a3ec-e0e9f31360a5",
  "coinSymbol": "BTC",
  "coinAmount": 1,
  "purchaseOrderDate": "2022-07-03T11:48:38.594Z"
}

From the moment in which we did a request to the controller, the application process this request in some phases, and one of these phases is to send it to the queue that we created: wallet-calculator-purchase-order

After request, we can check the queue and see the message that was sent by the application like this

{
  "walletId":"d7541e18-f54b-4202-a3ec-e0e9f31360a5",
  "coinDTO":{
      "id":1,
      "symbol":"BTC",
      "purchaseOrderValue":19053.0 
      // Bitcoin price in 2022-07-03:14:25
  },
  "coinAmount":1.0
}

To check the message that was sent, access the amazon console and get the message like the gif below:

Conclusion

In this story, we had a brief about a communication style, communication by queues, in which this communication type is between two microservice, the benefit is more:

Scalability and decoupling: we can scale a piece of the application (module) rather than scale the application fully, bringing an economic infrastructure

Disponibility: with the DLQ mechanism and the broker as middleware between services you gain more power with temporary storage until the message is consumed

Reliability uses AWS SQS to send messages a lot, while not losing increasing the fault tolerance of the system.