Queues have always played an integral role in software architecture. They allow asynchronous communication among the systems having different throughput. With the recent trend towards microservices, queues have become more important than ever before. To produce robust software architectures, Amazon also offers its own version of queues in the form of AWS SQS (Simple Queue Service).
In this AWS SQS tutorial, we are going to discuss the following topics:
- What is AWS SQS?
- How Does AWS SQS Works?
- Benefits AWS SQS
- Features AWS SQS
- Amazon SQS Pricing
- Amazon SQS vs Amazon SNS
- Comparison between Amazon SQS and Amazon Kinesis streams
What is AWS SQS
- Amazon SQS is a message queue used by the processes to communicate with each other and carry out the operations.
- SQS supports both types of queues - standard (unordered) and FIFO (ordered).
- Amazon Simple Queue Service automatically deletes the messages once the maximum message retention period is over. By default, the retention period is 4 days.
- Using the SetQueueAttributes action, a user can set the message retention time to a value from 1 Minute to 14 days.
Learn more about AWS
How Amazon SQS works
Amazon SQS is a distributed queue system that allows applications to queue messages that are generated by one component and consumed by another component. SQS acts as a temporary repository for messages and is used in situations where the messages are produced at a higher rate but get processed at a lower rate.
There are three parts in the SQS messaging system - components of a distributed system, a queue and messages in the queue.
From the given image, you can see that a system consists of many producers (components that generate messages and send it to the queue) and consumers (components that retrieve messages from the queue and process it). Stay tuned with AWS SQS tutorial to learn more concepts of SQS.
Lifecycle of a Message
- Component 1 generated message A and sent it to the queue. The message gets distributed among the SQS servers.
- After that, component 2 consumes the message A and processes it. Once the message is received by some component, SQS sets a visibility timeout to prevent other components from receiving and processing the message. The default visibility time is 30 seconds, the minimum is 0 second and the maximum is 12 hours.
- Then, component 2 deletes message A from the queue so that it does not process it again after the visibility timeout expires. See the below-given image to understand the concept more clearly.
Benefits of AWS SQS
Eliminate administrative overhead
SQS queues can be created and scale automatically based on the demand. With SQS, a user need not install any messaging software to carry out operations and maintaining the infrastructure.
Reliably deliver messages
Amazon SQS decouples the application components which makes them run or fail independently and increase the overall fault tolerance of the system. Copies of all the messages are stored across multiple zones to avail them whenever needed.
Keep sensitive data secure
With the help of Amazon server-side encryption (SSE), we can encrypt our data and share it between the applications. Integration of Amazon SSE with AWS KMS (Key management service) allows a user to centrally manage the keys responsible for protecting SQS messages and other AWS resources.
Scale elastically and cost-effectively
Subscribe to our youtube channel to get new updates..!
Amazon SQS scales dynamically based on demand, therefore, you do not have to pre-plan the capacity and worry about pre-provisioning. There can be an unlimited number of messages per queue and the cost is based on usage. So, we can say that it is a cost-effective and elastic service.
Now the next section in this Amazon SQS tutorial article will help you know about the amazing features of AWS SQS.
Features of Amazon SQS
No Data Loss
SQS ensures that the messages sent are not lost and secure. Hence, it stores them on multiple standard queues.
SSE (server-side encryption) allows transmission of highly confidential and sensitive data by offering protection to the messages present in the queues with the help of Amazon key management service.
Each Request is handled independently
In SQS, every buffered request is processed independently ie. each request is scaled transparently to handle the increase in load without any explicit instructions.
SQS locks the user messages when it is being processed so that various producers can send messages to various receivers and receivers could receive messages from multiple recipients at the same time.
Two queue types
Amazon SQS offers two types of queues - standard and FIFO queue for different application requirements.
Unlimited queues and messages
A user can create unlimited SQS queues processing an unlimited number of messages in any region.
Messages payload can be of maximum 256Kb size in any format. Each 64KB of a payload is considered as 1 request, hence, a single API call with 256KB payload is considered as 4 requests. If a user wants to send a message having a size larger than 256KB, he can use Amazon SQS extended client library for Java.
Amazon SQS Pricing
For the first 1 million, monthly requests are free. But after that, the user will be charged which may vary depending upon the region. For eg. for the region, US (east), Amazon SQS pricing is as follows-
- Standard queue - $0.40 ($0.0000004 per request)
- FIFO queue - $0.50 ($0.0000005 per request)
SNS Vs SQS
Amazon SNS (Simple Notification Service) is a fast and flexible push notification service that allows a user to send messages to a large number of mobile users, email recipients and also other distributed services. Amazon SQS is a distributed queuing system where messages are not pushed but receivers have to poll SQS to receive messages.
|Entity Type||Topic (Pub/Subsystem)||Queue (Similar to JMS)|
|Message Consumption||Push Mechanism||Pull Mechanism|
|Use Case||Fanout (the same message is processed in different ways)||Decoupling two applications and allow parallel asynchronous processing|
|Persistence||No persistence. If a customer is unavailable, the message is lost||Message is persisted for some time if the consumer is unavailable|
|Consumer Type||Consumers are supposed to be identical and process the message in the same way||Consumers process the message in different ways|
Comparison Between Amazon SQS and Amazon Kinesis streams
When it comes to creating a message based cloud application having distributed architecture, AWS offers two options - the Kinesis stream and the SQS queue. Which one to choose - SQS or Kinesis? Both the technologies seem similar but they have vastly different use cases. Kinesis is designed to manage and process real-time continuous data streams whereas SQS makes it easy to decouple and scale serverless applications, microservices, and distributed systems.
Kinesis Use Cases
- Real-time Analytics
- Mobile Data Capture
- Log and Event Data Collection
- “Internet of Things” Data Feed
Benefits of Kinesis
To get the absolute maximum throughput for data processing, Kinesis is the best choice. The delay time between writing a data record and reading it from the Stream is less than a second, regardless of how much data a user needs to write.
If you have to process terabytes of data on a daily basis in a single Stream, Kinesis is the best option to go with. A user can push data from various data producers, which is stored for later processing or read out in real-time.
SQS Use Cases
- Application integration
- Decoupling microservices
- Batch messages for future processing
- Allocate tasks to multiple worker nodes
- Decouple current user requests from intensive background work
Benefits of SQS
Ease of Use
SQS is very easy to use. In contrast to Kinesis, the user does not have to read special libraries or write to an SQS queue. Also, no need to coordinate among consumers.
SQS easily scales to manage and process a large volume of messages, without user intervention and no pre-provisioning of AWS resources. Amazon SQS buffer requests to manage the spikes in load with ease.
Hopefully, this simple walkthrough has helped you find out what is Amazon SQS and how it fulfils the queuing needs in software architecture. Let us know how did you like this AWS SQS tutorial article and comment down if you have any query regarding Amazon SQS.
If you interested to learn AWS and build a career in Cloud Computing? Then check out our AWS Certification Training Course at your near Cities
AWS Course in Ahmedabad, AWScourse in Bangalore, AWS course in Chennai, AWS course in Delhi, AWS course in Dallas, AWS course in Hyderabad, AWS course in London, AWS course in Mumbai, AWS course in NewYork, AWS course in Pune
These courses are incorporated with Live instructor-led training, Industry Use cases, and hands-on live projects. This training program will make you an expert in AWS and help you to achieve your dream job.