Amazon Simple Queue Service (SQS) - Simple in good and bad times
Now I am biased having just dealt with IBM's MQ product line and the walled garden approach to their tool suite, so I might be over complementary to Amazon AWS SQS. Okay, one more thing about IBM MQ, just look at the website, tired, old, selfware; some might argue stable, but hype is the name of the game. On the other hand AWS SQS fresh and simple, by name, approach, use, and in features. However, I believe there is something simply excellent about that.
Simple Setup - I had a queue up and running in 30 seconds. I clicked on the AWS Management Console tab, a browser/java based tool and entered some stuff in and bam queue. Then I right clicked on it, snappy, I can write a message to it. So, again I right clicked on the queue entry listed in the AWS SQS console and found a real-time queue reader/listener, and hey I can even delete messages.
Simple Developer Tools - Sure enough I flip over to the eclipse based tools and as expected I could access the queues from there as well. Nice I can write my 64KB per message max from eclipse, integrate with a plug-in, create a new queue, or whatever.
Simply Service Enable - Most important to me is that every feature is web service enabled. So, anything Amazon SQS has not automated or simplified can be further refined and utilities written. To some this may see interesting or novel, but let me assure you that large companies are slowed by walled gardens and management consoles that only the privileged or support companies have access to. If I dynamically want to create a queue for every batch so I can keep track of them, fine. If I want a queue for each and every of the million customers, fine, go ahead. The simple way to put it is that life cycle and direct control matters in large organizations period. Want to integration just an HTTP call with some XML error handling, which of-course is documented and has a schema, see for yourself, SQS API Documentation.
Simple Documentation - The documentation is short and simple. I understood how the main drawback or assumption AWS SQS was built on within 5 minutes. The examples drove every question I had, only so many you can have when you can list the functions on two hands. Read the architecture document, here.
Simply a message - I can't knock IBM here either, but this is about AWS SQS. A message can be any text up to 64 KB and can live for up to 14 days. I have spend weeks trying to wrestle simple non-functional requirements or constraints from product vendors or existing deployed environments. Not much to question when it is documented and any specific queue can be checked via the management console or a simple service call.
Simply Assumes Idempotent - Which is smartly pants for safe from replay attacks of messages due to outages or glitches. They chose the word Idempotent and they chose to saddle the consumers of the queue with this to ensure simplified recovery of nodes in their queue clusters. Now don't get me wrong as designing a consuming system to be Idempotent is not extremely hard, but it pushes the process and data locking onto your application performance. So you get fast queues, but certain transactions or types of applications will be hard to write. For example updating a Facebook status or tweeting twice might be fine, or an Order system that commits a transaction, then it is easy for an consuming application to be Idempotent. However, what about a complex relational multi-transaction rules engine driven process? Not so easy now. Now because I cannot trust the AWS SQS to not make a mistake the consuming application could have an architecture performance flaw, which could be avoided by a more rich queue management capability.
Simply distributed and centralized - There is a constant tension in an Enterprise to centralized or disperse. This is very true for enterprises with countless systems. AWS SQS is hosted at Amazon, period, and cannot even be easily distributed. If that data center disappears there could be issues. If an application wants to use it for internal communication without having to leave the network or memory, AWS SQS will let them down.
Simple Summary - So simplicity is a double edged sword that can provide architectural nirvana or nightmares. Don't get me wrong I am a huge fan of Amazon's approach to creating service oriented resources for the masses. They even give you thousands of free messages a year which inspires me to learn more and develop demos with SQS, but I cannot use it for real enterprise demands; darn.
Simple Developer Tools - Sure enough I flip over to the eclipse based tools and as expected I could access the queues from there as well. Nice I can write my 64KB per message max from eclipse, integrate with a plug-in, create a new queue, or whatever.
Simply Service Enable - Most important to me is that every feature is web service enabled. So, anything Amazon SQS has not automated or simplified can be further refined and utilities written. To some this may see interesting or novel, but let me assure you that large companies are slowed by walled gardens and management consoles that only the privileged or support companies have access to. If I dynamically want to create a queue for every batch so I can keep track of them, fine. If I want a queue for each and every of the million customers, fine, go ahead. The simple way to put it is that life cycle and direct control matters in large organizations period. Want to integration just an HTTP call with some XML error handling, which of-course is documented and has a schema, see for yourself, SQS API Documentation.
Simple Documentation - The documentation is short and simple. I understood how the main drawback or assumption AWS SQS was built on within 5 minutes. The examples drove every question I had, only so many you can have when you can list the functions on two hands. Read the architecture document, here.
Simply a message - I can't knock IBM here either, but this is about AWS SQS. A message can be any text up to 64 KB and can live for up to 14 days. I have spend weeks trying to wrestle simple non-functional requirements or constraints from product vendors or existing deployed environments. Not much to question when it is documented and any specific queue can be checked via the management console or a simple service call.
Simply Assumes Idempotent - Which is smartly pants for safe from replay attacks of messages due to outages or glitches. They chose the word Idempotent and they chose to saddle the consumers of the queue with this to ensure simplified recovery of nodes in their queue clusters. Now don't get me wrong as designing a consuming system to be Idempotent is not extremely hard, but it pushes the process and data locking onto your application performance. So you get fast queues, but certain transactions or types of applications will be hard to write. For example updating a Facebook status or tweeting twice might be fine, or an Order system that commits a transaction, then it is easy for an consuming application to be Idempotent. However, what about a complex relational multi-transaction rules engine driven process? Not so easy now. Now because I cannot trust the AWS SQS to not make a mistake the consuming application could have an architecture performance flaw, which could be avoided by a more rich queue management capability.
Simply distributed and centralized - There is a constant tension in an Enterprise to centralized or disperse. This is very true for enterprises with countless systems. AWS SQS is hosted at Amazon, period, and cannot even be easily distributed. If that data center disappears there could be issues. If an application wants to use it for internal communication without having to leave the network or memory, AWS SQS will let them down.
Simple Summary - So simplicity is a double edged sword that can provide architectural nirvana or nightmares. Don't get me wrong I am a huge fan of Amazon's approach to creating service oriented resources for the masses. They even give you thousands of free messages a year which inspires me to learn more and develop demos with SQS, but I cannot use it for real enterprise demands; darn.
Comments