Designing and implementing applications that make use of Amazon Web Services such as S3, SQS and SimpleDB, require extensive testing to ensure a robust integration. In addition to scenarios for the “happy flows”, test scenarios should be created to handle the critical exceptional situations. How does my application respond to failures in communication, checksum errors, service unavailability, never ending transactions and extremely frequent or heavy traffic?
In the spirit of test driven development, I see a need for writing up such scenarios and simulate the unexpected behavior of AWS just to verify the correct handling of each exceptional event. Such a Test Drive service would implement the API of services available at Amazon Web Services. It should provide extra control services to put a Web Service in some “testing mode” such as “simulate disk failure”, “simulate terminated http connection” or “simulate invalid access key"or a combination thereof.
To implement the persistent feature available in SQS, S3 and SimpleDB, a basic filesystem or relation database could be used to support test scenarios that involve more then one request. And of course all actions are properly logged and event information is made available to verify that a scenario is completed with success, i.e. the exceptions are handled correctly.
Finally, if I had such a Test Drive Service, I could startup multiple instances on different machines and run scenarios that involve combinations of AWS services SQS, S3 and SimpleDB. How much effort is needed to build such a Test Drive Service in e.g. Ruby on Rails? Isn’t this a common testing practice for all public service APIs out there on the Web?