AWS SimpleDB – what’s it good for?

Actually – it’s a nice and handy part of your cloud toolbox is what.

First of all – it’s a non-relational data store.  Which means you are not going to have highly normalized table structures and you are not (for the here and now) going to be able to join two tables in a SQL like query.

BUT!  It does work faster and more efficiently that trying to get access to a XML config file; it provides persistence that an in-memory data structure can’t; you can query it with SQL type statements; you can either insert or update data using a decent API; plus you can add data fields on the fly in your code.  Best of all – it’s very easy to use, and it’s cheap.  Far cheaper than either the MySQL or Oracle offerings – you get charged for every hour those guys are up and running.  For SimpleDBthe pricing structure is a lot more attractive.

Some things to square away first.  Forget the idea of tables – each unique data structure in SimpleDB is called a domain.  It’s a non-relational/quasi no SQL data structure that does allow SQL queries to get data.  Think tree and you have it – the primary node for each data row is called an itemName – the rest of the data fields are elements of the primary node.

So for my very simple Java servlet – I needed a user data structure that had to be persistent.  With SimpleDB the domain (think table) name became MyUsers – and the primary node for each row – the itemName in SimpleDB syntax – was the unique user ID.  I had one other data element – the child node – it was a URL for the user.

I have covered the making the connection to AWS in an earlier post so I’m not going over that again.

A great resource is the sample code that Amazon provides – available for more than just Java.

Next couple of posts I’m going to cover creating a domain, putting data into it, and then getting the data back out.


Amazon Web Services and SimpleDB Part Two

Next item: adding the AWS jar file libraries to the ant build.xml file.

Actually – this gets a bit round about.  Amazon supplies the SDK files – and if I had more experience with Eclipse this probably would not have been an issue.  In any event – I downloaded the AWS SDK for Java files into a directory on my C drive – C:\AWS_HOME to be exact.  I added the files to my path and class path – didn’t matter for building a servlet as the code wouldn’t compile.

Next step – I added a folder in my project, copied the AWS jar files into that file and set the path in my ant build.xml file.  Okay – got it to compile but Jetty would bomb.

Onto step number three – I added the following files to my Jetty ..\lib\ext folder:

From \aws-java-sdk-1.2.10\lib – aws-java-sdk-1.2.10.jar, aws-java-sdk-1.2.10-javadoc.jar and aws-java-sdk-1.2.10-sources.jar.

From \aws-java-sdk-1.2.10\third-party\commons-codec-1.3\commons-codec-1.3.jar.

From \aws-java-sdk-1.2.10\third-party\commons-logging-1.1.1\commons-logging-1.1.1.jar.

And lastly from \aws-java-sdk-1.2.10\third-party\httpcomponents-client-4.1.1\httpclient-4.1.1.jar and httpcore-4.1.jar.

My code would now compile – but then I got a weird error attempting to connect to SimpleDB.  I’m working with a 64 bit version of Windows 7, and Amazon supplies a 32 bit encrypt/decrypt codec from Apache.  I downloaded the commons-codec-1.5 jar file from Apache and added that to my Jetty directory – now it all worked fine.

Issue four – I naively thought that the Tomcat environment I was using on Amazon would have the library files included to connect to SimpleDB.  I was wrong – I had to include the files in my build file.  I created a WEB-INF\lib directory and placed copies of the above files into it.  My ant build file now included:

<path id=”compile.classpath”>

<fileset dir=”${env.JETTY_HOME}/lib”>

<include name=”**/*.jar”/>


<fileset dir=”${env.AWS_HOME}/third-party” includes=”**/*.jar”/>

<fileset dir=”${env.AWS_HOME}/lib” includes=”**/*.jar”/>

<pathelement location=”.”/>

<pathelement location=”../lib/aws-java-sdk-1.2.10.jar”/>

<pathelement location=”../lib/aws-java-sdk-1.2.10-javadoc.jar”/>

<pathelement location=”../lib/aws-java-sdk-1.2.10-sources.jar”/>

<pathelement location=”../lib/commons-codec-1.5.jar”/>

<pathelement location=”../lib/commons-codec-1.5-javadoc.jar”/>

<pathelement location=”../lib/commons-codec-1.5-sources.jar”/>

<pathelement location=”../lib/commons-logging-1.1.1.jar”/>

<pathelement location=”../lib/httpclient-4.1.1.jar”/>

<pathelement location=”../lib/httpcore-4.1.jar”/>


Success!  Now not only could I connect to SimpleDB on my client using Jetty, but also on my Amazon environment.

Next post – some more on using SimpleDB.

Amazon Web Services and SimpleDB

Recently finished a simple Java Servlet using Amazon Web Services which required a simple data persistence layer.  The natural choice was SimpleDB.  And I found some gotchas…

First of all – a thumbnail sketch of SimpleDB.  It’s a schemaless data structure consisting of hash tables with sets of key value pairs.  A nice write-up is here.  The AWS SimpleDB site is here.

My typical dev process was build and test local, then move up to AWS.  I’m using Java version jdk1.6; Apache Ant 1.8.1; Jetty 8.0.1x.

Start by downloading the AWS Java SDK .  Amazon provides a good resource of Java programs providing examples of using their various cloud products – the two I used as reference for this project was in the samples\AmazonSimpleDB and samples\AwsConsoleApp directories.

There were four issues I ran into before being able to successfully use the product:

1.  Connect to SimpleDB thru the supplied AWSCredentials API.

2. Adding the AWS jar file libraries to the ant build.xml file.

3. Add the AWS jar files to Jetty for development and testing.

4. Make sure the AWS instance had access to the jar files.

First issue, connection using a credential set.  You create your credentials when you set up your Amazon Web Services account.  And a blank credential file is provided in the SDK demo programs.  Since this is a servlet program, the code resides as child directories of a WEB-INF folder.  I placed my credentials file into the WEB-INF\resources folder.

Next thing is to bring the credential file into the various methods that need access to the SimpleDB.  First I defined a String variable containing the path to the credential file:

String filename = “/WEB-INF/resources/”;

The file is brought in as a stream:

InputStream is = context.getResourceAsStream(filename);

An AWS credentials object is set using the data from the stream:

AWSCredentials credentials = new PropertiesCredentials(is);

And finally an AWS SimpleDB object is created, using the AWS credentials.  This SimpleDB object is what you use to connect to your SimpleDB domain:

AmazonSimpleDB sdb = new AmazonSimpleDBClient(credentials);

I’ll cover the other issues in future posts.

%d bloggers like this: