Amazon cloud instances (EC2) as performance test load generators

Recently I was given a task of executing a performance test for one of our customer's applications.

Test was simple. It was primarily to test a "Maps download" scenario (each file size to be around 30 MB). A dedicated Maps generating/downloading server is there to cater to the need and our task was to find out how well the server responds to the user need.

JMeter (2.9) was used for this test and it was ideal for the scenario.

Creating valid/ realistic user load

Doing some calculations/statistical analysis, we estimated a figure for "Downloads/Hour". Our test was around assessing the performance for the above.

Load Generation

For the purpose of load generation, we decided to go for a cloud solution, and we choose Amazon AWS services, after doing some researches using its "Free tier" provision.

Some reasons why we go for a Cloud solution

1. The hosting Map server is on the Internet - We, as test team is located far away geographically as opposed to the real uses who are using it. So we needed to perform the test as close as possible . Amazon gives the flexibility of choosing a closer location because their data centers are located all over. (Or at least in our case they had regions which we wanted)

2. We wanted to test the application at "last mile" as opposed to "First mile" because, no body knows what's beyong your firewall. You need to try out your entire delivery chain including your ISPs, 3rd party applications like CDN, Ad servers, other web services that your application consumes which are basically beyond your control.

3. We were afraid to do it using our internal network since it could generate heavy traffic inside which could have an impact on the rest. (Still we do not have a separated test lab for this purpose)

4. Since we do not have an internal test lab in place, there was an issue in finding right hardware to generate the anticipated load for the test.

Support from Amazon  

I was impressed by the support given by Amazon team (Support service) for some account related issues I had initially. It was prompt and useful.

Using EC2 Instances

For the test we used "On-demand" extra-large standard instances having 8 compute units (4 cores) and 15 Gig RAM. The IO performance relatively OK, having intermittent lagging.

Standard Instances comes with 30 Gig Hard disk and the operating system consumes 75% if it (It was Windows OS) and left over with 7GB usable. So had to add a new volume using the Management console and attach it to the instance. After restarting the instance, you will see a new partition being added in My computer.

Distributed Load testing using JMeter - Problem using EC2 instances

JMeter can make use of several load generators to empower its load generating capability. But, it is required that all the load generators need to hook to the same network in order to communicate among them. You might experience that your started instances belong to different subnets and thus  cannot ping to each other.

To solve this issue, we can create- VPC or Virtual Private Could in the Amazon cloud. I used to have a VPC having "VPC with a single public subnet only" configuration, and it serves the purpose.

Once you use a VPC, you can start-up instances inside this VPC so that they will be assigned IP addresses in the same subnet fulfilling our requirement. Also in order to communicate these instances with the Internet, you will have to allocate some ELASTIC IPs, and associate them to the instances.