#1 Create a valid load (Load validity)
The first challenge is the selection of a subset
of different transactions from the set of all
transactions that can be executed by clients.
The spread of transactions, and the percentage
of each transaction in relation to each other is
of the essence here. Whenever there are backoffice
transactions that have to be taken into
account, it might be necessary to verify the
impact of their execution upon the response
time experienced by users (more on this in the
second article in this series).
#2 Create realistic users (User Realism)
The second challenge is to have the users behaving
as realistically as possible, and thus
generating as realistic a load as possible in
terms of spread of data within a single transaction
(i.e. not always looking for the same set of
www.testingexperience.com The Magazine for Professional Testers 25
data in a table).
#3 Validate the responses (Response validity)
A standard practice for testing is to test the
actual results against expected results. In performance
tests, what often happens is that the
transaction responses are timed. This relies on
the responses being of adequate quality (functionally).
This is contrary to a basic tenet of
software testing that implies that you cannot
rely on an answer to remain the same if conditions
change. When one is stressing a system,
such a reliance is unwarranted, and the
response validity is to be verified.
#4 Reduce the number of injectors 1(Injector
reduction)
Simulating a large number of clients is the
fourth challenge. Here a fine balance exists
between adding more machines and simulating
more clients per machine. The first option
has an impact in terms of hardware (adding
injectors, connecting and synchronizing them
during campaigns), while the second has an
impact in terms of load on the client (injector)
machine, as simulating a large number of
transactions and validating the responses obtained
creates a load on the client hardware.
#5 Transactions coherence (Transaction realism)
A fifth challenge comes in terms of the validity
of the stream of transactions coming from
a client. This is the case when the transactions
involve different screens linked to each other,
such as when purchasing an item from an online
store, or processing banking transactions.
This requires the server to keep data linked to
the opened session through means other than
cookies on the client side or inclusion of the
full information in the request 2.
#6 Encryption (Encryption impact)
The use of virtual private network (VPN) and
dynamic encryption methodology to ascertain
that the transactions are secure can have an impact
on the ability of capture replay to create
transactions that will be correctly interpreted
by the server and not rejected due to invalid
encryption keys (that would happen in case
of encryption based on keys synchronized between
server and client).
Extracted from testingexperience magazine "Agile testing" Sept,2009 http://www.testingexperience.com/
The first challenge is the selection of a subset
of different transactions from the set of all
transactions that can be executed by clients.
The spread of transactions, and the percentage
of each transaction in relation to each other is
of the essence here. Whenever there are backoffice
transactions that have to be taken into
account, it might be necessary to verify the
impact of their execution upon the response
time experienced by users (more on this in the
second article in this series).
#2 Create realistic users (User Realism)
The second challenge is to have the users behaving
as realistically as possible, and thus
generating as realistic a load as possible in
terms of spread of data within a single transaction
(i.e. not always looking for the same set of
www.testingexperience.com The Magazine for Professional Testers 25
data in a table).
#3 Validate the responses (Response validity)
A standard practice for testing is to test the
actual results against expected results. In performance
tests, what often happens is that the
transaction responses are timed. This relies on
the responses being of adequate quality (functionally).
This is contrary to a basic tenet of
software testing that implies that you cannot
rely on an answer to remain the same if conditions
change. When one is stressing a system,
such a reliance is unwarranted, and the
response validity is to be verified.
#4 Reduce the number of injectors 1(Injector
reduction)
Simulating a large number of clients is the
fourth challenge. Here a fine balance exists
between adding more machines and simulating
more clients per machine. The first option
has an impact in terms of hardware (adding
injectors, connecting and synchronizing them
during campaigns), while the second has an
impact in terms of load on the client (injector)
machine, as simulating a large number of
transactions and validating the responses obtained
creates a load on the client hardware.
#5 Transactions coherence (Transaction realism)
A fifth challenge comes in terms of the validity
of the stream of transactions coming from
a client. This is the case when the transactions
involve different screens linked to each other,
such as when purchasing an item from an online
store, or processing banking transactions.
This requires the server to keep data linked to
the opened session through means other than
cookies on the client side or inclusion of the
full information in the request 2.
#6 Encryption (Encryption impact)
The use of virtual private network (VPN) and
dynamic encryption methodology to ascertain
that the transactions are secure can have an impact
on the ability of capture replay to create
transactions that will be correctly interpreted
by the server and not rejected due to invalid
encryption keys (that would happen in case
of encryption based on keys synchronized between
server and client).
Extracted from testingexperience magazine "Agile testing" Sept,2009 http://www.testingexperience.com/
No comments:
Post a Comment