Agile Performance Testing the mBrace way (1)
As I described in the first article of this series (Rethinking Performance Testing - Why do it?) performance testing is done to reduce the risk of poor performance. It is a risk mitigation process. The risk can be split in three areas: 1) code with insufficient performance potential; 2) inadequate capacity; 3) defects in the stack of hardware, software and parameter settings of the application’s production environment. Through performance testing and subsequent optimisation we may reduce these risks as close to zero as possible. Most of the damage from poor application performance originates from postponing cutover, which hurts the time to value.
Let’s have a closer look at how performance testing can reduce such risk in an Agile setting, starting at the first risk area,1) performance potential of the code. Here you have a choice between load & stress testing and a model based approach like mBrace.
Load & stress testing
In a nice article Rebecca Clinard describes an example (Shift left Performance Testing . . .), where the software developers create the script for each transaction they produce. The script is run in performance tests during the night in the build procedure controlled by Jenkins.
https://www.linkedin.com/pulse/shift-left-performance-testing-all-aboard-rebecca-clinard
For scripting I calculate 1.5 hours per transaction and I consider that a waste of time. Determining the time behaviour of a transaction takes a load test of at least 15 minutes. That’s probably why it is done in the nightly build procedure I guess. Though this provides much improvement to waterfall testing, it adds considerably to the complexity of the software development process.
mBrace Agile
A software developer can do the entire test of a transaction immediately after its completion without any scripting within 3 minutes.
On average 80 % of transactions meet the performance norm leaving 20% of them to be optimized. Load & stress testing provides feedback the next morning with hardly any application intelligence to show how. With mBrace it takes three more minutes to apply the drill-down functions that help find out how to optimize the code. This allows developers to experiment with the transaction being built and optimise them right the first time without disrupting their productivity.
When the transaction is ready it has optimised performance potential that meets the requirements. Applications are built transaction by transaction, sprint by sprint. The code is delivered with guaranteed performance. When all transactions are done, the risk mitigation process is ready to deal with the next risk area: capacity.
To get an idea of the look and feel of the mBrace Agile Performance Test Tool, there are three short videos at:
https://www.mbrace.nl/en/documentation/video-presentation/