mBrace Performance Modelling - Data Gathering

One,  Multi & Stacked User Measurements

Single User Measurements

For the purposes of definition,

•    an application is composed of a number of business cases which directly relate to specific business objectives which need to be accomplished by the application, for example, the placement of a customer order.

•    Each business case is composed of a number of transaction types which relate to specific activities which must be completed by a user to perform the business objective. Such actions may include:

•    Start the application

•    Log On

•    Navigate to Orders Page

•    New Order

•    Search Customers

•    Enter Order Details

•    Commit Order

•    Log Off

The single user measurement procedures are aimed at the collection of metrics for each transaction type, which will be used to establish the profiles of transaction types of the application under investigation. The transaction type profiles are an important input for the simulation model. The measurements collected should also include the single user transaction type response times which are important as they are needed to validate the simulation model.

With single user transaction measurements, transaction types are executed in a single serial manner (with long pauses between each transaction type execution) against an unutilised application infrastructure. This requires that the engineer collecting the measurements have exclusive access to the test environment. Whilst transactions are being executed against the test environment, various capacity elements of the application server infrastructure and the network will be utilised.

These metrics will be collected in such a manner as to establish a relationship between:

•    the transaction types executed,

•    the response times for each seperate transaction type execution,

•    the network traffic generated during the transaction type executions, and,

•    the server capacity utilisation metrics encountered with each transaction type execution.

The final aim is therefore, on a transaction type execution basis, to establish the specific infrastructure service demands incurred by the application, and the contribution that specific server capacity elements had to the overall response time for that execution. This requires that server component metrics be collected at the smallest possible time interval, namely 1 second.

Whilst it is ideal that existing test scripts (functional or non-functional, eg. Loadrunner)  be used to execute the transaction types, it is possible to effect the transaction type executions manually.

NB -A common misunderstanding exists that the single user, single transaction type executions mentioned are sufficient to understand the performance of the application under various loads. This is not the case. Whilst they do allow for an understanding of the performance impacts of specific transactions, analysis of multi-user, multi-transaction type loads requires the use of the analytical mBrace model.

The purpose of the single user, single-transaction type execution is to allow for the collection of specific metrics which are then used to populate the mBrace analytical model. Once populated with the basic transaction type data, the performance of the application can then be established under a wide variety of application transaction loads.


The Stacked Single User Measurement

Stacked single user measurements are used where the impacts, due to a specific transaction type, are so small that they become difficult to establish. The transaction types are therefore “stacked” either vertically or horizontally and executed as a transaction type group. The transaction groups are executed in a similar manner to the Single User measurement method, by executing transaction type groups in a serial manner with a long pause between group executions.

The transaction type groupings can be structured in a number of ways, depending on the specific circumstances encountered:

•    A single user can execute mutliple transaction types as a group, without pauses or with small pauses

•    Multiple users can execute single transaction types as a group, with or without pauses

•    Multiple users can execute multiple transaction types as a group, with or without pauses

During the parsing process, the total resource impact is evenly distributed amongst the number of transaction types executed, allowing for the calculated average Single User Measurement result. The response time of the Single User measurement is established using the average response time of all executed instances of the transaction type.


Multi User Measurements

The Multi User measurements deliver application platform resource utilisation and transaction type response time information, with greater user and transaction volumes.  These measurements are required to perform the second model validation process.  Validation is required in order to ensure the accuracy and reliability of the mBrace simulation modelling results.

It is usually required that a transaction coincidence matrix be completed with every measurement exercise.  This requires an identification of the execution paths of transactions in order to understand which server tiers have been active during the processing of a specific transaction.  Whilst it is expected that application developers understand their application mechanics and therefore know the transaction execution paths of specific transactions, this is not usually the case in practice.  Further, it is important to know which transactions were processed by which system interfaces, and this is also normally difficult to establish through the developers.  If the measurement plan has been well executed, and the measurement functionality allows, the coincidenc matrix can be established with reasonable accuracy following an analysis of the measurements.

Also note that the only guaranteed means by which the coincidenc matrix can be established, with absolute certainty, is through an analysis of the entire network traffic trace of the application during the execution of the transaction types.