mBrace Performance Modelling - Data Gathering

Measure Single and Multi Users

The measurement scenarios are always initiated by establishing the time differences between the clocks of the various server platforms and the load generation PCs. The Network Time Protocol (NTP) is usually used to synchronise all servers and PCs in the environment where the measurements are collected. This is not always practically possible.

The engineer controlling the LoadRunner testing environment is responsible for establishing any time differences which are encountered in practice. Any time differences encountered should be determined with reference to the load generation PCs and these differences should be identified in terms of a degree of accuracy of 1 second. The parsing process can usually accomodate degrees of accuracy of 2 seconds, but greater deviations usually lead to a dramatic increase in the amount of time required to complete the parsing process. It goes without saying, that if the load generator clocks are out of synch, the best practice would be to ensure that they are synchronised with the rest of the environment.

Single User Measurements

These measurements should be collected under exclusive access conditions, namely that no other application activity should be taking place on the servers whilst the transaction types are being executed. If the application under test uses batch processing transactions, these should be scheduled to occur at intervals which do not coincide with the single user measurement activities. If possible, all batch activities on the application under test should be temporarily halted whilst the single user meeasurements are taking place. 

In clustered environments, it is recommended that only a single server node be active during transaction type execution and measurement activities.

Whilst conducting the single user measurements, the collection engineer should ensure (if possible) that the transaction types required for a test or business case, be stipulated in a single script, in the order required for the successful execution of the entire business case. The script is to be executed six (6) times in a repetitive manner, with ten (10) second user sleep time intervals between script iterations. Transaction types within a script should be seperated by fixed user sleep times of five (5) seconds.

The 5 and 10 second sleep time spacings are required to facilitate the parsing process, in that transaction activites can be clearly identified and relationships established between transaction execution and the resource utilisation measurements obtained from the various application server platforms. It is recommended that inter-transaction sleep times be limited to a maximum of 5 seconds in order to ensure that total measurement durations and data sizes remain as small as possible Note that extensive and detailed measurements are gathered at 1 second intervals during the measurement collection process.

Stacked single user measurements are similar to single user measurements, with the exception that instead of executing single transaction types, transaction type groups are executed. Each transaction type group is composed of 10 transactions of the same transaction type which are executed in a serial manner, one after the other with no user sleep times.

Multi User Measurements

In a similar manner to single user measurements, multi user measurements require exclusive access to the test environment whilst execution is taking place.

In clustered environments, it is recommended that all cluster server nodes be made active. Care should be taken to ensure that load balancing components are functioning correctly in order to ensure that the application cluster nodes experience balanced loads. This is not usually the case in practice.

The multi user measurement scenario provides for a gradual stepped increase in transaction volumes and may contain a number of load “steps”.

The steps may be composed as follows:
-    Step 1 –  5 users executing transaction scripts for 10 minutes.
-    Step 2 – 10 users executing transaction scripts for 10 minutes.
-    Step 3 – 15 users executing transaction scripts for 10 minutes.
-    Step 4 – 20 users executing transaction scripts for 10 minutes.

The stepped process can increase after every 10 minute period. The 10 minute period is required to ensure reasonably stable transaction response times and server resource utilisation metrics. The transaction response times and server resource utilisation metrics for each 10 minute “step” is used to validate the mBrace model at load.

Multi-user measurements are collected at 1 minute intervals and the focus is on resource utilisation. Scripts used for Multi-User measurements should be identical to scripts used for single user measurements.

If possible, at least two multi-user measurements should be taken. The first should include all scripted transactions in equal transaction rate proportions and the second should run transactions in the transaction mix expected to be encountered in production. This will enable the analyst to determine the relative impact of transactions on each other.

Post-Measurement Activities

Following the conclusion of the measurement exercise, all collected data and data files should be delivered as soon as possible to the mBrace Parsing Engineer.

Following delivery, measurement collectors are requested not to delete measurement data located on the respective application server platforms unless specifically requested by the mBrace Performance Analyst.