Non Functional Testing Requirements

The Process

It is recognised that Non Functional Testing is a highly specialised area of expertise; to be able to clearly identify requirements for a project requires specialised skills.  There have been many occasions where a project manager has not identified all the costs and complexities associated with conducting Non Functional Testing.  In order to help project managers clearly identify their project’s Non Functional Testing requirements and to have clear perspective on the Non Functional Testing Group's deliverables, a standardised test strategy that centralises Non Functional Testing expertise is often a good approach.

 

The process consists of four separate phases of activity:

•    Requirements Gathering

•    Resourcing

•    Test Strategy Development

•    Test Development, Execution and Reporting Stages


Requirements Gathering

Senior Non Functional experts gather information from project documentation as well as a series of meetings.  These could include:

•    A high level meeting with the project manager to qualify his requirements

•    A series of meetings with the Project Group’s Technical Architects

•    A series of meetings with DBA’s and Business Analysts

•    Review of design documents and system testing test plans


This is followed by a proof of concept using an appropriate automated Non Functional Testing tool.  From the information gathered, from both the meetings and the proof of concept, a Non Functional test strategy would be produced and presented to the project group.  This Non Functional test strategy ideally includes clearly defined deliverables, timescales, resourcing needs and test environment requirements.  This is a robust method for engaging with a Non Functional test requirement but is demanding on both time and resources.


A more streamlined approach for gather the information whilst maintaining the integrity of the requirements gathering is:

Step 1:    For each project the project manager is required to complete a “Requirements Questionnaire”.  This will clarify, qualify and quantify any requirements for Non Functional Testing


Step 2:    The Non Functional experts review the completed “Requirements Questionnaire” and where applicable, confirm that a requirement for Non Functional Testing exists


Step 3:    A second “Technical Questionnaire” is passed to the project’s Technical Architect for completion. This questionnaire will be used to gather and consolidate the technical requirements for Non Functional Testing


Step 4:    A Proof Of Concept is carried out using an appropriate Non Functional Testing tool


Step 5:    A test strategy is then produced based on the information contained in both questionnaires


Step 6:    The Non Functional Test Strategy is delivered to the Project Manager for review and refinement as required.  The Non Functional Test Strategy will provide the Project Manager and Non Functional Testers with definitive objectives and deliverables.  The Non Functional Test Strategy should include the following:

•    A  volumetric model against which test scenarios are developed and Non Functional results are measured

•    The requirements for the Non Functional test lab to produce test scripts, inject the load and monitor servers under test

•    A breakdown of in-scope activities

•    A breakdown of out-of-scope activities

•    Details of the scripts and load scenarios

•    Details of required user access information for monitoring sub-system components such as web, application and database servers

•    Details of the data required for each Virtual User such as user ids and passwords

•    Details on any application data that is required to be built prior to or during any testing activities

•    Details of responsibilities for each group involved in the Non Functional Testing effort

•    Details of  resource, timelines, costs and deliverables

•    A schedule of test execution including testing and tuning cycles