Recommended Reading Archive

1

Introduction to Requirement Driven Testing (RDT)

Software testing is a process of finding defects to ensure that the system being developed meet the requirements. In most cases it is likely that some of the requirements are changed or become obsolete. Therefore it is very important to always work from requirement to select which test to execute. Quite often not all test cases are up to date. Working from requirement will ensure that the they are reviewed regularly. Existing requirement and test cases might be updated, removed or added during each review. Requirement Driven Testing (RDT) is project methodology independent test approach that can be used in any project methodologies such as the Waterfall, Agile and SCRUM and focuses on the following:

  1. Building business requirement list;
  2. Requirement is used to select test case(s) and;
  3. Report pass/fail on business requirements.

The following diagram (click to enlarge image) illustrates testing and development activities are based on predefined requirements which encourage team member to review the requirement list on regular basis (even though no defect is found) to ensure maximum number of defect can be detected before releasing to production.

Building Requirement List

As the name suggests, RDT work is around building and refining requirements. The process continues throughout the project life cycle involving everyone in the team especially project owner, subject matter experts and/or application vendors.

RDT’s success is based on the fact that without well defined and testable requirements, test executions become subjective and are based on unsubstantiated assumptions

Having an up to date and testable list of requirements will make testing process much easier and productive. It is important to keep reviewing existing requirement especially during the following activities:

  1. Executing tests
  2. Creating defects
  3. Regression testing
  4. Reviewing business requirement documentation
  5. Retesting defects

Requirement is used to select test case(s) to execute

In most cases, the number of requirements will grow over time but it does not mean that the time required for testing will increase the same way. Priority classifications (High, Medium, Low) will be used to select what to test first and to manage time effectively. The priority classification also changes throughout the project life cycle as what is currently critical might not be so the same in the following iteration(s). The classification can also change when defects are detected during testing.

Although it is not mandatory, in general RDT follows the following guidelines:

  1. Each scenario should have at least one business requirement
  2. Each business requirement should have at least one test case
  3. A test case can be derived from business scenario
  4. Each defect should have one requirement and test case
  5. Each business requirement should have priority classification
  6. Each defect should have severity classification
  7. A separate test case should be created for each scenario

Report on Business Requirements

Closely coupled with the RDT approach is reporting on requirements. It focuses on providing up-to-date test results, test progress, test statistics in requirement term. The test reports can be as simple as manual update excel worksheet to a real-time online reports.

Imagine a business owner want to know the progress of the software being developed, it makes more sense if there are 10 out of 100 requirements still not tested than 10 out of 100 test cases not tested. In other words, RDT emphasis on the “what” factor rather than the “how” factor.

The quality of reports depends on the quality of inputs that have been provided. It is very important to plan what information is to be included to produce different reports. Some of the useful reports for monitoring and managing progress are:

  1. How many requirements without test case
  2. How many test cases without requirement
  3. How mane defects created without requirement and/or test case
  4. How many scenarios without requirement

Summary

RDT is a test approach that focuses on building requirement list, use the list to prioritize test execution to manage time efficiently and report on requirement status. RDT is project methodology independent and can be used in Waterfall, Agile or SCRUM project methodologies.

Next topic: Each scenario should have at least one business requirement

If you have any question, feel free to send email to admin@requirementdriventesting.com.

7

Each sofware defect should have severity classification

One of the most important criteria working as a Software QA is the ability to manage defects (bug in software) to ensure critical issues are fixed in the right order.  Software testing and prioritizing defects come hand in hand and if it is done correctly can reduce development and testing efforts significantly. 

To monitor defects efficiently with minimum effort, each defect should have severity classification and usually rated High, Medium or Low.  Rating severity can be subjective and should involve business owner(s) and/or project owner(s).  Defect severity is not the same as priority classification for requirements and is not required for managing defect.  Having both severity and priority will complicate the defect management process.

The following guidelines can be used when assigning severity to software defects:

  • High – System crashes, causes non-recoverable conditions, loss of key component functionality, database or file corruption and potential data loss are all examples of a High severity defect.  A new build with fixed high severity defects is required.  Testing cannot continue while a High severity defect exists.
  • Medium – Major system component unusable due to failure or incorrect functionality. Medium severity defects cause serious problems such as a lack of functionality that can have a major impact to the user. Medium severity may prevent other areas or components of the system from being tested.  A work round may exist, but the work around is inconvenient or difficult.
  • Low – Incorrect or incomplete functionality of a component or process.  Low business impact if this is not delivered or implemented.  This is usually cosmetic issues raised during User Acceptance Testing (UAT).

Ability to manage defect effectively is one of the most important skills to have in computer careers especially if you are a Software Tester.  So, take advantage of this information now and give another review on the existing defects.  Remember to involve key stakeholders such as business users and business owners to agree on the severity rating for each defect.

0

Each business requirement should have priority classification

It is generally true that there are more test cases to run but there is not enough time.  Imagine that you have the time to do 3 out of 10 test cases how would you choose or know which ones are more important that others?  How do you apply risk based assessment and choose which ones to test first?  Here is a list of possible answers:

  1. Choose high priority business requirements
  2. Choose high priority test cases
  3. Perform exploratory testing by someone who is familiar with the system
  4. Retest high severity and/or high priority software defects that have been fixed
  5. Combination of any or all the above

Each of the above answers is applicable but it is also important to have a direction and a quick way to identify all relevant requirements to test.  In Requirement Driven Testing it is recommended to have priority classification for each business requirement since each test case is derived from the requirement.  This way testers can quickly produce a list of high priority requirements to test.

In summary, business requirement and its classification require ongoing review to ensure that they are up to date and to reduce testing requirement to focus on high priority tasks.  It is recommended that stakeholders especially business users to review the existing requirements at the end of each development cycle.

0

A separate test case should be created for each scenario

A scenario consists of a set of actions or a sequence of events to be completed to achieve a specific outcome.  For example in order to start a car 1) a driver uses a key to start the engine 2) release the hand brake and 3a) shift to Drive gear for automatic or 3b) shift to 1st gear for manual car.

In this example the first 2 steps are the same but the 3rd step is slightly different depending on the type of car.  This also shows dependency that certain condition must be met before the next action can be done i.e. start the engine first before releasing hand brake or shift the gear.  From this example there are two scenarios to be tested
1) For people who drive manual car
2) For people who drive automatic car

Similarly, scenarios in software development can be regarded as normal day-to-day business activities.  Each scenario can be further extended to the following areas:
1) state transition – what scenarios will be handled differently by the system?
2) decision – different outcome(s) based on different input(s);
3) dependency – what condition(s) to be met before the following steps can be executed.

It is recommended that a separate test case should be created for each scenario to provide clear test objective.  Each test case should have the following information:
1) the test objective i.e. what business requirements are to be verified
2) pre condition (if any) for the scenario
3) steps or procedures to execute the test
4) Expected result

If you have any questions, feel free to send email to admin@requirementdriventesting.com.

0

Each defect should have one requirement and test case

A software defect by definition is a flaw in the system that can cause the system to fail to perform its required function.  In Requirement Driven Testing requirement is first selected to short list which test cases to execute.  It only makes sense that for something to be called a defect should have been tested against a requirement using test case(s) derived from the same requirement.  Hence, every defect should be linked to at least one requirement together with the test case that proving the test to fail.

In reality many functional defects are being reported without matching functional requirements.  What is the best way to deal with this situation?  The following are possible reactions from the development team (including testers):

  • Fix the defect
  • Create change request
  • Defer the defect
  • Reject the defect
  • Ignore and wait for escalation from business owner or users

Each of the above actions could be a correct outcome when an appropriate procedure is followed.  Different projects in different organizations have different approach but Requirement Driven Testing test approach uses the following guidelines:

  1. Prove that the defect can be reproduced
  2. If yes, check if there is an existing requirement.  Else add a new business requirement and review and request additional time if needed.
  3. If yes, check if there is an existing test case.  Else add a new test case.
  4. If yes, create a defect.
  5. Note that in Requirement Driven Testing each defect is linked to requirement and test case

The following diagram shows the workflow of defect management lifecycle.

Although it is not mandatory to maintain each defect with requirement and test case as one package, this method will surely give developer, business owner and other stakeholders clarity and help everyone in the team understand the ‘what’ and ‘why’ factors of the issue.

Next topic: A separate test should be created for each scenario.

2

Test case can be derived from scenario (and business requirement)

Previous article talks about test case is a way of proving that certain conditions or requirements have been met.  Test case are also created to prove that the software can handle certain business scenarios.  You can think a scenario consists of a set of business requirements put together to generate a different outcome.  It is recommended that each scenario should have a separate test case to avoid confusion and better way of analyzing business risk(s).

In Requirement Driven Testing, the following guidelines can be used when creating test cases for scenarios:

  1. Who performs day-to-day business activities and how they interact with the system?
  2. Restrictions on business activities;
  3. Changes on the existing business requirements;
  4. Changes on the existing technology or infrastructure .. and so on.

Writing test cases is a good opportunity to review the existing business requirements and scenarios.  After all software testing is about using your imagination and creativity how users interact with the system being developed.

Next Topic : Each defect must have at least one requirement and test case

If you have any question, feel free to send email to admin@requirementdriventesting.com.

0

Test Case should be derived from requirement

Every system will get upgraded occasionally for different reasons such as change of policy, technology or product just to name a few. Requirement should be used as a baseline for testing to ensure that the existing and new functionality are working.  Test case is derived for each requirement as a way of proving that the system or changes being implemented is ready for day to day business use without error and/or adverse impact on the existing functionality.

Requirement itself changes from time to time and some become obsolete and to be removed together with the test case(s).  Although the main activity of being tester is to execute test, maintaining requirement list is crucial to be able to find real defects in the shortest time.  Associating every test case with requirement, as a source, is a good way to manage your time efficiently.  I have seen examples where test cases were created without requirement can lead to hours, days and weeks of testing efforts without clear objective.

There is no hard and fast rule how to derive test case but I find it easier to start thinking from test types.  For example,

Requirement: User must have a valid username and password to login.

Question to ask yourself when developing test case would be:

  1. what ‘functional’ capabilities I need to test?
  2. what ‘non-functional’ features to be included?
  3. what ‘security’ constraint?
  4. what ‘negative’ tests should be considered i.e. invalid username and/or password .. etc.

As you can see it is very easy to come up with at least two test cases, one being positive and one being negative test, from one requirement.  Note that test type is context specific which means specific type(s) are more suitable for specific application. I would normally finish first round of static test with two aspects without taking too much time and move to the next requirement.  As you understand more how some requirements are linked together scenario based test case can be developed (how to derive test case from scenario is out of scope in this topic).

Summary

Test case should be derived from requirement and if there isn’t one you can verify whether or not new requirement(s) need to be added regardless if it is functional or non-functional.  Maintaining requirement list is important so that your testing efforts have clear objective(s).  Lastly, try thinking different types of testing to create different test cases this way you 1) see the situation from different perspective 2) have a good understanding of the system when several requirements are put together.

0

Each scenario should have at least one business requirement

The objective of this article is to explain that every business scenario should have at least one business requirement.  Business scenario is a hypothetical story that helps us to think the system behavior for a given condition or set of conditions. The following examples show different ways of looking business scenarios:

  • A scenario can be considered as a process flow from point A to point B.
  • Every alternative path can be consider as another scenario.
  • Variables that can produce different results can be considered as different scenarios.
  • Negative test on how the system responds when invalid inputs or sequence is applied.

How do we select which requirement(s) for a scenario?

In Requirement Driven Testing, each scenario should have at least one business requirement.  There is no hard and fast rule on how to link scenario and requirement together.  You can associate any requirements with one or more scenarios and vice versa.  Maintaining scenario-requirement relationship is an ongoing process.  Quite often the existing scenarios and/or business rules are changed, moved or removed during the Software Development Life Cycle (SDLC).   It is recommended to review all requirements when business scenario changes to maintain the quality of test cases for each requirement.

You can ask yourself the following questions when assigning requirements to scenario(s):

  1. Which scenario(s) would this requirement applies to?
  2. If this requirement is changed or removed which scenarios will be affected?
  3. If this requirement is changed or removed which other requirement(s) will be affected which are linked to other scenario(s)

In most cases it will be one-to-many relationship, that is one requirement might be applicable to more than one business scenarios.

Lets do one example

Scenario 1: Actor logs in using a valid username and password

Scenario 2: Actor perform search using item description

Requirement 1: only authorized user can use the system

Requirement 2: search result page should only return the first 50 records

In this example it is clear that Requirement 1 is applicable for Scenario 1 an Scenario 2 where Requirement 2 is only applicable to Scenario 2.

Next Topic : Each business requirement should have at least one test case

If you have any question, feel free to send email to admin@requirementdriventesting.com.

Related Posts Plugin for WordPress, Blogger...