Testing Experience Archive


Insights to Software Testing Career

New to Software Testing? Been in the Software Testing career for a while? No matter how much or little experiences you have, you will always find different challenges regardless of which industry you are in, or the size of your team.  I would like to share with you some things I have learned throughout the years and hopefully this will give practical perspective of software testing.

Software Testing is very much similar to research and development where planning and analysis must take place.  There are three main factors that contribute to successful testing

  1. Testing scope must be clearly defined;
  2. The knowledge of the system under test;
  3. Interpersonal skills

It is very important to know ‘what’ first before we can start thinking about ‘how’.  When I said ‘what’ doesn’t mean knowing everything up front but it is something that every team member is aware the work to be done in the next days, weeks or months.  By knowing where to start we can prioritize deliverable tasks to a point where the system ready for acceptance and released to production.

Effective testing is not about executing a large number of test cases but it all about finding as many defects in the shortest time possible.  Every tester should have a good knowledge of the system functionality.  The learning process usually starts from spending time with the Subject Matter Expert then followed by reading software specifications.  Learning phase should be taken into considerations when estimating the effort to complete testing phase.

Last but not the least testers should work closely with every team members from business users, Business analyst and development team.  Good interpersonal and communication skills will surely help working in an environment where we need to liaise with people from different background and expertise.  The right answer comes from asking the right question to the right person.

In summary, career in Software Testing can be challenging yet rewarding.  Regardless the size of your organization or project, you will need to focus on these areas and everything else will fall into place.  You must first know what you need to do, learn what you need to do and communicate with your team members what you do.  I hope you find this article is useful and all the best with your career.


Software Testing Approach Only Hours Instead of Days Available

Imagine that you only have hours left instead of days to complete testing.  Does this sound familiar?  No matter how well we have planned for the resources, estimating testing efforts, prepared the test scripts and scenarios, it would be useless if we don’t have the time to execute or a testable system.  There are so many factors can contribute to the delay but how do you approach the situation so that critical functionality is tested?

Before we start discussing what can be done in this situation, I would like to emphasize that software testing is a process of finding defects to ensure that the system under test meets the requirements.  Thorough planning test efforts and monitoring test progress should not be taken lightly.  The time component in software testing is generally divided into two:

  1. Test to validate the requirements.  This is where you execute test cases and scenarios you have prepared during planning/development stage.  It is very common that test cases get updated as we me make progress especially working in Agile project methodology where changes are allowed (within reason) as end users see fit.  New test cases might be created as the result of better understanding of the system.
  2. Finding defects.  Once High and Medium priority test cases have been executed, more complicated test scenarios can be developed such as varying the inputs or negative scenarios that might occur on regular basis.

This seems to be a good plan if you have enough time to complete.  However, different approach should be taken when there is only a fraction of time remains from initial plan.  User Acceptance Testing (UAT) checklist would be the best place to start.  This will ensure that the main functionality are working.  Exploratory testing should also be considered to detect any obvious defects.  Any administration work such as updating test cases and recording test results can be postponed after UAT.  it is also important to make business users aware any limitations or defects that might arise and provide solutions and steps to work around until issues have been resolved.

In summary, going through UAT checklist as a starting point is not recommended and should be avoided.  It is important to have UAT checklist done as soon as possible to give us a practical perspective of which areas to focus.


How do you estimate testing efforts?

As a software tester one of the most difficult questions I get asked is how long does it take to test something?  My mind goes 1) will the system ready for testing 2) will there be any critical defects in the first cycle 3) has development team delivered complete (or near complete) functionality as per specification .. etc the list goes on.  To most people testing is a task to complete but to me it is all about finding as many defects in the shortest time possible.

Whether it is for Agile or Waterfall methodology testing work becomes easier when every improvement, fixed defects and other updates are testable.  There are cases where changes might not be testable such as changes to non system related functionality such as back end improvements, system architecture or best practice.  Acceptance criteria could be checking the functionality from user interface, records in database or log files.  Having something testable is far more important than planning itself or how much time has been allocated.

The question we often get asked is how much time is required to test given changes or system upgrades?  To me this is an open-ended question because unlike development or analysis work, testing is to ensure what the system is supposed to do as well as negative testing.  In addition, new test cases might be added to find new defects.  In other words, testing is more towards time management to find relevant defects in the shortest time.  It shouldnt be focussing on the number of cycles or how many test cases have been executed.  The following guidelines can be used to find more defects in less time:

  • Business requirement or acceptance criteria should be up to date;
  • Test case should be derived from each requirement not based on other test cases;
  • Each requirement should be given priority to manage testing time effectively.

Finally but not the least we can estimate the testing efforts using one or more of the following steps:

  • Identify which test cases to execute based on what requirements are in scope;
  • Group these test cases into two or three different levels such as Easy, Medium and Hard;
  • Estimate how long will it take to do the easy ones and twice as long for Medium and three times for Hard ones;
  • Allow 50% – 100% factors for administration work and verification for unexpected outcomes;
  • Review this estimate at 25% progress and see if the estimate still stands;
  • Once the full cycle is compelete we can use this as a reference for the next release but usually shorter as people are more familiar.

In summary,  testing should not be a measure of efforts or time but it is about time management to find the maximum number of potential defects in the shortest time possible.  It is important to have up to date business requirements with priority so that testing can be done effectively.  Although test estimate is crucial for planning activities how we spend the time and in which order is far more important than completing the testing itself.  In my years of experience time has always been the limiting factor and how much testing is ever enough?

If you are interested in receiving a FREE test estimate template please email us to admin@requirementdriventesting.com.


Lessons learned working in SCRUM methodology

I have been involved in many ‘Agile” projects with different organizations and amazingly each project was different from another in term of management style, how each team member ‘collaborate’ with each other and documentation style. If you are about to start a new SCRUM team or already implementing SCRUM this article might give you some ideas of potential issues to address sooner than later. Refer to Wikipedia on SCRUM for more details but in general SCRUM shares similar characteristics with Agile methodology emphasizing on the following:

  • Working software is valued more than documentation
  • Team collaboration is more important over contract negotiation
  • Responding to change is more important than following a plan

The first week of the first first sprint was to work out high level requirements such as system architecture and business needs with business owner, operators and/or subject matter expert (SME). We managed to produce product backlog and sprint backlog i.e. project scope by the end of the week. We also covered team responsibilities and expectations from each other as well as operational activities such as daily stand up meeting, user story format and acceptance criteria.

At the end of the first sprint (4th week of 4 week cycle) some things did not turn the way as planned and we had to push some of the user stories for the next sprint due to changes on requirements i.e. scope creep. However, this wasn’t a real issue because we work closely with business owners and they understand the situation. Besides SCRUM / Agile methodology is about responding to changes as it is required.

The following table summarizes some key issues that could be prevented:

Issues Solutions
Developers experienced difficulty trying to complete

tasks with changing functional requirements.

Scrum master will liaise with developers and

better manage the change request from users.

Testers did not get incremental builds in Test

environment because of user story interdependency

from one to another.

Will aim for smaller and shorter user story so

that Testers will get weekly build and also

clearly defined the exit criteria/acceptance

for each user story.

Developers need better functional specification

before they can start coding to avoid changes

and loss of time.

Project Manager suggested that business users

together with Business Analyst should have

daily catch up with developer.

The aim for delivery and time estimate were

very optimistic and should include time for

meeting, research and ‘collaboration’

into sprint backlog.

Scrum master will reduce the ‘optimal’ working

hours down from 6 to 5 hours per day and try

to aim less user stories (at least for the first sprint)

In summary, it is important to keep business owner informed on the progress on weekly or even daily basis depending on the situation. Any impediments should be prioritized accordingly to minimize delay, hence reduce project risks and costs.


Requirement Driven Testing for Agile vs Waterfall process

I have worked on many projects for different organizations and found there are two main streams of project methodology: 1) waterfall and 2) agile process.  Other methodologies such as SCRUM share similar characteristics from either one of them.  People often asked “How different is the test approach between Waterfall and Agile methodology?”  Before we discuss the different lets quickly go through the characteristics for each methodology and here is the summary:

Waterfall methodology is a classically linear and sequential approach through the phases of requirements analysis, design, implementation, testing (validation), integration, and maintenance.  This method focuses on planning, time schedules, target dates, budgets and implementation of an entire system at one time (source: wikipedia.org).

Agile methodology on the other hand is an incremental software development emphasize on the following aspects:

  • Working software over documentations
  • Team and customer collaboration over contract negotiation i.e. Service Level Agreement (SLA)
  • Quick response to change i.e business requirements.. etc refer to wikipedia.org for more information

The above summary shows different values from each methodology.  How is this going to impact the test approach?  My short answer to this would be the same test approach can be used for Waterfall and Agile methodologies.  Regardless what method is used test cases should be derived from 1) business requirements (functional and non-functional) but can also 2) derived from business scenario (business process).  One of the main differences is the planning for each iteration is a lot smaller in Agile methodology compared to Waterfall.  For example, it might take ten iterations to complete a complete system in Agile methodology compared to two or three iterations in Waterfall methodology.

One of most common issues I have while working in Agile project is all testing work are left right the end of iteration which I think defeat the purpose of the “incremental software development” concept.  Just like development or business analysis work, testing is no exception requires incremental testable items to ensure the product being developed meet customer expectation.  Regardless which method is used testing should start as early as possible i.e. static testing is to validate the requirements or system specification.

In summary, one of the most important lessons I have learned is to always think of the exit criteria with testablable conditions for each development task.  This will prevent a situation where testing is left to the end of iteration and delay the discovery of defects.


How do you measure a successful SCRUM?

I have worked in many AGILE/SCRUM projects and today we completed the first sprint.  Like many projects the first few weeks were usually spent on analyzing system architecture, infrastructure and gathering high level business requirements.  It is also quite normal slower progress in the first couple of sprints because of continuous changes in discovery phase but things will improve as soon as basic functionality and business needs have been identified.

We had a sprint review meeting and one of the interesting question was from business users asking how to measure a successful sprint?  This is quite a common reaction in the early development stage especially for someone new to Agile methodology. Scrum Master on the other hand sees this from different perspective and always consider any progress, visible or not, as a good outcome and closer to finish product. 

It is very important to set the expectation right from the beginning by informing business user(s) what to expect in the first couple of sprints. This will normally be focusing on the preliminary work such as system architecture, database, business layer and user interface prototype which provide the overall ground of the whole system.  The success of SCRUM is measured rapid delivery of ‘working’ software.

In summary, delivering some of the functionality is still considered a progress and adding values and a step closer to complete product.  It is also important to have something testable so that software or system owner(s) can see the progress and gain confidence on the overal project.  A checklist of how many tasks have been completed by the end of sprint can be used as  a form of signoff.

Related Posts Plugin for WordPress, Blogger...