Issues with the Software Testing Outsourcing

Improper and ambiguous documentation is one major problem that can make the outsourced testing a difficult task. The testing team will develop their test cases by understanding the requirements (Baresi & Pezze, 2006). So the Software requirement specification (SRS) document must be precise and unambiguous. Some of the questions that should get answered by reading the SRS would be (Ramesh, 2004):

1. What are the modules (sub systems) that make up the system?
2. What are the inputs and outputs of each module?
3. What is the business logic in each module that transforms the inputs into outputs?
4. Are there any external industry standards that the software should adhere to?
5. What are the interfaces between the modules?
6. What are the external interfaces that the software should present?

These questions will help the external testers to understand the software in a better way and can plan out the strategy to test the product. The effectiveness of the SRS can be judged if the testing team is able to prepare a Traceability matrix, which is a mapping between the requirements and the test cases (Horch, 2003). Understanding the SRS is also important when the Black box testing (testing a specific functionality as an end user) and Integration testing (testing the functionality of the module as a whole) is performed. In order to perform the white box testing which is primarily to test the code, its logic, conditional paths the design document plays a crucial role. Class diagrams, Data flow diagrams, and Sequence diagrams, help the testing teams to develop automated test cases to perform the tests. In short the documentation plays a crucial role in ensuring that the outsourcing team will be able to perform a good testing.

Lack of face-to-face communication is always a serious issue outsourced jobs where the two teams (developers and testers) are sitting in different physical locations. The client company might not be able to put their point properly when communicating via emails or phones. To make the things worse, the two teams might be speaking some other languages.

One of the main reasons why American companies generally outsource their software-testing task to India is because Indian engineers are well versed in English communicating with them is easy. Documentation, especially design documents if written using UML standards can be useful in scenarios where the two teams speak different languages to explain the software design efficiently. Since documentation seams to play a crucial role to make the two teams to understand each other properly, companies have started enforcing strict policies for documentation. The companies have defined standards about the level of documentation that will be done before the software can be considered as complete (Horch, 2003).

 In case the two companies are geographically separately like in case of American client companies and Indian outsourcing companies different time zones is another problem. The twelve-hour time difference makes it difficult for the testers and developers to find a common time to interact. Also any issue if reported by Indian team in the morning can only get addressed during night when the developers in the American company are up. So the resolution of issues can get really slow and time consuming. Sometimes the network infrastructure becomes a limitation when huge data has to be transferred between the two locations. Imagine a scenario where the client company wants to transfer their sample database running into gigabytes to the testing team over a low speed network. It will take days to get the transfer done.  

References:

Baresi, L. , & Pezze, M. (2006). An introduction to software testing. Electronic Notes in Theoretical Computer Science, 148(1), 89-111.

Ramesh G. (2004). Managing global software projects. New Delhi: Tata McGraw-Hill.

Horch, J.W. (2003), Practical guide to software quality management. Norwood, MA: Artech House Computer Library.

 

Benefits of outsourcing software testing

Outsourcing has become very popular in software industry. Outsourcing is often seen as way for organizations to reduce costs and investment, while focusing on what they do well. Outsourcing decisions affect the overall project cost structure. Software testing is one such activity which as we discussed is time consuming and expensive. Outsourcing the software testing to companies in India and other third world countries is very common. There are multiple advantages to that:

1. Outsourcing Companies provide skilled team of testers

One of the reasons why companies decide to outsource software testing is to eliminate the learning curve.

Employees of specialized software testing companies are not only aware of different software testing tools, but they already know how to use them very well. This helps the client company to have some assurance for a quality product and also lower their cost of hiring and training the testing engineers. In case the company cannot hire dedicated testing engineer, it is really difficult to train the development engineers to learn new tools and processes. Tight schedules are another reason why the engineers who are involved in writing code are generally reluctant to find time and learn testing tools and processes. The developer need not have the interest in testing and may not do it properly raising concerns on the quality of the software. Also developer might not have the knowledge to decide the correct type of testing to apply which again leads to error prone testing. Companies generally will agree to the outsourcing since “anything that saves time helps keep projects on schedule is generally acceptable”.

2. Reduce Expenses with Resources

To perform quality testing lot of processes have to be automated. Automated testing techniques involve buying tools, which are quite costly. The testing companies will have the infrastructure ready to perform various kinds of automated testing. Sometimes it is required to test for multiple platforms like Windows, Linux, Solaris. Also the hardware can vary from Intel to ARM to Macintosh. It’s difficult for the client company to set up and maintain the entire testing infrastructure. On the other hand the outsourcing company will have the capability to carry out the desired testing. So the fact that client company does not have to buy them any more is major cost saver.

With the testing being outsourced, the client company can concentrate on the other core activities.

3. Better Software quality

One of the key benefits of outsourcing testing practice is better software quality supported by the fact that the outsourcing companies are fully dedicated to test the product thoroughly. They do have the technical expertise in carrying out the various testing activities, which ensure a certain level of quality. Another factor involved in software development, which makes outsourcing of testing and QA feasible is independent reporting. The testing professionals will be unlike the in-house testers, and they would objectively report the bugs right on time, also this would reduce the communication overhead sometimes observed when testing and development are carried out at the same place.  The testing is all about finding the bugs so that they can be corrected before the product is out in the market. Outsourcing the process is a win-win situation for both the client company and the outsourcing company. The client company wants to make sure that it can make the product as robust as possible. The outsourcing company wants to prove their efficiency in the testing by reporting lot of bugs. So together they both work to make high quality product.

Also the testing companies will test the software against the quality standards set by the appropriate regulatory bodies, which will help the client companies to sell the product easily. The client companies can with confidence mark their product as xyz compliance since the testing company has certified the product.

The outsourcing company will test the software without having any bias for it. Their job is to find out as many bugs as possible so they will be ruthless when it comes to testing the software. This ensures that the software has gone through in depth testing. On the other hand if the same company is testing the software, which had developed it, then the company might have a bias towards the software and might not want to report too many bugs for their own software. However, in case of offshore testing the personnel’s would test them with the consumer perspective.

4. Faster time to market

Testing is a time consuming activity. It becomes more time consuming when the engineers are not skilled with the testing techniques and tools. A company that has testing experts can perform the same task in much time than the company where the testing related experience is less. Also the experienced testers know how to find out bugs in the software. They will create test cases that can reveal the potential problems in the software. The time taken by the testers to perform all these tests and to report the bugs is much less than the time taken by inexperienced developer turned testers. Also the skilled testers can predict the schedule for the testing pretty accurately compared to unskilled ones, so the client companies can estimate properly the time for other activities involved in the product development.

 Sooner the software gets tested is better since the time to market will reduce which indirectly means earlier the client companies can sell their products and make money.

Why testing is ignored by companies?

Some of the reasons why companies feel reluctant to do quality testing are:

1. A major problem in software testing is that it often over-runs its budget and schedule. Witttaker (1999) reports that software project management fails more often because of schedule over-runs than budget over-runs. If a schedule over-run is not possible, the time reserved for testing is often reduced (Karhu, Taipale, & Smolander, 2009). Good practices recommend testing to be given equal time as the actual code development.


2.  Most of the companies don’t have the enough resources and money to perform quality testing. They cannot bear the cost of maintaining a separate test team.


3. Its not only time and money, the companies might not have the expertise to perform the testing. As discussed in the previous section, the automated testing might require some sophisticated tools, which might require special training and learning. Companies might not be ready to do such an investment.


4. Companies might not have the dedicated test environment to test for specific configurations. It involves lot of money to set up test bed for multi-platform testing


5.  Companies feel that the time involved in testing could be better use for developing more features in their product and so try to minimize the task of testing. 

References:

• Whittaker, R. (1999). What went wrong? Unsuccessful information technology projects. Information Management and Computer Security 7(1) 23–29.
• Karhu, K., Taipale, O.,&  Smolander, K. (2009). Investigating the relationship between schedules and knowledge transfer in software testing. Information and Software Technology, 51, 663–677

Test Driven Development in outsourcing environment

In case of small software Development Company which has outsourced the development to another company, the role of the first company is like a customer who is paying for the services provided by the outsourcing company. The role of Test Driven Development (TDD) is of great advantage in this scenario.

 The client company (one who outsourced the project) ideally would like to make sure that the requirements are clearly understood by the outsourcing company. Test driven development approach involves that the test should be written first before the actual code is developed. The test serves as documentation for the functionality the code is going to deliver as the test code will be testing for some of the features to be provided by the code. The outsourcing team can discuss the tests with the client to ensure that the tests cover meets the requirements.  The tests are technical in nature with no fancy English words which gives room for multiple interpretations, so it is easier for both the parties to come to the same conclusion and avoid confusions in later stages of development.

Test driven development is usually done following Agile methodology of short sprint cycles. So the outsourcing team can give the updates on the progress of the software more frequently. This also reduces the chances of getting lost away from the requirements which is an important issue in case of outsourcing. Number of passing tests can serve as a measure for the progress of the outsourcing team. Also “Pass or Fail” measurements of the tests provide a clear picture of whether the solution meets the test and hence the requirement specification, so at any given time the client has an idea of how many requirements have been met by the outsourcing company. This is a better indication of progress compared to the traditional approach of tracking the percentage of the tasks completed.

Requirements keep changing. TDD helps the outsourcing team to easily accommodate the changing requirements from the client. So the development is more flexible. The automated tests developed as part of TDD help the outsourcing team to undergo regression testing and correct the sections of code that stopped working with the changes in the requirements.

So TDD is beneficial for the client company to effectively communicate the requirements and to track the progress on the development. For the outsourcing company TDD is beneficial from the overall software development perspective.

In case of the Big company the outsourcing is not of the entire project but of certain modules. The organization is more like geographically distributed teams working for the same project. In this situation communication is a big barrier. The language disparity might make the things worse. The TDD methodology of writing the tests first before the actual code development help in making sure that the requirements are properly understood by all geographically distributed team members.  The tests act as the requirement specification document which was the case of small software company  too.

The outsourcing company is generally in different time zone to get the development happening round the clock.  So the availability of all the employees working on the project at the same time is difficult to get this makes the project management a bit difficult. But the scrum updates in the form of mails/online scrum tools can be a solution to understanding the progress on the project.  

TDD enables anybody to just run the test cases and find out what sections of the code are failing. So if a shared version control system is being used then the coordinator can “get” the tests and run them to find out the progress and potential problems in the code. The failure cases can be later discussed over phone or email.

TDD is very helpful at the time of integration. Integrating the modules that are being developed by the outsourcing team with the other modules can be easily done with TDD. TDD ensures that at the time of module completion all the sections of code are well tested and fully functional. So it is easy to put everything together.

Thus TDD seems to be a good methodology when opting for the software outsourcing.