Testing as a Service

Automation, robotics and crowdsourcing are part of the latest wave of test capability and are driving major change in the way that test services are delivered. Are you benefiting from these latest innovations and delivering maximum speed and significant cost savings in your test services?

There are growing pressures on IT departments across all market sectors to deliver with agility and flexibility, simultaneously delivering projects faster in order to improve time to market and driving cost savings.
 
Testing is a critical part of the applications lifecycle, typically accounting for a quarter of the total spend on any project. However many businesses and test automation offerings are still focused on the traditional approaches to test and not keeping up with the pace of development demands.
 
Historically outsourced test services have been provided on an offshore labour (T&M) model, exploiting the benefits of labour arbitrage through low cost locations. This approach tends to lead to labour intensive approaches (with benefiting from the availability of high numbers of skilled resources, to be engaged in the process).
 
Historically test automation tools focused on the test execution phase, scripting and executing the test, many internal test teams take this approach combined with heavy man hours to complete the rest of the lifecycle.

More recently, outsourced testing services have developed into an As a Service model. Testing as a Service (TaaS) can be priced on either a time and materials basis (as with the traditional model), but can also be based on an outcome based unit pricing for successful tests completed. TaaS models leverage the latest technologies and tools and the typical cost reduction of adopting TaaS is 30%.

Increased opportunities for speed and cost reduction are derived from leveraging recent innovations in robotics and crowdsourcing, as explained below:

Robotic Process Automation (RPA) is taking over the role of humans running test scripts using test tools.  Always available and very easy to use RPA presents good opportunities to automate far more of the lifecycle.

Robotic process automation (RPA) tools have learnt from the foundation that was created by test automation tools and taken them to the next level, allowing people with very little software development skills (and in some cases none)  to build out complex, rules driven business processes.

Versatile by design, RPA is anytime and fast; there are no limits when it comes to scheduling robots. RPA scheduling also allows automated start times so no humans are needed.  They also work as fast as it is possible to do. Robotic testing has a zero error rate; rule based robots do not make errors, they never get tired, need a rest or get bored of repetitive tasks. There are no limits to scalability when it comes to robots. Suited to design once and repeat many times testing.

RPA is cheaper and easier to use than test automation tools; there is no need for software development skills it is easily accessible by the in house team without expensive resources, it is also cheaper than typical testing tools.
However, RPA is not typically used for usability testing, mobile app and real life testing.  It is not suited to one off tasks – this is where crowdtesting comes to the fore.
 
Crowdtesting; has also successfully penetrated the test market. Using crowdsourcing, this approach provides a totally flexible source of testers used primarily for user acceptance and usability testing on a wide range of devices.

Crowdsourcing testing or ‘crowdtesting’ is the practice of sending out prototype software to large groups of flexible resources for test, typically offshore. First leveraged by Wikipedia to fill their content, crowdsourcing is now becoming a major source of advantage to the test lifecycle.

Crowdtesting is being used primarily for functional/user acceptance, localisation, usability and security testing. It provides the opportunity for testing under real world conditions; testing software, websites or mobile apps on a wide range of devices becomes possible with crowdsourcing as they BYOD. There is also no limitation on testing any language, anywhere.

It has the benefit of always on resource; there are no 9-5 hours or issues with vacations and public holidays. Time to wait for resource can be reduced from weeks/months to hours/days and flexed up and down as required (crowdtesting is estimated to be 10 times faster than traditional user testing and delivers results in real time).  For example 90 man days can be reduced to a 72 hour duration. Often testing is broken down into micro tasks to avoid issues of leakage of IPR or confidential data.

Bug Bounties (vulnerability rewards programmes) are using this approach to enhance penetration and vulnerability testing.  Apple, Google and Microsoft are some of the biggest names taking advantage of this approach.

Crowdtesting is not typically used for all other aspects of testing and requires good process and clarity as to requirements; you have to be very definite about your instructions to these testers.

Whilst RPA and crowdtesting can be adopted by internal client test functions, the knowhow and maturity of process required to benefit from these options generally exists in the supplier base, or very large technology based companies such as Google and Wikipedia. As a result, clients are often choosing to contract for a TaaS arrangement with a service provider, with the following benefits:

• Guaranteed savings can be baked in to the contractual arrangement.
   
• The scope and scale is provided by the TaaS suppliers mixing and matching the sources of automation and crowdsourcing, to deliver the solution best suited for the client test scenario.
   
• Investment capability – most client organisations find that one project does not want to pay to automate testing and benefit the next project down the line. Suppliers are prepared to invest in automation, in the context of an outsourcing contract.
   
• The ability to purchase the automation services and aggregation into a complete testing solution, is complex and requires high levels of process maturity and vendor management capability.
   
• Crowdtesting is currently an emerging market, using TaaS in a properly constructed commercial arrangement, de-risks the “bleeding edge” for the client organisation.
   
• Accessing economies of scale – TaaS service providers will offer fixed price or consumption based pricing – de-risking the financial exposure for the client.

Is your testing service fit for today’s changing environment?

If you are reflecting on whether your testing delivery capability is able to leverage the opportunities and innovations in today’s IT world, why not take some time to talk to us and understand how we have helped other organisations to achieve significant savings through effectively implementing a TaaS solution.
enquiries@quantumplus.co.uk