DevOps: Development Automation Testing

Our global technology client’s product development team wanted to reduce the time required to manually perform regression tests, increase test coverage performed by developers, and improve the quality of the product delivered to the testing team. With a lack of adequate regression tests performed by development, the testing team was finding an increasing number of defects, which caused delays in the delivery of features into production. The client needed to identify a suitable automation tool, framework, and test strategy.

group of people at computer

Challenge

Our global technology client’s product development team wanted to reduce the time required to manually perform regression tests, increase test coverage performed by developers, and improve the quality of the product delivered to the testing team. With a lack of adequate regression tests performed by development, the testing team was finding an increasing number of defects, which caused delays in the delivery of features into production. The client needed to identify a suitable automation tool, framework, and test strategy.

 

Experis Solution

After a brief assessment, Experis conducted a proof of concept program to validate the selected automation tool and framework. We then established the correct test methodology and created a test plan for integrating automation into the development portion of the software development lifecycle.

In this engagement, Experis delivered:

  •  A customized Ruby/Cucumber framework that satisfied our global tech client’s extensive testing needs and requirements.

           o Functional UI/Web (Java) testing: selenium-webdriver, watir-webdriver; Page Object Pattern (page-object)

           o PL/SQL API testing: ruby-plsql

           o Database (Oracle)/Data Validation testing: activerecord

           o Web Service (SOAP) testing: SoapUI integration with framework with the use of SoapUI’s testrunner

  • Seamless and efficient script development with the use of the industry leading Ruby IDE, RubyMine.
  • Selective and parameterized test execution with Rake build automation tool.
  • Framework version control for easy control and distribution of source code and files, leveraging existing TFS (Team FoundationServer) infrastructure; increased productivity by allowing multiple teams and users to contribute simultaneously
  • A continuous integration/continuous deployment model using Jenkins (CI Server) and dedicated VM test node. Supplied an interactive test dashboard using cucumber-reports plugin to monitor test activities, report on progress, find gaps in test coverage, and identify test areas that might require additional investigation.

To transition responsibility for automation scripting, framework maintenance, and following continuous integration testing principles to the client’s development team, Experis conducted a week-long training session at the conclusion of the project.

 

Results

Using Ruby/Cucumber, Experis designed a framework that allowed for front-end GUI layer web application regression testing, API testing, and backend services/data validation testing. The framework is quickly updated to address the constant changes in feature requirements and increases test coverage in each sprint. This engagement:

  • Eliminated time on task to manually perform regression tests - approximately 700 hours per release
  • Increased test coverage by emphasizing developer contribution and development quality
  • Improved the overall quality of the product, decreasing the number of defects and expediting delivery of features into production
  • Various project teams in different geographies were able to contribute to a centralized automation solution, increasing communication and collaboration
  • Various domain tests (user interface, web service, database) were reused and combined to make full end to end test cases, reducing repetitive work
  • Increased testing frequency to nightly, detecting defects as they were introduced, decreasing the cost and time on task to fix defects