Test and Continuous Quality: Enterprise Test Automation Strategy and Implementation

Our client, a leading, captive financial services company that provides financing for automotive, commercial vehicle dealers, and their retail consumers in the United States, Canada, Mexico, Brazil, and Argentina wanted to establish an enterprise automation solution to ensure ongoing excellence of their product suite. Specifically, the company wanted to implement a Behavior-Driven Development (BDD) test automation strategy and framework for its enterprise application portfolio of 150+ applications. There were no existing automation tests, and manual regression tests were growing with each subsequent sprint and project. The application development teams didn’t have enough time to perform manual tests, thus lacked adequate test coverage before the delivery of each project.

man leaning in office

Challenge

Our client, a leading, captive financial services company that provides financing for automotive, commercial vehicle dealers, and their retail consumers in the United States, Canada, Mexico, Brazil, and Argentina wanted to establish an enterprise automation solution to ensure ongoing excellence of their product suite.

Specifically, the company wanted to implement a Behavior-Driven Development (BDD) test automation strategy and framework for its enterprise application portfolio of 150+ applications. There were no existing automation tests, and manual regression tests were growing with each subsequent sprint and project. The application development teams didn’t have enough time to perform manual tests, thus lacked adequate test coverage before the delivery of each project.

Experis Solution

Experis developed and implemented a full automation solution including strategy, tool evaluation, proof of concept, and a long-term implementation plan. The solution was developed in a collaborative, innovative approach. It included:

  • Conversion of priority manual tests to a Gherkin (Given/When/Then) format to maintain a business-readable user story and serve as requirements documentation
  • Customized automation framework (Cucumber JVM) with detailed results reporting
  • Automation of 14 applications to cover the depth and breadth of application types in the application portfolio – web (AngularJS), mobile (iOS & Android), mainframe, web services (REST & SOAP), and database (DB2, Oracle, MySQL)
  • Iterative scripting, validation, and deployment of test scripts
  • Integration of the automation process with the continuous integration build (DevOps) process (Bamboo)

Technologies

  • Cucumber JVM (Java/Cucumber) tool with Gherkin
  • Selenium for web application testing; Appium and Sauce Labs for mobile application testing
  • Bamboo for building, deploying, and executing automated scripts and test cases
  • JavaScript to handle complex, out-of-the-box situations with IE Driver
  • Maven for build and project management
  • GIT for version control of automated scripts
  • Automated tests triggered from JIRA/XRAY including test results and screen shots
  • Selenium Grid with a master-slave architecture to execute tests in parallel

Results

Experis provided a cost-effective, comprehensive test automation solution that met the client’s
objectives and requirements. Results included:

  • Cost savings of 50 hours each time the regression suite is fully executed
  • Suite of 300+ automated tests across 14 applications
  • Testing across multiple environments (Dev, QA, Production)
  • Cross-browser support (IE, Firefox, Chrome)
  • Device support for desktop, tablet, and smart phone
  • Support for local and remote execution of test scripts
  • Code analysis metrics that ensure test quality management (SonarQube)
  • Data management through automation of core functionality (XML)
  • Full test results reporting in HTML with embedded screenshots for analysis
  • Standard guidelines and documentation around automation workstation set up, environment variables, framework architecture, sprint and release-level automation made available in Confluence so the entire organization has access
  • Provided training on the tools, framework, and test processes for the application development teams, plus post-engagement support
  • Conducted a “TestIT” hackathon event to promote, measure, and implement the capabilities of the solution across all teams in the organization