Embedding Mobile Compatibility Testing Into The Longer Term Development Roadmap For A Complex eCommerce Platform

A leading retailer that has a presence throughout the UK through supermarkets and via an online store. It has an increasingly mobile-focused customer base and were looking to expand their digital services to keep pace with competitors. The client had limited mobile compatibility testing experience and resource in-house.

The client had a complex eCommerce platform that allowed consumers to purchase a range of groceries and other goods.

The application prior to our testing only included a traditional desktop view of the site, restricting the functionality and usability of the site when accessed from mobile devices.

Its aim was to convert the core functionality on the web application to a responsive design with a staged rollout required over several releases.

Ten10 was engaged to perform mobile compatibility testing following these steps:

• Collaboration with DevOps engineers to understand usage.
• Identification of target devices for compatibility testing.
• Ten10 device lab to be used to avoid up-front device costs.
• Mobile user-friendly reporting with prioritised actionable recommendations.

The first step in the project was to carry out a discovery phase; this involved working with the DevOps engineers to understand the usage of the existing application in the live environment. From this, the range of target devices, operating systems and browsers were identified, then prioritised by the most used and the highest conversion rates for the client by type of device.

The second phase was the design of test sessions. Here the appropriate list of combinations for devices and browsers was planned for. The test team also worked together with the developers to ensure that the design and implementation of the application were well understood in order for tests to be accurately devised.

A structured feedback process was setup for the client, ensuring the information being provided was easily categorised and managed. Issues found during mobile compatibility testing were generally clustered against specific types of device, operating system or browser. This enabled the client to prioritise and action the application changes needed.

During the testing, Ten10 identified that a specific browser was exhibiting issues rendering form fields that resulted in the checkout functionality not working as expected. Not only was this issue reported to the client, but it also enabled Ten10 to adopt a risk-based test approach to the ongoing and future testing, prioritising this particular browser and allowing resources to be allocated most efficiently based on the design and compatibility issues which were being found.

The client was upgrading its existing eCommerce product to an AngularJS site; converting the core functionality on the web application to be responsively designed as well as integrating the up and downstream systems.

The rollout of the updated site was taking place over several releases spanning several months, with the rollout following a logical order, for example, the product pages being made responsive before the basket and checkout pages.

Some mobile compatibility testing was being performed, but it tended to be ad-hoc and carried out on the personal devices owned by the development, product and ops teams. Whilst this found some defects, structured reporting of the discovered defects was limited and not consistent.

The client was becoming aware of a number of reports from users where their new AngularJS site was not working well on some less common mobile devices. There was particular concern that more issues would come to light on pages essential for the primary revenue stream of the business.

Moreover – they had experienced issues before where an upgrade to a new version of the operating system (OS), would raise a number of reports from users that the system did not work well. Whilst limited to a small number of users, there were concerns that in the event of a problem, users could take their business elsewhere.

Using Ten10’s Digital Test Lab gave the client confidence that the necessary test coverage could be achieved without incurring additional expense to purchase the necessary devices.

As well as the testing of existing devices, Ten10 took responsibility for insights and testing recommendations relating to all new devices, operating systems and browsers coming onto the market. This provided the client with peace of mind that the right devices were continuing to be identified and tested against on a continuing basis via Ten10’s cloud-based test environment.