Performance Testing Custom Code In Microsoft Dynamics CRM Online

The housing association was in the process of upgrading its existing CRM system and replacing it with Microsoft Dynamics CRM Online. The CRM system manages the association’s customers, properties and tenancies. It allows the handling of cases raised by customers such as enquiries, complaints, or other issues relating to the properties.

Ten10 was tasked with discovering and reporting on
the reasons behind the inconsistently poor
performance of its custom CRM code in a variety of
different end-user circumstances. We successfully
tracked down the root causes of the performance
issues and achieved resolution for the client whilst
building a good relationship

The previous version of Dynamics CRM had demonstrated poor responsiveness across test environments, with the same number of nodes as the system that would be released to production. For specific transactions, such as saving a case, response times were observed to reach values of up to a minute, yet this was not consistent and response times varied throughout the day.

Ten10 proposed to carry out systematic performance testing across the association’s various locations. This would allow a sample of measurements to be acquired to identify the performance behaviour of the latest releases, whilst diagnosing and supporting the resolution of the observed performance issues.

We identified three key goals for the project. The first was to design and implement a solution for acquiring consistent performance metrics for the transactions of critical use-cases for Dynamics CRM. We also needed to understand the performance behaviour of Dynamics CRM Online when accessed by the association’s users from within the intranet as well as via a Citrix virtualised environment from a selected number of key locations. Finally, we needed to analyse the results, diagnose potential performance issues, recommend actions following the performance tests, and investigate the effect of the solutions we had identified.

The key challenge related to the intermittent performance issues that appeared at seemingly random times. Monitoring random performance issues is never simple, especially in a complex environment such as the one used by the client.

The issues were exacerbated by limitations on the extent to which the server-side of the cloud-based CRM system could be monitored. This was due to it being provided as a sealed, turnkey service. Additionally, the service, which had been customised by the client, was being accessed from multiple locations, sometimes from within the virtualised environment.

We chose to take an incremental approach to performance testing, using a repeatable method that would be consistent for direct access to CRM, for access to CRM via Citrix from a location next to the client’s data centre and from access to CRM via Citrix from remote locations. We used visual automation with hundreds of samples to allow for the consistent gathering of metrics across the various access methods.

Based on our analysis, we identified issues in both the application’s customisation and the infrastructure layer. We were able to identify the specific transactions which suffered very high transaction times and provided the development team with this data for investigation.

Very high overhead across all transactions was also identified when we ran tests from a virtual host servicing a Citrix session. This was possibly caused by the contention of multiple Citrix workers on the same physical host.

An incremental testing approach was suggested to identify the level of contention at the virtual hosts that were detrimental to performance.

Ten10 used its testing expertise and proven methodologies to pin down performance problems – building a good relationship with the client in the process – and was able then to convey that information to the client for action.

Being an independent software testing partner enabled us to take a holistic view of the issues, which proved beneficial due to the number of separate organisations involved.