At Rounds, like most other software companies, we are always looking at the best way to test our products. We have a mobile app for iOS and Android and also a web (Chrome extension) app. As a small company, we do not have the resources (either human or hardware) to test our apps on all of the devices and networks that are out there. We have 15 developers and only 1 QA engineer, so how are we dealing with this issue?
There are a few crucial components that make up our solution to this problem:
The first line of defence is our developers who never commit a line of code without testing. They test not only the issue that was just fixed but also think about how it can affect different use cases and test those as well.
Our second line of defense are uTest services that we use to test both our mobile and web apps. We define what is needed to be tested and how many testers we want and – almost instantly – we have an army of testers to our disposal. All issues are connected to our internal bug tracking system and reported to our Jira servers. These reports include all the actions that were performed, the expected result from these actions, the actual results and then the relevant logs, screenshots and even a screencast. All bugs are going via a screening process by our QA engineer before they are assigned to a developer.
For our Android app we use TestFairy, which injects some code into our app and in return we get a video of the user actions as well as his logs in case of a crash. Upon every crash a Jira issue is opened automatically. Getting the video helps us find exactly what was the specific usage flow that caused the crash. An additional benefit of the video, is that we know what exactly the testers at uTest are doing :) (We do not use TestFairy in production builds)
We are using Crittercism (Android) and Crashlytics (iOS) that provide us real-time crash data and handled exception ,so that we can match with the tester’s report , see how frequent it is, prioritize it accordingly and quickly solve the report and close the loop.
Another service that we are utilizing for mobile testing is Perfecto Mobile. By using Perfecto Mobile we can test on a wide range of devices and network carriers. We do not use it on a daily basis, but if we get a report on a problem with a specific device or a specific network we will try to reproduce it using Perfecto Mobile.
We also are great believers in eating our own dog food, so everyone in the company has the latest versions of our mobile apps installed on his phone and we use it regularly. As the number of Android and iPhone lovers is divided almost 50/50 in the company, both platforms get the respect they deserve. We also use our web app for group video chats with our partners or for internal conferences daily.
All of the layers above are helping us build a robust software testing solution with only 1 QA engineer.