Automated Exploratory Tests with Firebase Test Lab for Android – A Tester’s Story

Of late, I got into a backbreaking situation, a familiar situation in most of my Android testing projects — changes updated, and the developer gave the build for regression testing.

Now, hours of lengthy and monotonous manual testing await me to ensure there are no basic technical snarls, before I move onto the next level of sanity or smoke test that I must execute tirelessly. Nevertheless, the need of the hour is a free tool that would run automated exploratory tests without any hassle.

Firebase Test Lab for Android runs tests against a test matrix that you create to validate your app across a range of device types and configurations. Although the lab allows us to load espresso scripts and use them on the instrumentation method, I preferred the Robo test method to automatically simulate user activities and do a quick exploration on the overall app.

I was sure that it would be a straightforward process to set up an automated script on the Robo test, given that the user already has a firebase account in the test lab. I followed the steps below.

  • Selected Robo tests,
  • uploaded the apk,
  • selected the range of devices (virtual/physical) on which I should run tests,
  • selected the orientation and locale, mostly I prefer portrait with English as I can also specify the test timeout and max. depth in the advanced options.

And, finally, whooo!!! Now, I’m all set to run my test.

Setting Up Automated Testing on Robot Test

Wait, suddenly I had a flash: did I miss something?

What if my app has a sign in and requires a username password to go in and then do an exploration? Where should I find the credentials?

Then, I discovered that the Robo test feature handles these under “Test Account” credentials. I followed the steps below.

  1. Entered the username and password resource names;
  2. entered the username and password for the test account; and, I was good to go.

Finding Test Account Credentials

I ended up finding that after completing the Robo test, logs videos and screenshots are available, which finally helped me understand how the Robo ran across the application and interpreted my next move.

Now this was extremely useful tool to me, in the sense, earlier I was relying on the old “MonkeyRunner” to perform a few random exploratory tests on my Android apps, but now I have the luxury of selecting my devices, giving text inputs, and getting more visual outputs rather than going through the painful process of skimming “adb logs” for results.

Robo Test Interface with Logs Videos, Screenshots

Finally, not only I found a crash but also added the script to my regression suite. With that, I can run it every time a new build comes in and visually verify how the script traversed through the application without breaking a sweat.

Put simply, I can now perform minimal monotonous testing and extensive investigative testing to deliver a flawless app rendering superior performance!

Are your testers finding the monotonous exploratory testing process grueling? Try automated testing with Firebase Test Lab. Don’t know how to get started? Talk to an expert tester now.

Get in touch