by Gopi Kannan

Comparison Matrix of Cloud based Mobile App Testing

Comparison matrix has been created for the cloud based mobile application testing providers; I have compared the Amazon AWS Device Farm, Sauce Labs, Perfecto Mobile and Xamarin and highlighted the key points in the below screenshot;

CM1CM2

10 thoughts on “Comparison Matrix of Cloud based Mobile App Testing”

  1. Hello Gopi,

    Just wanted to confirm, What do you mean by Supported Language row in above table ?
    Moreover I wanted to confirm about AWS device farm, how it supports C#? can we write tests in this?

    Like

    Reply

    1. Supported language is used to write your automaton test scripts in IDE. You can write your automation scripts in core java etc

      At this point of time, AWS Device Farm will support Java and Python in respect to Appium framework based on the below reference URL and Calabash framework will support Ruby language to prepare your automation scripts;

      http://docs.aws.amazon.com/devicefarm/latest/developerguide/test-types-intro.html

      Custom Android Test Types
      Appium Java JUnit
      Appium Java TestNG
      Appium Python
      Calabash
      Instrumentation
      UI Automator

      Custom iOS Test Types
      Appium Java JUnit
      Appium Java TestNG
      Appium Python
      Calabash
      UI Automation
      XCTest
      XCTest UI

      Custom Web Application Test Types
      Appium Java JUnit
      Appium Java TestNG
      Appium Python

      Like

      Reply

  2. hi, few queries….

    Can saucelabs work with any other tool apart from appium?

    real device support: can you elaborate more on limited. how is limited on this front?

    -Gaurav

    Like

    Reply

    1. Can saucelabs work with any other tool apart from appium?
      GK: Yes, it will support Selenium for mobile web automation, ROBOTIUM & ESPRESSO for Android Native/Hybrid App Automation.

      real device support: can you elaborate more on limited. how is limited on this front?
      GK: This post was created on MARCH 4, 2016 and at that time, Saucelabs was not supported with real devices.
      Now everything got changed and saucelabs supporting real-devices in their cloud environment. Please read as “Yes” instead of “Limited” 🙂

      Like

      Reply

  3. Hi,

    Will you please suggest a device cloud for Appium Java with TestNG ?

    Xamarin supports Appium Java with TestNG ?

    Thanks,

    Lincy

    Like

    Reply

  4. Hi Gopi,

    Thank you for the suggestion. I was asked to perform a comparison of different device clouds which are compatible with Appium Java with Test NG and come up with best one.

    Perfecto mobile cloud with appium will have any advantage over Sauce labs?

    Can you please help me to find the best mobile cloud for Appium?

    Thanks

    Lincy

    Like

    Reply

  5. Hi,

    This is a very helpful comparison. But I see this was done in 2018. So wanted to ask, 2 years later, do you any difference in the performance and features of these cloud based mobile app testing services ?

    We are looking for a good option where web app automated testing (with Appium, Java and TestNG involved). Also we have some external files like excel that is part of the framework. So you see any challenge in reading them in any of the app testing services noted above ?

    Thanks in advance for your inputs and for creating this thread.

    Regards,
    Kimaya

    Like

    Reply

Leave a comment