Appium Vs Selenium Vs Calabash

Manual testing requires a human being to sit in front of the computer and execute each testing step carefully. In automation testing, an automation tool executes the test suite instead. In the technique of automated testing, the application implements the entire life cycle of the software in a very short time. The testing software offers the necessary efficiency and effectiveness.

In this article, you can learn an introduction, what is appium, selenium, & Calabash, types of automation testing, appium architecture, etc. which will help you understand About Appium vs Selenium vs Calabash.

Appium Vs Selenium Vs Calabash - Find the Similarities

• What is Automation Testing?
  • Why is Automation Testing required?
  • When should Automation Testing be employed?
  • Types of Automation Testing
• What is Appium?
  • What is Appium Philosophy?
  • How does Appium work?
  • Appium Architecture
  • Communication between Appium Client and Server
  • Prerequisites for using Appium
  • Limitations to using Appium
• What is Selenium?
  • Selenium IDE
  • Selenium Remote Control
  • Selenium WebDriver
  • Selenium Grid
• What is Calabash?
  • Behaviour Driven Development
  • What are the advantages of using Calabash?
  • Prerequisites for using Calabash
  • Limitations to using Calabash
• Appium vs Selenium vs Calabash

Q1) What is Automation Testing?

Ans: Automation testing is a technique where the tester uses suitable software and writes test scripts for the automation of the tests. It actually automates the normal testing process. 

The application is tested from loading to termination, and every aspect from performance to the stress points is checked. The automation tools are used to write and execute test cases during the execution of the automated test suite. 

The testers create the test scripts and test cases through the tools and group them into test suites. The main aim of automation testing is to use the tool to group them into test suites. It is possible to record the test suite and replay them. No human intervention is required once the test suite is recorded.

Are you interested in taking up Appium Training? Enroll for Free Demo on "Appium Online Training"

Q2) Why is Automation Testing required?

Ans: When software comes up with bug fixes or new releases, you might be unsure about the fact that there could be new bugs that weren’t present in the previous versions. The best way to know is to check the entire software thoroughly. Manual testing certainly has a lot of drawbacks, which can be overcome by automation. The following are the reasons why automation testing is required:

• No human intervention is required in automation testing and the tests can be run unattended.
• Manually testing multilingual sites can be very difficult.
• The speed of test execution is increased through automation.
• Automation testing will not be prone to errors.
• Automation helps to increase test coverage.

[ Related Article: Appium Interview Questions & Answers ]

Q3) When should Automation Testing be employed?

Ans: The following scenarios could require automation testing:

• The application is accessed for load and performance with lots of virtual users
• There are huge and serious projects
• The projects need to check the same spots regularly
• The software is steady with respect to manual testing
• The requirements do not change frequently

Types of Automation Testing

Automating testing depends on the frameworks which define a set of rules or practices that can be followed to get the desired results. The following are some of the most common types of testing frameworks.

1. Linear Scripting Framework: It is a basic test automation framework that works in a linear fashion in a 'Record and Playback' module, which gives it the name 'Record and Playback' framework. The framework tests small-sized applications. The creation and execution of each test script are done individually in this framework. Though it saves time and does not require much planning, it has drawbacks like hard coding and reusability.
2. Modular Testing Framework: The testers write the test scripts by breaking down the complete application into smaller and independent tests in this framework. The application is divided into multiple modules and the test scripts are created individually. Larger test scripts can be created through their combination using master scripts and the required scenarios can be achieved. The master script invokes individual modules to end the test scenarios.
3. Data-driven framework: The data-driven framework separates the test script logic and tests them against each other. Automation test scripts can be created by passing different sets of test data. The scripts are connected to external sources which are why they can be used for different sets of test data. The number of test sets required is reduced significantly.
4. Keyword Driven Testing Framework: A table format is used to define the keywords or action words for each function or method. Testers can work with the keywords to develop the automation script and work on the test scripts. Though programming skills are not required, the initial setup can require expertise.
5. Hybrid Testing Framework: The hybrid testing framework uses the characteristics of two or more frameworks. This is very common in the current scenarios.
6. Behaviour Driven Testing Framework: A platform is created where everyone including developers, analysts, and testers can collaborate. Acquaintance with a programming language is not required. Non-technical, natural language is required for the purpose.

These are some of the most popular automation testing frameworks today and prove to be useful in different cases.

Q4) What is Appium?

Ans: Appium is a mobile application UI testing framework that is open source and freely distributed. Hybrid, native, and web application testing is allowed by Appium. Automation testing is supported on physical devices, simulators, and even emulators.

Cross-platform application testing is allowed, which includes the APIs which work for both iOS platforms and Android. There is no dependency on the mobile device operating system. Selenium Webdriver commands can be translated through the Appium framework or wrapper into UIAutomator (Android) or UI Automation (iOS) commands. The commands do not depend on the type of device or operating system.

All the languages that have Selenium client libraries like Java, PHP, C#, Python, Ruby, javascript with node.js, and others are supported by Appium.

Learn Appium Tutorials

Q5) What is Appium Philosophy?

Ans: Appium has the power to automate and no recompilation of the app is necessary. This means access to the application code is not required. Appium is built to follow a few philosophies. It is important to look at them as it helps to understand Appium.

• There is no need to recompile the app or change it for the automation process: Appium doesn’t require an extra agent for the application code. Hence, there is no need to compile the code again or modify it.
• The user should not be bound to a few certain languages or frameworks to run the tests properly: Appium allows all users to perform the test processes. Everything from Python, Node.js, PHP, Objective-C, Ruby, and C# are supported. There is no need for users to learn a new language or framework in order to use Appium.
• The automation frameworks would not need to start defining new concepts for automation APIs: The WebDriver API is quite popular among the testers today is used by Appium. Thus, there is no reinventing the wheel with the introduction of a totally different API.
• The mobile automation framework needs to be open-source: Appium is freely available and users can modify the software as and when needed. The Appium testing tool is open-source both in practice and in spirit.

Q6) How does Appium work?

Ans: Written using a Node.js platform, Appium is an HTTP server. It drives the Android and iOS sessions through the Webdriver JSON protocol. Node.js must be pre-installed before the Appium Server is initialized. A server is set up on the machine that exposes the REST API when Appium is downloaded and installed. The API receives the connection and command request from the client and the commands are executed on mobile devices.

The responses are HTTP requests. The mobile test automation framework is used to drive the interface of the apps and the request is executed. The frameworks include Google UIAutomation for Android API level 16 or higher, Apple INstruments for iOS, and Selendroid for Android API level 15 or higher.

Appium Architecture

The architecture of Appium is fairly straightforward. One of the first things to know about architecture is the client/server nature. In the client/server architecture, it is necessary to understand the roles of both the client and server of Appium.

1. Server: Appium functions in a similar manner as Selenium. Appium is a server in the core just like Selenium RC. Communication between a mobile device and a client is facilitated by this server. The Appium server has been coded in Node.js. This makes it important for the users to have Node.js pre-installed on their systems.
2. Client: According to Appium terminology, the clients represent the client libraries like Python, PHP, Ruby, Java, C#, and JavaScript. The libraries offer commands specific to the mobile devices in addition to the Selenium commands.

Communication between Appium Client and Server

As the Appium server is started, a REST API is exposed. The client begins a session using the JSON 'desired capabilities' object. The server also initiates a session. The responses to the client require a session ID.

The result is the creation of a session between the client and the server. After the session has been established, the client would automatically send commands to the server. The commands are executed by the Appium server on the mobile device. The necessary responses are offered to the client.

The Appium commands are not executed directly on mobile devices. Automation frameworks offered by the vendors are invoked which are specific to the platform. There are vendors like Google, Microsoft, and Apple who provide the automation frameworks for the platforms.

The Appium server hooks itself to the frameworks and executes the device automation. Appium uses different frameworks for various platforms.

• iOS: UIAutomation from Apple
• Android 2.3+: UIAutomator from Google
• Android 4.2+: Instrumentation from Google
• Windows: WinAppDriver from Microsoft

Prerequisites for using Appium

The following are some of the things that should be present in the system:

• Java Development Kit or JDK
• Android SDK Studio
• Eclipse
• TestNG for Eclipse
• Appium Client Library
• Selenium Server JAR
• APK app info on Google Play
• js
• Appium Desktop

Limitations to using Appium

• Testing of Android versions lower than 4.2 is not supported by Appium.
• There is limited support for hybrid app testing. It is not possible to test the switching action of the application from the web app to native and vice-versa.
• There is no support to run the Appium Inspector on Windows.

Q7) What is Selenium?

Ans: Selenium is a popular web-based automation tool that is entirely open source and is meant for web apps over different platforms and browsers. Selenium cannot be thought of as a single tool. It is a  suite of services and software, and each caters to the different needs of the organization. The four components include:

• Selenium Remote Control
• WebDriver
• Selenium Integrated Development Environment
• Selenium Grid

At the time when Selenium was created, QTP was a famous automation tool and was developed by Mercury Interactive before being acquired by HP. Selenium was used as an antidote in the treatment of mercury intoxication. Jason Higgins was an engineer at ThoughtWorks who suggested the name of Selenium.

Selenium IDE

The Integrated Development Environment of Selenium or the Selenium IDE is a Firefox plugin - a simple framework offered by Selenium. The testers can record and play the scripts using this.

Though people can create scripts using Selenium IDE, Selenium WebDriver or Selenium RC would be required to design more advanced and robust test cases. It is supported on Windows, Linux, and Mac OS. The browser support is Mozilla Firefox.

Selenium Remote Control

Selenium RC had been the main Selenium project and was known as Selenium 1. It is still actively supported in maintenance mode and relies on JavaScript automation. Almost every browser and language like Ruby, Javascript, Java, Perl, Python, PHP, and C# are supported.

The WebDriver merge brought around Selenium 2. The operating systems that are supported include Mac OS, Linux, Windows, and Solaris. The browsers that are supported include Google Chrome, Opera, Safari, Mozilla Firefox, and Internet Explorer.

Selenium WebDriver

Selenium WebDriver, also known as Selenium 2, is a browser automation framework that accepts the commands and sends them to a browser. It is implemented through a browser-specific driver.

Selenium WebDriver manages the browser and directly communicates with it. PHP, C#, Perl, Ruby, Python, and Java are supported. The supported operating systems are Linux, Solaris, Mac OS, and Windows. The supported browsers are Google Chrome from version 12, Mozilla Firefox, Opera from version 11.5, Internet Explorer and Safari.

Selenium Grid

The Selenium Grid is used in conjunction with Selenium Remote Control in order to execute tests on the different machines against browsers in parallel. Multiple tests are run at the same time against different machines through different browsers and operating systems.

Q8) Which of the Selenium tools should be used in what cases?

Ans:

Selenium IDE should be chosen in the following cases:

• To know about Selenium and the concepts related to automated testing which include locators like name, id, XPath, CSS selector, and others. The users can also learn about Selenese commands like open type, verify, assert, click and wait, explore test cases in the various formats and execute JavaScript code with the help of runScript.
• If test are to be created with meager or no knowledge in coding at all
• To test a web application only against Firefox
• To create test suites or basic test cases that can be later exported to WebDriver or RC.

Selenium RC can be chosen if:

• the testers want to test web applications with complex AJAX-based scenarios
• to test an application against a new browser that supports JavaScript
• to deploy tests over multiple platforms with the help of Selenium Grid
• to run a test against different browsers on different operating systems

Selenium WebDriver can be chosen in the scenarios:

• Executing tests on the HtmlUnit browser
• Testing applications that have rich AJAX-based functionalities
• Using a particular programming language in the design of the test cases
• Creating a customized test result

Selenium Grid can be chosen if you would like to:

• Run Selenium Remote Control scripts in different operating systems and multiple browsers at the same time
• Run a grand test suite which must be completed as soon as possible

Prerequisites for using Selenium

• Java Development Kit or JDK should be present on the system
• Eclipse is another necessity
• You would need to have FireBug and FirePath
• Selenium component should then be installed

Limitations to using Selenium

• Automation for desktop applications is not supported by Selenium.
• Built-in object repositories like UFT/QTP are not present that maintain the objects in a centralized location.
• Performing tests on images are not possible.
• The inbuilt reporting capability is not present.
• Knowledge of at least one programming language is necessary to create the test scripts

Q9) What is Calabash?

Ans: Calabash is an open-source testing framework, where tests can be written and executed for the Android and iOS apps. This Automated User Interface framework allows tests that are to be written in Cucumber and Ruby.

UI interactions are enabled by Calabash within a mobile application through input text, buttons, and by validating responses. It can be configured to different iOS and Android devices. Real-time feedback and validations are offered. Calabash also offers a great foundation for cross-platform testing strategy.

Behaviour Driven Development

Calabash is Behavior Driven Development or BDD. This framework is quite similar to Test Driven Development, but application behavior is specified instead of creating tests to describe the shape of APIs. BDD is a process where multiple stakeholders weigh in to create a common understanding of what needs to be built. This strategy helps to build the right software and design it from the perspective of the owner.

Calabash Framework

Calabash is a test automation framework enabling mobile developers to develop anything without the necessary coding skills and execute automated tests for iOS and Android apps. UI interactions are enabled by Calabash. Though this is a great first step in the UI acceptance test automation, the real benefits can be seen when the tests are executed on mobile devices. Calabash testing can be configured on hundreds of iOS and Android devices to get real-time feedback and validation.

The Behavior-Driven Development Calabash is based on defines a philosophy where development takes place from the outside-in. It isn't much different from creating functional specifications and building them to shape, Calabash takes the specifications and makes them executable, and provides the means to test the actual specifications.BDD is meant to be a process where multiple stakeholders can create a common understanding of the project to be built.

Q10) What are the advantages of using Calabash?

Ans: Calabash is quite easy while writing tests and makes the entire process quite fun. The language syntax is easy to read and coding experience is not really necessary. The advantages include:

• Calabash helps to improve throughput or productivity.
• It helps to obtain improved robustness or consistency of the product or processes.
• There is an increased consistency of the output and labor costs and expenses are reduced.
• The quality of the products is improved and can be predicted.

Prerequisites for using Calabash

• One of the first things necessary on the computer is Ruby. The latest versions are recommended.
• Bundler is to be installed to take care of the dependencies.
• For Android users, the latest SDK needs to be installed.
• It is recommended to have at least Mac OS 10.10 with Xcode 6.x. The Calabash needs to be installed for iOS.

Limitations to using Calabash

• It requires proficiency to write the automation test scripts.
• Debugging the test scripts is often a major issue.
• The maintenance of the test data files would be difficult if more screens are tested by the scripts.
• In the case of the playback methods, test maintenance would prove to be costly.

Appium vs Selenium vs Calabash

Though none of the automation tools would offer a universal solution for the native and hybrid mobile applications, Appium and Selenium can offer good language support and a comprehensive platform, while calabash gets good testing but lacks stability. Often addressing one testing challenge introduces another one. Thus, automated testing tools should be chosen depending on the project. If necessary, two or more of them can be combined to address the challenges.