Around 2014, Inneractive’s R&D department made a strategic decision that would significantly streamline the company’s QA process. After realizing the extent to which automation shortens QA cycles, it was decided to channel resources into developing an automated platform for product testing. This would replace the company’s legacy platform.
Given the extensive product and technology portfolio, including backend, web, mobile and data components, the automation process began with backend, expanding out to include other products. Backend tests are divided into APIs, request generation and response extraction, server logic and UI for analytics and configuration. Inneractive’s main SDKs are Android and iOS, both native apps. When a publisher integrates the SDK, he can reap the benefit of the extensive preparatory work Inneractive has already done. Our SDKs support display ads, VAST video ads, rich media ads, expansion and collapse ads, native ads, as well as Story video an innovative in-feed video with storytelling abilities.
Let start with: What is Appium?
Appium is an open-source tool for automating native, mobile web and hybrid applications on both iOS and Android platforms.
Why we chose Appium?
Appium is standalone tool, meaning it can be reached no matter where you install the server. Placing the server on one central computer with connected devices or emulators allows QA testing to be run. The server can be on AWS or any alternate location of your choosing. Having proved a match for SDK tests, Appium can also help us with backend tests and mobile web.
With third party platform independency, a key pain-point is Appium installation, which is not always smooth sailing. When the installation is not going well, you can find yourself alone with no help in sight.
Running tests on iOS requires a Mac which can be challenging if you work in a startup and a Mac computer is too costly. In this case a third-party platform offers the best solution since it doesn’t require installing. With Appium, there is no need to buy a computer and devices; you only need to change the server URL.
The third-party platform divides into two areas:
- Firstly, you gain round-the-clock access to the Appium server and the ability to run your tests in real-time.
- Secondly, when compiling and uploading the tests, they go into a que, running backend tests, we need to talk with Appium in real-time.
When we started the development process, a series of questions popped up, such as: Where and when will the tests run? Are there any limitations? Phases? What is the budget for this project? Should the tests be part of the tested project or a standalone project?
After careful consideration, we developed the following practice:
Our tests run nightly and after the development team releases a version to QA team.
We tried several options such as install Appium in the office, install Appium in AWS.
Since Jenkins run our tests and is located in AWS, installing Appium in the office was not an option.
We successfully installed Appium on AWS, but given that the machine in AWS runs Linux this only provided a solution for Android. Testing iOS requires Mac OS.
The next step was to start testing third party platforms and we explored several companies’ solutions.
Video verification is a constraint, and there is no solution currently available in the industry.
We began by covering sanity, and then we asked the QA team what their biggest and most difficult tests are in regression: Areas where automation can most help save time and allow testing of new features.
My top tip:
During the first part of 2016, I decided to open a Meetup group named: “Appium Israel”.
I wanted to create a community around Appium, a forum where people from different companies could help each other. Appium is not very intuitive, and when you start developing, many issues arise that you need to deal with, and the answers are not always available in the Appium community (or anywhere else). After I first opened the Meetup group, I was approached by several people, they all wanted to begin developing their own platform and were keen to hear more about the Inneractive process and practice.
One of the biggest mistakes that I saw, was made by companies who wanted to start with a POC and see if Appium matched their needs. Such companies would allow a member of the QA team, without any knowledge in development, to perform the POC. To me this as a big concern, as Automation developers, especially when mobile is involved, are different from server developers. They need to have a solid background in basic mobile development, iOS provision and certification for real iOS devices, Jenkins, Junit, HTML, JS, understanding CI/CD and more.
I won’t say that the QA or development team shouldn’t be the ones to do it – simply that you need to find the right person with the right skill-set for an automation project to be successful.