Check out Inneractive’s popular ad tech blog.

‹ Go Back

Appium at Inneractive: a Tale of QA Automation

Appium at Inneractive: a Tale of QA Automation

Around 2014, Inneractive’s R&D department made a strategic decision that would significantly streamline the company’s QA process. After realizing the extent to which automation shortens QA cycles, it was decided to channel resources into developing an automated platform for product testing. This would replace the company’s legacy platform.

Given the extensive product and technology portfolio, including backend, web, mobile and data components, the automation process began with backend, expanding out to include other products. Backend tests are divided into APIs, request generation and response extraction, server logic and UI for analytics and configuration. Inneractive’s main SDKs are Android and iOS, both native apps. When a publisher integrates the SDK, he can reap the benefit of the extensive preparatory work Inneractive has already done. Our SDKs support display ads, VAST video ads, rich media ads, expansion and collapse ads, native ads, as well as Story video an innovative in-feed video with storytelling abilities.

Inneractive JavaScript Ad Tag integrates on your mobile browsers and includes display ads and VAST video ads. When Inneractive first began developing the ability to test mobile, most available mobile automation tools were very basic, so we set out to find a solution that could cater to most Inneractive products. Following market research to identify the best fit for our needs, we decided to start working with a tool called Appium.

Let start with: What is Appium?

Appium is an open-source tool for automating native, mobile web and hybrid applications on both iOS and Android platforms.

Why we chose Appium?

Appium is standalone tool, meaning it can be reached no matter where you install the server. Placing the server on one central computer with connected devices or emulators allows QA testing to be run. The server can be on AWS or any alternate location of your choosing. Having proved a match for SDK tests, Appium can also help us with backend tests and mobile web.
With third party platform independency, a key pain-point is Appium installation, which is not always smooth sailing. When the installation is not going well, you can find yourself alone with no help in sight.
Running tests on iOS requires a Mac which can be challenging if you work in a startup and a Mac computer is too costly. In this case a third-party platform offers the best solution since it doesn’t require installing. With Appium, there is no need to buy a computer and devices; you only need to change the server URL.

The third-party platform divides into two areas:

  • Firstly, you gain round-the-clock access to the Appium server and the ability to run your tests in real-time.
  • Secondly, when compiling and uploading the tests, they go into a que, running backend tests, we need to talk with Appium in real-time.

When we started the development process, a series of questions popped up, such as: Where and when will the tests run? Are there any limitations? Phases? What is the budget for this project? Should the tests be part of the tested project or a standalone project?

After careful consideration, we developed the following practice:

When
Our tests run nightly and after the development team releases a version to QA team.

Where
We tried several options such as install Appium in the office, install Appium in AWS.
Since Jenkins run our tests and is located in AWS, installing Appium in the office was not an option.
We successfully installed Appium on AWS, but given that the machine in AWS runs Linux this only provided a solution for Android. Testing iOS requires Mac OS.

The next step was to start testing third party platforms and we explored several companies’ solutions.

Limitations
Video verification is a constraint, and there is no solution currently available in the industry.

Phases
We began by covering sanity, and then we asked the QA team what their biggest and most difficult tests are in regression: Areas where automation can most help save time and allow testing of new features.

My top tip:

During the first part of 2016, I decided to open a Meetup group named: “Appium Israel”.
I wanted to create a community around Appium, a forum where people from different companies could help each other. Appium is not very intuitive, and when you start developing, many issues arise that you need to deal with, and the answers are not always available in the Appium community (or anywhere else). After I first opened the Meetup group, I was approached by several people, they all wanted to begin developing their own platform and were keen to hear more about the Inneractive process and practice.
One of the biggest mistakes that I saw, was made by companies who wanted to start with a POC and see if Appium matched their needs. Such companies would allow a member of the QA team, without any knowledge in development, to perform the POC. To me this as a big concern, as Automation developers, especially when mobile is involved, are different from server developers. They need to have a solid background in basic mobile development, iOS provision and certification for real iOS devices, Jenkins, Junit, HTML, JS, understanding CI/CD and more.

I won't say that the QA or development team shouldn’t be the ones to do it - simply that you need to find the right person with the right skill-set for an automation project to be successful.

More blog posts

Ad Fraud Myths Debunked

Ad Fraud Myths Debunked

Methbot’s discovery in late 2016 brought ad fraud to the forefront of the digital advertising agenda. With fraudsters reportedly...
Read More
The changing climate of native programmatic advertising in China

The changing climate of native programmatic advertising in China

Native has been touted by industry leaders as the future of the advertising industry. This is especially true (if more challenging) for...
Read More