The case for using open source tools for test automation

One of the strong signals coming from the test automation community lately is the increasing popularity and adoption of open source tools and libraries. And you’d be wrong to think it’s only because they come for free. Read on, to find out about some of the less obvious benefits of open source.

Some years ago I stepped into the intriguing world of test automation as the technical lead in a project that involved end-to-end functional testing of a mobile application and a proprietary point of sale system. Although I had 15 years of software engineering practice, my experience with test automation was very limited. There was also an additional complication we had to deal with: one of the applications we were supposed to test was a Windows native application using custom drawn controls, so we could not rely on object selectors for automation. We realized that whatever tool we were to settle on, there was going to be some heavy custom logic involved. There wasn’t too much room for error either - changing our mind one year down the line, would mean throwing away all the investment we made in time, effort and money, damaging the team’s morale, our credibility and impacting the timelines of the projects involved.

This is when we first started to throw around the idea of building our own automation tool and, after going back and forth a couple of times, we decided this was indeed our best shot at success. We were very much aware of the amount of hard work involved, but we were optimistic, because we had a few good things going for us: we had good software engineering and test automation expertise in our team and the leadership was very supportive, which is a crucial factor in large companies. The outcome of this effort was the OpenTest functional test automation framework. Luckily, the product proved to be a success and, after the first year, it started to see adoption from numerous teams across four continents. I know this wouldn’t have been possible had we not made — at the very start of the project — the strategic choice to only use dependencies released under a liberal open source license. The core projects we incorporated into the product were: Selenium for web testing, Appium for mobile testing, Apache HttpClient for API testing, OpenCV for visual validation and Tesseract for OCR capabilities. It’s amazing to think about all the work that went into these projects, that we can just download and use, no strings attached. We later decided to pay it forward and release the OpenTest framework itself as open source.

If you’ve been spending time around a test automation team for a while, I should have no problem convincing you that automation is hard work. But doing test automation while dealing with license-related issues and the artificial limitations imposed by vendors is like trying to do your manicure while being chased by wolves. Please don’t do put your testers through that. Wouldn’t it be nice if they could install the automation tool in about 15 minutes, on however many physical or virtual machines they need to, without having to ask anyone for permission and without having to get anyone to agree to a $3000+ expense? That would free up time and mental resources that can be redirected towards dealing with the real difficulties of test automation.

One other thing I witnessed first-hand is that teams using commercial tools tend to hold very strong opinions about the tool of their choice. And it makes perfect sense: when you spend thousands of dollars on a test automation tool, you’re going to do everything in your power to defend the soundness of that decision long after you’ve admitted to yourself the tool is not all that great. The net effect is that different teams end up using different tools which stifles collaboration and encourages the formation of silos.

To be sure, using open source software is not without its risks and perils, so you need to do the due diligence to vet a project before adopting it. I will not elaborate on this, as there are many good resources out there that you can reference.

Here’s a non-comprehensive list with the benefits of open source that mostly focuses on the aspects relevant to test automation:

  • Zero cost. Everybody likes free stuff, but the zero cost comes with even more benefits: when your teams can install the automation tools for free, they will be able to do their jobs better and easier (e.g. run more parallel jobs to minimize execution time, install all components of the test tool on both their Mac and their Windows machine, etc.)

I hope I convinced you to give some serious thought on using open source for your next test automation project. Whatever the case, one thing is sure: open source is no longer the orphan kid it used to be until not so long ago, and that just fills my heart with joy!

It would be great if you can share your own experience with open source software. What projects and libraries have you found the most useful and what mistakes did you make that the rest of us can avoid?



Passionate software engineer, proud parent of two, author of the OpenTest functional testing tool for web, mobile and APIs.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Adrian Theodorescu

Passionate software engineer, proud parent of two, author of the OpenTest functional testing tool for web, mobile and APIs.