Nove Markets

The Customer

Nove Markets is taking on an important challenge at a critical time – creating jobs through the use of technology. As a startup out of the US with remote engineering in India, they have a global team committed to their mission. With a wide array of opportunities in front of them, they mainly focus on Charitable Giving, Health, and Wellbeing.  

The Environment

Nove Market’s first app being launched is called Cuisine Crafters, and it is a marketplace app to connect culinary providers with culinary clients. With both a web app and mobile app, it is a cross platform application with interfaces for 2 distinct user bases. As a new app coming out in beta soon, it is under rapid development and is in need of automated test coverage to keep up with the pace of development.

The Vision

The need to be able to move quickly and deliver changes rapidly to customers has required the team at Nove Markets to automate testing from the get-go. Ultimately, the goal is to achieve nearly 100% automated test coverage of application functionality. Driving this goal is a desire to do more with the limited resources of a startup, where a full QA team won’t be an option until far beyond go live.

The Challenge

When working with the most common open source automation tools, such as Selenium, too much time was spent on non-value added tasks like test framework maintenance. Additionally, as the team continued to scale and new members were added, there was a steep learning curve. Furthermore, the lack of cross platform support in open source drivers meant managing and maintaining Selenium and Appium, as well as various Mac machines to facilitate iOS testing.

The Process

Sailaja Banu from Nove Markets embarked on a journey to investigate tools that could provide the basis for a robust test automation framework. She began by selecting a short list of automated testing tools to evaluate from the wealth of tools on the market. Up for consideration was building a bespoke framework from popular open source drivers like Selenium and Appium, as well as acquiring a solution built on top of those drivers such as TestProject or Katalon.

The Requirements

After meeting with various members across the teams, Sailaja assembled a comprehensive list of requirements for Nove Markets’ new testing tool. Some of the main requirements highlighted in her research included: 

• Easy to learn and adopt, even for those with less technical training or coding experience
• Ability to support web and mobile applications (including Android and iOS) 
• Preference to execute all automation on available Windows machines
• Support for testing on cloud devices, simulators, and emulators
• Easy to understand reports on test coverage, failures, and release progress
• Robust support and documentation to allow for easy onboarding and troubleshooting
• Affordable, so the cost remains reasonable as the team and organization grows

With these features in mind, Sailaja would be able to narrow down the list of tools to those which support Nove Markets’ test automation strategy over the years to come.

The Search

Sailaja kicked off a thorough search of tools on the market, perusing various forums, articles, and other sources of test automation trends. Shee came across TestProject through her research, and it quickly became one of the leading tools in the evaluation. It eventually came down to a short list, TestProject, Katalon, and building an in house framework from Selenium and Appium. After consideration, TestProject became the favorite, due to some key considerations: 

• TestProject supports web as well as mobile in one interface (Android and iOS)
• TestProject allows for testing iOS devices from Windows machines (no Mac required)
• TestProject offers integration to cloud device providers like BrowserStack and Sauce Labs when testing needs to be done on devices the customer does not own
• TestProject provides an interface for those who wanted to script as well as those who lacked technical skills
• TestProject offers quick support via live chat as well as a thorough forum and documentation to ease onboarding
• TestProject is 100% free, including the platform, hosting, as well as live support

Now that the choice to implement TestProject had been made, Sailaja was ready to embark on the process of implementing TestProject within Nove Markets.

The Implementation

The process of getting TestProject up and running at Nove Markets was relatively painless, with initial results coming quickly. Sailaja started by mapping out and planning the more than 200 test scenarios that needed to be automated across the web and mobile applications. With the ramp up period only taking a matter of days, test scripts were being automated quickly, and by the end of the first 3 months, most had been automated. TestProject has now been used to automate all of the current test scenarios, with ongoing work to continue to keep pace as the application gains more and more functionality.

The Results

With TestProject fully deployed, Sailaja has been able to see tremendous improvements to her automated testing, including: 

• Improvement in tester efficiency, with all users now able to contribute fully to automated test coverage, regardless of technical skillset
• Reduction in onboarding time, with users fully enabled and contributing in one month or less, as compared to several months previously
• Increase in test coverage, with 100% of functionality now covered with automated tests, across more than 200 test scenarios
• Reduction in test framework maintenance, from 20-30% of time spent on it previously to less than 5% of time spent on it with TestProject

Overall, the benefits of TestProject have been instrumental in supporting the team’s efforts to get their new applications launched in beta. 

The Advice

Overall, Sailaja would recommend TestProject to other organizations looking for a similar solution to tackle Web, Mobile, and even API automation. Her advice to ensure a smooth implementation is as follows: 

1. There is a desire to use specific tools like Selenium due to their popularity in the market. However, for many users, these tools will be more work than they will provide benefit. Find the best tool for your organization specifically – don’t follow the masses.
2. Focus on getting the tool working and adding value first, then look into how to customize the tool further to fit your needs. Nove Markets is constantly improving their workflow with TestProject even today, but overcomplicating things too early can risk success.
3. Share your success to increase buy in and support for your test automation initiatives. We share our testing reports daily with a wide audience, to give them comfort in our test coverage and win them as allies supporting our vision.
4. Integrate test automation closely with development. The value of test automation lies in getting quick feedback to developers on their code, especially when new features are being created.  Our integration of TestProject to the CI/CD process has been critical to our success.
5. Don’t hesitate to reach out to TestProject support. Though the documentation and forum are very complete, the support team has been exceptional! Especially in the beginning when you are working to learn the tool and get everything set up, the TestProject team’s help is very beneficial.