Regression testing ensures that any change introduced to the software does not break anything that was previously working well. Visual regression testing ensures that any GUI issues haven’t cropped up because of the changes made. The application should still look like what it used to before the modifications were introduced. A visual regression testing tool captures screenshots of the current UI and compares them with the original screenshots. It looks at the historical polarities and confirms that the web pages still render as expected across various browsers even after the modifications. Applitools Eyes is one of the most popular Visual Testing Tools available in the market that is powered by its proprietary cognitive vision technology.

Applitools has developed a cloud-based software testing tool, Applitools Eyes, that automatically validates all the visual aspects of any web, mobile and native app. These validations are similar to that of the ones a manual tester does. Applitools technology automatically validates the correctness of the UI layout, content and appearance, and enables you to automate tests that can only be done manually without it. It can test apps on any given browser, device, OS and programming language, on any screen resolution and form-factor.

An Overview of the Applitools Eyes System

Various test environments interact with the various Eyes Components of Applitools to implement visual UI testing. The system consists of the following components:

• The Application Under Test (AUT).
• A test suite that runs for the AUT and verifies its accuracy.
• SDKs are called with in the test suite code to trigger screenshot capture and visual checking.
• Application Drivers like Selenium or Appium which provides a framework to build test suits which can be send to the AUT. It also uses these to capture screenshots.
• The server receives and stores the screenshot send from the SDKs. It compares them to the stored baseline images and reports any differences.
• The Eyes Test Manager provides the user with test results they can review and helps user to report bugs and manage the baseline images.

The below is an example of how the components interact to run a test, allowing the Testing team to view and manage the test results.

Credit : Applitools

STEP 1: The testing team runs the test suite. The code usually repeats the following steps for different application states.

STEP 2.1: Using driver such as Selenium or Appium, the user actions such as mouse click, keyboard entry etc are simulated.

STEP 2.2: Along with that the Eyes SDK API is called to perform a visual checkpoint simultaneously.

STEP 2.2.a: The Eyes SDK uses the Driver to collect the screenshot.

STEP 2.2.b: After which the Eyes SDK sends collected image to the Eyes Server. Here the images and the other checkpoint images are compared to the baseline images which were previously stored on the server.

STEP 3: Once the images in the test have been processed, the Eyes Server provides with information regarding any differences observed. Along with this a link to the Eyes site is provided where results can be viewed.

STEP 4: Testing team uses the Eyes Test Manager to view the test results, to update the baselines, report bugs and annotate regions that need special handling. After viewing all of the results, testing team saves the baseline. This becomes the basis for comparison in the next test run.

Applitools have SDKs that supports a wide range of popular - web, mobile and desktop test automation frameworks, various application driver infrastructures, programming languages and all the common platforms, browsers, and operating systems. These SDKs don’t interact directly with the AUT, so Eyes is completely independent of how the application is implemented and deployed.

Compatibility of Applitools Eyes

Applitools has SDKs for all the popular programming languages and test runners providing full test coverage for web, mobile, desktop, PDF, TV OS's, IoT, Images, and more.

WEB SDKs

The WEB SDKs support all major web testing tools and programs:

1. Selenium for -Java, JavaScript, C#, Ruby, PHP, Python
2. WebdriverIO4, WebdriverIO5, WebdriverIO6
3. Selenium IDE
4. Cypress
5. Playwright (New addition)
6. Puppeteer (New addition)
7. TestCafe
8. Storybook CSF React (New addition), Storybook -React, Angular, Vue
9. Watir
10. Capybara
11. Protractor (New addition)
12. UFT/QTP
13. Lean CFT#, Lean JavaScript
14. Coded UI

Mobile SDKs

The Mobile SDKs support all major Mobile testing tools and Programs:

1. Appium Native –C#, Java, JavaScript, PHP, Python, Ruby
2. Appium Web - C#, Java, JavaScript, PHP, Python, Ruby
3. XCUI Objective-C, XCUI Swift
4. Espresso
5. Calabash iOS and Android

Screenshot SDKs

CLI, C#, Java, JavaScript, PHP, Ruby, XCTest Objective-C, XCTest Swift

Desktop SDKs

Windows UFT, Windows Coded UI

Codeless Tools

PDF Forms, Codeless IDE

The Test Manger in Applitools Eyes

The Eyes Test Manager is a browser-based tool that provides access to visual UI test data. It allows users to manage the test results, baselines, and other metadata.

Eyes Test Manager provides the following facilities:

• Access the results of visual UI test runs
• Accept or reject the differences recorded
• Annotate steps with comments
• Add match level and ignore regions
• Update the baseline
• View and manage the application and the test names
• View and manage the baselines and the branches
• View and manage the execution environments
• Compare and merge baseline branches
• Manage your account, teams, and users

There are various other tools and concepts that has been integrated into Eyes that helps user to organize testing and test cases. One such tool is the Baseline. The visual inspection protocol compares the captured images to the ones that were stored in the baseline from the previous test run. Each test run can have a different look and feel. Eyes have many baselines per test, where each baseline represents a selected environment. Each of these baselines can store series of image references.

When visual test runs, the application is driven through a sequence of application states, and for each state a checkpoint is execute which captures an image of the application for that state. Eyes compare the series of captured images to the series of reference images stored in the baseline of that test and report any significant differences. The image within the baseline depends upon on the test runs, but they could also depend upon other factors, like operating systems, browser type and viewport size. These factors are called the execution environment.

The viewport is the visual area of any application within the application window. The viewport’s size can also impact the layout of any page. For instance, changing the dimension of the browser might change the appearance of rendered text. Responsive pages with elements like menus may appear different for different dimensions. The font and image size may change. Note:  the viewport is platform specific.