Problem Statement:
Most GUI automation tests redo the same pattern repeatedly:
Controls are represented by a selector
Page objects encapsulate selectors and often methods which make use of the controls
Either tests or page methods manipulate these controls
Either tests or page methods validate the result
And we write these little pieces of code over and over for each action or validation we want to do
There is a lot of duplication in this model, even when it's done really well.
Justification Intent:
By building an engine to validate all the controls on a page, and validating them whenever the page is loaded or changed, automation can catch things otherwise missed.
Description:
Reduce automated testing to writing descriptions of pages and data, which are then processed by an engine. No additional coding is required for most tests.
A broadly applicable special class of data driven test, where the data provided describes not only the test conditions, but also how the page is manipulated given said data.
Detail:
In engine driven testing, we provide lists of controls to check, with not only the attributes for the control, but also attributes that described how the control was to be tested. The engine would then process the control lists.
Example:
Controls become collections of data that include not only identification data (eg a selector string) but also includes the information required in order to determine how to test the control. Tests provide test data (eg username, password, first name, and so on). An engine steps through the set of controls, applying or capturing data as directed.
Test data includes information like:
{'first_name': 'FQAUser', 'last_name': 'LQAUser'}
Control data, on the other hand, includes data similar to that shown in the example below:
my_control = SWADLControl( selector="blahblah", is_text="Please enter your first name", test_data_key="first_name", actual_value_key="actual_first_name", validations={ VERIFY_EXIST: True, VERIFY_VISIBLE: True, VERIFY_ENABLED: True, VERIFY_UNIQUE: True, VERIFY_VALUE: "Please enter your first name", VERIFY_PROPERTY: ("selenium", "first_name"), VERIFY_INPUT: True, }, )
Using the test_data_key as the pointer into the test data, this then is enough to scan the input control and take a value from the test data to apply to the control.
Based on the control type, additional information is included in the control such as the expected value for labels, link targets for links and so on... And this then tells the engine how to manipulate and validate the control and it's contents.
Values to capture for later evaluation are captured if a actual_value_key is specified in the control's data.
The engine then uses data control's data to determine which tests to perform:
Verify exist
Verify visible
Verify enabled
Verify unique
Verify text value
Verify attribute value
Verify clickable = verify that it exists, is visible, is enabled, and is unique.
Verify Input = Set a value (from the test_data_key, for relevant types)
Verify Click = Click on the thing
Or other tests as needed. Some instructions are implicit... Eg you don't need to test the default text on a label if it's expected value isn't in the verify data.
Control information can also contain flags such as "any error with this control is fatal", or "this control is optional/required, and so on.
By adding another test to the engine, all the existing controls that are eligible for the new test also get tested, provided they have all the data needed.
Of course not all tests will fit into this pattern, but in our experience, most will.
Collections of controls then can simply iterate through all the controls, calling the engine for each one.
Example:
selector="blahblah",
is_text="Please enter your first name",
test_data_key="first_name",
actual_value_key="actual_first_name",
validations={
VERIFY_EXIST: True,
VERIFY_VISIBLE: True,
VERIFY_ENABLED: True,
VERIFY_UNIQUE: True,
VERIFY_VALUE: "Please enter your first name",
VERIFY_PROPERTY: ("selenium", "first_name"),
VERIFY_INPUT: True,
},
)
This defines a control (in python). No flows or page methods will typically access this control other than via an inherited process_control(passed_dict) method.
In the example above, self refers to the page.
While this example is in python, it would even be possible to implement these descriptions as xml instead of code.
Engine:
The engine is simply a set of conditionals, like if control.validations["verify_visible"]==True then assert control.get_visible().
Some control classes will take data and some won't. Those if statements will prevent the wrong thing from happening with the wrong control class. For example, if it's a label type control, we won't go looking to see if there's a value to put in the control.
For any special case control, we just replace the process_control() method with a custom one.
Personal Experience:
At Home Depot, we implemented this using wrapper objects for the controls, where the wrapper carried the selector, but also flags to indicate what was to be tested and what the expected result was. Then we just ran a loop to validate all the controls, calling the engine. The engine then read the flags so it knew what to test.
When confronted with significant changes to the AUT, we nonetheless were able to turn around our fixes in a matter of moments.
