Although it's not required to use a test framework to organize and execute your Selenium WebDriver tests, a good test framework does make this task more convenient. As a Python programmer, I'm a fan of Pytest.
Selenium WebDriver supports a wide variety of web browsers, but is limited to those browsers installed on the computer where WebDriver runs. That's where a service like Sauce Labs comes in. Sauce Labs runs Selenium WebDriver on virtual machines in the cloud, offering many combinations of device, operating system and browser.
So the combination of Selenium WebDriver, Pytest and and Sauce Labs enables testing of a web application across many platforms. There's just one catch. While our web application is under development, we likely wish to run it on our local machine, with a localhost URL. A Sauce Labs virtual machine can't access the local machine to run such tests -- unless we install Sauce Connect. Sauce Connect is free software from Sauce Labs that creates a secure tunnel between Sauce Labs' servers and our machine.
What follows is a step-by-step guide to creating and running tests using Selenium WebDriver, Pytest, Sauce Labs and Sauce Connect. This guide refers to Python 2.7 running on Ubuntu 15.04, but the same principles apply to any programming language and operating system.
Let's start by creating a web app to test. There are many ways to do this. Our example uses a "hello world" Python Flask app, based on the sample from the Flask Website. Here's the code, which saved as hello.py:
# Source of hello.py
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello World!"
if __name__ == "__main__":
app.run(port=8080)
Once we run
python hello.py
we can visit http://localhost:8080 in our browser and see a web page that displays "Hello World!" Now we can write a test to verify that the web application is working as expected. Selenium WebDriver supports almost any programming language and web browser with Selenium. This example uses Python and Firefox. Let's make a directory named tests and create a file in it named test_hello.py:
# Source of test_hello.py
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://localhost:8080/')
body_text = driver.find_element_by_css_selector('body').text
assert body_text == 'Hello World!'
driver.quit()
As long as Firefox is installed and hello.py is still running, we can execute
cd tests python test_hello.py
Selenium WebDriver will open Firefox, navigate to http://localhost:8080, and verify that the text "Hello World!" is displayed.
Let's introduce Pytest to make the test a little more convenient to run. Pytest's discovery feature automatically finds tests based on naming conventions, such as files and functions whose names start with test_. We'll make a small change to test_hello.py, placing everything inside a function named test_1:
# Source of test_hello.py
def test_1():
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://localhost:8080/')
body_text = driver.find_element_by_css_selector('body').text
assert body_text == 'Hello World!'
driver.quit()
Now, as long as Pytest is installed, we can simply run
py.test
Our test will run, and we'll get a handy summary of the results:
This test ran on Firefox. Suppose we'd like to test our web application on an operating system and browser that aren't installed locally, such as Windows 8.1 with Internet Explorer 11. For such remote testing with Selenium WebDriver running on a virtual machine, we'll need a Sauce Labs account. We'll need to supply our Sauce Labs username and an access key in order to connect to a Sauce Labs virtual server. We start by changing test_hello.py, replacing driver = webdriver.Firefox() with several lines of code that connect to a remote WebDriver.
# Source of test_hello.py
# Using a remote WebDriver
import os
def test_1():
from selenium import webdriver
desired_capabilities = {
'platform': "Windows 8.1",
'browserName': "internet explorer",
'version': "11.0",
'screenResolution': '1280x1024'
}
sauce_url = 'http://%s:%s@ondemand.saucelabs.com:80/wd/hub' %\
(os.environ['SAUCE_USERNAME'], os.environ['SAUCE_ACCESS_KEY'])
driver = webdriver.Remote(
desired_capabilities=desired_capabilities,
command_executor=sauce_url
)
driver.get('http://localhost:8080/')
body_text = driver.find_element_by_css_selector('body').text
assert body_text == 'Hello World!'
driver.quit()
The code above assumes the Sauce Labs username and access key are stored in environment variables named SAUCE_USERNAME and SAUCE_ACCESS_KEY, respectively.
Once again, run the test with
py.test
The test runs, but doesn't pass:
Why? Because the Selenium WebDriver instance running on a Sauce Labs server can't access http://localhost:8080, which is running on our local machine. The error message provides a hint how to fix this: install Sauce Connect, which we can download here. This example uses Sauce Connect v4.3.14 for Linux. Versions for other operating systems are also available.
Installation on Linux is as simple as unzipping the downloaded file and ensuring that the resulting bin/sc file is on the path and has execute permission. Don't forget that, as above, our Sauce Labs credentials must be stored in the environment variables SAUCE_USERNAME and SAUCE_ACCESS_KEY.
Prior to running the test again, we run sc, which creates a secure tunnel between Sauce Labs' servers and our local machine. The tunnel remains available until we terminate sc (e.g by pressing Ctrl+c). Leaving sc running (either in the background or in a separate terminal window), we run py.test once more. This time, our test should pass.
At this point, we have everything we need to run remote tests of localhost URLs. However, there are some best practices we can implement to make such tests more convenient and reliable. Sauce Labs recommends that a tunnel be established and terminated for each test run, rather than leaving the tunnel open indefinitely. We'll accomplish this in two steps. First, we'll pass command-line arguments and add a few more lines of code to run sc as as daemon (that is, in the background). Second, we'll use a Pytest fixture to ensure the tunnel is ready before any test runs.
The full list of Sauce Connect command-line arguments is available here. The ones we want are:
- -t: Controls which domains are accessed via the tunnel. For improved performance, we'll tell sc to use the tunnel for localhost URLs only.
- --daemonize: Runs sc as a daemon.
- --pidfile: The name of a file to which sc will write its process ID (so we know which process to kill when we're done).
- --readyfile: The name of a file that sc will touch to indicate when the tunnel is ready.
- --tunnel-identifier: Gives our tunnel a name so we can refer to it.
We could place the code to launch sc at the top of function test_1() in test_hello.py, and the code to terminate the tunnel at the bottom of the same function. However, this has a drawback. Presumably, we plan to write many test functions, not just one. We need to tunnel to be created before the first test runs, and terminated after the last test finishes -- regardless of the order in which the tests might be executed. Pytest provides a powerful feature known as fixtures that can accomplish this.
We could create a module-scoped fixture that applies to every test function in our test_hello.py file. But we can do even better. Let's create a session-scoped fixture. By defining our fixture in a specially-named file, conftest,py, and using a decorator to give the fixture session scope, the fixture will automatically apply to every test function in every test file we choose to create. The comments in the source code below provide more details.
# Source of conftest.py
import os
import pytest
import signal
import subprocess
import time
@pytest.fixture(scope="session") # Session scope makes the fixture apply to any test function in any test file
def tunnel(request):
sc_pid_file_name = '/tmp/sc_pid.txt' # File where sc will store its PID
sc_ready_file_name = '/tmp/sc_ready.txt' # File sc will touch when the tunnel is ready
sc_pid = None
def fin(): # Function that executes when the last test using the fixture goes out of scope
if sc_pid:
os.kill(int(sc_pid), signal.SIGTERM) # Kill sc's process, terminating the tunnel
try:
os.remove(sc_ready_file_name)
except OSError:
pass
# Sauce Connect reads credentials from environment variables SAUCE_USERNAME and SAUCE_ACCESS_KEY
subprocess.call([
'sc',
'-t', 'localhost', # Use tunnel for localhost URLs only
'--readyfile', sc_ready_file_name, # Name of the "ready" file
'--tunnel-identifier', 'my_tunnel', # Name of the tunnel
'--daemonize', '--pidfile', sc_pid_file_name # Run as daemon; store PID in specified file
])
with open(sc_pid_file_name) as sc_pid_file:
sc_pid = sc_pid_file.read() # Read the PID
request.addfinalizer(fin) # Register the finalizer function
# Wait for the tunnel to be ready
start_time = time.time()
while True:
if os.path.exists(sc_ready_file_name):
break
if time.time() - start_time > 30:
raise Exception('Timed out waiting for Sauce Connect')
Now we just need to make two small changes to test_hello.py, passing the name of the fixture function as an argument to the test function, and including the tunnel identifier in the desired_capabilities dictionary that controls Selenium WebDriver's behavior.
# Source of test_hello.py # Using a remote WebDriver
# Using a Pytest fixture
import os
def test_1(tunnel):
from selenium import webdriver
desired_capabilities = {
'platform': "Windows 8.1",
'browserName': "internet explorer",
'version': "11.0",
'screenResolution': '1280x1024',
'tunnelIdentifier': 'my_tunnel'
}
sauce_url = 'http://%s:%s@ondemand.saucelabs.com:80/wd/hub' %\
(os.environ['SAUCE_USERNAME'], os.environ['SAUCE_ACCESS_KEY'])
driver = webdriver.Remote(
desired_capabilities=desired_capabilities,
command_executor=sauce_url
)
driver.get('http://localhost:8080/')
body_text = driver.find_element_by_css_selector('body').text
assert body_text == 'Hello World!'
driver.quit()
That's it! We now have a framework for running any number of test functions, with a tunnel automatically created before the first test, and terminated after the last test.
