--
EricKessler - 18 Sep 2020
A Tale of Two Tests
From zero to test in multiple languages
This guide will walk through the process of setting up an automated browser test, starting from an empty project and going to a finished and passing test. Each stage of the project will be split into sections for each language so that you can see how each language achieves the same goal in its own way.
Prerequisites
- Java installed on your machine
- Ruby installed on your machine
- An IDE of your choice (optional) or other text editing program
- A command terminal (your OS shiould have one)
- A browser (your OS should really have one)
Project Setup
To get our project ready we will need a suitable project structure, a tool to manage our various code dependencies, and a tool to help build and run the project. Commands will be run in a terminal and it is recommended that you start in a separate empty directory for each language.
Java
For Java, our dependency manager and build tool are one and the same:
Maven. We will even be using Maven to generate our initial project structure. First,
download and
install Maven. Next, run the command
mvn -B archetype:generate -DgroupId=co.ultranauts.app -DartifactId=ultra_demo -DarchetypeGroupId=io.cucumber -DarchetypeArtifactId=cucumber-archetype -DarchetypeVersion=6.6.1
This should result in a new project being created that is shaped something like this
ultra_demo/
├─ src/
├─ test/
├─ java/
|├─ co/
| ├─ ultranauts/
| ├─ app/
| ├─ RunCucumberTest.java
| ├─ StepDefinitions.java
├─ resources/
├─ co/
├─ ultranauts/
├─ app/
├─ .gitkeep
pom.xml
Because we used the 'cucumber-archetype', Maven has created all of the folders and files needed to minimally run the project. 'RunCucumberTest.java' and 'StepDefinitions.java' are used by Cucumber when executing tests and 'pom.xml' is used to keep track of the project's dependencies.
RunCucumberTest.java
package co.ultranauts.app;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;
@RunWith(Cucumber.class)
@CucumberOptions(plugin = {"pretty"})
public class RunCucumberTest {
}
StepDefinitions.java
package co.ultranauts.app;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import static org.junit.Assert.*;
public class StepDefinitions {
}
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>co.ultranauts.app</groupId>
<artifactId>ultra_demo</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<properties>
<cucumber.version>6.6.1</cucumber.version>
<junit.version>4.13</junit.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>${cucumber.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>${cucumber.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<encoding>UTF-8</encoding>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
Ruby
For Ruby, each tool that we need is its own separate code library (we will be using
Bundler and
Rake for our dependency managment and "build" script, respectively) and there is far less "auto-magic" setup done for us. First, we need to create the root folder for our project with
mkdir ultra_demo
Next, we need to install Bundler with
gem install bundler
and then go into the 'ultra_demo' directory and run
bundle init
This will create a 'Gemfile' which is what Bundler uses to keep track of needed dependencies. Inside of that file, add dependencies for the 'rake', 'cucumber', and 'rspec-expectations' gems that will be used for our build tool, test framework, and assertions, respectively. Your final file should look something like this
Gemfile
# frozen_string_literal: true
source "https://rubygems.org"
git_source(:github){|repo_name| "https://github.com/#{repo_name}"}
gem 'cucumber'
gem 'rake'
gem 'rspec-expectations'
Run the following two commands to install the two new dependencies and generate the folders used by Cucumber.
bundle install
cucumber --init
This will have created an empty 'env.rb' file and some additional folders such that your final project structure looks something like this (except for the Rakefile, which we'll make in a moment)
ultra_demo/
├─ features/
|├─ step_definitions/
|├─ support/
| ├─ env.rb
├─ Gemfile
├─ Gemfile.lock
├─ Rakefile
Rake reads its build tasks from a file called 'Rakefile'. Create that file and add a task for running Cucumber.
Rakefile
require 'cucumber'
require 'cucumber/rake/task'
Cucumber::Rake::Task.new(:features) do |t|
t.cucumber_opts = "--format pretty"
end
task :default => :features
Running Tests
At this point, you should be able to run your (non-existant) tests.
There's not much to see, it may not be in color (depending on what kind of terminal you are using), and, for newer versions of Cucumber, there is a big message about the new publish feature. Let's take care of all of that.
Java
In the 'src/test/resources/' folder, add a 'cucumber.properties' file with the publish notice turned off.
cucumber.properties
cucumber.publish.quiet=true
Ruby
In the root project folder, add a 'cucumber.yml' file with the publish notice turned off and color turned on.
cucumber.yml
default: --publish-quiet --color
Run the tests again and the output should be nicer to look at this time around.
Writing the Tests
Because Cucumber features are written in Gherkin, they are the same no matter what programming language is used to implement their execution. Add the following 'search.feature' file to the 'src/test/resources/co/ultranauts/app' folder for Java and to the 'features' folder for Ruby.
search.feature
Feature: Search
In order to learn All The Things
As a curious person
I want to search the internet for information
Scenario: Search for something
Given I am on the search page of Google
When I search for "chocolate milk"
Then I am shown results related to my search
Running the tests again, you should now see that Cucumber did find and execute the feature but its steps were not defined. Helpfully, Cucumber provides code snippets for steps that are missing so that we can easily define them.
Java
Add the following methods to the 'StepDefinitions' class
StepDefintions.java
// Other code
// …
public class StepDefinitions {
@Given("I am on the search page of Google")
public void i_am_on_the_search_page_of_google() {
// Write code here that turns the phrase above into concrete actions
throw new io.cucumber.java.PendingException();
}
@When("I search for {string}")
public void i_search_for(String string) {
// Write code here that turns the phrase above into concrete actions
throw new io.cucumber.java.PendingException();
}
@Then("I am shown results related to my search")
public void i_am_shown_results_related_to_my_search() {
// Write code here that turns the phrase above into concrete actions
throw new io.cucumber.java.PendingException();
}
}
Ruby
Create a 'steps.rb' file in the 'features/step_definitions' folder and add the following code
steps.rb
Given('I am on the search page of Google') do
pending # Write code here that turns the phrase above into concrete actions
end
When('I search for {string}') do |string|
pending # Write code here that turns the phrase above into concrete actions
end
Then('I am shown results related to my search') do
pending # Write code here that turns the phrase above into concrete actions
end
Now when you run the tests, Cucumber will see that the steps are defined but the tests will be in a "not really passing, not really failing" state called "pending". This is a result of all of those pending commands in the step definitions and they are a convenient way to make sure that the person writing a test doesn't just forget to finish it and thereby cause the test framework work to count it as a passing test due to not having any failing assertions.
Automating the Tests
So now we come to the last major tool that we need: some way to control a web browser.
Selenium is a popular tool for doing this and it is what we will be using for this exercise. Using Selenium requires two things. It it needs a way for Selenium to talk to the browser and it needs a way for our program to talk to Selenium.
In order to talk to a browser, Selenium uses a driver program that is specific to each different browser. Download the driver for whichever browser you want to use for the test.
- Chromedriver (for Chrome)
- Geckodriver (for Firefox)
- Other drivers (Chrome and Firefox drivers tend to be the most up to date/stable, so I recommend using one of those two instead)
Create a new "drivers" folder in the root folder of your project and put the driver file (e.g. "chromedriver.exe") in that folder.
In order to talk to Selenium, our program needs another dependency added.
Java
pom.xml
<dependencies>
<!-- ... -->
<!-- other dependencies -->
<!-- ... -->
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.141.59</version>
</dependency>
</dependencies>
This addition will get picked up the next time that you run "mvn test".
Ruby
Gemfile
# Other code
# ...
gem 'selenium-webdriver'
You will have to run "bundle install" again in order to use this new dependency.
Now we can move on to the test framework itself. Our first goal will be automating the setup and teardown that needs to happen around each test.
- A hook that runs once before all of the tests that will configure Selenium to use the diver file
- A hook that runs before each test that will create a browser instance
- A hook that runs after each test that will get rid of the browser instance
Now is also when
state is going to become important because we will need some way to share information between the different stages of a test, of which these hooks are the first and last.
Java
Because Java is highly class based, the hooks, like the step definitions and tests themselves, will be their own class. This means that they will use seperate objects and thus some work will have to be done in order to share information between one class and another. There are several ways to do this in Java but, for this exercise, we'll be using
dependency injection via
PicoContainer. Unfortunately, the Java implementation of Cucumber does not have suite level hooks that only fire off once at the very beginning and once at the very end and so we'll have to use regular test level hooks with a conditional workaround so that it only triggers once. All together, there are three files to create/update.
pom.xml
<dependencies>
<!-- ... -->
<!-- other dependencies -->
<!-- ... -->
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-picocontainer</artifactId>
<version>6.6.1</version>
<scope>test</scope>
</dependency>
</dependencies>
And the two new class files, both of which will live in the same "app" folder as your other Java files.
World.java
package co.ultranauts.app;
import org.openqa.selenium.WebDriver;
public class World {
WebDriver driver;
}
The world object is what will be used to hold state that is passed from class to class. Currently, we only need to keep track of the instance of the browser driver that will be used during the test.
Hooks.java
package co.ultranauts.app;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
public class Hooks {
private World world;
private static boolean seleniumConfigured = false;
public Hooks(World world) {
this.world = world;
}
// Before All hook
@Before(order = 0)
public void configureSelenium() {
if (!seleniumConfigured) {
seleniumConfigured = true;
String projectPath = System.getProperty("user.dir");
System.setProperty("webdriver.chrome.driver", projectPath + "\\drivers\\chromedriver.exe");
}
}
@Before
public void createBrowserDriver() {
world.driver = new ChromeDriver ();
}
@After
public void closeBrowser() {
world.driver.close();
}
}
The three hook methods manage Selenium configuration and browser setup/teardown and having a constructor that requires an instance of our World class causes
PicoContainer to automatically provide one. Also note that the Selenium configuration hook was given an explicit order priority so that it always triggered first.
Ruby
Like the Java implementation, Ruby does not have explicit "Before All" and "After All" style hooks. However, the desired code can simply be placed in one of the files that will be read before any tests run and the same effect will essentially be achieved. Traditionally, this file is "features/support/env.rb" (which Cucumber prioritizes reading first when it loads) or in whatever file you place your hooks in (but with the one-time 'hook' code being just regular code in the file instead of inside a hook declaration, thus causing it to be executed just the one time when that hook file is loaded in order to create all of the other hooks). For this exercise, we'll go the "env.rb" route, just to get in the habit of using that file for the general environmental setup that it would be used for in larger projects.
Below are the new files that need to be created. Note that no additional framework effort is need in order to store state information like was need in the Java implementation because, in the Ruby implementation of Cucumber, all tests and hooks are executed within the scope of the same test object instance (which is also referred to as the "World" object). This means that we can just use normal instance variables to save information between test stages.
features/support/env.rb
require 'selenium-webdriver'
project_root = "#{__dir__}/../.."
Selenium::WebDriver::Chrome::Service.driver_path = "#{project_root}/drivers/chromedriver.exe"
features/support/hooks.rb
Before do
@driver = Selenium::WebDriver.for :chrome
end
After do
@driver.close
end
If you run the tests again, they will still report as having undefined steps but now you should also see a browser being created and destroyed when the test runs and, if you were to have more than one test (feel free to write additional tests in the project for practice) you would see this happen multiple times.
Adding implementations for the steps is the only thing left to do at this point. The browser interactions that we need to add code for are
- Navigate to Google's search page
- Perform a search
- Verify that search results appear
For longer or more complex tests, it is recommend to implement one step at a time and then rerun the test to ensure that the new step is working as expected but the test in this example is simple enough that we can just implement all three steps at once.
Update the files as shown below
Java
StepDefinitions.java
//Other imports
// …
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.ui.ExpectedCondition;
import org.openqa.selenium.support.ui.WebDriverWait;
public class StepDefinitions {
private World world;
public StepDefinitions (World world) {
this.world = world;
}
@Given("I am on the search page of Google")
public void i_am_on_the_search_page_of_google() {
world.driver.get("https://www.google.com");
}
@When("I search for {string}")
public void i_search_for(String searchTerm) {
//Save the search term for the next step
world.searchTerm = searchTerm;
WebElement element = world.driver.findElement(By.name("q"));
element.sendKeys(searchTerm);
element.submit();
}
@Then("I am shown results related to my search")
public void i_am_shown_search_results() {
new WebDriverWait (world.driver, 5L).withMessage("Expected to have search results for " + world.searchTerm).until(new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver d) {
return d.getTitle().toLowerCase().startsWith(world.searchTerm);
}
});
assertTrue(world.driver.findElements(By.cssSelector("div.g")).size() > 0);
}
}
World.java
public class World {
WebDriver driver;
String searchTerm;
}
Ruby
steps.rb
Given('I am on the search page of Google') do
@driver.get "https://google.com"
end
When('I search for {string}') do |search_term|
@search_term = search_term
element = @driver.find_element(name: "q")
element.send_keys search_term
element.submit
end
Then('I am shown results related to my search') do
wait = Selenium::WebDriver::Wait.new(timeout: 5, message: "Expected to have search results for #{@search_term}")
wait.until { @driver.title.downcase.start_with? @search_term}
expect(@driver.find_elements(css: 'div.g').count).to be > 0
end
Just like we had to use the World object save the browser driver object that we created in the hooks so that the steps could use it, we are saving the search term data that was only defined in the second step so that the third step can also have access to it.
The only other noteworthy thing is the distinction between the two checks in the final step of the test. The first check (waiting for the title bar to change) is a "polling" style check. Many asynchronous tests (which most tests that use a browser are) are written with extra code in order to stabilize an otherwise unpredictable application execution time between a user action and the application response. Although it is verifying that something has happened, it is not the actual verification of the test. Even if both checks were written using assertion style methods, only the second check (asserting a non-zero number of search results) is proving that the applicationbehaved correctly. In other words: The second check is the goal of the test and every other check was just helping us get there more easily.
Success!
And there you have it. Running the tests should now result in all of them passing. Congratulations! Feel free to add a few more tests to the project or even go through the process again using another language in order to see how the project structure and Cucumber framework are different. Every difference found is an opportunity to understand the similarities that each implementation shares.
--
MayaMcKela - 18 Jun 2021 (Added tabs to separate languages)
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
Automation