Automated testing with CucumberJS with Selenium Webdriver using JavaScript

I’m currently picking up something new at work. And I guess the title of the post gives it away. It’s not a brand new thing altogether, just something that I haven’t tried out before. And in one of the blogs I stumbled upon, there was a line that captured what I wanted to do in this post (and I guess in a lot of other previous posts too): helping me remember what I figure[d] out. Anyways, this is just a gist of the basic setup. Much of the learning is in the actual scripting, and most of it is when things don’t work as expected.

First things first: Setting up

  • Download and install Node from https://nodejs.org/en/download. This will also install the Node Package Manager (NPM).
  • Download the drivers into a folder in your local.
  • Update your environment variables to include the drivers into your PATH variable.
    • Add SELENIUM_DRIVERS as a system variable and specify the folder where you placed the drivers
    • Update the PATH system variable by appending “;%SELENIUM_DRIVERS%;”
  • We used a starter found here. Download the files from there, and that’ll give us stuff that I have no idea how to write from scratch at this point:
    • package.json
    • Gruntfile.js
    • features/support folder contains world.js, env.js and hooks.js
  • Install the needed dependencies.
    • Dependencies like CucumberJS, Chai, Selenium Webdriver, etc. are indicated in the package.json file.
    • Open the Command Prompt, then go to the folder where the starter files are placed. Run “npm install”.

Writing our Feature File and Step Definitions

Since we’re using CucumberJS, we’ll write up a feature file in Gherkin format. This means that scenarios that we’ll be testing for are written in a Given-When-Then format. Given includes the needed setup for the test. When refer to actions taken from the initial state (as setup in the Givens). And Then includes the expected outcomes that will be checked for.

For each Given, When and Then, corresponding code has to be written to actually perform the automated steps. These steps are stored in step definition files stored under features/step_definitions.

Feature: Searching for cucumbers
  As an internet user
  In order to find out more about cucumbers
  I want to be able to search for information about cucumbers

  Scenario: Google cucumber search
    Given I go to the Google website
    When I search Google for "cucumbers"
    Then I should see some results
'use strict';

var expect = require('chai').expect;

module.exports = function() {
  this.World = require('../support/world.js').World;

  this.Given('I go to the Google website', function (next) {
    this.browser.get('http://www.google.com').then(function () {
      next();
    });
  });

  this.When('I search Google for "$searchQuery"', function (searchQuery, next) {

    this.browser.findElement(this.webdriver.By.css('input[name="q"]')).sendKeys(searchQuery);
    this.browser.findElement(this.webdriver.By.css('input[name="q"]')).sendKeys(this.webdriver.Key.ENTER)
    .then(function () {
      next();
    });
  });

  this.Then('I should see some results', function (next) {
    this.waitFor('div.g', 10000);
    this.browser.findElements(this.webdriver.By.css('div.g')).then(function (elements) {
      expect(elements.length).to.not.equal(0);
      next();
    });
  });
};

Okay, actually writing our step definitions

To interact with the web site under test, we use Selenium Webdriver. While working on a POC, we had to look for elements within the page under test, and then interact with those elements by either triggering an action like clicking or sending keys or we also interact by extracting information about the element I found. For the checks in the Then steps, we use Chai to do the comparisons. For the POC at least, I pretty much managed to get by with simple scenarios using the items below.


//Finding Elements with Selenium Webdriver
//Find Element by ID
this.browser.findElement(this.webdriver.By.id('sp-search-input'))

//Find Element by CSS
this.browser.findElement(this.webdriver.By.css('#sp-search-input'))
this.browser.findElement(this.webdriver.By.css('.ui-chip'))
this.browser.findElement(this.webdriver.By.css('.ui-chip[data-val="hello"]'))

//Interacting with Elements with Selenium Webdriver
this.browser.findElement(this.webdriver.By.id('sp-search-input')).sendKeys(searchText+"\n");
this.browser.findElement(this.webdriver.By.id('sp-add-widget')).click();
this.browser.findElement(this.webdriver.By.css('.ui-chip>span')).getText();
//looks for element with class "ui-chip" and then looks for a <span> and gets the text in that <span>

this.browser.findElement(this.webdriver.By.id('sp-search-input')).getAttribute('placeholder');
//e.g., <input id="sp-search-input" placeholder="Add Search Tags" …>
//above will return "Add Search Tags"

this.browser.isElementPresent(this.webdriver.By.css('.ui-toggle.checked'));
//returns boolean value

//Comparisons using Chai
expect(actualText).to.equal(expectedText);
expect(textFound).to.contain(expectedString);
expect(toggleState).to.be.ok;

Running our scripts

Using the Command Prompt, go to your test project folder, then run “cucumber-js”. There’s also an option to run it with other options. I find the tags to be most helpful. What I do is I tag the scenarios that I’m working on in the feature file. The format of the tag is “@tagname” in the line before the scenario you are tagging.

cucumber-js --tags @dev               //tagged with @dev
cucumber-js --tags ~@dev              //NOT tagged with @dev
cucumber-js --tags @foo,@bar          //tagged with @foo OR bar
cucumber-js --tags @foo --tags @bar   //tagged with @foo AND bar

Well, that’s about it. Again, the learning really comes when you’re actually scripting. Sometimes, you just need to dive in knowing you don’t know everything, and hopefully you’ll figure it out as you go along.

Summary of references

Example, Starter – https://github.com/Matt-B/cucumber-js-selenium-webdriver-example
CSS Selectors – http://www.w3schools.com/cssref/css_selectors.asp
Chai – http://chaijs.com/api/bdd/
Selenium Webdriver – http://www.seleniumhq.org/docs/03_webdriver.jsp
CucumberJS – https://github.com/cucumber/cucumber-js
Gherkin – https://github.com/cucumber/cucumber/wiki/Gherkin
Cucumber – http://cukes.info
Cucumber tags – https://github.com/cucumber/cucumber/wiki/Tags
Blog post – http://www.tcias.co.uk/blog/2014/09/03/writing-our-first-functional-test/ (2014 Sep 3)
Blog post – http://transitioning.to/2012/01/cucumber-and-js-getting-started-with-cucumber-js/ (2012 Jan 27)

Web accessibility, testers’ start for building it in

So I haven’t had any first-hand working experience on web accessibility testing. I’ve read about it in passing, and once I attended this weekend testing session that tackled the subject. In my previous company where I did a lot of functional web testing, I recall some standard test cases that were related to usability or accessibility. This week though, the topics of accessibility testing and WCAG 2.0 came up, and I had to google a bit on the subjects.

One of the interesting things I found is this blog post on how expensive is accessibility. And what the writer, Karl Groves, said makes sense — that if it’s already part of how you do things, then it comes at no extra cost. And inversely, if it’s something you haven’t factored in at all, then pushing for accessibility would require changes and those changes would undeniably entail some cost.

Now I googled WCAG 2.0 and found really looong references. There’s a quick reference page, but seriously, that page felt anything but quick. In another post by the same person:

What if I told you that the WCAG 2.0 recommendation by the W3C is 36 pages, printed? In addition, “How to Meet WCAG 2.0” is 44 pages and “Understanding WCAG 2.0” 230 pages. Not only that, but the accompanying Techniques and Failures for WCAG 2.0 is 780 pages, printed.

That’s a bit daunting!

But thankfully, he also shared a post on the 6 simplest web accessibility tests anyone can do. And the Web Accessibility Initiative (WAI) has a reference on easy checks for web accessibility. These could be a start and could be things we start looking for in our web testing projects even though we weren’t really asked to do accessibility testing. It’s a chance to add value. Let’s check these out:

 

Reads: Creating a Modern Mentoring Culture by Randy Emelo

I finished a couple of books last week, though I’m not sure if the 20-page “infoline” counts. So anyways, one is this infoline I read on “Creating a Modern Mentoring Culture” by Randy Emelo over at Books24x7. Here are just some of the stuff that I’ve highlighted for myself and for sharing:

Modern mentoring is connecting people across an organization to share critical knowledge and skills. Everyone has something to learn and something to teach, regardless of age or title, and people can be both mentees and mentors at the same time.

Key Pillars of Modern Mentoring

  • Open and Egalitarian
    • everyone has something to learn and something to teach
  • Diverse
    • different perspectives within mentoring communities and relationships help novel ideas and approaches arise in answer to organizational problems or issues people are facing
  • Safe and Judgment-Free
    • people don’t want to show perceived weaknesses by asking for a mentor
  • Independent and Autonomous
    • no need to try to control the amount of time people spend engaged in mentoring, the topics they connect around, or the people with whom they connect.
    • Too much rigid control will only create unwanted barriers to knowledge flowing from those who possess it to those who seek it.
    • Once you have created an enabling structure for modern mentoring, let your employees take the reins of their own learning.
  • Asynchronous
    • technology-enabled communication (email, online communities of interest, business social networks, mentoring and social learning software) is only on the rise and is a key enabling structure that supports modern mentoring
  • Self-Directed and Personal
    • Self-directed learning also allows individuals to learn what is applicable to them right now, gain skills that can help them with their unique work context, and make them more productive.
  • Technology-Centric
    • means to connect with others and a space to collaborate and communicate
  • Flexible
    • allowed and encouraged to shift in and out of your mentoring program and of the mentee-mentor roles themselves, as learning needs and knowledge strengths evolve

If the open nature of modern mentoring is compromised by too much organizational involvement, the quality of mentoring connections and the caliber of learning that takes place as a result of these connections will be degraded.

Creating a Modern Mentoring Culture

  • Re-Educate Leaders
    • need to help organizational stakeholders understand the expanded and broad vision of modern mentoring and its associated benefits
    • must be re-educated to understand that modern mentoring is a productive activity that won’t detract from employees’ effectiveness, but rather will help to strengthen it.
  • Get the word out
    • webinars or e-briefings, various media (podcasts, webinars, or newsletters), brief “commercials” at other training events
    • Sponsor roadshows or lunch-and-learns where mentoring participants share their experiences. Offering a venue for mentoring participants to meet and mingle can help energize your program and provides another opportunity for people to network and make learning connections.
    • Leverage employee resource groups, town hall meetings where a brief presentation could be followed by a question and answer session, Leverage your program’s evangelists.
  • Modernize Current Mentoring Programs
    • expanding your current mentoring programs and making them modern
    • Onboarding – new hires
    • High-potential development
      • brightest talent pull from an array of mentors and knowledge resources [instead of just one mentor]
      • allow high-potentials to be mentors themselves and share their knowledge with others while concurrently learning how to be a leader
    • Augment your formal training initiatives with mentoring cohorts
      • alumni of training programs mentor and advise a group of people currently going through training
      • Peers going through the same training can also connect and share stories around application of concepts learned in class to help cement the newly attained knowledge.
  • Amplify Using Technology
    • Let employees use technologies you have available to communicate and collaborate.
    • Make online employee directories or other skill profiles available to help participants see who would be a good mentoring connection.
    • Allow people to join your mentoring program at any time.
    • Acknowledge the efforts of those in the program.

On retrospectives

When things slip, a typical reaction is to add processes especially for tracking and monitoring. Just take the overhead effort for effort tracking for example. I’ve been in a lot of projects that at one point or another have gotten so delayed. A knee-jerk reaction is to have the team prepare more reports — daily status updates from everyone, summarized daily progress reports from the leads, reports for upper management, reports for client, etc, etc. This has its benefits — it provides more visibility on what’s going on in the project, and it can pacify stakeholders since it gives the impression that you’re on top of things. What it doesn’t do is get more of the actual needed work done. It doesn’t get more work coded. It doesn’t get more tests executed.

dilbert-19950217

Instead of just diving into the “more reports” bandwagon, what I’d like is for the team to look into how we’re currently doing. What’s working for us? What isn’t? What do we need to do differently? These are questions that usually get asked in a retrospective meeting. If we google “retrospective”, a definition that turns up is “looking back on or dealing with past events or situations”. Reflecting on what has happened is but a part of it. In retrospectives, you also try to identify what you need to bring forward to keep your project successful or to recover so that your project will (hopefully) be successful.

I think I can pretty much go ahead and do a little retrospective on my own. We all can do it individually. But that just won’t suffice. I can plan all I want but if the folks who will execute are not aligned with the plan, then the plan will just fail to materialize. And you can come up with all brilliant sorts of ideas and workarounds but if you can’t get the other guy to execute then it wouldn’t matter so much (but I’ll credit your brilliance, of course). Retrospectives need to be a team thing. WE need to come up with plans for improvement that WE are ALL willing to execute or follow through. In the end, there is no one else to drive the project’s success but US as a team.

Slideshare: 26 Time Management Hacks I Wish I’d Known at 20 by @egarbugli

I stumbled upon this deck again and it’s really something that I want to share to the younger ones. Time management wasn’t exactly something I learned when I was starting out. I came from projects where OT would become the norm at certain points, and we even had Saturday work. I was so time-poor. As I grew older, I came to realize how valuable my time is. How I could make up for losses for some things, but I can never get back time I have lost or wasted. And that the more efficiently I manage my time will allow me to spend it on things that matter more.

Saved myself some time by not writing my own presentation covering the same topic.

One of the points I like is #18 which had a quote from Jason Cohen (@asmartbear):

Only ever work on the thing that will have the biggest impact.

I think most especially for the younger ones, we often get sidetracked by initiatives or other non-project related tasks. We fill up our plate with a lot of things. We say “yes” to this and that. But then you have to think about it, step back and look at the big picture, and reflect whether the things that you are doing are really the things that you need to grow or achieve your goals. As an aspiring tester/technologist, are these tasks really relevant to making myself more technical and capable in my craft?

On being overwhelmed (an open letter to the new guys)

Hi, guys…

We’re three weeks in our current project. I know it’s still a period of adjustment for you. You have a lot to take in, and I’ve been asking you to do stuff that is new to you. I know things can be a bit difficult at first. You might feel overwhelmed. You might just want to curl up into a fetal position, hug your favorite stuff toy, and just wish everything away. I know, I’ve been there. And I’m here to tell you to not despair.

Sometimes it’s the fear of having to do something big that freezes us into inaction. We end up not accomplishing anything because our minds are too caught up at the scope of what we have to accomplish.

It is not because things are difficult that we do not dare, it is because we do not dare that they are difficult. ~ Seneca

When you feel like there’s too much in your plate, here’s what you can do. Take a step back, breathe in, breathe out. Don’t try to tackle everything at once. That’ll just be crazy. Find out what you need to prioritize at the moment, then focus on those. Try to break up the work into smaller bite-sized portions. Taking it one step at a time will make it less stressful and less daunting for you. Sometimes it’s just hard to get the ball to start rolling, but once kicked off, things get easier as you go along.

When you feel like you’re sinking, do not hesitate to call out for help. You must remember that you are not alone in this project. Your team mates are here, we’ve got our senior test automation engineer (Sr. TAE), and I am here (I’m not just a pretty face, you know). We also have support from our team leads and our manager. Don’t take asking for help as a sign of defeat or something that will be taken against you. That may be the case in other cultures or other teams, but I assure you that it’s not the case with me. But don’t take this as a cue that you can just ask for help anytime and every time. What I’d want is for you to learn how to help yourself first. And if you come to the point when you’re already doing your best and things are still not working out, that is when you reach out.

So there. I hope you don’t feel too overwhelmed. Know that you can rise above that feeling and that you have my support.

In the end, everything will be okay. If it’s not okay, it’s not yet the end. ~ Fernando Sabino, translated from Portuguese

Cheers,
KC

Basic QTP Training: Enhancing our script

The past couple of days of our QTP training covered a lot more ground. The most nifty stuff are on tools like the Object Repository, the Object Spy (which is an icon of a guy with a top hat reminding me of Professor Layton), the Active Screen, and the topics on parameterization. I created a short video covering the last 2 topics. So there I demonstrate two things:

  • Updating the script with the help of Active Screen — instead of having to record all over again
  • Updating the script to make use of parameters instead of the hard-coded values.

Basic QTP Training: Creating our first script

For our Application Under Test (AUT), we made use of the sample application called “Flight” which was bundled with QTP.  We were given the test script that we had to automate. Essentially, the script involved:
1. Logging in
2. Creating an Order
3. Faxing an Order
4. Logging out

To accomplish this, here are the basic steps:
(1) Create a new test by selecting File -> New -> Test from the menu (or CTRL+N).
(2) Update the Record and Run Settings by selecting Automation -> Record and Run Settings… from the menu. We needed to add our AUT in the list of Applications.
(3) Create the script — including steps, checkpoints and output values.

You can try recording the steps and it will be able to produce a script.  However, playing it back will fail due to some hard-coded items that are no longer applicable upon rerunning (e.g., window title included the Order Number which would be different in the next run). It also wouldn’t have the needed checkpoints (or the test/assertions) and it wouldn’t gather output data.

For the first script created, QTP’s record function was used to record the steps. Inserting checkpoints and output values were also done during the recording. For instances wherein the object title or text being checked is dynamic (e.g., window title containing the Order Number), we made use of regular expressions e.g., instead of “Flight No. 19” as originally recorded, we made use of “Flight No.*” with the regular expressions checkbox marked.

Here’s a link to a video recording of the script creation: http://youtu.be/DwayCg1M5IY

Alternatively, here’s a similar script creation video without step 3 though, but it does have audio. It covers basic recording and inserting checkpoints and an output value.

Basic QTP Training: Setup

Earlier today was Day 1 of a 5-day Basic QTP Training that I’m attending at work.  Putting my notes here just in case I have to relearn or I have to echo the training. So initially, we did some setup (see details below), and then there was a demo where a manual test case was automated (will post this separately).

Install HP QuickTest Professional

This involved an installer that was nearly 4GB in size. The setup with the wizard was pretty straightforward. After installing, additional configuration included specifying the license and going to Tools -> Options to modify the Run settings such that the “Allow other HP products to run tests and components” checkbox is marked. This is probably since we’re integrating with HP ALM.

Set up ALM integration

QTP can manage to work on its own, but since we plan to be working with ALM for storing and managing the tests we had to install the following items:

  • ALM Explorer (ALMExplorerAddIn.msi, alternative to viewing ALM in IE)
  • HP Quality Center Connectivity (TDConnect.exe, an ALM add-in)
  • QTP for Business Process Testing (Quick-Test_Add-in_for_ALM-QC.msi)

5th World Quality Report

An interesting find… Capgemini, Sogeti and HP co-sponsored the 5th edition of the World Quality Report.

It examines the current state of application quality and testing practices across the globe, as well as emerging trends that may affect the future of testing within the next two years.

The report is available at http://www.hp.com/go/capgemini or at http://www.worldqualityreport.com/.

On page 9 of the 64-page report, it lists the following recommendations:

  1. Improve reporting of business-oriented metrics to demonstrate the value of QA
  2. Innovate and optimize QA operations for return on investment
  3. Establish a specialized function for mobile testing
  4. Make greater use of the cloud for testing
  5. Establish test environment and test data management practices
  6. Integrate testing into agile projects