Automated testing with CucumberJS with Selenium Webdriver using JavaScript

I’m currently picking up something new at work. And I guess the title of the post gives it away. It’s not a brand new thing altogether, just something that I haven’t tried out before. And in one of the blogs I stumbled upon, there was a line that captured what I wanted to do in this post (and I guess in a lot of other previous posts too): helping me remember what I figure[d] out. Anyways, this is just a gist of the basic setup. Much of the learning is in the actual scripting, and most of it is when things don’t work as expected.

First things first: Setting up

  • Download and install Node from https://nodejs.org/en/download. This will also install the Node Package Manager (NPM).
  • Download the drivers into a folder in your local.
  • Update your environment variables to include the drivers into your PATH variable.
    • Add SELENIUM_DRIVERS as a system variable and specify the folder where you placed the drivers
    • Update the PATH system variable by appending “;%SELENIUM_DRIVERS%;”
  • We used a starter found here. Download the files from there, and that’ll give us stuff that I have no idea how to write from scratch at this point:
    • package.json
    • Gruntfile.js
    • features/support folder contains world.js, env.js and hooks.js
  • Install the needed dependencies.
    • Dependencies like CucumberJS, Chai, Selenium Webdriver, etc. are indicated in the package.json file.
    • Open the Command Prompt, then go to the folder where the starter files are placed. Run “npm install”.

Writing our Feature File and Step Definitions

Since we’re using CucumberJS, we’ll write up a feature file in Gherkin format. This means that scenarios that we’ll be testing for are written in a Given-When-Then format. Given includes the needed setup for the test. When refer to actions taken from the initial state (as setup in the Givens). And Then includes the expected outcomes that will be checked for.

For each Given, When and Then, corresponding code has to be written to actually perform the automated steps. These steps are stored in step definition files stored under features/step_definitions.

Feature: Searching for cucumbers
  As an internet user
  In order to find out more about cucumbers
  I want to be able to search for information about cucumbers

  Scenario: Google cucumber search
    Given I go to the Google website
    When I search Google for "cucumbers"
    Then I should see some results
'use strict';

var expect = require('chai').expect;

module.exports = function() {
  this.World = require('../support/world.js').World;

  this.Given('I go to the Google website', function (next) {
    this.browser.get('http://www.google.com').then(function () {
      next();
    });
  });

  this.When('I search Google for "$searchQuery"', function (searchQuery, next) {

    this.browser.findElement(this.webdriver.By.css('input[name="q"]')).sendKeys(searchQuery);
    this.browser.findElement(this.webdriver.By.css('input[name="q"]')).sendKeys(this.webdriver.Key.ENTER)
    .then(function () {
      next();
    });
  });

  this.Then('I should see some results', function (next) {
    this.waitFor('div.g', 10000);
    this.browser.findElements(this.webdriver.By.css('div.g')).then(function (elements) {
      expect(elements.length).to.not.equal(0);
      next();
    });
  });
};

Okay, actually writing our step definitions

To interact with the web site under test, we use Selenium Webdriver. While working on a POC, we had to look for elements within the page under test, and then interact with those elements by either triggering an action like clicking or sending keys or we also interact by extracting information about the element I found. For the checks in the Then steps, we use Chai to do the comparisons. For the POC at least, I pretty much managed to get by with simple scenarios using the items below.


//Finding Elements with Selenium Webdriver
//Find Element by ID
this.browser.findElement(this.webdriver.By.id('sp-search-input'))

//Find Element by CSS
this.browser.findElement(this.webdriver.By.css('#sp-search-input'))
this.browser.findElement(this.webdriver.By.css('.ui-chip'))
this.browser.findElement(this.webdriver.By.css('.ui-chip[data-val="hello"]'))

//Interacting with Elements with Selenium Webdriver
this.browser.findElement(this.webdriver.By.id('sp-search-input')).sendKeys(searchText+"\n");
this.browser.findElement(this.webdriver.By.id('sp-add-widget')).click();
this.browser.findElement(this.webdriver.By.css('.ui-chip>span')).getText();
//looks for element with class "ui-chip" and then looks for a <span> and gets the text in that <span>

this.browser.findElement(this.webdriver.By.id('sp-search-input')).getAttribute('placeholder');
//e.g., <input id="sp-search-input" placeholder="Add Search Tags" …>
//above will return "Add Search Tags"

this.browser.isElementPresent(this.webdriver.By.css('.ui-toggle.checked'));
//returns boolean value

//Comparisons using Chai
expect(actualText).to.equal(expectedText);
expect(textFound).to.contain(expectedString);
expect(toggleState).to.be.ok;

Running our scripts

Using the Command Prompt, go to your test project folder, then run “cucumber-js”. There’s also an option to run it with other options. I find the tags to be most helpful. What I do is I tag the scenarios that I’m working on in the feature file. The format of the tag is “@tagname” in the line before the scenario you are tagging.

cucumber-js --tags @dev               //tagged with @dev
cucumber-js --tags ~@dev              //NOT tagged with @dev
cucumber-js --tags @foo,@bar          //tagged with @foo OR bar
cucumber-js --tags @foo --tags @bar   //tagged with @foo AND bar

Well, that’s about it. Again, the learning really comes when you’re actually scripting. Sometimes, you just need to dive in knowing you don’t know everything, and hopefully you’ll figure it out as you go along.

Summary of references

Example, Starter – https://github.com/Matt-B/cucumber-js-selenium-webdriver-example
CSS Selectors – http://www.w3schools.com/cssref/css_selectors.asp
Chai – http://chaijs.com/api/bdd/
Selenium Webdriver – http://www.seleniumhq.org/docs/03_webdriver.jsp
CucumberJS – https://github.com/cucumber/cucumber-js
Gherkin – https://github.com/cucumber/cucumber/wiki/Gherkin
Cucumber – http://cukes.info
Cucumber tags – https://github.com/cucumber/cucumber/wiki/Tags
Blog post – http://www.tcias.co.uk/blog/2014/09/03/writing-our-first-functional-test/ (2014 Sep 3)
Blog post – http://transitioning.to/2012/01/cucumber-and-js-getting-started-with-cucumber-js/ (2012 Jan 27)

Advertisements

Testing Mindset

At work, I’m helping a colleague in developing a material on ‘testing mindset’ which is to be shared to our fellow testers. Initially, we have this list of one-liners — catchphrases that we hope would stick like: Always ask the question why, Assume the product is broken, Trust but verify, etc.

I tried going over them one evening. And while they do emphasize some qualities that are important for testers — like being persistent, curious, attentive to detail, critical-minded — I felt that it was still lacking. Some qualities that I deem important were missing — like having good communication skills, being a team player, being technically skilled, striving for self-improvement, having pride and ownership of your work/craft. I came up with an 8-point bullet list describing qualities that I would like in my test team, until I cut myself short since I was working time-bound. And as I’ve taken a step back from the one-liners to come up with this incomplete list of desired qualities, I took another step back from these qualities and thought of what is the driving force behind the need for these qualities.

Why must testers have those qualities I’ve been enumerating?  Why must they keep those catchphrases in mind?

In pausing and taking a step back, I realize it’s all for making things better. I guess that’s my personal testing mindset: I try to make things better.

When I started testing, I thought it was my nitpicking skills and slight OC-ness that made me such a good fit for the job. I just loved finding bugs (and still do)! Over time and over many projects, I found that my purpose in the team isn’t just to find bugs. Essentially, it’s to try to make things better. By “things”, I don’t only mean the products under test. But also my working relationship with my team mates, work loads, schedules, processes, the team itself, and even myself.

  • Say, I try to expose the bugs and report them, so that they’d get fix. Product gets better.
  • I try to provide suggestions for improvement like a comment on usability. Product gets better. If fixed, that suggestion could cut short the work that the end-user has to do.
  • I try to report bugs clearly and concisely. Dev’s life and mine are better than it would have been if i had given a vague report. Dev will hopefully be able to replicate the bug so he won’t have to nag me for details.
  • I try to find shortcuts and tools, and share them with the team. Tester’s lives are better. Even just the little things that could help minimize the tedium of some of the tasks we do is something I appreciate.
  • I try to read up, and try to continue learning and improving myself. I (hopefully) get better. (Well, i try.)
  • I try to encourage knowledge sharing among team members. The team (hopefully) gets better.
  • Etc.

Yoda might disapprove, because after all he said “there is no try.” But still, we must persist. The things we do, the qualities we instill in ourselves, our values — these must all drive towards the betterment of our team, our product, and ourselves.

Browsing through 2010

I went through my public 2010 blog posts and summarized what I had written about down in the list below. Not much for this year, not that it’s the quantity of posts that matters. But back in 2008 and 2009, I averaged about 4 entries a month. This has now dwindled to 2 entries a month.

Is this thanks to microblogging? Have I less to gripe, er write, about? Are the iTouch apps too much of a time-sink than I would like to admit? Has having to censor my work due to confidentiality clauses become a deterrent? Have I become too busy with work?

Whatever the reason is, I’ve found 2 entries a month to be somewhat manageable. So my goal for 2011 is to have the discipline to keep at it. Not just the writing, per se; it’s more of the learning. Testing blogs, social networks like twitter and the software testing club, and – more recently – the weekend testing sessions have provided me with invaluable opportunities for learning and exposure to new technologies, techniques, tools, ideas. I reckon I’ll continue tapping on to these resources in the coming year.

 

Continue reading

Learning shouldn’t be a chore

Over the weekend, I met up with my friend Barry (not his real name) the baker (not his real profession). He used to tell me about how he’s not getting any new training at work. I asked him how things were with him, and he was quick to jump onto the topic of training.

“Well, we now have a lot of training lined up for us,” he said in a tone that was far from enthusiastic. “Just not the ones that I was expecting,” he added sullenly.

I asked him to elaborate. He told me he was expecting more technical training. Something along the lines of baking design patterns, introduction to other baking frameworks, new technologies, or essentially training that would help them learn to bake better. What they got instead were mostly soft skills training that had nothing to do with baking altogether.

I asked him a few other questions like do they have a say about the classes they’re assigned, do they have a means to feedback on the value they get from the training, how do they gauge whether the training had been worth it, were the topics something you can just as easily find better if not more concise resources on from the net, what do your peers think about the training, etc.

From his responses, I got reminded about Dan Pink’s talk about motivation being best driven by autonomy, mastery and purpose.

  • Autonomy – getting to choose for yourself the classes that you’d take, or having a say on that matter
  • Mastery – getting trained on areas that you want to improve
  • Purpose – getting training that’s relevant to your you, your craft, your role, your career path; getting some alignment on why you’re getting a particular training especially if it’s something you’re not interested in to begin with

While it was a good move to provide more training, I guess it wouldn’t hurt to look into these three things so that attending those classes wouldn’t be such a chore and so that they won’t be as forgettable either (as in the case of Barry’s co-baker who remembers nothing else but having good coffee on training day).

Help yourself

It’s a bit ironic that I’ve become friends with a fellow tester only when she was about to leave the company. We still get to have dinner occasionally and I get to find out about the training and learning activities her new company offers. For instance, she’s got some training lined up on Perl. They also have book reviews with the costs for the books and for reading them shouldered by their company. I think the last book they’ve read was on scrum which I find interesting since I would like to have some experience with Agile methodologies. Their upcoming book for review is on Agile testing, which made me blurt out, “by Lisa Crispin?” and just left me with my mouth agape. Lucky lucky lucky!

As for where I work, well, the last official training I received was way back in 2007. It wasn’t even testing-related. It was an upgrade training on CMMI version 1.2. Some books on software development and testing were purchased last year (or was it the year before). We don’t have time allowances for reading them though. You can, but it would just have to be on your own time. That’s fine, but only if the workload and schedules given to you afford for you to have your own time.

Anyway, the lack of training offered isn’t — and shouldn’t be — an excuse for turning stale. It sucks if you aren’t provided training to keep you up-to-date. But it will still be your fault if you let that stop you from picking up new things altogether. With social media on the rise, one can turn to blogs and testing networks for materials. Web sites covering Agile methodologies, offering programming lessons, giving testing tips also abound. Online shopping has also made it possible to buy books from overseas, but it’s just too heavy on the wallet.

Unavoidably, there are things that impede or limit us — financial constraints, lack of time, no internet at work, lack of available training (although this one’s expected to change soon), etc. But what is important is that we don’t limit ourselves. We can learn if we want to. We can train ourselves. We can seek mentorship outside the confines of our workplace. The potential to grow is there.

Quite aptly, I received a text message from my friend: Obstacles don’t have to stop you. If you run into a wall, don’t turn around and give up. Figure out how to climb it, go through it, or work around it.

3 stages; shu-ha-ri

Recently, I’ve been putting the waiting time for eclipse/tomcat to start-up and restart to good use by doing some reading (actually, more of skimming).  So far, I’ve been leafing through Alistair Cockburn’s Agile Software Development.  I came across a part of the book on stages of learning that reminded me of the Dreyfus Model which I’ve picked up from Pragmatic Thinking and Learning.

The book describes three stages of behavior of people who are learning or attempting to master new skills:

  • Following – At this level, people simply try to follow and copy whatever works. They only seek out explicit instructions. I suppose the predominant mantra at this stage is: “At least it works.”
  • Detaching – Here, they learn the “limits of the procedure”. They start looking into other ways, and figure out how to make adjustments when things go wrong. I reckon, at this point, they’re no longer bound to explicit instructions and they manage to make workarounds when needed.
  • Fluent – Here, at the third level, they’ve somewhat achieved a zen-like stage wherein they just know the goal and they just know what needs to be done to reach it.

Alternatively, there’s Shu-Ha-Ri, which roughly translates to learn, detach, and transcend.

  • At Shu (godblessme), “the student builds the technical foundation of the art… the student should be working to copy the techniques as taught without modification and without yet attempting to make any effort to understand the rationale of the techniques of the school/teacher.”  Mmm’kay, I do a backswing like so… what do you mean another 30 rounds?!
  • At Ha, “the student must reflect on the meaning and purpose of everything that s/he has learned and thus come to a deeper understanding of the art than pure repetitive practice can allow.”  Oh, right… locking my wrist does make for a better angle upon contact.
  • And lastly, at Ri, “the student is no longer a student in the normal sense, but a practitioner. The practitioner must think originally and develop from background knowledge original thoughts about the art and test them against the reality of his or her background knowledge and conclusions as well as the demands of everyday life.” You want to do some baseline volleys? Sure, no sweat!

Now, what exactly is the relevance of these different stages.  As a learner or a receiver of information, it reflects the type of information that you seek out.  As someone from the other end, it teaches you to be more wary of the needs of someone who’s just learning. As someone who communicates, it teaches you that whatever you said that you think is crystal clear could be any of the following:

  • Unnecessary chatter, esp for someone from a higher level
  • Undecipherable, esp for someone from a lower level
  • Clear enough, but it will never be crystal clear.

One other thing that the chapter expresses is that communication is imperfect.  You can’t do away with its imperfection.  Having an understanding of one of the possible causes of miscommunication (i.e., the different stages) would hopefully at least make you tolerant if not make you strive to bridge the gap.

The toxic mentor

Here’s someone NOT to emulate, along with some stuff he does gives him that discredit.

The toxic mentor…

1.  Expects his mentee to render overtime.

A 10+ workday should NOT be considered as the norm.  Overtimes should only be considered as a last resort, or as something to get into when you’re feeling productive or enjoying what you’re working on.  And as uncle bob martin (no actual relation) tweeted, the tenet of professionalism is to work 40 hours for your employer, and spend 20 hours on improving yourself and increasing your own value.  Now, you can’t do that if you’re slaving away and putting in all those 60 hours at work.

Excessive overtime also tends to burn people out, or it can make people slack off.  One can already do so much (especially if uninterrupted) during the normal 8 hours of work.  What you should do is push yourself to be more focused and productive in those 8 hours.

2.  Thinks UAT without overtime is impossible.

That’s learned helplessness in action. Don’t try to poison young minds.  And if you think it’s impossible, then you wouldn’t try to attain an OT-less UAT. Management ought to provide an incentive for teams to not be on OT’s or ON’s during UAT. Maybe that would push the teams to improve the quality of their work during CUT and System Test phases where the problems are less costly to fix.

3.  Undermines his mentee’s estimates.

It would be best for the one who’d actually work on the task to provide the estimates. Don’t jump the gun and tell your mentee that the task can be done in just 4 hours when you haven’t even picked up on the requirements of the task. That will just pressure your mentee to do the work within the time you stipulated at the risk of quality.

time_to_do_things_right

4.  Causes delay and does not bother to apologize.

Don’t be too proud to say you’re sorry when you’ve inconvenienced your mentee. Just because you’re the mentor and she’s the mentee doesn’t mean she’s not entitled to the same respect and courtesy.

5.  Keeps making promises he can’t keep.

Stick to your word. Keep on breaking it and your mentee will learn to distrust you or, worse, respect you less.

Hello SQ3R

In reading Pragmatic Thinking & Learning, I came across a reading technique called SQ3R.

Survey – scan the table of contents and chapter summaries for an overview
Question – note any questions you have; to be answered as you read
Read – uh, read
Recite – summarize, take notes, put it in your own words
Review – Reread, expand notes, discuss with colleagues

With regard to “Recite”, it is one of the things that help etch whatever it is I’ve read into my brain.  Instead of just shoving in the info into your brain, it requires some more effort to actually process this info to produce your own output. It makes the learning process into a more active experience.

Continue reading

Sheep dip training doesn’t work

Between one of my best buds and I, some of the parts below on learning is already nothing new to us. But I appreciate its being mentioned in the book. It kinda validates our perceptions on learning i.e., you’ve got to be willing to learn and no matter how hard anyone force-feeds information to you, if you yourself aren’t willing and taking up measures to learn, then nothing will happen.

Excerpts from Chapter 6: Learning Deliberately of Pragmatic Thinking & Learning

Education comes from the Latin word educare, which literally means “led out,” in the sense of being drawn forth… we don’t generally think of education in that sense… Instead, it’s far more common to see education treated as something that’s done to the learner as something that’s poured in, not drawn out.

Sheep dip training – You line up unsuspecting employees, dunk them in an intensive three-to-five-day event in an alien environment devoid of any connection to their day-to-day world, and then proclaim them to be Java developers, .Net developers, or what have you. It wears off, of course, so next year you need to have a “refresher” course another dip.

Sheep dip training doesn’t work because

  • Learning isn’t done to you; it’s something you do.
  • Mastering knowledge alone, without experience, isn’t effective.
  • A random approach, without goals and feedback, tends to give random results.

A single, intense, out-of-context classroom event can only get you started in the right direction, at best. You need continuing goals, you need to get feedback to understand your progress, and you need to approach the whole thing far more deliberately than a once-a-year course in a stuffy classroom.