Capturing timestamps

An often unconsidered scenario here is when testing saving of records are cases wherein you try to save a record that had already been modified by another user. Or when trying to view a record that is no longer available. Often folks are focused on the CRUD of the screens they are testing, and forget that it’s possible for multiple users to update the same record (or it’s also possible that the single user updates the same record using multiple browsers).

In most of the web systems I tested, they are able to better handle such cases through the use of timestamps. There are fields capturing info like who created the record, when the record was created, and more importantly, who last updated the record and when the record was last updated.

Just to illustrate… So when you load the record on screen, the last update information is also retrieved (let’s say 10/13/2016 3:40 PM). Just in case, the system or another user updates the same record, that update would change the last update information. Let’s say that happens, and the last update info becomes 10/13/2016 3:45PM. That means you’re no longer viewing the latest version because what you’ve got is the 3:40PM version. So when you save, that should throw an error saying the record has been modified by another user or that you should refresh.

So aside from testing the cases wherein you consider multiple users viewing and updating the same record, you also need to check whether the timestamp fields are indeed updated accordingly.

READ MORE:

  • You can try googling for timestamping.  A related item is on locking (oft-missed).

Back-end and Front-end validations

When it comes to validations, it’s safer to handle it both in the front-end AND also the back-end. You can’t just rely on front-end validation because the request that gets sent to the server can be tampered/modified. Typically, you have the same set of validations implemented in both the back-end and the front-end. We typically test via the app/system’s user interface only so we don’t get to directly test the back-end validations… most of the time, the front-end has already filtered our inputs / validated such that by the time the request is sent to the back-end, more or less, the data should also pass back-end validations.

Case 1

The case below shows how the back-end validation captured the comparison between the start and end dates. Although, there shouldn’t have been such a comparison to begin with because the end date field was supposed to be hidden, but the front-end had passed on the incorrect information.

Just an elaboration of the case below:

What we see on screen

What we expect

What actually happened

  1. User enters the following for experience:
    • start date Apr 2016
    • end date Feb 2016
  2. User checks box to indicate he’s currently working there
    • That hides the end date
  3. User proceeds with saving the profile

Saving should be successful.

Start date = Apr 2016

isCurrent = 1 (yes)

Error on saving occurs.

This is because the information on end date is still sent. Back-end validation fails because Apr 2016 > Feb 2016.

Case 2 [10/28/2016]

Here’s another case wherein the back-end validation got triggered because the front-end validations didn’t get triggered. Our application works with migrated data from an existing system. So what we tried was to open a migrated record and save it using our edit form. Normally, front-end validations would have prevented a particular field from being null, but since we didn’t use the front-end ui to create/edit the record that field ended up being null. On save, the error was still captured thanks to back-end validation.

READ MORE:

Some basic stuff

I noticed that last month I wrote one note a week related to some issue encountered or a test case that needed to be considered. They seem a bit basic but then I realized they didn’t really teach that to us in school, and I don’t exactly recall having this in the onboarding at work. So basic as they may seem, I’ll go right ahead and post them. I don’t have (m)any readers anyway so I doubt anyone will be complaining. I’ll tag them as #fundamentals

Bug Magnet Chrome Extension

I just remembered that I have this Chrome Extension installed called Bug Magnet. It comes in pretty handy while exploratory testing to try to see if the app breaks when different inputs are entered. It allows you to enter various inputs — could be valid, invalid, with special characters, super long text, etc. So just in case you get asked to very, very quickly try out a new application / website that allows user inputs, then this Chrome Extension could also help refresh your memory for cases/inputs to try out.

20170222-184501-register-supercar-showdown

July 2016 Meetup of Software Testing Philippines

I attended a local software testing meetup yesterday. Typically, I’d only join if the venue was near work, but I had the day off so stressing out over getting to the venue wasn’t an issue. What also lured me in is that there would be a talk on chartered exploratory testing and mind mapping. It’s not the first time I’ve ever heard of those things, but to hear about them in a local context was something that got me interested.

Actually, hearing things in a local context is why I generally bother to attend these meetups. You can google the concepts, you can google for tools, but you can’t google how other peers are doing software testing locally.

So the meetup had 2 parts — 3 if you count the part where we got fed free pizza and soft drinks. There was the talk on Collaborative Test Design by Ian Pestelos (I’ll link to the slides if the deck gets shared), and there was an Open Space discussion. Since “Open Space” was in title-case, I went ahead and googled it and found this (from Martin Fowler’s site) and this (how to run an Open Space event). An interesting take-away from googling about Open Space is The Law of Two Feet:

If, during the course of the gathering, any person finds themselves in a situation where they are neither learning nor contributing, they must go to some more productive place.

Overall, I enjoyed Ian’s talk, and I wouldn’t have second thoughts about recommending it to peers. The Open Space discussion, not my favorite thing. Would I go to one of these meetups again? Sure, if there’d be talks on topics I find interesting and if my schedule permits.

(Just some notes after the read-more)

Continue reading

Not every new thing is an innovation

My friend Alice shared that she conducted a talk and shared that they were using so-and-so tool in their project, whereas typically projects use this other so-and-so tool. That seemed to have wowed her audience and they said that could be considered as an innovation. Alice and I agreed that it felt like it wasn’t. Unless you’d call it an innovation if I suggested to use Google Docs as opposed to Microsoft Office.

But isn’t innovation simply a new idea, device or method? Something new that will make things easier for yourself and others? Yes and yes. But taking that definition kind of opens the floodgates where anything new could be taken as innovation — which shouldn’t be the case because not every new thing can be regarded as truly innovative.

In googling what is not innovation in the hopes of getting more insights on this, I found this whose points I do understand:

First, an improvement that only meets the market standard or reacts to innovation that your competitors have already introduced into the market is NOT innovation. It’s playing catch-up.

Second, introducing an improvement that does not significantly differentiate you from your competitors is NOT innovation. It’s simply just an improvement—evolutionary, not revolutionary.

And finally, introducing improvement that may give you a competitive advantage but also can be easily copied by your competitors is NOT innovation. It’s just a temporary advantage.

…Here’s the takeaway: It’s easy to confuse improvement with innovation. But only innovation creates a unique outcome that, despite the superior financial returns resulting from the action, competitors are either unwilling or unable to match.

Now, innovation or not, I still believe in looking for and sharing things that helps us make things easier for ourselves and others. Whatever ideas we put onto the table and help realize, just push for it if it holds the promise of making things better. And you can just let other people decide if it’s an innovation or not.

Trying out Postman

One of the tools that we’ve used in our project is Postman. It’s a Google Chrome App wherein we can send the web service request and then view the response in a prettier, much readable style.

In one of our user stories for this Sprint, we needed to come up with the list of profiles that needs to be deactivated because the employee had become inactive. To verify whether the employee had become inactive, we had to check against an employee database. To check, you can go visit a search site, search for that user, and view the search result. Using their provided API, you can pretty much do the same with Postman. Submit the request with the email passed as parameter and evaluate the result.

01-single-request

But then, sometimes you have to check several employees at a time and it turns out we can use Postman to test in batches. To accomplish that what I did was the following:

1. Whether you run it one at a time or in batches, I defined my tests. Below is the snippets I used in my tests which checks for a 200 status being returned, and for whether the employee status field has value not equal to Active.

tests["Status code is 200"] = responseCode.code === 200;

try {
    var responseJSON = JSON.parse(responseBody);
    tests["Employee is NOT Active"] = "Active" != responseJSON.result[0].empStatus;
}
catch (e) { }

2. I used a variable for the parameter e.g., {{email}}.

02-parameter

3. I saved the parameter values in .csv format where the first line (treated as the header) use the variable used as parameter, in this case “email”.

4. I saved the request (with the tests and params) into a collection.

5. Then I used the Collection Runner to select the collection I want to run, the csv file with the data to pass as parameter values, and to indicate the number of iterations. Results are then displayed at the rightmost panel.

3-collection-runner

So there. I probably haven’t even scratched the surface of how else Postman can be used. But it’s been pretty nifty for this particular use case.

Automated testing with CucumberJS with Selenium Webdriver using JavaScript

I’m currently picking up something new at work. And I guess the title of the post gives it away. It’s not a brand new thing altogether, just something that I haven’t tried out before. And in one of the blogs I stumbled upon, there was a line that captured what I wanted to do in this post (and I guess in a lot of other previous posts too): helping me remember what I figure[d] out. Anyways, this is just a gist of the basic setup. Much of the learning is in the actual scripting, and most of it is when things don’t work as expected.

First things first: Setting up

  • Download and install Node from https://nodejs.org/en/download. This will also install the Node Package Manager (NPM).
  • Download the drivers into a folder in your local.
  • Update your environment variables to include the drivers into your PATH variable.
    • Add SELENIUM_DRIVERS as a system variable and specify the folder where you placed the drivers
    • Update the PATH system variable by appending “;%SELENIUM_DRIVERS%;”
  • We used a starter found here. Download the files from there, and that’ll give us stuff that I have no idea how to write from scratch at this point:
    • package.json
    • Gruntfile.js
    • features/support folder contains world.js, env.js and hooks.js
  • Install the needed dependencies.
    • Dependencies like CucumberJS, Chai, Selenium Webdriver, etc. are indicated in the package.json file.
    • Open the Command Prompt, then go to the folder where the starter files are placed. Run “npm install”.

Writing our Feature File and Step Definitions

Since we’re using CucumberJS, we’ll write up a feature file in Gherkin format. This means that scenarios that we’ll be testing for are written in a Given-When-Then format. Given includes the needed setup for the test. When refer to actions taken from the initial state (as setup in the Givens). And Then includes the expected outcomes that will be checked for.

For each Given, When and Then, corresponding code has to be written to actually perform the automated steps. These steps are stored in step definition files stored under features/step_definitions.

Feature: Searching for cucumbers
  As an internet user
  In order to find out more about cucumbers
  I want to be able to search for information about cucumbers

  Scenario: Google cucumber search
    Given I go to the Google website
    When I search Google for "cucumbers"
    Then I should see some results
'use strict';

var expect = require('chai').expect;

module.exports = function() {
  this.World = require('../support/world.js').World;

  this.Given('I go to the Google website', function (next) {
    this.browser.get('http://www.google.com').then(function () {
      next();
    });
  });

  this.When('I search Google for "$searchQuery"', function (searchQuery, next) {

    this.browser.findElement(this.webdriver.By.css('input[name="q"]')).sendKeys(searchQuery);
    this.browser.findElement(this.webdriver.By.css('input[name="q"]')).sendKeys(this.webdriver.Key.ENTER)
    .then(function () {
      next();
    });
  });

  this.Then('I should see some results', function (next) {
    this.waitFor('div.g', 10000);
    this.browser.findElements(this.webdriver.By.css('div.g')).then(function (elements) {
      expect(elements.length).to.not.equal(0);
      next();
    });
  });
};

Okay, actually writing our step definitions

To interact with the web site under test, we use Selenium Webdriver. While working on a POC, we had to look for elements within the page under test, and then interact with those elements by either triggering an action like clicking or sending keys or we also interact by extracting information about the element I found. For the checks in the Then steps, we use Chai to do the comparisons. For the POC at least, I pretty much managed to get by with simple scenarios using the items below.


//Finding Elements with Selenium Webdriver
//Find Element by ID
this.browser.findElement(this.webdriver.By.id('sp-search-input'))

//Find Element by CSS
this.browser.findElement(this.webdriver.By.css('#sp-search-input'))
this.browser.findElement(this.webdriver.By.css('.ui-chip'))
this.browser.findElement(this.webdriver.By.css('.ui-chip[data-val="hello"]'))

//Interacting with Elements with Selenium Webdriver
this.browser.findElement(this.webdriver.By.id('sp-search-input')).sendKeys(searchText+"\n");
this.browser.findElement(this.webdriver.By.id('sp-add-widget')).click();
this.browser.findElement(this.webdriver.By.css('.ui-chip>span')).getText();
//looks for element with class "ui-chip" and then looks for a <span> and gets the text in that <span>

this.browser.findElement(this.webdriver.By.id('sp-search-input')).getAttribute('placeholder');
//e.g., <input id="sp-search-input" placeholder="Add Search Tags" …>
//above will return "Add Search Tags"

this.browser.isElementPresent(this.webdriver.By.css('.ui-toggle.checked'));
//returns boolean value

//Comparisons using Chai
expect(actualText).to.equal(expectedText);
expect(textFound).to.contain(expectedString);
expect(toggleState).to.be.ok;

Running our scripts

Using the Command Prompt, go to your test project folder, then run “cucumber-js”. There’s also an option to run it with other options. I find the tags to be most helpful. What I do is I tag the scenarios that I’m working on in the feature file. The format of the tag is “@tagname” in the line before the scenario you are tagging.

cucumber-js --tags @dev               //tagged with @dev
cucumber-js --tags ~@dev              //NOT tagged with @dev
cucumber-js --tags @foo,@bar          //tagged with @foo OR bar
cucumber-js --tags @foo --tags @bar   //tagged with @foo AND bar

Well, that’s about it. Again, the learning really comes when you’re actually scripting. Sometimes, you just need to dive in knowing you don’t know everything, and hopefully you’ll figure it out as you go along.

Summary of references

Example, Starter – https://github.com/Matt-B/cucumber-js-selenium-webdriver-example
CSS Selectors – http://www.w3schools.com/cssref/css_selectors.asp
Chai – http://chaijs.com/api/bdd/
Selenium Webdriver – http://www.seleniumhq.org/docs/03_webdriver.jsp
CucumberJS – https://github.com/cucumber/cucumber-js
Gherkin – https://github.com/cucumber/cucumber/wiki/Gherkin
Cucumber – http://cukes.info
Cucumber tags – https://github.com/cucumber/cucumber/wiki/Tags
Blog post – http://www.tcias.co.uk/blog/2014/09/03/writing-our-first-functional-test/ (2014 Sep 3)
Blog post – http://transitioning.to/2012/01/cucumber-and-js-getting-started-with-cucumber-js/ (2012 Jan 27)

Looking for that singular statement

There’s been some recent talk about vision and goals where I work, and that has somehow led me into thinking about what I personally stand for (at least professionally). Or maybe it could also be because I watched Batman v Superman last weekend and there was this scene wherein the lady senator wants to know what Superman stands for. Anyways, I tried to give it some thought. The truisms and platitudes that I often say crossed my mind. I thought of my default thoughts and my fave quotes. But it’s hard to capture what I stand for or what I’d like to stand for in a singular statement.

Be kind. Play fair.

Let your work speak for itself. Keep on learning. Keep the saw sharp.

No, not quite. Each statement doesn’t quite cut it.

Then something hit me. 16 years out of college, and it hit me! I don’t think I’ve ever been the type who’s so overly gung-ho about graduating from our state university. But I realized that the singular statement I was looking for was in our school motto all along:

Honor and Excellence.

Honor. This captures how it’s not about winning all the time, but more importantly it’s how you played the game. It covers honesty, integrity and respect. It covers selflessness, doing what is right, and giving credit where it’s due.

Excellence. This highlights competence. It covers knowing your craft and continuing to learn to be better than you were previously. It covers setting the bar high. It covers uncompromising standards. This also extends to paving the way for others to excel and being a spotlight to others when it’s their time to shine.

Honor and Excellence. You need both. It’s not one or the other. This means being good at what you do without the needless assholiness. This means rising above others while still remaining grounded. This means expanding your mind, capabilities, scope, or whatever; without getting so full of yourself.

This is a good reminder of what I’d like to see more of in myself, and what I should reward or encourage more in others.

Read: Leading the Transformation

Our product owner is one of the rare few individuals I know at work who actually still reads books. Last month, he recommended that we read Leading the Transformation: Applying Agile and DevOps Principles at Scale by Gary Gruver and Tommy Mouser. It’s a thin book with only 112 pages on paperback and around a 3-hour read. It’s intended for leaders/executives so it gives a high level overview of the changes teams and the organization need to make and the benefit of those changes, and it repeatedly emphasizes management’s role in pushing for those changes. In particular, the changes that they want to drive at center around Agile, DevOps and Continuous Delivery (CD).

At work, small teams have now been shifting to Agile, our own team has been in this Agile project since January of last year, and I’ve heard of proposals wherein the methodology they suggest is already Agile instead of Waterfall. But then, I pick up from the book that trying to scale up Agile adoption across the board with small teams as the starting point doesn’t quite work for large organizations. Whoops. The book suggests that if you want an enterprise-level change, you have to plan for it and drive it from the management level down to us lowly minions. A key difference though is that within our organization (at least locally that I know of), we don’t really have hundreds of developers working on the same product or code base. And in our case, we’re only under 20 in the team, but even so the book still offers a good introduction to a lot of mature development practices that we need to look into.

Key items highlighted in the book that I’d like to reiterate further:

Importance of having quick feedback loops

Unit tests and static analysis tools can already weed out a lot of problems so that defective code won’t even get committed to the repository to begin with. And having those fixes done even before passing it to the test team — instead of fixing them only after the code has been deployed and testers found issues that were caused by those defects — will definitely help reduce the turnaround time.

Quick feedback loops will also help the team work and resolve issues while the code or user story is still relatively fresh in their heads. It’s more difficult for both the devs and testers to fix and retest an issue on a behavior that they’ve pretty much forgotten about.

Having builds as release- or production-ready as possible

With regular and stable builds in place, it’ll be easier to identify when a commit breaks the build. Since you don’t have to backtrack through days or weeks of commits, it’ll be easier to narrow down and identify the problematic commit.

Having dev/test environments as close to production as possible

One problem that we’ve personally encountered in not having a test environment in sync with the production version was that whenever we encountered an odd behavior in the test environment we had to double check whether the issue was also in prod. We also had to be mindful of issues that were already resolved in prod but not in the test environment. But I guess this problem is a combination of why it’s good to have the test environment as close to prod as possible and the next item related to why it’s good to have good deployment procedures in place.

Having repeatable build, deploy and test processes

From experience and the example above, having a reliable and repeatable deployment process could’ve saved us all effort and heartache. It could be so frustrating to test the same build (supposedly) but then get different outputs even if you’ve done the same steps using the same test data. In the same vein, you’d hate for a feature not to work in prod even if it had already been thoroughly code reviewed, tested and signed-off in UAT/PO review.

And last, but not the least, having test automation

You simply will never achieve the full benefit of Agile development until you get your automated testing properly built out and integrated into the development pipeline.

Test automation is key to the first item I mentioned since it enables quick feedback loops. It also allows repeatable tests to be executed across the different environments, and it allows repeated execution of the regression tests which you might not be able to afford to do so manually.

Having test automation, by itself, will not suffice. Tests have to be designed such that it’ll be easy to localize the cause of failed tests should any be encountered. Maintainability of the automated tests also have to be considered. Otherwise, the benefits of test automation won’t be realized since the team ends up ignoring the test results on account of being not sure whether the issue encountered is a code issue or a test issue.

One last thing… it’s a cultural shift

You can’t just invest on tools for CD or test automation or announce “Let’s do the Agile thing”, and expect the benefits to magically follow right away. This kind of thing takes time because there’s the technical learning overhead, plus shifting to a new way of doing things requires discipline and resolve so that folks won’t revert to the old habits that they’re trying to change.

It is important for executives to understand early on if the organization is embracing this cultural change, because if it doesn’t, all the investments in technical changes will be a waste of time.

It’s not going to be enough for the project team alone to be invested in the changes. The management and executives need to be aligned with this. In fact, they should help drive it. Otherwise, they might give demands that would bypass the adoption of change and instead force people back to their old habits (just because it might appear faster but only in the short term).

The book, after all, isn’t entitled “Leading the Transformation” for nothing. Management’s presence and push isn’t merely a suggestion; it’s a necessity. Sure, the project teams are the ones making the technical changes; but management needs to understand and support the changes. Essentially, people need to be in the same page in order to move in the same direction.