I went through my public 2010 blog posts and summarized what I had written about down in the list below. Not much for this year, not that it’s the quantity of posts that matters. But back in 2008 and 2009, I averaged about 4 entries a month. This has now dwindled to 2 entries a month.
Is this thanks to microblogging? Have I less to gripe, er write, about? Are the iTouch apps too much of a time-sink than I would like to admit? Has having to censor my work due to confidentiality clauses become a deterrent? Have I become too busy with work?
Whatever the reason is, I’ve found 2 entries a month to be somewhat manageable. So my goal for 2011 is to have the discipline to keep at it. Not just the writing, per se; it’s more of the learning. Testing blogs, social networks like twitter and the software testing club, and – more recently – the weekend testing sessions have provided me with invaluable opportunities for learning and exposure to new technologies, techniques, tools, ideas. I reckon I’ll continue tapping on to these resources in the coming year.
Since the seventh session, the WTANZ topics have all been geared towards test automation.
- WTANZ07 (Jul 25) offered an introduction to watir. Here we tried to automate simple tasks like logging in, finding a forum and then posting a reply to it.
- WTANZ08 (Aug 22) was sort of a continuation of the previous session. Here the mission was to find a bug and then write a failing test for it. This session also provided a mini refactoring lesson for me since I placed the task of calling a particular function and the actual test under just one function. In retrospect, (I think Marlena also pointed this out) this session would have been a good time to try out assertions.
- WTANZ09 (Sep 19) was another session on watir. I missed this session but I did get to read a bit about it through Marlena’s blog post.
- WTANZ10 (Oct 17) earlier today was an introduction to Cucumber and Gherkin. From what I understand, Cucumber is the tool that does the automated test execution, whereas Gherkin is the language used to define the tests and it’s the language that cucumber understands. With the use of gherkin, test scenarios can be described in the format Given-When-Then-. And being in plain English, the business analyst can then supposedly be requested to write the test scenarios that he expects. Gherkin reminded me of Fitnesse wherein non-programmers are said to be able to write test cases in wiki format. For both tools, I guess the technical part of the actual automation is obscured from the non-programmers allowing them to focus on the scenarios or inputs that they would like to test.
For this Sunday’s weekend testing session, Oliver introduced the group to a web test automation tool called Watir which works with Ruby. I also found out that this session will be a precursor for yet another session for a test automation tool called Cucumber but the next session will be four weeks away though. I’m digressing. Anyways, as a prerequisite (supposedly), we were asked to install ruby and watir onto our workstations following instructions laid out in http://watir.com/installation/#win.
The session started off with Oliver walking us through with some basic commands, then afterwards he let us work on our own on a mission to use watir for posting a forum reply in weekendtesting.com. I had some trouble at first. For one, when I worked with IE7, one of the commands (
b.text.include? "<text>") just kept returning
false. I’m guessing watir was still looking at the blank tab rather than at the tab where the test web page was launched. I then tried using Firefox instead but I was getting an error message on jssh even though I’ve just installed the plug-in as indicated in the watir installation site. I later realized that I had two versions of Firefox and the command
Watir::Browser.new seemed to be opening the older one which didn’t have the plug-in installed. After getting that sorted out, it was pretty much smooth sailing. 🙂
Here’s a summary of commands I used in the session:
Watir::Browser.default = "firefox"
b = Watir::Browser.new
# Find text
# Click links
b.link(:text, /Next Weekend/).click
# Log in
b.text_field(:id, "user_login").set "<username>"
b.text_field(:id, "user_pass").set "<password>"
b.button(:value, "Log In").click
# Post a reply for the WTANZ07 topic
# Would only work if the topic is still in the list of Ongoing Discussions
# Searching for "WTANZ session #07" didn't work (even thru manual approach)
b.link(:text, /WTANZ session.*07/).click
b.text_field(:name, "message").set("Test reply yada yada blah blah")
# Log out
Right after the session, I found some more links on watir to check out later when I have time. :p
One thing I’ve found with these weekend testing sessions is that there’s always something new to learn. Last Sunday’s session was no different as Marlena introduced us to modeling — not the kind with runways or with strutting involved. Here we had a chance to try out modeling a few suggested applications using state diagrams. Our mission for the session was posted over at Marlena’s blog.
I reckon that as testers, we have this mental picture of how we expect our programs under test to behave. I guess preparing test specs is a way to translate this mental picture into something more tangible, and alternatively, capturing the system into a model does the same thing. In modeling, we get to break things down into bite-sized chunks that are less overwhelming as opposed to dealing with the system as a whole. This activity also potentially allows us to capture gaps in our thinking.
With state diagrams, we try to depict the system in terms of states and trans(actions) with more focus on the former. I came to find out that Anne-Marie tends to think more in terms of transactions than of states. I shared that I did so too and gave out an example of a textual (rather than visual) model that I previously used. Unknowingly, as Vivek had pointed out, I had created some sort of state transition table. 🙂
Takeaways from this session:
- Gliffy – a new tool! It’s a browser-based tool that uses flash for creating diagrams.
- Somewhat similar to what Anne-Marie said… as someone who also focuses on transactions over states, this exercise challenges our traditional way of thinking. It reminded me of Tim Toady (TIMTOWTDI) 🙂
- All the more interested in HWTSAM. I still have a lot of books (and comic books) lined up to read though. [Subtle hint: But if Roy buys me a digital copy, I wouldn’t mind ;)]
- Tim Toady again… aside from taking textual notes, there’s an option to go visual through the use of models.
To a lot of folks, drafting a blog post with the unaided eye might seem like a very mundane task — no biggie, easy as pie! As for me, as I type these words, I cannot read what’s being displayed on screen thanks to my less than perfect vision. I usually wear contact lenses with PWR -4.00 and so without them my face would have to be around a hand span away from the monitor for the text to be readable. Using my browser’s zoom function isn’t really of much help. Even at maximum zoom, at my usual resolution and with the monitor at arms’ length, the text still looks blurred and the site’s layout gets badly compromised.
For some, I guess an option is to use screen readers. And last Sunday, that (and accessibility) was actually the focus of the WTANZ session. For the session, I used a Firefox add-on called Firevox and one of the built-in apps in Windows called Narrator. Our mission was to go to the weekend testing website and log in, with posting a comment reply as a bonus — without looking for most parts.
One downside of Firevox is that its keyboard shortcuts seemed to have been in conflict with my other add-ons. So to enable it, I had to look on screen to find and click the corresponding icon. Once enabled it went on to read text from the website including some stuff which sounded like css properties. It also gets triggered by mouse actions so whenever I move my mouse around, it starts reading the section where the pointer is at. I suppose it does this by design but it was awfully distracting whenever I moved the mouse pointer by accident.
As for Narrator, I wasn’t really able to use it as a screen reader although I think it could function as one. It’s use for me during the exercise was for telling me where the focus was as I shifted between windows or between fields using the keyboard. Sometimes it gets annoying though since it seems to repeat some stuff over and over again.
Some other things:
- I found myself inclined to use the keyboard instead of the mouse. This means shortcut keys and correct tab ordering come in very handy.
- It’s also helpful to have a default focus on first field to be most likely used — e.g., Search text box in Google, Username text box in wordpress.
- The field descriptions proved to be quite useful. — For Narrator, it reads out these descriptions when the focus is on the field. Without these descriptions (as in the case of the post reply form), I couldn’t easily tell when I could actually start entering inputs. Downside though is the Narrator took around 2-3 seconds before it read it out loud.
- Breadcrumbs are not screen reader friendly. Sometimes I just wanted to know which page I was at but the reader would go on to read the entire trail.
- Same thing with some of unnecessary links on top of the page (as pointed out by Oliver). These aren’t screen reader friendly as well.
- An interesting point raised during the discussion was that it would be helpful if there was a b.<whatever.com> version similar to the m.<whatever.com> for mobile sites OR some way to tone down the site content to be more apt for screen readers.
- We thought ALT text would be read by the screen reader. I tried Firevox on xkcd but the ALT text wasn’t read.
I guess what Marlena said during the discussion sums it up best: “This was, ironically, eye-opening.”
Last Sunday afternoon, for the WTANZ session, we were asked to go through http://jarlsberg.appspot.com/part1 at our own pace and then trade notes afterwards. The site, through Jarlsberg (/yärlz’·bərg/) which is this cheesy app with known vulnerabilities, aims to show how to attack an app using common web vulnerabilities. On hindsight, part1 was only upto familiarizing yourself with Jarlsberg but most of us (i think) went on to the XSS topics. I think one or two even made it up to the next part on elevation of privilege.
Back where I used to work, some workmates and I used to tinker and find some bugs in the internal apps. We used to find the html we entered in some input field was rendered rather than escaped. And occasionally, there were alert messages and ruined layouts that were triggered. Previously, I thought they all fell under XSS. Through the Jarlsberg codelab, I found out some distinctions.
For instance, reflected XSS is when the hack is in the actual request e.g., when you create a link that points to some URL with a malicious script (although not really malicious) like this. A stored XSS is when you store the hack where it would be retrieved when the page gets requested e.g., when you post something like
alert(1) in some input field, and the alert gets displayed when that post is retrieved. There’s also file upload XSS for apps that allow the upload and retrieval of file attachments. The uploaded file could contain some scripts that aren’t expected to be executed.
As for elevation of privilege, I guess I had stumbled on to something that could be categorized as that in an internal system that we used in a previous company. It was a forum and I was able to access a certain functionality that wasn’t supposed to be available to me. They may have hidden the button to access it, but that was all they did to keep me off of it. It was still possible to access the functionality by modifying the URLs and there were no validations when I submitted the request.
Anyway, this weekend’s session tells me something I’m well aware of and that is I know so little about web security testing. The upside is there’s this codelab that I could explore further. Through the session, I also found out about a couple of interesting sites. One’s another site for learning about web app security — Web Goat — through which I came across an XSS cheat sheet. Another site was Cornify which was suggested as a more colorful and humorous alternative (imagine unicorns and rainbows on load) to pesky alert messages.
[Edit: July 17, 2013] I’m taking this Technical Web Testing 101 course by Alan Richardson on Udemy and found out that the Jarlsberg is now Gruyere (http://google-gruyere.appspot.com/).
Last Sunday’s weekend testing session was quite unlike the previous two that I’ve joined. Usually, the session is split into two parts wherein we do testing at the first half and then have the discussion at the second half. This time, we spent the entire two hours on a sharing session on tools. There a 21-page chat transcript of the session, and alternatively there’s my summary which I approximate to be around 2 to 3 pages long. The summary’s not comprehensive, of course… I simply listed out the tools that had been mentioned.
Anyway, the session’s topic also had me thinking of the bare minimum set of tools that I’d want in my workstation. Just a couple of months ago, I got a new laptop for work and right after the required installations I went on to install or download the apps/tools that I felt I couldn’t do without. They’re listed out below, along with other handy stuff from my home laptop (well, only those at the top of my head).
- Tiddlywiki – I’ve been keeping a personal wiki for my notes, to-do’s, etc. It’s my paste bin for ideas and links to revisit later on. Initially, I wanted to use OneNote, but back then I only had it on my home laptop and not on my work PC. Plus, I wanted something easily portable.
- Notepad++ – My preferred text editor. It also has syntax highlighting and has a tabbed layout, and you can configure it to run applications e.g., i can run ruby scripts directly from n++.
- Gadwin Printscreen – A screen capture tool so that I won’t have to use Print Screen + Paste to Paint. It automatically names and saves the screenshot into your directory of choice.
- Winmerge – Used for comparing files. Notepad++ also has a compare function but I guess I’m just more used to Winmerge.
- 7zip – For file compression; It has better handling of files with Chinese characters than the default in Windows.
- Wordweb (free version) – An English dictionary and thesaurus. Its advantage is that it can be used offline.
- Firefox add-ons like Firebug and Web Developer Toolbar had been helpful when I wanted to play around with form data. Fireshot is another screen capture tool, and it allows one to capture the entire page including those that can only be seen by scrolling down). Echofon, I use for following tweets. Delicious, for my bookmarks.
- Other stuff… I’d most likely tag them as nifty stuff if I posted them here. 🙂
A couple of days ago, I joined in on another weekend testing session with WTANZ. “The mission: Exploratory testing of how easy it is to get data in different formats about education in the United States and the United Kingdom from data.gov and data.gov.uk.”
I wanted to go right ahead and jump into those sites to get an idea of what kind of data is available from them. But the fates weren’t cooperating with me. I tried Chrome and couldn’t load both sites. I tried Firefox and was only able to load the US site. I felt resigned that I had no choice but to do my testing on only one of the two given sites. All was well for a short while as I was able to navigate around the site. But moments later, I couldn’t access the US site at all anymore. I tried pinging it, and was able to connect when I used the IP address. At some point, at some random try, I was eventually able to connect to both sites using Firefox.
With that out of the way, it was now on to testing in line with the mission… here were the comments I shared during the session:
comments on the US site
+ advanced search to filter out by file type, category, agency
+ i can already proceed with the download of some file types from within the search results list [edit: as compared to the UK site wherein you’d get redirected to other sites before you can actually download]
– got confused by the behavior of clicking on the file type link in the header of the search results list. Initially, i had 4 search results (csv and xls). I clicked the xml link thinking the results would be filtered out and I’d get no rows, but it didn’t turn out like that. It’s effect was more on the ordering.
– [not included in the session] i also came across Oliver’s finding that the csv file got downloaded as an exe file which wouldn’t help if you were using a mac. It’s probably a self-extracting compressed file.
comments on the UK site
+ “Data” used as label, as opposed to “Federal” in the US site, was more helpful to me
+ “Request new data” link was more noticeable than the “Suggest datasets” in the US site
+ alternative means for searching (didn’t get to try these out much though)
–> http://education.data.gov.uk/ – i got here from http://education.data.gov.uk/ but clicking search got me an “invalid or empty query parameter”. i also tried specifying “high school” into one of the search criteria but i got a not-so-helpful search results list.
– [not included in the session] agree with Oliver’s point about their data in PDF format cannot be processed
In the end, I was able to download a few files from the US site (xml, csv, xls) and a couple of PDFs from the UK site. I guess it was easier for me to meet the mission with the US site since it had the search function to filter according to file type. While in the UK site, I’d have to go to the actual search result, then go to the download page before I get to find out the format of the available data.
Other highlights I’d like to take away from the session:
- Using frustration (or emotions) as an oracle for something wrong, as a cue to question further
- First time to hear about KML data format
- [4:06:38 PM] Allmas Mullah: @marlena dev complain about testers asking ‘a lot’ of questions
[4:06:47 PM] Marlena Compton: That means u r doing ur job.
- [4:31:10 PM] Ajay Balamurugadas: It is the TESTER who has to set and clarify the mission.
I’ve been hearing about Weekend Testing for quite some time now. From what I’ve heard, the WT sessions usually last for two hours. Participants are given their mission and they conduct their testing on the first hour, after which they have a discussion for the second hour covering stuff like bugs found and lessons learned. WT initially started out in India, and has quickly gotten the attention of testers from all over the globe. I’ve heard of Cem Kaner and James Bach joining in on these sessions. Soon enough, a European chapter of WT was formed. And just recently, the ANZ chapter was formed and they had their first session this Sunday.
I didn’t exactly had my mind set on joining today’s session. My Sundays are usually reserved for visiting my parents’ home and for errands. But since I didn’t go home to my folks’ place today, I had the afternoon free and so I gave it a shot.
For today’s session, we tried out a mortgage calculator. The primary mission was to verify the functional correctness of the calculator. Other missions were to check on usability and browser compatibility. What follows is a hodgepodge of some of the stuff that was tried, bugs found and tools.
- (tool) http://bugrepository.com for logging bugs
- (tool) I used an excel spreadsheet to plot my inputs, expected and actual outputs
- (tool) Some compared the outputs to the outputs of other online mortgage calculators
- (bug) Interesting finds were the lack of data validation e.g., negative loan amounts, negative interest rates, no maximum limit for the number of digits, etc.
- (bug) Something I wasn’t able to explore further was a weird behavior wherein my input for the interest field “10000000000000000000000000” became “10,000,000,000,000,000,905,969,664.00”
- (bug) I triggered a division by zero error by setting the start and end dates as the same date
- (usability / accessibility) Lack of color contrast of the calculate button
- (bug) $100, 10%, 10999 years as input gives “$inf” for the Total interest you will pay in today’s dollars
- (bug) Loan period lasts for 10998 years by entering -999 and 9999 in the year fields
- (tool) http://browsershots.org for multiple browser testing
- (tool) I tried out http://ipadpeek.com which distracted me a bit from the mission :p
- (usability) The “nominal dollars” and “today’s dollars” links didn’t seem to do anything. I had a tendency on clicking on the “today’s dollars” link instead of the provided check box.
- (usability) This is out of scope but I think the links at the side look too much like plain text.
- (tool) Suggested tool: VirtualBox.org for testing multiple browsers
- (tool) Suggested tool: AllPairs tool at James Bach’s website for combinations to test
- (oracle) Some used personal data — “data that is very familiar to you so errors would be easily spotted.”
- (tool) Suggested tools: HTML Validators — http://validator.w3.org/ or HttpLiveHeaders plug-in for Firefox
- (tool) Suggested tool: Windows Live Writer, a note-taking tool used by Ram who was able to post a blog write-up within the WT session
- (bug in test tool) When logging an issue in bugrepository, the only option for the mandatory Category field combo box is “(select)”.