Web accessibility, testers’ start for building it in

So I haven’t had any first-hand working experience on web accessibility testing. I’ve read about it in passing, and once I attended this weekend testing session that tackled the subject. In my previous company where I did a lot of functional web testing, I recall some standard test cases that were related to usability or accessibility. This week though, the topics of accessibility testing and WCAG 2.0 came up, and I had to google a bit on the subjects.

One of the interesting things I found is this blog post on how expensive is accessibility. And what the writer, Karl Groves, said makes sense — that if it’s already part of how you do things, then it comes at no extra cost. And inversely, if it’s something you haven’t factored in at all, then pushing for accessibility would require changes and those changes would undeniably entail some cost.

Now I googled WCAG 2.0 and found really looong references. There’s a quick reference page, but seriously, that page felt anything but quick. In another post by the same person:

What if I told you that the WCAG 2.0 recommendation by the W3C is 36 pages, printed? In addition, “How to Meet WCAG 2.0” is 44 pages and “Understanding WCAG 2.0” 230 pages. Not only that, but the accompanying Techniques and Failures for WCAG 2.0 is 780 pages, printed.

That’s a bit daunting!

But thankfully, he also shared a post on the 6 simplest web accessibility tests anyone can do. And the Web Accessibility Initiative (WAI) has a reference on easy checks for web accessibility. These could be a start and could be things we start looking for in our web testing projects even though we weren’t really asked to do accessibility testing. It’s a chance to add value. Let’s check these out:


Ah, estimation!

For as long as I can remember, I’ve been asked to provide test estimates for project proposals. More often than not, the details are very high-level and there is never enough time given to you to digest the materials. The latest I received was this morning with 104 bulleted requirements (not yet including other background references) and they expect a reply by today otherwise they’d assume the test effort would be 25% of the development effort. Such short notice demands just make me wince. Didn’t they ever do math problems as a kid?

Juan renders 8.5 hours a day to Project X. Last week, Juan was on sick leave and is still feeling slightly under the weather (but this doesn’t really matter). How many minutes must Juan spend on each 104 requirements (not part of Project X) so that he does not have to do overtime?

Try as I might though, I can’t do away with test estimation. Estimation is part and parcel of the software engineering process, and is highly essential when you’re trying to bid for a project. Business needs to put a price. There’s no going around it. You, as a tester, will eventually be asked to estimate!

Lesson 1: Estimates are wrong most of the times

There are so much unknowns. You have to deal with the cone of uncertainty and all that jazz. Try googling why estimates are always wrong and you’d find a lot of results on the topic. So knowing this? What does it tell us? Nope, this doesn’t mean we just go off throwing numbers just for the sake of having a deliverable. Personally, it just tells me to be efficient with the effort that I put into estimation. I may only have half a day or half an hour even to come up with the numbers. I can’t develop an estimation model in that short a time, and even if I could the output can’t be validated and would most likely be wrong. Essentially, just estimate — don’t kill yourself over trying to come up with something perfect, and put aside your worries that it will be wrong (because it most likely is anyway). Chill! Don’t sweat it!

Lesson 2: Estimate like you’ll be the one to deliver

A reality is you often don’t have the authority over the final estimates. Countless of times, I’ve seen my initial estimates whittled down with each review. You have to remember that those who are working on those proposals are out there to win the bid. But as someone who got pulled in to do the estimates, do whatever’s in your power to not short change the team who will actually execute. Just keep adding buffers whenever and wherever you can!

Lesson 3: Best case scenarios hardly ever happen

In an ideal setup, there could be a team of 2-3 testers going over the requirements — each giving their estimate for the optimistic, most likely and pessimistic scenarios. In instances where there’s a huge discrepancy, you guys can talk it out until you come to a point where you all agree. From experience though, it’s almost always a lone effort. You don’t get the ideal situation for estimation, so it is safe to assume that you won’t get the ideal situation when it comes to delivery. Buffers and factoring in the pessimistic scenarios are always a good idea.

Lesson 4: Ask for models

For mature organizations, it’s possible that they’ve already collected some past metrics and have already crafted some magical software estimation models. Enter the size in function points and voila we have the estimates! Enter the development estimates and ta-dah you have got estimates for everything else! Just enter the parameters and it spews out the number! What number? THE number!!! Now I don’t want folks to be all reliant on these numbers since the parameters in which they may have worked in the past might be totally different from your current situation. I am wary of when they say that testing is just 25% of the dev effort when I know from first-hand experience that the effort in testing could go far more than that. What you can do is ask around and use these numbers as references. Only as references. You have to exercise judgement before deciding these numbers could be applicable.

Lesson 5: How I estimate all by my lonely self

So after making you painstakingly read through the previous items, here’s the actual content. When pushed into an estimation task for a new system to be built, I’m typically given a list of requirements or functionalities that the new system or app has to cater for. I’d go over each requirement to estimate for the following:

  • Test Execution – This covers the execution of the test cases, reporting the status of the tests, and logging bugs when found.
  • Test Planning – This includes studying the test basis, drafting test cases, identifying the needed test data.
  • Test Plan Rework – This is for reviewing and revising test case drafts.
  • Retest – For verifying bug-fixes.
  • Test Execution using a 2nd, 3rd, nth platform – It’s possible that you need to test the system/app using a different platform or device.

Now that’s for when you have the time to go over each and every requirement. Plan B would be to do the estimates for what would constitute as a Simple, Average, or Complex requirement. Then instead of estimating for each requirement, you just categorize them and then use the corresponding estimates for them.

Then you also add buffers to consider things like:

  • Things generally go bad, and there’s a general communication overhead
  • Technical difficulty — like if it is relatively unknown how to test a particular function, or if you’ll need to have some learning curve since you’ll need a tool to augment testing
  • Adjustments in case testing will be done by someone less experienced

Now, that was only for the functional test estimates. You’ll also need to estimate for the following:

  • Test Management – Especially for a large testing project, there’ll be management overhead to consider. There are also project deliverables like the Test Plan or Test Strategy document. There would be management meetings to attend to, reports to report, and project issues or escalations to deal with. There’d be onboarding of new folks into the team and essentially setting up the test framework, processes, standards, guidelines, etc. to keep all the testers in the same page, and the test team in the same page as the rest of the development team.
  • Integration / System Test – The focus of this test is for the end-to-end scenarios or business flows.
  • User Acceptance Test – The involvement of the test team during UAT will also need to be identified. Will they be facilitating (or would that be handled by the BAs)? Will we just provide support e.g., replicate issues reported, verify change requests, etc.?

And in the process of going over the requirements, you’ll also need to identify other types of testing which might be needed for the project and estimate for those that you are capable of or have experience on. It’s possible that the project could go with usability testing, accessibility testing, performance testing, etc.

Anyways, you’ve read this far. Thank you! I guess it’s too late to give out the disclaimer that I’m no expert. In case you have magical ratios or percentages that you’d like to share, I’d like to hear about them. Please share in the comments! And if there’s one take-away that you get from this super long post, please let it be: Do not short change the team who will actually deliver.

Why aren’t we… why don’t we have…

Warning: A lot of mention of “innovation”, “innovate”, “innovative” in this post. Here’s a photo of my two cute dogs if you need a break from those buzzwords.

A question got raised one evening while we were in Tinola (that’s the name of the conference room, which is also the name of a local viand), and we ended up having a full-blown discussion over it complete with a mindmap on the white board.

So the question was: Why aren’t we innovating as much as they want? We decided the key points were: Time, Skills, Motivation. Experience is essential for the insights and the intuition it brings for being able to distinguish what could be of value and what would most likely be a waste of time. We had a dotted line to “Management support” as they’d be the one most qualified to allocate time and skills at least. But that didn’t make the cut on what’s at the top of our list. Looking at the image, it kind of generally addresses any other question on why we don’t have something that’s being demanded of us.


I’ll try to go over each item from what I can recall.

Time – To produce something or create something generally takes time. I think that’s a physical, realistic, and basic requirement. Just sitting down on a problem can’t solve it — you have to think about it, process it, rack your brains for solutions or ideas. That by itself takes time. And implementing the solution all the more. So when folks are so busy in their projects, it’s a bit hard to expect them to focus on anything else.

But say you free up people so they can work on “innovations”, there’s this other aspect on skills.

Skills – If the expected solution is an implemented application or system, then that requires skills in software engineering — design and analysis, then there’s implementation or coding, testing, etc. These are skills you don’t just build overnight so with the learning curve, it’s going to take a lot more time to come up with a minimum viable product (i had to use that term*). Then there’s this concern on which skill do we need to build up further — shouldn’t we focus on strengthening our testing chops since we’re testers after all? Learn more about other aspects of testing that could be relevant to our craft/career? Or should we use that hypothetical free time learning to code so we can create applications?

And then again do we know how to innovate to begin with? Maybe there are some workshops to develop one’s skills in problem solving or creative thinking which could increase one’s chances in coming up with something innovative.

So say we have the time, we have the skills. It doesn’t mean we’ll already be working on innovating. We need a reason why to drive us and that’s the third point which is motivation.

Motivation – This is what would compel us to innovate. It’s so important to have a why! It’s emphasized so much even in the Jillian Michael’s workout video I’ve been regularly watching where she says “If you have a why, you can tolerate any how.”

It’s but natural for us humans to ask what’s in this for me? What would I get out of this? This could take on different forms — it could be financial rewards for some, fun for others, opportunities to learn for the rest, etc. It depends on the person’s individual priorities on what would be rewarding and in turn motivating for him.

Then there’s also the question on whether there is even a need to innovate. Of course, there are always things that could be improved, but there has to be a good problem to solve — the kind that’ll feel like you have an itch to scratch, the kind that will merit the need to spend time on it.

So that’s pretty much it for what we discussed that evening. Thanks for reading!

*Long story — maybe i can share some other time. I’ve rambled on for too long already. Thanks again for reading!

Challenge: focus on value

I’m so tired of things that waste or unnecessarily demand so much of people’s time. You’ve got that meeting where people don’t bother to come in on time. You sit through meetings seeing people kill time on their mobile phones (either that or they’re also people-watching just like you). You have all these goals and expectations thrust upon you, and you can only shake your head over how un-SMART a lot of them are. You’ve got young, impressionable team mates working overtime to prepare for game shows, plotting surprises for people they hardly know, making fancy props for who knows what, etc.

Maybe that’s aligned with what they like. Maybe noontime show antics is what floats their boats. Maybe I’m the boring, cultural misfit who values people’s time (mostly, my own), how it should be the individual’s choice on how they would rather spend it, and how they should have a say if other people are wasting it for them.

Maybe we should challenge ourselves to find focus, and focus on just one simple thing: providing value.

What if our focus is on producing quality interactions with whoever we deal with. We set up meetings that people don’t dread going to and they actually find value in attending. We hold general assemblies where people will get key take-aways other than free food, and they leave feeling inspired or motivated. We hold activities where participants would feel they are better or they’ve grown — even just a little teeny bit — for having been there; rather than have the feeling that they’ve just killed off 30 minutes or more of their lives.

I don’t think it’s possible to get it right off the bat and all the time. But wouldn’t this be a better direction worth going for?

More on When in Rome

One of the things I often say is “When in Rome, do as the Romans do.” I have some team mates who admittedly know me better than most that jokingly say what I really mean is “When in Rome, invade the Romans.” While I have no plans for domination (that’s just too much trouble), a lot of jokes are half-meant.

“When in Rome, do as the Romans do” is not a call for conformity, submissiveness, or withdrawing your own beliefs to follow someone else’s. To me, it’s about flexibility and just being plainly realistic that when you’re thrust into a new environment (be it a new company, a team, or a project) you can’t expect things to go your way or the old way that you know. You need to “do as the Romans do” to survey the environment, find out how things are being done, and more importantly to find out why things are being done a certain way.

There’s a saying that goes first learn the rules then break them. “Doing as the Romans do” allows you to learn the rules. This gives you the context and is essential for finding out which rules you can break or to how much extent can you push the limits of the rules. I don’t generally condone rule-breaking (I hate jaywalkers), but sometimes there are just BS rules (Google: sacred cows) which were set in place ages ago that are no longer relevant to the current situation. To me, rules (just like tools and processes) should be there to help make things easier or move things along more easily. If it’s more of a pain in the ass, then something’s wrong.

And I guess this is where “invading Rome” comes in. Armed with what you’ve learned from “doing as the Romans”, you are in a better position to trigger change. You also pick up on how to go about it, say who you need to talk to effect change more easily. But before we all go on a changing spree, one other important thing for me is to choose your battles. Not everything is worth rallying change for. I am not religious but the serenity prayer* says it best: Grant me the serenity to accept the things I cannot change, the courage to change the things I can, and the wisdom to know the difference.

[* N/A when I’m driving.]

Must I take on extra work?

At work, there’s this initiative asking people to do research work on some testing tools. There was a meeting where the topics were laid out and the facilitator asked folks who’d like to own them. There were a few takers but not everyone grabbed something to own. And I guess I can’t really blame them.

This is “on top” work (or as I’d like to start saying “in my personal time”). Most folks are already full-time in their projects and could use whatever slack there is (no matter how small) to take a breather and do other stuff that they want/need to. Personally, I need slack to be able to step back and think about how things are going, whether things can be improved, what do I need to plan for, how do I actually plan to do something, etc. Of course, a really small fraction of that slack goes into looking at cute creatures on the Internet for a boost of good vibes.

On the other hand, I want to think out loud a bit on why folks would want to do this extra work. What’s in it for them if they take on this extra, unbillable research work that they’ll have do in their personal time? Thomas A. Edison said:

Opportunity is missed by most people because it is dressed in overalls and looks like work.

The thing is if you do the work well enough, it doesn’t go without merits. If you’re hungry for learning, then the reward is that you learn. If you’re hungry for distinction, then the reward is visibility to your manager or your peers (which is a tougher crowd). If you’re hungry for promotion, then the reward is you get outputs/results to help increase your chances.

So back to the question, must you take on extra work? Sorry to disappoint, but I have no definite YES or NO answer to this. I go for IT DEPENDS. Not because it’s a safe answer but that’s the reality of how it is. It depends on what your goals are or what you’re hungry for. You can’t say yes to everything that falls on your plate. Say yes to opportunities that could bring you closer to your goal or just generally make you happy.

Innovation schmation

I was asked to write an article on innovation sometime ago to be included in our team’s newsletter. It’s finally out so I can finally post it here. I guess my basic thinking about the whole innovation thing is to get over the hurdle that it’s this major monster of a buzzword that only special geniuses can overcome. As Seneca puts it, “It is not because things are difficult that we do not dare, it is because we do not dare that they are difficult.” So really, just try and start doing.

Have you ever had a pretzel from Auntie Anne’s? I’d typically have a Cinnamon & Sugar pretzel, request for that to be cut into pieces, and order their cream cheese dip. Then one day, I was about to order my usual when I noticed something posted on the side. “Cinnamon Stix,” it read. The pretzel dough was shaped into short sticks coated with the Cinnamon & Sugar flavor, and each stick had a cream cheese filling. That “new” item was pretty much my usual order — rolled into a different package. “Ooh, innovation!”, I thought as I took my first bite.

It’s probably not what folks would even consider as an innovation. “Innovation” is such a buzzword nowadays. When you say “innovation”, people think of something mind-blowingly drastic with tremendously huge impact. The type that if you give a keynote speech about it you’ll get at least 5 minutes of standing ovation.

One other common notion is that you have to automate in order to innovate. Or that you need to produce a new tool that will save a measurable number of hours.

Or that you produce an even newer (hopefully, better) tool than the one just recently deployed. That happens. There was one time I attended a tools demo, and there were several tools that were shared but they were pretty much just different implementations of the same thing.

Discard those notions. Putting those limits in your head counters the chances of you coming up with one. So for starters, open up your mind to the possibility that you can come up with something that will make things easier for yourself and others. Take a look at how things are currently being done, and question whether they can be done better. It can be something as simple as introducing a minor change in one step of the process, or finding a useful shortcut key combination.

Your innovation doesn’t have to be big. You can start with the little things and it could be just as valuable because little things can add up to something bigger.

It doesn’t have to make a universal impact. Start with something that helps yourself, and then maybe work your way to helping a bigger audience.

You don’t need to automate. Automation is a tool that can be used to innovate; it is not innovation itself. If coding is not your thing, you can do research. For all you know there could be an existing solution to what you’re trying to address that is already out there.

It doesn’t have to be something totally new altogether. Just take my cinnamon stix example. Also, you don’t need to come up with a new tool every time. Sometimes it could just be a matter of tapping some unused feature of the existing tool, or looking into the current process rather than the tool.

Specialization schmation

So I was at this meeting with a team mate, and one of topics that got mentioned was specialization. So while we were on the topic, I kinda heard Roy’s voice in my head saying “Specialization is for insects!” There was also this quote I remembered:

If all you have is a hammer, everything looks like a nail.

On one hand, I reckon there’s just too much to learn in software engineering that there really is a need to focus. But on the other hand, specialization has its own traps. I read somewhere that specialist is just a fancy way of describing someone who doesn’t know a lot on other matters (crap, I can’t remember where I saw that). Using specialization as an excuse to not learn anything else is just too egotistical. And to relate to the quote I mentioned, just imagine a specialist on a particular solution who pushes that solution for every and any problem he encounters even if it doesn’t really fit.

Here’s a similar (and probably more thought of than this brain dump) take to it: Why Office Gurus are Bad (And the Buses Who Hit Them).

Semi-off topic. TIL that quote is called Maslow’s hammer or Baruch’s Observation.

Sprint retro: Testing worked!

So my friend is in this other agile project doing 1 week sprints implementing a web-based system. Until last week, their team didn’t have a tester. Of course, they probably dev-tested their own work and they did have sprint reviews with their product owner. But, as they’ve also brought up in their retro, and what’s also great about their team is that they recognize the need or value of testing (I quote: “need proper testing”).

A couple of weeks ago, my friend asked me to take a look at their site, give it a quick run through, and give feedback. So I sat down on it for an hour or so, and did some exploratory testing on a few available features. After that session, I had spewed out as many comments (bugs) as I could and consolidated them along with screenshots. For the sprint they were doing a retro on, they fixed some of the items I raised and finally got a tester in their team. It felt great to see this dev team post under “what worked”:

  • Tester
  • Bugs are raised
  • Some bugs are fixed / completed
  • KC (that’s me!)

Yay for testing!

Reads: The Myth of Epiphany

The myth of epiphany is that great ideas dawn upon you in an a-ha moment. Take for example the popular story of an apple falling on Newton’s head when he discovered gravity or Archimedes’ eureka moment in the bathtub. But what those stories seem to miss out is the significant amount of work that they’ve poured into solving related problems, and that it’s only when they took a break and let their minds wander that the answer came to them. We mustn’t overlook that there is work that leads up to those a-ha moments. There is a period of incubation where we try to digest the information that we’ve observed as we work on things, and our brains are catching up with all that’s been observed. Then if we’re lucky, the answer or the great idea comes to us at an instance that seems so out of the blue that it makes for a good story.

One quote from Ted Hoff (inventor of the first microprocessor, Intel’s 4004) said it best:

“… If you’re always waiting for that wonderful breakthrough, it’s probably never going to happen. Instead, what you have to do is keep working on things. If you find something that looks good, follow through with it.”

The Myth of Epiphany is from a chapter in Scott Berkun’s book, The Myths of Innovation.