QA Hates You

Clicking the Void

The Director — Wed, 31 Dec 1969 17:00:00 +0000

I think that big manufacturers often have the worst customer service sites. Some of that cannot be helped, I suppose, since "big manufacturers" are often big brand names whose products are manufactured

Sez Who?

The Director — Mon, 09 Jul 2007 17:58:59 +0000

Sez me. I'm a veteran of the quality assurance industry and have experience vexing developers in myriad industries as both a technical writer who logged more defects than most of the full time QA team, a QA analyst who logged more defects (1000+ in nine months) than the rest of the team together, as a Director of Quality for an interactive agency who demanded all things be done right, no matter what tinny little arguments the designers or developers offered, or as a mercenary whose destructive talents go to the highest bidder. Ask yourself, "Why do they hate us?" Because developers and designers get all the glory, and QA comes along like a broom behind them to make sure that the feckless work they slop out between YouTube videos and brainstorming sessions at the local sushi joint comes together to please the client and make sure that the end users can actually use it. We get lower pay, longer hours, less glory, and the joy of watching our testing time cut every time that the cool kids decide to restart everything from scratch with a week left in the project. Then, when the customer complaints roll in, who gets called onto the carpet for missing things? Not the people who did it wrong in the first place; no, it's the people who didn't catch everything in the twenty minutes before the thing went live. Even when QA identifies a problem, management, development, and designerment offer a variety of reasons why things should remain broken to maximize profit or to make sure that they have enough time to properly render the next project into an emergency. Clients trip over these problems, or users trip over these problems, and either they learn to mistrust the software or they complain--and QA takes the rap, again. A lot of QA employees burn out quickly when dealing with that sort of rap. We at QAHY.com prefer gallows humor and mockery. Although no one will learn a lesson from a good shaming, at least we can enjoy ourselves as much as possible. You got a problem with that? You can always reach the helpful staff of QAHY by sending an e-mail to thedirector at qahatesyou-dot-com.

Nancy Pelosi Fails QA

The Director — Mon, 09 Jul 2007 18:06:54 +0000

Well, not Nancy Pelosi herself, but her Web site has gotten the wrong sort of attention on the blogs recently (here and here and so on). It's a simple Macromedia Flash presentation embedded within a Web site, but it has a number of problems that a trained eye would have caught. First and foremost, whomever created the presentation used stock imagery in the most sloppy manner; they chose, to represent a story on American military medical care, a stock image of someone with a uniform featuring an epaulet talking to a doctor. Unfortunately, that epaulet said "CANADA": Her political opponents (of which I am one, don't get me left) were quick to seize upon this as something more than a failure (or lack) of quality assurance, but they're just looking for something to make noise about anyway. Still, someone who reviewed this with any degree of exactitude would read all text and identify any extraneous logos within stock photography. And someone would have read "Canada" and said, "Uh, no....."This particular failure has been remedied, as the slide that offended the bloggers no longer appears. However, the site still fails QA in the following manners. At the change of each slide, the text from the first slide ("Green the Capitol") displays during the transition. Now, unless you're actually trying subliminal advertising, perhaps you don't want this to occur. Perhaps you want a smooth fade of the words and the fade in of the new slide. Still, unrelated text shouldn't appear: Next, the embedding of the Flash object is faulty. It gives the user too much control over the behavior of the object, including the ability to zoom so that the images appear pixellated or the text displays outsized. Since the Flash object has a certain set size, only a portion is visible, like this: Finally, as you should know if you build Web sites for a living (or pretend to), Macromedia Flash Player is a plugin whose presence should not be taken for granted on the user's Web browser. Any time you provide animation or other documents through plugins, you should provide a handy mechanism so that those users without the plugin can get them if needed. Does Nancy Pelosi? No: Instead of a static graphic or a link to Macromedia Flash Player, we get empty space. That Other America that I'm always hearing about, the one without Flash, gets left behind.I am tempted to go into metaphors about legislators whose Web sites aren't checked before they're put up and the implications for legislation, but I'll save that for another blog post and will point out that a couple of hours' worth of time of a trained Quality Assurance professional would have ferreted these issues out before releasing it to the public, sparing embarrassment and also sparing someone the "emergency" of fixing it. But, hey, if you don't want to spend that money or budget on quality assurance, you roll the dice. Sometimes they don't come up snake eyes, but when they do, you'll pay for it. (Cross-posted at Musings from Brian J. Noggle.)

That’s No Way To Mail Merge, Son

The Director — Mon, 09 Jul 2007 19:13:27 +0000

It must be difficult running weekly e-mails and all, but someone should probably review the friendlies to make sure that your variable names aren't coming through instead of, oh, I don't know, values:

Those In The Software Quality Industry Snicker At Headline

The Director — Mon, 09 Jul 2007 19:15:10 +0000

Disapppointing quality results can spur real change: Yes, do tell us about quality.(Cross-posted at Musings from Brian J. Noggle.)

Sissy Load Tests

The Director — Tue, 10 Jul 2007 14:00:03 +0000

In this week's SD Times, Larry O'Brien identifies a problem that load testing didn't uncover:

Perhaps the only thing worse than a slow uptake of your application is a smash hit. Users have a way of outfoxing everything, including load tests, and the imperative to respond to existing customers can absorb all the working hours of a team that is scheduled to move on to the next version. Worse, when a product is exposed to an order of magnitude more users than planned and when the product is used more intensely than anticipated, the defect list grows rapidly, potentially panicking the team into treating the symptoms, not the causes. The resulting chaos can easily derail a team, especially one new to agile processes, where “the customer is always right” and being responsive were the values that led to the success in the first place. Not long ago, I witnessed this very problem. I was engaged to work on the requirements and architecture of The Next Phase, which didn’t seem to have a lot to do with The Current Deployment, whose two big features were a comprehensive audit trail for management and a Web-based “dashboard” that gave users a much better view of their own context. Following the principles of “You Ain’t Gonna’ Need It” and “Don’t Repeat Yourself,” the dashboard and the auditing facilities used the same messages to request information; the dashboard, of course, stripped out the huge blobs of auditing data and presented a much-compressed summary. What was not anticipated (note the use of the passive tense to avoid blame) was that the users found the historical perspective of the dashboard very valuable and configured their dashboards to retrieve not just a day or two of history, but often everything they did in the past month.

Listen, there's "load testing" and then there's LOAD TESTING, and I would wager this organization did some "load testing." "load testing" occurs when someone, usually in "QA," runs a set of scripts in a nice, Bob Ross way of having happy little virtual users do happy little things and then sign off. The friendly sets of scripts behave according to anticipated user workflow and ramp up nice and slowly. Like here's five users, and then ten minutes later, here's five more users. Because that mirrors workflow. LOAD TESTING, however, is a nice, Tom Zatar Kay way of throwing everything at once, having the virtual users do resource-sucking tasks and the most processor-intensive things possible. Then, you launch all virtual users at once and see how the server handles a spike. In most cases, it will bomb out because it wasn't designed to handle that sort of load. Suddenly, passive voice will be deployed in lessons learned documents. The difference is between using load testing to benchmark or to break software. When you first roll it out, you want to see how it breaks and make sure it doesn't break early and often. Once it's in production, you can run a series of basic, friendly scripts to see how it behaves at certain load levels. That way, you can retest over time and after modification to see how it's running with the changes or optimizations. But if you don't blast the hell out of it, you're letting the users do it for you. And then the passive voice will be used.

Book Report: How to Break Software Security by James A. Whittaker and Herbert H. Thompson (2003)

The Director — Tue, 10 Jul 2007 19:50:38 +0000

After I read How to Break Software (which a quick Google check indicates I have not reviewed, gentle reader, but most of you wouldn't have read it anyway), I bought the companion volumes. This book, which I bought off of Amazon.com at its retail price, disappointed me where How to Break Software did not. Both books run off of a quick list of fault-model testing (a term I learned from the first book). I had a ball with the first book, laughing at seeing some of my favorite dirty tricks encapsulated in someone definitive's book. This book, however, didn't hold the same glee for me. The first book dealt with a broad subject and offered some very concrete things to try to attack software. This second book deals with a similarly broad subject (security testing), but is more abstract. The attacks it discusses aren't as narrow and easy to recreate; they're more methods and abstract ideas to try rather than concrete shortcuts to finding issues. I know, there's something to be said for a broad, ranging methodology, but the first book wasn't that way, and I didn't expect this one to be that way. Additionally, the book is sized similarly to the first, which doesn't allow it to go into a lot of detail for each of the abstract things it talks about. Finally, I don't know that the book focuses enough on actual security attacks; rather, it focuses on attacks that could be construed as security breaches. However, in many cases, they're not specifically security attacks, but rather regular tests that could, if applied to applications needing security, be security attacks. Maybe that's all security testing is, but this book wasn't different enough from the first book to make me wonder if it wasn't really a sequel given a better title. On the other hand, it does come with a CD and a tool which looks to be pretty cool, if I could get some professional time to play with it. So buy the first book, How to Break Software, and apply its attacks to secure software. Buy this book if you're really into it or if the company is buying it for you. Books mentioned in this review:

(Originally posted on Musings from Brian J. Noggle)

You Love The Favicon; You’ll Love The T-Shirt

The Director — Wed, 11 Jul 2007 19:50:48 +0000

Buy it: Or you're on QA's list.

Inspiration

The Director — Thu, 12 Jul 2007 14:45:57 +0000

Here is the ultimatum of our camp: what can be smashed, must be smashed; whatever survives a blow has value, whatever flies to smithereens is rubbish; in any case, smash right and left, it will and can do no harm.
Dmitry I. Pisarev

With Requirements, Without Requirements, It’s All The Same

The Director — Thu, 12 Jul 2007 17:39:07 +0000

I've encountered the question in interviews before, "How do you test without requirements?" The SD Times offers an article entitled "Testing Without Requirements—Impossible?"

As crazy as it sounds, there are actually teams of software testers that perform their job without the slightest clue of what the application under test is supposed to do. How is this possible?

Jumping Jack Sprat, how do you test with requirements? I've heard mysterious talk about these mythical beings upon some pantheon of BusinessAnalympus somewhere, where the gods of projects have come together and thought about how things should go, but I've never seen them myself. Well, okay, I did see, once, a thick binder of what were allegedly requirements for a financial service company with a robust product that a co-worker waved around from previous employment that purportedly identified screens down to the field/control level and explicitly said what could and could not go into said control, but it might have actually been his history notes from high school for all I know. No, in many of the projects I've worked on, the developers have started coding based on nothing more than some stake holder saying, "Wouldn't it be cool if...." and maybe a couple of inferences. Or perhaps someone has made the effort to put together specs, user flows, and detailed documentation about how the software is supposed to act, and all of that documentation lasts only until such time as the client takes a look at it and says, "Wouldn't it be cool if....." or "That's not what I wanted/signed off on...." At which point, it's off to the miscommunication and misinference races. Even if you've got requirements (as in the second case above), they're going to be incomplete or completely inaccurate by the time the application is ready for testing. So you do the same basic things to an application:

You get a basic idea of what it's supposed to do.Every application is a tool designed for a user to do something. So find out what that something is, and then find out if the application does it.
You get the basic idea of how the user achieves the goal.That is, the application has a set of controls and a set of pages or screens into which the user should enter some information that the computer computes. So you look at those steps, and say, "Hmmm." Once you've sussed out the extent of the application, that is, the playing field upon which you can operate, you can go to work.
Do what the user isn't supposed to do.Once you know what the user is supposed to do, you can subvert that and do what the user shouldn't be able to do. The user shouldn't be able to edit records he or she hasn't created yet or delete records that he or she has already deleted. Everyone else is working to make sure that the application sort of does what the client demands; however, the absence of requirements also often means that no one has planned for the unexpected. And by unexpected, I mean "normal user negligence."Sure, it's a lot of infinity to try to cover, but with some experience, you'll get a sense of what to try.
In the absence of standards, use common sense.If your organization does not have standard set of limitations for data the user can enter, apply common sense. You'll no doubt catch many of your developers unaware if you start trying to enter 30-digit zip codes, "I Can Has Cheezburger" strings for someone's age, or gas station prices for currency values (2.999).

The buzzwords to use in your interview, gentle reader, are investigate the application goals and exploratory testing and consult stakeholders and boundary analysis and, if you're feeling really buzzed, black box testing. Since these words mean something slightly different to everyone who hears them, you'll get some point across.

Unleashing the Hamlet

The Director — Fri, 13 Jul 2007 15:42:29 +0000

Ladies and gentlemen, the infamous Hamlet Test, explained for your edification and as a tool for your arsenal. Some might call it a mechanism for Boundary Analysis, but it's more than that. It's just plain mean. The use case, if you need one (oh, and how you'll need one since "rock star" developers will tell you this would never happen in real life so he can, instead of writing flawless code, can get back to YouTubing): Back when I was a technical writer, I used to write the documentation by using software. Hey, I know, that's an odd concept; most technical writers, if they exist for an organization, will take what the developers give them and put it into a serifed font and call it a day. Not me, I actually used the software, which also explains why I had the second highest defect count in the company, above most of the full time QA people, but that's another story. So there I am, swiping and pasting data from the application into my text editor (UltraEdit, don't you know?) while I'm rearranging a user's guide weighing in at about 250 pages. I'm reorganizing procedures and how-tos, building new chapter intros, and whatnot, and I'm swiping and pasting from a massive Microsoft Word document at the same time as I'm swiping and pasting shorter strings from the application. You can see where this is going, right? A never-in-the-real-world situation occurs. Instead of pasting a short string into an edit box, I dumped an entire chapter of the user guide into it. And it took it. Friends, this was a desktop application written in Java. You know, Java. A couple years ago it was what all the cool developers did because it was a badge of honor to code in such an esoteric non-Microsoft language. The smart people used Java. Also, Java did not have the color-by-numbers safeguards offered by Microsoft IDEs to help developers not do something stupid. Like automatically accepting SUSs (Strings of Unusual Size). So they were playing without a net and without talc on their sweaty little developer palms. It took the chapter, and although it took forever for it to return (or even repaint the screen), nothing crashed. So of course I wanted something bigger. Hamlet is bigger. I went out onto the Web and grabbed the full text of Hamlet by William Shakespeare (I do have an English degree, after all). I stripped it of all of its punctuation and line breaks and ended up with a string of 135,014 of the best characters ever strung together in the English language. Why, when one of the infinite monkeys cranks this string out instead of the complete text of the play, he'll be my hero, damn that copycatting monkey who gets the whole thing correct down to the last parenthesis. Yeah, well, that did the trick. Good night, and thanks for all the fish. I logged a defect, of course, and no doubt it was right at the top of the developers' list (right after watching the complete video of a guy going through all the levels of Pac Man without dying once). So true boundary analysis means you check to make sure that values up to and just exceeding the limit are handled appropriately, right? So an edit box that's supposed to have a limit of 255 characters can handle 254 and 255 characters without puking stack traces on my screen and 256 should either be limited by the control or should produce a nice validation message for the user. That's how it works, I'm sure, in those smoking-jacket, pipe-smoking Victorian English QA shops where they've got time to drawl with accents about unfortunate events. In the real world, you've got 4 hours before the tech team is promoting it because the client set a hard deadline and, whoops, the tech team missed its internal deadline by 2 weeks (sorry, old chap). Hamlet will blast right through all of their mistakes. Want to know if it's going to fail? If you see a peal of ordnance is shot off in your desktop application means something's going to give. If you're testing a Web application, many browsers will just show a blinking cursor to the right edge of the edit box. This means someone has decided to save $5.99 by not installing a MAXLENGTH on the edit box. Click Save and let the fun begin. And let the passive voice e-mails begin.

What We Have Here Is A Failure To Incrementate

The Director — Sat, 14 Jul 2007 15:02:42 +0000

So I was looking at the pictures on the Today Show's Web site that someone was trying to use to blackmail Miss New Guernsey or Miss New Jersey or some beauty pageant participant/winner. I was doing this for research purposes, understand. And I noticed something special about the slide show viewer. Here's one of the photographs: You can click it to see it full size in a new browser window, gentle reader. I've even circled points of interest to simplify things if you're a developer, although I couldn't constrain my syllable count in explanation. You see that the querystring identifies this as page 7, but the iteration at the bottom says it's image 1 of 10.So what happens if you click the Next button? A new picture, a new pg value on the querystring, but it's still image 1 of 10.What happens in Internet Explorer? Wonder of wonders! It's working in Internet Explorer; that is, Internet Explorer knows it's image 8 of 10.Yeah, so some developer came along and, since IE is his or her preferred browser, he or she tested it in IE and it worked, so everything's copacetic. Except that a large portion of the public doesn't use Internet Explorer.It's one thing to build a Web application for your internal company network where you mandate everyone use Internet Explorer 6.0 until such time as we're all implanted with cybernetic surfboards. It's another to build a Web site for the general public that lacks very basic cross-browser compatibility testing. In many cases, your common developer fancies himself smarter than the average man, so he's running Firefox and you'll find that he's developed a Web application or a Web site that looks like crap in Internet Explorer. Other times, he'll be the artistic, Macintosh type developing for Safari which can look okay in Firefox, but again like crap in Internet Explorer. Sometimes, just to cheese them off, I like to fire up Opera or another lesser (less than 1% of the browsing public uses it) browser and log defects about how things don't align correctly or work just right. Or AOL on the Macintosh.....oh, yeah. Fix that, you developing scum!

Ignoring the Peril of the Badmins

The Director — Mon, 16 Jul 2007 18:25:09 +0000

A commenter recently asked if I had heard the excuse, "It's impossible to write idiot proof code because idiots are so damn inventive." Well, I don't hear that one very often because the excuse itself implies a flaw in the code, and developers rarely admit that outside, perhaps, the circle of other developers. Because THEY ARE LIKE THE GODS! However, developers often justify their sloth for administrative applications. "Admins will use this," they say, explaining why they're not bothering to build a functioning user interface or enforcing any sort of data validation. As such, the tools in question often look like something that got an incomplete grade in a community college Introduction to Programming class. Web applications are missing images, fields and controls are laid out in the order in which the developer thought of them, and you can enter bad data directly that will have an immediate adverse impact on the application the adminster is adminstrating. You developers sneaking a peak at this Web site are shrugging your shoulders or nodding in agreement right now, aren't you? Well, dear reader, if you're in QA or otherwise have a head upon your shoulders, you understand that the developers hubristically think that their intent will actually direct the users' and clients' actions. However, actual deployment of many systems shows that those incorrigible users will use the applications with only the constraints in the software, not the constraints in the developers' minds. Once the programs are out in the wild, people get assigned administrative privileges because they're administrative staff, not because they're computer power users who like to open terminal windows for the comfort of the dark background and monospace font. Assistants and receptionists are unleashed on things that have no safety nets, and all sorts of havoc ensues. Not fun havoc, which is what QA does, but emergency klaxons sound havoc that costs the software company money to fix. Because the developers think that admins are like themselves, that is, omnipotent and infallible. I'm reminded of a project that happened when the developers created an administrative interface for themselves. Nobody in the organization, particularly the developers themselves, wanted to expend the lucre and the time to make the administrative interface since the only people who would use it were the developers themselves. Then those developers left the organization. Suddenly, other people had to use the ill-conceived and undocumented interfaces for day to day operations. A good time was had by all, at least those in QA who got to repeat, "We told you so." So remember these lessons, QA, because developers and other IT staff will not: Administrative applications need to hold to the high standards as other applications should, regardless of the bleating of the developers who would rather slowly make their way through the show with zefrank's archives than do things the right way.

Antagonize Me

The Director — Mon, 16 Jul 2007 18:49:59 +0000

I guess it's an improvement. The first time I went to the Simpsonize Me Web site, I got a PHP error message. As you know, this is only a single line of text on the screen. The next time, though, I reloaded the page, I got the Web site with only a JavaScript error: Click for full size I guess that's progress. I'd suppress the Flash context menu, too, but then again I am a stickler.

(Link seen on Outside the Beltway.)

UPDATE: It looks as though they failed to anticipate load and have had to throw up a temporary page until such time as they can accommodate the load. Double oops.

Proofreaders Apparently Not Invited

The Director — Mon, 16 Jul 2007 20:06:39 +0000

Software Test & Performance magazine calls for nominations for its 2007 TESTERS CHOICE AWARDS. [sic]

Screenshots: A Primer

The Director — Tue, 17 Jul 2007 17:53:42 +0000

Screenshots are the QAer's best friend, gentle reader. Appended to a defect report or printed on a color plotter at 20" by 30" so you can wave them two-handedly in front of the developer, they can prove conclusively that something very wrong happened with the application. Because otherwise the developers will fix something and claim it was never broken in the first place, or the humidity within the office will change, altering the flow of electrons through the network cable, producing different results when you try to recreate it, or something. It's always something. So, at any rate, a good screenshot adds a lot of value to a defect report or other communication. As you might know, gentle reader, I used to be a technical writer before I gave up trying to explain the inexplicable and decided to seek and destroy it instead. So I have some advice for taking screenshots effectively. Most of the advice that follows talks in terms of Windows-based computers, but if you're in QA, you should be smart enough to apply the lessons to other platforms. First off, you need a program that can handle bitmap images, such as Microsoft Paint (standard with a Windows install), Microsoft Word (if you have Microsoft Office installed), or a custom graphics program. I've always been happy with Paint Shop Pro because it's under $100 and I've used it for almost a decade. However, Photoshop or what have you will work as well. Next, there are two ways to take a screenshot. You can simply press the Print Screen key, which takes an image of the entire desktop including taskbar and whatever bits of your wallpaper might display behind the application you're testing. This is typically too much information, so you'll only use it occasionally. I mean, here's a busy screenshot of my whole desktop, including windows open beneath the application I want to call out: Click for full size.Attach that to a defect report, and you'll overwhelm a poor, simple developer.Now, if you press the ALT key and the Print Screen key, Windows will take a screenshot of just the active window. Like this: Click for full size.Now, be careful if you've got a floating window that's not maximized and it's tucked under the taskbar, as this will capture the taskbar, too: Click for full size.That's just poor craftsmanship.Now, keep aware of the various and sundry things that pop unbidden onto your screen while you're taking screenshots. Some applications grab focus or display notifications in the corner of your screen on events, and if you get any of those things, retake the screenshot. Geez, this particular screenshot appeared in the June issue (PDF at the link) of Software Test & Performance magazine. The QA guy himself and any editors at the magazine just let this go. In the magazine itself, you can even make out who sent him that offending e-mail: Click for full size.Now, you've got the screenshot and you've pasted it into your graphics package. It's okay to use some of the graphics tools within the graphics program to highlight or add explanation: Click for full size.Additionally, you can judiciously use the cropping feature of your graphics package to bring the issues to prominence.Now, remember I said there was one reason to do a Print Screen shot that doesn't just capture the active window? If you're showing a modal dialog box, such as a message box or other sub-window functionality, ALT+Print Screen will only capture the modal dialog box and not the underlying state of the application. Like this: That doesn't tell the developers much, and they like it that way, since it presents less challenge by their subatomic consciences in rejecting the defect.Instead, you should take a full desktop screenshot and then crop it down to just the application you're testing: Click for full size.That way, you can show more about the state of the application behind whatever modal dialog you show.Now, about saving those screenshots. Hopefully, you've got a real graphics package and can save the images as either a JPG or a GIF. This makes the image easy to review in other graphics programs or in Web browsers if your defect tracker is Web-based. Of course, if you don't have a graphics package, you can save the image within a Microsoft Word document. You want to avoid proprietary image formats (such as Paint Shop Pro images with .psp extensions) or more esoteric image formats (TIFF or whatnot) because they will require image programs to review. You probably want to avoid bitmaps or Windows file formats as they can be large in size.Now you can attach your stunning, hard-hitting evidence of application failure to your defect report. Not that it's going to matter one difference to your technical staff; they will receive the report, ignore the image, bang randomly on the keys while making monkey sounds, and when that fails to cause the problem to occur, will mark this issue as not reproducible. But you have proof. And you're not crazy. (As a footnote, if you're testing on a Macintosh, you can do a screengrab in different ways recounted here.)

Oh, We Did?

The Director — Wed, 18 Jul 2007 19:38:49 +0000

This is Web 2.0 at its finest:

Oh, we have, have we? Somehow, I suspect if you had encountered the error, I would not have encountered the error. Or at least I would hope so. It's Web 2.0, though, because we encountered the error together, and I am invited to freely contribute to the community by submitting a bug report.

Wherein QA Fixes Its Own Issues

The Director — Thu, 19 Jul 2007 15:25:40 +0000

As commenter The QA Slayer pointed out in a preceding post, the comment text area was susceptible to the Hamlet test. Not so much that it dumped a stack trace, mind you, but in that it allowed the user to enter unbroken strings such as Hamlet, and the cascading style sheets and basic logic of WordPress didn't have a mechanism for handling it gracefully; instead, the unbroken string would burst through the right side of the body image like an alien from Kane's chest: Click for larger Well, all right, then, I guess that should be fixed.I've often logged issues like that in various and sundry positions where I've worked, and the people in charge of fixing things have often declined to fix this issue even though it's a recurring issue with pretty much any Web site that we built. It was a kind of tradition; I would test a Web site, I would log the same freaking issue, and it would be resolved with a response that they wouldn't fix it. Even though, as you know, gentle reader, some URLs run long and don't have a handy place for word-wrapping. Oh, but no, that's only an issue QA would find. Armed with the Internet and a book about PHP, your humble Director tore into the code for WordPress and spent two hours devising a fix. That would have been one hour, but I took some advice from a developer that led me down a blind alley for an hour. So here it is, gentle reader, a piece of server side validation that should keep your WordPress blog from accepting too long of an unbroken string:

if ('' !=$comment_content){
   $string = $comment_content;
   $i_length = 50;

   $tok = strtok($string, " nt");
   while ($tok !== false) {
      if (strlen($tok) >=$i_length){
        wp_die( __('Error: please keep your individual words to fewer than
        50 characters, as the developers who read this blog are not that
        smart.') );
      }
    $tok = strtok(" nt");
   }
}

Change $i_length to match a number of characters that can display well within your template and add this to your wp_comments_post.php file and it will display the error page if the user tries to enter a single unbroken set of characters that would go outside the boundaries of your nice layout. Now that I have that, what do you call it, algorithm thing ready, I can use the logic anywhere I want; I could even bundle it into its own function to reuse throughout WordPress. So what keeps developers from fixing these simple, recurrent issues? Because ThinkGeek.com won't browse itself!

That Headline Needs a Punchline

The Director — Thu, 19 Jul 2007 20:25:23 +0000

CIO magazine offers an article entitled How to Spot a Failing Project; that begs for a punchline, such as The client is still paying the bills, unlike a failed project, where the client has stopped paying the bills.

I can't really find too much fault for the actual information in the article that identifies symptoms, including a lot of overtime/resources throw at the project, a lack of communication among project members, missed milestones, and so on. However, these are mere symptoms that let you identify when something is off the rails, but doesn't really explain what you should do--take the legal hit of breaching the contract? Buy some of the project management tools from the people interviewed for the story?

Probably the latter.

However, most of those steps apply to most projects, including the ones that don't fail (and by "don't fail," we mean "end and the customer pays anyway"). So we thought we would provide with an additional set of signs you can use to spot a failing project:

Another project manager commits seppuku.
QA walks around snickering or smiling.
The coffeepot in the communal pot has begun to have a Ritalin flavor.
The account directors show up in a neck brace from rigorous nodding in agreement with irritated clients.
Developers are seeking work visas for Tuvalu.

Punishing the User of Buggy Code

The Director — Sat, 21 Jul 2007 16:10:24 +0000

It's bad enough to subject users to your software, but it's another to hold them criminally liable for your bugs:

Prosecutors are considering criminal charges against casino gamblers who won big on a slot machine that had been installed with faulty software. More than the Target usability lawsuit, developers are watching this legal proceeding because, in developer logic, if one is prosecuted for using buggy software, one will not use buggy software; that is, the software developers write will suddenly be bug-free under penalty of law.

Fortunately, Attention to Detail Remains Annoyingly High

The Director — Sun, 22 Jul 2007 01:31:07 +0000

I, like many, have tried the Personal DNA test, and it revealed something about me that I had never known: At least the title and alt attributes are consistently misspelled.

Simple Quality Problem Now A Security Problem

The Director — Mon, 23 Jul 2007 20:27:59 +0000

New hacking technique exploits common programming error:

Researchers at Watchfire Inc. say they have discovered a reliable method for exploiting a common programming error, which until now had been considered simply a quality problem and not a security vulnerability.

Jonathan Afek and Adi Sharabani of Watchfire stumbled upon the method for remotely exploiting dangling pointers by chance while they were running the company's AppScan software against a Web server. The server crashed in the middle of the scan and after some investigation, the pair found that a dangling pointer had been the culprit. This wasn't a surprising result, given that these coding errors are well-known for causing crashes at odd times. But after some further experimentation, Afek and Sharabani found that they could cause the crash intentionally by sending a specially crafted URL to the server and began looking for a way to run their own code on the target machine.

Funny, most security holes exploit a common software problem: developers wrote it.

Suddenly, all of those issues marked not reproducible or low priority (fix when hell freezes over) would suddenly get new importance. Never mind that, developers, you better check Woot! right now, or you could miss a deal!

The Querystring: Soft Target

The Director — Tue, 24 Jul 2007 17:42:49 +0000

Gentle reader, I want to let you in on a little soft underbelly your Web sites and Web applications might have. The querystring. As you might know, gentle reader, the querystring is that junk in the Address bar of the Web application. It includes the URL/pagename of the page your user is accessing, but it also can include parameter/value pairs that server-side applications process. That is, it's a way that your developers can ignore inserting error-catching logic and show the world the stack traces they're so proud of. Here's the basic anatomy of the querystring:

protocol:	domain:	page:	separator:	parameter:	equals:	value:
http://	www.site.com/	page.aspx	?	pid	=	4

This particular querystring uses server-side scripting using ASP.NET (aspx as the page extension gives this away). It also shows that the application will accept at least two parameters, pid and sec, and it will attempt to process values you pass in the querystring. Of course, the application should check the validity of the values passed into the page, but because the developers are too busy refreshing Calico Monkey over and over to see if the new episode is finally up, they don't have time. To really make your developers' inattentiveness shine, you should apply all the normal boundary analysis sorts of mean things to the parameters on the querystring as you would to any control or edit box on the pages themselves:

If the value appears to be a number, replace it with letters.
Try to put as many characters/numbers into the value as you can; if it's currently 49, change that to 7435378347425675432676790676767674676.
Try to access the page without a value specified. That is, remove the value from the querystring and send that to the Web server.

You'll find, as I mentioned, that many times the developers just trust the querystring to have valid information on it. When you log your defects with the pretty, pretty screenshots of stack trace after stack trace with Input string was not in a correct format and Microsoft VBScript runtime error '800a0006' kinds of errors, your developers will no doubt try to put you off with the standard, "Only QA would do that" sorts of responses. Sure, circumstances never would occur where another site/application would have to pass values to your application like that. Say, a banner ad or e-mail campaign or integration with a third party widget somewhere in the ether. Oh, no, Mr. Developer, only QA thinks that the application should be constrained to behave correctly in all circumstances, not that it should behave correctly in correct circumstances. But then again, who listens to QA?

Every Single One of Us, The Error’d Outside

The Director — Wed, 25 Jul 2007 19:38:06 +0000

Over at Worse Than Failure, they've captured some error messages on marquees and message boards. Although I haven't seen any of these particular errata, I have found a number of ways to break kiosks that use Web pages to deliver their content. One of my favorites is to navigate the links to external sites provided that give local details and color and whatnot; within them, you can sometimes find the mailto: links that trigger Microsoft Outlook Express to open up or you can get access to forms and whatnot that you can manipulate with the tap of the screen. Actually, there's this kiosk down at a certain building in downtown St. Louis where you can double-tap a row on the screen of the building's residents, triggering a stack trace. Seems the developer forgot to account for that action; after all, who double-clicks on a kiosk? Ah, the worst of both worlds.

Reminder to Project Managers, Account / Engagement / Client Relationship Managers

The Director — Fri, 27 Jul 2007 17:43:20 +0000

Remember, Project Managers, No is an unlimited resource; you can use it as often as you like to deal with timeline compression and feature creep, and you will always have more for the next time. And here's some new information for everyone in direct contact with the client: that word that everyone's always talking about is pronounced noʊ. Try to use it in conversation with the client sometime.

D-E-F-E-C-T, Find Out What It Means To Me

The Director — Mon, 30 Jul 2007 21:07:54 +0000

If you're in the software industry and you actually, you know, run the software (which excludes most managers, client facing people, technical writers that produce the manuals, many developers, and some quality assurance professionals whose jobs rely upon creating pretty reports with neat statistics), you'll probably encounter an issue, otherwise known as a defect or a bug, with your application or Web site. What should you do? If you ask a developer, he'll probably tell you don't do that. However, you should probably log an issue or otherwise report the problem so that your organization can earnestly act as though it's going to fix the problem. Many organizations create elaborate procedures to trace the defect's accountability and to standardize defect report information. Many software packages ensure that users enter the same sorts of information for a defect report, but those expensive applications do nothing to audit the quality of the report that the users enter. Some organizations just use spreadsheets and misplaced e-mails in lieu of spending money or installing Bugzilla. Regardless of what technology your organization uses, the quality of the content within the defect report is more valuable than the most rigorous procedures in handling the defect report. For no matter how stringent the processes imposed by the software package, some users use the defect tracker software as a personal Post-It™ notepad, logging such shorthand defects as this:

dlgEntry not updating tblUsers

Although this defect means something to the developer or data architect who logged it, the rest of the team who might not be as familiar with the code as the this developer cannot evaluate or investigate, much less retest, the defect after a developer, probably the one who logged the defect, corrects the problem. No doubt the developer thinks that this is not a bug, it's a feature of the bug report. By creating a well-formed defect report, you can ensure that everyone will understand the defect you have uncovered and ensure that all other people who need to deal with the defect, whether a manager who assigns the defect, a developer who recreates the defect to fix it, or a software tester who retests the fix. To properly capture the user mindset and expectations as well as the behavior of the application, you should include the following sections in a defect report:

A relevant title that briefly identifies the problem and expresses it in a parallel fashion to other issues.
A summary that describes the current behavior of the application and what you were trying to do.
Steps to recreate the defect that describes exactly what you did and what values you entered to create the defect.
Expected behavior that describes what you think should have happened.

Many people often overlook the importance of the title of the defect, but the title displays in many table views when looking at lists of issues, so a properly formatted and written title will make it easier to understand and identify the underlying problem. It might behoove you to determine and apply a standard titling convention to make sure that the titles contain information relevant to your organization and use parallel structure for sorting purposes. In the past, I have promoted and used the following sort of title naming convention:

Section > Window/Page Title > Issue

The title for our sample for the preceding example, focusing on the dlgEntry, would look something like this:

User Entry > Doesn't save user data

This way, when your defect lists become longer, you can sort by the title and all issues with the User Entry section or dialog will fall into place. Most titles have strict limitations on character count, so heavily nested dialogs or pages will require some creativity to limit the breadcrumb trail in the beginning of the title. Also, remember that in many cases you can search lists of defect titles, so using standard keywords could help navigate large lists of defects. The summary component of the defect report should contain a sentence or more about what happened. You should summarize the Web page, dialog box, or window you were using, the task you were trying to perform, and the actual result of your action. For example, the defect sample given the preceding section:

dlgEntry not updating tblUsers

Should include a fuller summary, such as:

The User Entry dialog box doesn't seem to save the user data; when I enter a new user and then try to review that user, the application cannot find the new user.

This sentence conveys a better sense of what has happened. Although a developer might understand what dlgEntry means, most users would more easily recognize it as the User Entry dialog. This summary also tells people who cannot directly view the tblUsers table in the database what is wrong and how to see the effects of the problem. Developers and testers will appreciate if you add the precise steps you took when encountering and recreating the defect, including test data you used. This lets the developers recreate the defects on their own, without requiring a phone call to you so you can walk them through the issue or without marking the issue as nonreproducible because they couldn't even bother to call you. The testers can use these steps as a test case to retest the defect once resolved. For example, to describe the steps for the sample defect, you might enter:

Run the application.

From the User menu, select New User.

User Entry dialog box displays. In User Name edit box, type NewUser.

In the Password edit box, type password.

In the Confirm Password edit box, type password.

Click Save.

Success message displays. Note user ID number.

From the User menu, select View User.

View User dialog box displays. In User ID edit box, type user ID number of new user.

Click View.

An error message indicates that the user cannot be found.

This comprehensive set of steps and the result provides a complete script that anyone can use to recreate the issue you found. Although it takes longer to write than shorthand, its clarity ensures that someone else can follow the same steps you did without your intervention. Finally, the defect report should contain an explanation of what you expected to happen or what you think should happen instead of the current behavior. If you're lucky enough to have requirements for the application, you can refer to those requirements specifically here. If not, you must capture what your expectations as a user. This expected behavior can serve as a starting point for business analysis to how the application should behave in the situation. If the issue represents a gap in your understanding of the application's behavior, the resolution can serve not only as education for you but also as a reference for future users. If the issue requires resolution such as a code change or configuration update, the expected behavior provides the successful test criterion. For example, our sample defect would only need the following expected result:

When you enter a new user, you should find the user in the View User dialog box.

You should make this description of expected behavior as detailed as possible to ensure that anyone else can understand and retest the defect as necessary. If you compare the sample defect reports, you can easily see that one provides better information. The cryptic defect report that looks like this:

dlgEntry not updating tblUsers

could better serve the team with greater detail:

The User Entry dialog box doesn't seem to save the user data; when I enter a new user and then try to review that user, the application cannot find the new user.

Run the application.

From the User menu, select New User.

User Entry dialog box displays. In User Name edit box, type NewUser.

In the Password edit box, type password.

In the Confirm Password edit box, type password.

Click Save.

Success message displays. Note user ID number.

From the User menu, select View User.

View User dialog box displays. In User ID edit box, type user ID number of new user.

Click View.

An error message indicates that the user cannot be found. When you enter a new user, you should find the user in the View User dialog box.

The second defect offers a user-friendly and easily comprehensible illustration of the issue. By making sure that defect reports contain good, informative descriptions of defects, you'll reap benefits that mere technology would provide. Complete, well-formed defect reports will ensure that all team members can review, recreate, and retest defects easily, so your team can pass tasks easily. Complete defect reports also prevent duplication of defect reports. Users can review existing defects and can more easily determine whether someone else has already uncovered the issue they encounter. A more detailed and properly written defect report will better serve your organization by creating an artifact that can stand alone and is less subject to interpretation or dependency upon a single person's unwritten knowledge of the issue or application for comprehension. The benefits aside, some resist or simply overlook creating a well-formed defect report because it can be time consuming during the issue's creation. This time, however, will be recouped at every other stage of the defect handling process and will ensure that, two years from now when some junior developer who's just started starts marking issues resolved en masse, you have the information required to actually retest the issue and reopen it.

Cheap Shot Your Application’s Import Feature

The Director — Wed, 01 Aug 2007 16:53:39 +0000

Yes, it's one little menu command or maybe button on a toolbar, but the Import... command exposes the soft underbelly of your application. You know how your developers always shirk adding validation logic to any administrative tools because only administrators will use it, and administrators never try bonehead things? Well, the import feature offers access to the actual data that users normally have to use one or more screens on your application to enter. One or more screens to which developers have possibly added data validation after much prompting and shaming from the QA staff. But short of actually corrupting the data in the database directly (which is fun, and I'd recommend trying it for its own sake), the import feature offers a means to enter crap into the database (or try to, anyway) that nature did not intend for that database.

Understanding the File

Now, to properly cheap shot your application through the import wizard, you have to know something about what it's trying to import. For the most part, this piece will talk about flat files that might get sucked into your database, but import commands sometimes work with the contents of other databases or pieces of hardware. Some of the following dirty tricks can apply in spirit, but not in the direct form. The import wizard will expect the file to come in some sort of standard format. Easy examples are comma-separated value files (CSVs), tab-separated files, and Microsoft Excel spreadsheets (.xls). Other, more specific file formats include fixed-width files, files delimited by things other than commas or tabs (such as the pipe character), XML files, or industry-specific files (such as MOL and SDF files for our chemistry friends or MARC records for our attractive librarian friends). I say that the spreadsheet-like files are the easiest to work with because you can open them up and look at them to understand what's going on. It's particularly easy if you've got header rows on the columns; then it's just a matter of altering the data within to test your import wizard completely. The other forms require a deeper understanding of the file format/schema/standard which you can probably find on the Internet. To test out the import functionality of the application, you're going to want to not only make sure that a properly formatted file will import correctly and its data will appear, magically, whenever you try to look at it. Oh, no, you have to make sure that corrupt or improper data in the file fails gracefully. Given the inherent laziness of developers, it will probably fail spectacularly and amusingly.

Punch The Data In The Kidney

Let's say, for example, we have a data file that should contain first name, last name, e-mail address, and zip code. A properly formatted comma-separatedfile would look something like this:

doug,white,dougwhite@qahatesyou.com,19891 pete,brown,petebrown@qahatesyou.com,19821

You run the import wizard, and it just might work. To break the application, take each element that the application can import (the column in a simple file, the elements in an XML file, and whatnot) and add invalid data to it. For example, if the column is supposed to contain a zip code, you'll want to enter an alphabetical string (ABCD), a number with a decimal point (12.34), a number with fewer than 5 digits (1234) and a number with more than 5 digits). For the first and last name, enter a string that exceeds the length limit allowed by the database (you should find yourself limited to this on data entry screens). For the e-mail address, enter e-mail addresses that are incomplete, that exceed the length limit, and that use characters not found in e-mail addresses. Ergo, your file will start having lots of lines in it:

,white,dougwhite@qahatesyou.com,19891 doug,,dougwhite@qahatesyou.com,19891 doug,white,@qahatesyou.com,19891 doug,white,dougwhite@.com,19891 doug,white,dougwhite,19891 doug,white,@qahatesyou.com,19891 doug,white,dougwhite@qahatesyou.com,ABCD doug,white,dougwhite@qahatesyou.com,1234 doug,white,dougwhite@qahatesyou.com,12.34 doug,white,dougwhite@qahatesyou.com,-19891 doug,white,dougwhite@qahatesyou.com,1989123

You want each error condition on its own line to make sure that you try a record that should succeed except for that problem with the individial datum you're trying to test. Hey, I wonder what the application does with white space in the record?

doug ,white,dougwhite@qahatesyou.com,19891 doug,white,dougwhite@qahatesyou.com,19891 doug,white ,dougwhite@qahatesyou.com,19891 doug, white,dougwhite@qahatesyou.com,19891 doug,white, dougwhite@qahatesyou.com,19891 doug,white,dougwhite@qahatesyou.com ,19891 doug,white, dougwhite@qahatesyou.com,19891 doug,white,dougwhite@qahatesyou.com, 19891 doug,white,dougwhite@qahatesyou.com,19891

Hey, what if those are required fields for a record? Shouldn't we test to make sure you can't load empty data? Why, yes, we should:

,white,dougwhite@qahatesyou.com,12342 doug,,dougwhite@qahatesyou.com,12341 doug,white,,12345 doug,white,dougwhite@qahatesyou.com,

You know, perhaps each record should be unique. In which case, this should be disallowed by the application during import:

doug,white,dougwhite@qahatesyou.com,19891 doug,white,dougwhite@qahatesyou.com,19891

You should also check uniqueness by trying to import a row of data that you've entered by hand to make sure that the application is not only checking the file for duplicates, but that it is also comparing the file to the existing records in the database. Would that be too much to ask? Well, for your developers who need to step away from their desks from time to time for a quick brainstorming session around the refrigerator with the Mountain Dew Code Red, yes, it is. Basically, that's what you do with the data elements of the files themselves. Whether they're encapsulated in XML elements, columns in a spreadsheet, or 040 $a , you can easily embarrass those rockstars with the big paychecks by putting the wrong kind of information in your import file.

Kick the File in the Knee

But let's talk about the file itself. You can also show up the developers by marring the files. If the file has a proprietary format or header information, corrupt it. I don't mean offer to loan it money to buy it a house which you'll then buy from the file after a year at 10x what the file is "paying" for it now; I mean replace the elements of the header with junk data. For example, a MOL file depends very clearly on having the data begin on the fourth line:

-ISIS- 02120300062D 17 18 0 0 0 0 0 0 0 0999 V2000

What happens if add an extra line at the top of the MOL file so that the information is out of place?

-ISIS- 02120300062D 17 18 0 0 0 0 0 0 0 0999 V2000

HAVOC! Working with XML files? Create data files that don't adhere to the schema but use the same element names. Nest things too far. Blast it all to heck! Now, don't forget the files themselves. Why, what happens if you have a file with the right extension and it has 0 rows of data in it? Empty files can bomb out applications because administrators, the developers know in their heart of hearts, would never choose to import an empty file What happens if your file has 1,000,000 rows of data? HAVOC! Let your conscience, and your operating system/PC be your only limitation. What happens if your file uses DOS line breaks instead of UNIX line breaks, or vice versa? HAVOC! So there you have it, the basics of testing an importer on an application. It's almost as much work as testing the GUI itself, which is appropriate since it's offering a quick and dangerous way to do the same thing the pretty little windows and dialog boxes are. Also, it's probably doing it without a net, since it's one little feature hidden behind a single menu command or button. But it gives your users an easy way to junk up their databases without a warning from the application for which they paid so much money. Also, it will help to get your personal defect count up and lead to fear and loathing amongst the C# dressed men. That, my friends, is worth the time and effort in itself.

Nor Proofreaders, Apparently

The Director — Wed, 01 Aug 2007 17:14:15 +0000

There’s a Right Time and a Wrong Time for Data Validation

The Director — Wed, 01 Aug 2007 21:01:12 +0000

And on the page load is probably the wrong time:

Tools of the Trade: UltraEdit

The Director — Thu, 02 Aug 2007 15:11:11 +0000

The following is not a compensated post; I'm merely extolling the virtues of a piece of software I found useful. I have been a fan of UltraEdit probably for about a decade; I cannot actually remember the last time I had a PC onto which I did not install this robust little text editor. Notepad and Wordpad come free with Windows and most workstations in offices come with Microsoft Word these days, so you might not see the value in adding another program to your desktop. However, UltraEdit adds a number of features not currently available in these other applications, including:

Syntax highlighting, which shows keywords in a number of languages in another color to make the files more easily readable. The list of languages and the keywords themselves are extensible, so you can make your own as needed (I once made keyword files for MOL and SDF chemical data files).
Tabbed view, which means I can have a bunch of files or empty files open and can work on them at once, seeing their contents at a glance. Neither Notepad or Wordpad offer the multiple files thing, and Microsoft Word allows multiple files, but not a tabbed view.
Find or search/replace in files, which lets you dig through all the files in a folder to find what you need.

At only $50 a throw, I install it on every PC I own, as I mentioned, and I've also introduced it to every workplace where I've been in the last decade. Friends, let me tell you that this includes a rather surreal experience I had with my last Corporate employer. You see, I was hired on as a low-level QA Analyst and a publicly-traded multinational company that had layers and layers of managers. The fellow who hired me, as a matter of fact, had an MBA but no experience in QA when he was hired to be the QA manager. He did, however, have plenty of experience with risk/benefits analysis and meetings, so when I asked for permission to purchase UltraEdit, he wanted to know why I needed it. I explained it briefly. He said he wanted to know more about it. I wrote a 7 page document that detailed the things you could use it for and why it beat Microsoft Word and Notepad. After reviewing the document, he scheduled a meeting with the QA team (him, a lead, and three testers/analysts) to discuss it. However, before the meeting, the team lead called a pre-meeting so we could discuss what we would tell the QA manager. So I spent like 6 hours on the document, 1 hour in the pre-meeting, and 1 hour in the meeting with the QA Manager to facilitate the purchase and installation of $35 (at the time) software. Ultimately, it came down to a conversation between the QA lead and the QA manager, and the QA lead said "XML," so the company bought a three license pack for $105 and I had my UltraEdit. I quit a couple months later, meaning no one else ever used it, but on one hand at least it was only $100 for UltraEdit and not $25,000 for an enterprise QA package. One the other hand, my reputation as Johnny UltraEditSeed lives! That being said, here are some of the things I told them you could use UltraEdit for, because you can (and I do):

Editing data files.
Editing .ini files.
Writing batch (.bat) files.
Reviewing log files.
Editing HTML files.
Writing blog posts.
Reviewing/editing XML files.
Editing other scripting language files, including Ruby and PHP.

As you can see, the uses are endless. If you need to do those things. If you're the sort of QA professional who spends all day running reports out of your defect tracker and holding meetings to discuss the reports you've run and then writing e-mails to summarize the meetings about the reports, I guess you won't understand.

Fuzzyoumang

The Director — Fri, 03 Aug 2007 21:09:36 +0000

The Mozilla Foundation plans to give away its own security tools, including a fuzzer:

Mozilla Corp. will release some of its homegrown security tools to the open-source community, the company's head of security said Wednesday, starting with a "fuzzer" it uses to pin down JavaScript bugs in Firefox. The JavaScript fuzzer, said Window Snyder, Mozilla's security chief since last September, will be handed over tomorrow morning, following a presentation at Black Hat, the two-day security conference that opened today in Las Vegas. "We're announcing that we'll be sharing our tools with the community," said Snyder, "and releasing the JavaScript fuzzer then." Other tools, she said, would follow, including fuzzers that stress-test the HTTP and FTP protocols. Those two, however, are not ready to offer up to outsiders, largely because Mozilla wants to wrap up talks with other browser vendors before they do.

So if you haven't been fuzzing your applications, you're running out of excuses.

Developers Unleash More Wild Magic

The Director — Sat, 04 Aug 2007 15:49:11 +0000

What, foolish mortals playing with things they don't understand because it's cool? Say it ain't so:

Software developers using Asynchronous JavaScript and XML (AJAX) techniques to jazz up corporate Web sites are failing to pay attention to some very fundamental security issues, researchers warned at the Black Hat USA conference here Wednesday. As a result, many companies that have rushed to AJAX-enable their sites may be dangerously vulnerable to a variety of Web-based threats they're not even aware of.

So once again, software "engineers" put on their pointy hats, mutter some incantations, and in most cases, the user gets data loaded into a Web site without a page refresh, but every once and again, a boy in India suddenly gets rich, an airplane drops 10,000 feet rapidly, a Martian rover does a doughnut, or an Eastern European crime syndicate steals the user's data. And the grown ups in QA have to try to simulate all of those situations to test for them.

More Developer Hubris; Sorry, Expert Developer Hubris

The Director — Tue, 07 Aug 2007 14:18:23 +0000

I thought an article entitled A Guide to Hiring Programmers: The High Cost of Low Quality would thoroughly explain why spending lots of money on developers was a bad idea, and how you could improve your process by putting development and the developer staff back into its place in the software development lifecycle. Unfortunately, while the article makes some good points about how a good developer is better than a bad developer (I mean, when isn't good better than bad?), it falls too easily into the trap of DEVELOPERS ARE LIKE THE GODS!

Companies need to stop thinking about their developers as cogs in the machine. They are more akin to artists, authors, designers, architects, scientists, or CEOs.

The rest of the piece explains why expert developers are worth top dollar: because they're ROCK STARS! Expert/experienced anything are worth more than less skilled/less capable employees because they have experience in their field as well as problem solving ability related to their job duties. In many cases, developers have seen the rudiments of software (presentation, data access, network communication) before so they can apply those lessons to new problems at hand. Big deal; any produce clerk at a grocery store who's been on the job for a while will develop a system for culling stuff left on the rack quickly, for optimal filling patterns, and for building appealing displays. But does that mean that he or the equally skilled meat clerk is the axis around which the whole store rotates? No. Likewise, having smart developers is better than having dumb developers, but it won't make or break your organization because smart developers are only cogs in your machine. If you blow all of your money on Expert Developers, you'll not afford smart project managers, smart QA, smart customer managers, and smart everything else. And an organization run by and for Expert Developers will do lots of cool and smart stuff, but that's rarely the same as profitable stuff.

Tools of the Trade: Paint Shop Pro

The Director — Wed, 08 Aug 2007 21:58:23 +0000

The following is not a compensated post; I'm merely extolling the virtues of a piece of software I found useful. As I've mentioned, ~~falsifying~~ taking screenshots is a good means to capture details for defect reports. Your basic Windows install comes with Microsoft Paint, which is a mechanism you can use to save and manipulate your images, but it's very clunky, with rudimentary tools and only the ability to have one file open at once. Some people use Microsoft Word or PowerPoint for their picture editing ability and save their screenshots as documents or slide presentations, but some of our outsourced friends might not have Microsoft Office on their workstations. Remember, you want to save those screenshots as an image format so the developers can ignore the obvious that's presented by an image editor or a simple Web browser. I've used Paint Shop Pro since version 7 (which I still have installed on my main workstation, since there's nothing I've needed since 2001. Jasc and then Corel have come out with newer versions every couple of years, and they're still priced under $70 a seat (unlike other, more expensive graphics editors). Like UltraEdit, I've spread it across several of my employers. Paint Shop Pro has a pretty good set of tools for circling or highlighting issues on screenshots, for adding text for emphasis, and for altering Web 2.0 user submissions to give cute little doggies red demon eyes to match your QA soul. You can do all of these at once because you can have more than one file open at a time. So if you haven't considered a graphics editor, consider this one. It costs under a hundred, so you're not exactly breaking the budget on it, either.

A Proper QA Playlist

The Director — Thu, 09 Aug 2007 16:09:39 +0000

Last evening, as they worked late into the night on a production deployment, a tech lead and his tester asked my opinion on a proper QA playlist. Well, not exactly; no one ever asks for QA's opinion, but they all surely receive it, often accompanied by expressive hand gestures or an icy stare. So what is a good playlist for your iTunes while you're breaking software? I offer the following which served as the sweet sounds by which I serenaded a previous collection of co-workers. The open floor plan ensured that everyone got to enjoy the music that properly captured the QA mood. So here's what I had on the playlist:

Artist	Album
Iron Maiden	Iron Maiden Killers A Matter of Life and Death No Prayer for the Dying The Number of the Beast Piece of Mind Powerslave Seventh Son of a Seventh Son Somewhere in Time Fear of the Dark
Metallica	Master of Puppets Ride the Lightning ....And Justice for All Metallica
Nullset	Nullset
Pink Floyd	The Wall A Momentary Lapse of Reason Animals Dark Side of the Moon
Motley Crue	Dr. Feelgood
Evanescence	Fallen
Faith No More	"Epic"
Queensrÿche	Empire Operation: Mindcrime
Dokken	Beast from the East Under Lock and Key Back for the Attack
Heart	"Barracuda"
John Cougar Mellancamp	"Authority Song" "When the Walls Come Crumblin' Down"
Lillian Axe	Psychoschizophrenia

Ah, the thundering threat of destruction that presaged QA. All it needs is some Wagner. Apparently, too, the word got out that if QA was playing Metallica, it boded a very bad day. One morning, when I had queued up Master of Puppets to start the morning, I found a number of project managers/customer managers stopping by to try to placate QA. YOU CANNOT PLACATE QA, YOU CAN ONLY SURVIVE QA (SOMETIMES).

Emily Dickinson, Project Manager

The Director — Sat, 11 Aug 2007 20:08:33 +0000

It sure sounds as though she's worked on a timeline or two:

The Days that we can spare Are those a Function die Or Friend or Nature — stranded then In our Economy Our Estimates a Scheme — Our Ultimates a Sham — We let go all of Time without Arithmetic of him —

Gallery of Stack Traces: Date/Time Fun

The Director — Tue, 14 Aug 2007 15:25:59 +0000

As you know, stack traces provide QA with the joie de vivre and esprit de corpse that get us through the day. As part of an ongoing series, I am going to present a series of some of my favorite stack traces that never fail to bring a smile to my face and a Dosso double-snap to my fingertips. From the Gallery today, we have two extra fun .NET stack traces that occur, frequently, when you do naughty things with dates and times. For example, the following stack trace occurs if the application lets you enter a string that it expects to be a date, but it's not: Click for full sizeString was not recognized as a valid DateTime; well, it's good to see the system recognizes the obvious. That is, there is no first of undecember (that is, 13/1/1991). The best part is that the application had some validation in place when the user clicked Submit, but that drop-down lists triggered page refreshes, so I could bypass validation and wreak havoc. You'll elicit this stack trace easily when presented with an edit box that allows you to enter a string without constraints that the application expects to be a date. Developers could easily prevent this problem with drop-down lists for month/date/year, separate edit boxes with appropriate validation, or calendar controls. But that would be much more work. The second stack trace occurs if the application doesn't check to handle the database's earliest possible date: Click for full sizeSqlDateTime overflow. Must be between 1/1/1753 12:00:00 AM and 12/31/9999 11:59:59 PM. If your name is William of Normandy, you can forget putting that invasion of England thing on your calendar if it uses Microsoft SQL Server as its back end. Sure, you snicker, you foul developer, about the folly of putting a date from pre-history in a database (to a hip developer, C is ancient history, and everything before Woodstock which was the coolest thing before OOP, he's been told, is pre-history). However, in certain applications, users might need to do this. Like in geneology programs and legal programs that trace land deeds from time immemorial. Developers could use different variable types to hold dates or could perform validation to prevent this stack trace, but instead often rely upon the good will and infallibility of keyboardists everywhere to hope this stack trace remains hidden. And just to be clear, I'd like to be the first to offer the Y10K bug consulting services. Because, as you in the field know, someone's still going to be using a Microsoft SQL Server and Microsoft IIS platform in 9998 because they paid for the software 7 millennia ago and didn't feel the need to update it since it worked passibly well since then. So I guess some thanks are in order; Developers, thank you for leaving off these simple steps so we can log these same issues over and over again for every last application or modification you write.

Sales Cred Is Not Street Cred

The Director — Wed, 15 Aug 2007 15:07:11 +0000

In an otherwise tolerable article entitled "Defect Tracking Lacks Appeal But its importance is at a premium" about how defect tracking can serve other useful functions in addition to the already useful function of taunting developers using a database backend, the author inserts this howler:

Nathan Rawlins is yet another bug-tracking pundit who declared process to be more essential than ever. Rawlins’ role gives him sufficient street cred to make that claim. He is a senior director of product marketing at San Mateo, Calif.-based Serena Software, a company that generated more than US$250 million in revenue in fiscal year 2007 selling application life-cycle management software, a category that includes defect tracking.

To those of us in the trenches, sitting in a corner office in San Mateo evangelizing a software product does not give one street cred in QA. Scars from lessons learned meetings, QA lab tattoos, and large number of bug-shaped decals on one's cubicle wall, each representing a stack trace eliicited, now that gives you QA street cred.

Make Your Site Look Like A Security Risk

The Director — Thu, 16 Aug 2007 15:47:18 +0000

There's nothing like making your Web site look like a security risk to the general user:

Especially since those in the know recognize the domains as delivering ads. So one of the ads on the page under review triggers this alert, making it look to Joe Consumer like....well, who knows what evil malware/spyware/spam Joe Consumer would think this message represents? Regardless, Joe Consumer thinks your Web site is the problem.

Sometimes, developers mock up certificates for dev or test environments so the company doesn't have to pay for an extra certificate. Sometimes, during deployment to production, mistakes happen. Did I say sometimes? I meant Always. Once in a while, they deploy an invalid certificate like this one.

How do you find this error? You look at the Web site. Which, and for the life of me I cannot fathom why this is, most people involved in developing Web applications or Web sites do not do. They leave it to QA, if they have QA, and to the client to find the obvious for them.

Meanwhile, Joe Consumer looks at this and thinks I just got a virus and never comes back to the ad-delivering Web site.

Third Party Disintegration

The Director — Mon, 20 Aug 2007 18:15:09 +0000

You know what's worse than your development team? Someone else's development team. At least, your development team has to deal with you. Whether you have the dev team frightened of you, fearing for their existential meaning and worth because you can hold them accountable, or you have the dev team ignoring you until such time as you can triumphantly claim, "I told you so!" when something fails spectacularly, many shops don't even have that. And they're proud of it. Chances are, if you have to work with some other group for any aspect of technology or design, it's going to be crap. Because that QAless organization has a better grasp on the whole "It's more profitable if we do it with eyes closed" and "Deliver the client's minimum expectations at maximum bill rate" things than your organization, which is why you're employed and why your company is at the mercy of slops. So when you're working with a third party product or service, you must demand that that product/software meet your level of quality. Get it in writing, because otherwise they're going to cheap out on you, and you know who your client is going to blame? You. A lot of organizations, even some with QA staffs, feel that as long as they can prove that someone else is to blame for the failure (that is, the third party vendor or solution), they don't have to worry. Even if the client has told you to integrate with crap, they'll expect the end result to be a rose garden, not deeper manure. That being said, here are three ways your "integration" might fail based on what you're integrating. 1. Integrating an off-the-shelf product. Sometimes, an off-the-shelf product offers some functionality that your organization needs. Why reinvent the wheel? A couple of API calls and a couple of hacks, and WALLAH! You've got some polished piece of functionality prebuilt to bundle into your project. The integration point between the two pieces of software is a tenuous link at best. If it's commercial software, the application probably has some internal validation controls to prevent users from doing bad, bad things within it. However, those API calls or whatnot are probably very, very trusting. And if your developers insist upon integrating some sort of open source software into it--well, God bless you, Howard; you're on your way into Hell. For example, I once worked on a chemical modeling piece of software that used a third party sketching component when the user wanted to draw a model. When the user wanted to draw a molecule, the application popped open this chemical sketcher so the user could draw bonds and chains and whatnot. Of course, I am not a chemist and don't play one on television, but I did learn enough about this third party component to get an idea of how it worked. When the user closed the sketcher or selected Transfer to Application Under Test from the File menu, the sketcher passed the molecule back to the application our developers were developing. Now, that commercial piece of software offered some hints and visual cues, such as special highlighting when you tried to create the seventh valence bond on a Carbon atom. Why, that flies in the face of all natural law. But pass it back to the application under test, and HAVOC! Or create a molecule that's a mile wide by a mile tall and pass it back to the application under test, and HAVOC! Because the application under test was accepting, uncritically, data from the third party software that the third party software knew was flawed or that wasn't flawed, just really, really big (sort of like the Hamlet test with atoms). The developers, lazy souls that they were, trusted their counterparts at the commercial software company to do their validation work for them. At a cost of many, many defects with my name in the submitter field. 2. Integrating with other service vendors. Sharing data and functions with other vendors is fraught with danger. Apparently, only 37% of companies using SOA are receiving positive ROI. And of that 37%, 100% of the survey respondents know their bosses review their survey responses (as estimated by QAHatesYou.com). Want to know why? Because integrating with other companies' development efforts squares or cubes the worst parts of the development lifecycle. While the ultimate clients' needs shift, those clients incompletely convey those changing requirements to you and your integration partner, and your integration partner won't convey the changes they made to the Web services or COM interfaces or whatever neat new technological mechanism erupts in the next couple years. They probably won't test them, either, or allow you direct access to their databases or build in any sorts of data validation mechanisms you can use to make sure your company's calls to their software work. Which would be helpful, since they probably lack a QA staff and the calls won't. Sure, sure, I realize that your company's projects will all have proper documentation, excellent communication, and conference calls that are not two minutes of greetings and eighty-eight minutes of recriminations, accusations, and denials. I also realize that you're telling me the same thing you tell your client. 3. Using an outside company for small projects. When you've got a tolerable Web site up and running and you outsource small portions of it--such as forms processing--to outside vendors, you can get poor results. For example, the St. Louis Post-Dispatch recently outsourced some contest form processing to an outside vendor. Here's the banner ad and the header for the newspaper's Web site: Click for full sizeClick the banner ad, and you'll go to the contest form (soon to be dead, no doubt. Well, I'll start out with quibbling about the mismatched copy and control here: Click for full sizeThe copy says to click the box, but the designers have added a radio button control instead of a checkbox. That's senseless, but no doubt it was quick and senseless. Unfortunately, the real problem lies with the thank you page that displays after the user completes the form. As often happens, the third party vendor tries to recreate the look and feel of the client site, often to disasterous results. Here's what this particular thank you page yields, with an incomplete banner and missing images on the mouseover: Click for full sizeIt's a quick one off, but ultimately this poor design and the missing images reflect poorly on the St. Louis Post-Dispatch's Web site, not the form processing third party. But they must have hit the minimum the Post-Dispatch would accept with the tight deadlines that sweepstakes and contests mandate. So what's my point? I'll sum it up, briefly, for the developers who might read this and be confused about my point. My mates in QA: be warned that other companies' developers are less trustworthy than yours. If your project is going to use someone else, you better make sure to show them the billy club early and often to make sure that your integrators do the work up to your standards and that your developers make the integration airtight.

If Only Alt Text Rendered HTML

The Director — Mon, 20 Aug 2007 20:38:53 +0000

One of the dangers of becoming too automated in your page generation:

You can automate junk right into your alt text.

A Good Defect Process Is Worth 1000 Swear Words

The Director — Wed, 22 Aug 2007 21:11:48 +0000

Well, you've found an issue, ungentle tester; now what? Well, we'll assume you have some sort of mechanism in place to track that issue, but the software mechanism (usually called a defect tracker, but sometimes known as the developer's junk e-mail box) only serves to provide a technological means to support a process that handles these issues. That is, your organization needs to have an effective idea of what to do when QA starts identifying what the developers have done wrong and how to make sure that the issues are addressed correctly. That is, the developers are brow-beaten into actually opening up their little script editors/Eclipse/IDEs and making things right. This post, then, will discuss various ineffective defect processes I've seen and how issue resolution should work. Here's a very common defect/issue resolution process, very popular in the world: This process is only the second-most popular though, since the "No QA at all" process which uses customers/users as the entities whose discovery of issues yields the result of "Nothing."Obviously, we can see the fundamental flaws in that process.I've seen this novel process in place in more than one project/product, unfortunately: Click for full sizeThat is, QA goes through all the trouble of finding and logging issues, but the developers sit on the issues until a deadline has passed, at which time they can just bulk close the issues. I actually did work once at a company that rearchitected and rewrote its enterprise level software annually, at which time all issues in the defect tracker were closed summarily.Sometimes, though, you have developers who do respond to issues logged, and the process mutates into something like this: Click for full sizeNotice what QA does when it uncovers an issue; it evaluates the issue to see if it's a real issue or a misunderstanding of requirements. Then, QA tries to recreate the issue on its own to make sure it can reproduce the issue. Because the developers won't put any effort into research, because if it's not on YouTube, a bug doesn't exist for the highly-touted technical rock stars. Then, QA logs the issue and assigns it to a developer in charge of the project or functional area.But, Director, shouldn't QA assign it to a manager or project manager? Well, you could do it that way, but QA is going to find a large number of issues, no doubt, and the management official in charge of vetting the issues will serve as a bottleneck. I've always favored keeping the lines of communication open between peers in different teams. Ergo, trust your QA member to assign it to a developer. Of course, trusting the developer would be a mistake, because as we can see in this sample process, the developer will go through any and all of your defect tracker's resolution statuses to try to get out of fixing the issue.This particular flow chart shows this process going onto infinity, but the QA/Developer "It's an issue"/"No, it's not"/"Is too"/"Is not" thing will last until the deadline, at which point the software will ship with a set of known issues that your organization could have fixed. Now, you can expect a certain amount of give-and-let-die between development and QA regarding issues. Instead of letting these disputes spin into infinity, you need to have some diamonds and arrows in place so that someone has the ultimate authority to adjure. Of course, I would prefer that QA have the ultimate say, but most people in the technology field would disagree. Well, a fie on them. However, you should probably not leave it to someone in the development chain of command unless you're comfortable with the following process: Click for full sizeTo be honest, I didn't work at the place this process describes (and I tell you, friends, I could not), but a QAmrade did. Apparently, when a tester found an issue, he or she then had to present it to the developer and show the developer; if the developer agreed that it was an issue, QA could log it in the defect tracker. That probably looked really good on the defect reports.No, instead of a developer or a QA person acting as that role of arbiter, someone facing the client/user should decide the issue's priority. This usually means a project manager or a customer manager person, perhaps someone at the help desk since that's who's going to answer the phones and explain why this happened.The following process shows a semi-ideal process that involves an arbiter who reviews disputes between QA and developers regarding issues and can ultimately decide whether to fix an issue. This works when the arbiter is quality-minded and has a good view of the whole project/product and an issue's relationship to the whole. If the arbiter is a developer-groupie, you might as well forget it; the process will just look like the one above with the "arbiter" instead of a developer waving his or her hand Jedi-like, trying to convince reasonable people that these are not the bugs you're looking for. No, the semi-ideal process looks like this: Click for full sizeIn this case, QA finds/recreates/logs the issue, and the developer tries to recreate it. If the developer has trouble recreating it, he asks QA for help recreating it. This flies in the face of convention, where developers who cannot understand issues simply mark them as non-reproducible. Once a developer defensively claims it's not an issue, then you bring in the arbiter who can determine if it's an issue or not. Hey, maybe you even have the developer and QA converse to see if QA still thinks the issue exists in light of development's rationalizing insight. Then, the arbiter talks it over and makes a decision. QA can appeal, but ultimately, a decision ends the resolve/reopen marathon dev and QA could have in the defect tracker otherwise.I call this the semi-ideal process, because the ideal solution would involve cattle prods. But it allows for rapid defect responses for those times when the developers cannot justify their glitches and for keeping the process moving when you reach impasses.

Remember Your Users

The Director — Thu, 23 Aug 2007 23:37:03 +0000

Remember, computer users are not all geeks, nor are they godlike rockstar developers who live on Web logs, twitter, usenet, or whatever today's cool means of intrageek chatter is. Here is your computer user:

Pensioners surfing the internet are spending more time online than their younger counterparts. So-called "silver surfers" dedicate an average of 42 hours a month to the World Wide Web, compared with 37.9 hours among 18 to 24-year-olds.

Those are the computer users who need the bumpers and the training wheels and all the mechanisms within your Web sites/applications. If it's good enough for those who quote Office Space or Hackers or Star Wars all day, it's not good enough to ship. It has to be good enough for your grandmother who's one of the 12 million people still dialing up through AOL. Pardon me for harping on this again, but I spent several hours last night trying to explain client-server technology, again, to someone who asked me how to move image files (my term, not hers) from My Documents to My Pictures.

Gallery of Stack Traces: Potentially Dangerous? QA?

The Director — Fri, 24 Aug 2007 19:21:52 +0000

As you know, you should always test your edit boxes to make sure that it can handle things like HTML and XML tags within it, particularly the dreaded and or or . At best, the application will handle it through any sort of mechanisms, such as:

Accepting the value, but using escaped characters so that the application, browser, and database know this isn't supposed to render.
Disallowing use of the < or > characters.
Performing client- or server-side validation to tell user to try again without the markup.

What the application should not do is this: Click for full sizeBy default, they tell me, Microsoft Internet Information Server will automatically "handle" HTML or doctype declarations by throwing up a stack trace. Thanks, Microsoft, for helping Web sites/applications abandon their dignity. Don't let your developers fail to handle this correctly.

That Sounds Painful

The Director — Fri, 24 Aug 2007 19:26:49 +0000

If it hurts to sign in, only people who have to sign in will sign in: Click for full sizeGreat idea for security, IBM!

Message: Columnist Doesn’t Want To Hear From You

The Director — Sun, 26 Aug 2007 18:52:30 +0000

In a column by St. Louis Post-Dispatch columnist Bill McClellan, dummy values shine through: Click for full size A little bit of logic checking to see if the data-driven information was the dummy/default information and hiding the block if it was would have prevented this unfortunately not-embarrassing gaffe, but because it wasn't embarrassing, the development team decided they could live without a couple extra lines of validation code.

IBM’s Cheap Banner Ad Tricks

The Director — Tue, 28 Aug 2007 16:19:49 +0000

When do you make the same sort of control do different things? In two situations:

You're lazy.
When you want to trick someone into exposing a banner ad and blaring audio at them.

For an example of this, let's look at this IBM banner ad I saw on ComputerWorld.com:

See that little Open link with the X on it. You've seen similar control types even on banner ads, mostly with the X Close thing to shut those BadBoys up, right? So one would assume that this banner ad requires a click to open it and expose its content.

Ha ha! Fool! This merely requires a roll-over to expose it and start its audio come-on blaring. Given that, at ComputerWorld.com, it sits in the right sidebar between the scrollbar and the content, the odds of the user inadvertently mousing over it are pretty high indeed.

When it's open, look what we have:

Click for full size

Does the banner ad close on mouseout? Oh, but no; now, you do have to close the button to shut it down.

The two similar-looking controls behave differently, and not only that, but they behave differently in the fashion that will prove most annoying to the disinterested user.

Think Of It As More Joss Stone To Love

The Director — Wed, 29 Aug 2007 18:46:33 +0000

Some songs do seem to go on forever, but not that long: Click for full size I have no idea how iTunes deduced that. Bad data in the Internet servers keeping track of the songs? Corruption in data transmission? All I know is that the song never seems to end.

Thanks For The Reminder

The Director — Thu, 30 Aug 2007 15:28:23 +0000

I love FTP Voyager by RhinoSoft; I've used it off and on for a decade now. However, during one previous workstation migration, I didn't get a license for it and relied on the command line until such time as I was managing too many Web sites relative to my typing speed. So I downloaded the trial version and ran it for 29 days (of 30) before purchasing. Immediately, the application popped up a modal dialog box whenever I opened the application reminding me that it was a trial version and I could purchase the application and register it. After a set number of day, it began popping up the dialog box while I was running the application (in addition to when opening the application). I think this was an event-driven reminder, as it often showed when I finished a file transfer. However, I think the reminder event logic could use a review, as it popped up the modal dialog box immediately after I successfully entered the registration code: Click for full size Classic.

Little JavaScript Errors Mean So Much

The Director — Fri, 31 Aug 2007 15:20:33 +0000

Here's a little JavaScript error courtesy of ComputerWorld.com: Click for full sizeWhen you're reviewing Web sites, you can easily sniff these out by watching your status bar on in Internet Explorer (it is by default, but if you've turned it off, turn it back on). When Internet Explorer finds a JavaScript error, it displays the error icon in the bottom right corner of the status bar: Double-click that bad dog to get the details as shown above. It will help you make the developers feel small when you can tell them the exact error message.

In an Ideal World, The Semicolon Would Be There

The Director — Tue, 04 Sep 2007 15:50:34 +0000

Ideals magazine and its publication division have a couple problems with their Web site (also at IdealsBooks.com); first and foremost, no one has looked out for JavaScript problems as I mentioned earlier, otherwise that person would have seen the problem with the missing semicolon that seems to occur on every page in the site: Click for full sizeAlso, this site was not reviewed in other browsers such as Firefox, or someone might have seen that the size of the edit boxes is different and sometimes throws off the layout: Click for full sizeThese are some very basic things to look for when building a Web site. However, in the tension between someone wanting to have a Web site fast and inexpensive and someone trying to build a Web site for as much money as possible with the least effort possible, basic quality goes out the window.

That’s What I Like To See On An E-Commerce Site

The Director — Wed, 05 Sep 2007 00:25:10 +0000

Hey, who wouldn't put a credit card number into an e-commerce site spitting out config errors on page load? Click for full sizeHey, me first. Someone else can have this identity, I'm tired of it. What probably happened here is that the Alexa Ray Joel Web site (AlexaRayJoel.com) activated the merchandise link before the "store" was configured on the partner site. But shouldn't someone working on AlexaRayJoel.com have found that instead of a stray Billy Joel fan looking for some new rock music with the Joel name on it anywhere he can get it?

Opening the Raincoat on Flash Version Testing

The Director — Wed, 05 Sep 2007 20:21:12 +0000

The cool kids like to use Adobe Flash to build rich Web applications because if you build it in Flash, you don't have to worry about compatibility with Web browsers. Oh, who am I kidding, they like to use Flash because it offers a bunch of effects out of the box and it does some right purty things with animation and pictures. It allows them to put forms and data submission stuff right into the flash really fast, too, often without any data validation, but that's another story. No, today's lamentation is about how those cool kids build Flash animations or applications using the latest version of Adobe Flash love to use the latest gimcracks and whatnot and how people who have a previous version of the Flash player installed with their browsers sometimes don't get to see and marvel at the genius of your design team. Sometimes, they will see nothing at all, which is very, very bad. And you know, some browsers/computers ship without Flash installed in the default browser, right? So what happens when that user tries to hit the page, he gets no message and no warning, which means he thinks the site is defective. Because it is. I am not saying that every Flash application should be compatible back to Flash 6 (although Flash 7 would be nice); instead, Web sites that use rich media should build detection logic into their sites to inform the user of the Flash thingamabob's compatibility; that is, if the designers had to have some widget from Flash 8 in the thing, the Web site should check to ensure that the user's Web browser has at least Flash 8 installed. It's not impossible, but it takes a little extra effort that's not dedicated to the gee-whizery that warms the cockles of your Flash developers' hearts. That being said, what should you do to test these things, my QAhort? What I did was to create a batch file that, with a single click of the mouse, would use the silent installers to uninstall the currently installed version of Flash and then installs the version you want. Just like that. Here's the basics:

Go to Adobe's Archived Flash players available for testing purposes page.
Download the Zip files to a central location, such as a mapped network drive. If you're only going to do this on a workstation, you don't need to do this step, but if you want to make the files available across workstations, you'll need a central location.
Go to Adobe's How to uninstall the Adobe Flash Player plug-in and ActiveX control page and download the uninstaller executable.
Unzip the contents of each version's zip file. They tend to have two installers, one for the Active X control used by Internet Explorer and one for the Gecko/Firefox/Netscape version.
Create, in a central location, your Flash uninstaller batch file. This batch file removes Flash from your system as described on the Adobe article page. This file, uninstallflash.bat, should look something like this:
q: cd \qa_files\Macromedia Flash uninstall_flash_player /s
The installer files that I mention below will call this file to clear existing versions of Flash before running the installers. The Adobe page says it's a good idea to restart between uninstalling and reinstalling, but in most cases it will work without; however, do keep in mind the caveats about instant message clients and whatnot holding onto the ActiveX control and preventing its installation.
Create the individual version installation batch file. I used a single batch file for each version, but if you're sophisticated and underutilized, you can put a menu in the batch file if you want. For example, installFlash7.bat would look something like this:
q: cd \qa_files\Macromedia Flash call uninstallflash.bat z:\ cd \qa_files\Macromedia Flash\macromedia flash\Flash Player Installer Archive\fp7_archive\r61 flashplayer7r61_winax.exe /q flashplayer7r61_win.exe /s c:
Again, this uses the silent installer so that you're not bothered with the prompts associated with a normal installer. When the installers run, though, you might see some progress bars on the screen.
Once you've got the uninstallers set up, you can create shortcuts or whatnot to their locations and run them easily. You can create desktop shortcuts if you want, but I prefer to create a centrally located HTML file that has nothing but links to them and then embed that HTML file into the Active Desktops of all the workstations upon which I would use the batch files. This way, I'd have access to the latest versions of the files and would see any changes I'd made to the HTML file list of batch files. But that's just my way.

In any case, what you have is an easy way to rapidly switch between versions of Flash--or uninstalling Flash completely--to see what your users will see when they are exposed to the creative genius of your organization's Flash designers. And the ensuing havoc.

Microsoft Warstrike Load Testing Software

The Director — Thu, 06 Sep 2007 20:29:53 +0000

Well, it's not really Microsoft Warstrike, it's Microsoft Web Application Stress Tool (WAST, which as you old Bard's Tale fans know was the keystroke combination for the Warstrike conjuror's spell, good for 4-16 points of damage on a group and a pretty potent weapon). Now, Microsoft Web Application Stress Tool lies buried in the Microsoft Downloads section (here). Screenshot:

Click for full size It's an old download (ca. 2002), but it looks to work with IE 7. So if you're looking for a free rudimentary load-style tester, here's something.

Gallery of Stack Traces: There Is No Spoon, Either

The Director — Mon, 10 Sep 2007 19:47:02 +0000

Ever feel like you're stuck in The Matrix?

Spoon boy: Do not try and bend the row at position 0. That's impossible. Instead... only try to realize the truth. Neo: What truth? Spoon boy: There is no row at position 0. Neo: There is no row at position 0? Spoon boy: Then you'll see, that it is not the row at position 0 that bends, it is only yourself.

Click for full size

Yeah, you can get this big nasty in a variety of ways:

You can specify an invalid value on a querystring as part of the parameter name/value pairs. For further reading, see The Querystring: Soft Target.
You can delete something and then use the Web browser's back button to get to the screen where the spoon still exists, and then you delete it again.

Embrace the nirvana, the state of nothingness. And open an issue.

You Wouldn’t Expect Firefox Compatibility, Would You?

The Director — Wed, 12 Sep 2007 00:56:59 +0000

Microsoft.com?

Click for full size

Who are you kidding?

Banner Ads Are Flash Applications, Too

The Director — Thu, 13 Sep 2007 18:34:02 +0000

The following banner ad, probably a PointRoll ad or some such, left detritus on the screen when its animation played and did not close when the user moused out: Click for full sizeFriends, that does not build good brand equity or what have you. But, Director, what are you supposed to do? Test the banner ads? Perish the thought! With the new technologies and dependence upon browser plugins, you need to test them. I've done it, and so can you. So could your damn hipster designers if they weren't so caught up talking about what they saw at Burning Man. Off the top of my head, here are some of the things you need to check:

Animations are smooth. After all, that's the point of all this wizardry, ain't it? Mouseover the hotspots that expose panels; make sure they slide out pleasingly. Make sure that the animation in the banner itself doesn't jerk (unless you're going for the Blair Witch effect).
Words are spelled right. Believe it or not, your crack design team might err here. I mean, perhaps they took the brown Ecstasy instead of the red Ecstasy at lunch. Or perhaps they just cannot spell, since being "creative" doesn't involve "being smart enough to not look like a fourth grader when writing."
Hotspots behave appropriately. When you click a link, it takes you to the right place. When you mouseover a spot that should trigger a panel rollout, I don't know, maybe the panel should roll out. When you roll off of a panel, the panel should close. I realize some ad specs allow the damn thing to remain open until the user clicks some sort of close control, but have pity on the user. Never mind, if you're trying to sell something, obviously your powers that be are happy annoying 1000 users to make it easier for the 1 who will click through and then abandon a transaction later.Regardless, though, links should link and panel triggers should trigger panels.
Hotspots are placed appropriately. This is more subtle. Make sure places where the hotspots are make sense. The whole banner should link to the advertised site, not just a "click here" bit of text. Make sure that the "close" control on a panel doesn't lie on a place where the cursor's presence will trigger the panel to roll out again; I've seen my share of banners that play that prank. Close it? Open it! Close it? Open it!
Banners work with different versions of Flash. As I have mentioned before, checking compatibility with earlier versions of Flash is always a good idea. Your banners should work in a backwards-compatible fashion as well.
If your banners have actual form controls, you test them like forms. I understand that validation logic and whatnot aren't normally the purvey of people who are only doing Web work until their mixed media installations pay the bills, but your users don't care. Neither does the art world. Remind your Flash designers of this fact constantly.
Make sure your banner ads have appropriate tracking. Most links in banner ads carry tracking data on the querystrings. Make sure that, when you click the links, your banner ad carries through data that the destination Web site can consume.
Make sure user interactivity is limited to what user should do. As with any Flash stuff, you need to make sure that the user cannot use the mouse or other keystrokes to make the banner ad behave differently than planned. Try clicking, right clicking, clicking and dragging the elements within the Flash animation.

Remember, the houses that dish up these banner ads do some encoding of their own, so it's entirely likely that your inhouse QA will pass, and then the content deliverer will muck it all up. So you should run one more set of tests after they've done their number on it. Sometimes, you'll pass placeholder tracking information in your hotspots for the end house to replace; make sure they replace them. And don't let them monkey with the controls because their designers are worse than yours. So remember, the banner ad is a Flash application in its own and deserves the same amount of testing you would dedicate to any other application. No, not the amount of testing your company would dedicate to any other application (that is, none). Use your better judgment.

Gallery of Stack Traces: This Is Not The Object Reference You’re Looking For

The Director — Sat, 15 Sep 2007 00:39:22 +0000

This stack trace occurs when the Jedi Master developers play mind tricks on the weak-minded code: Click for full size Well, that's nice. I'd tell you how to elicit this error, but it occurs when the developers have done something so boneheaded that it's inexplicable.

UK House of Lords Readies Stake for Heart of UK Software Industry

The Director — Sat, 15 Sep 2007 00:57:40 +0000

The UK House of Lords might be looking at ways to put software companies on the hook for their quality negligence:

The Science and Technology Committee of the House of Lords published a report in August on personal Internet security, which concludes that it is all too easy for vendors to “dump risks” onto consumers through licensing agreements to avoid paying the costs of insecurity.

The report stated that efforts to promote best practices have been hampered by a lack of commercial incentives to make products secure. The committee’s solution is to propose transferring the cost of insecurity onto demonstrably negligent hardware and software manufacturers, with the long-term goal of establishing a framework for vendor liability across Europe.

This sounds like a good idea, but it runs counter to my antigovernment leanings as well. We've seen in America what happens when you make company executives spend lots of money on financial compliance and sign off on it personally; you'll see the same thing and other unforeseen consequences of this action, and I'll be hanged if I sign off on the quality of any product I test if my neck is in the noose.

Sounds Like Some Landlords

The Director — Tue, 18 Sep 2007 00:49:16 +0000

Deep in the heart of Redmond, danger lurks as you try to download service packs for Windows 2000: Click for full sizeGiven that the error occurs in Internet Explorer 5, perhaps Microsoft could be forgiven for this travesty. However, since Windows 2000 comes with Internet Explorer 5 installed and fresh installs of Windows 2000 would represent those people actually hitting this page in 2000, perhaps they're misunderstanding the target audience.Aw, suck it, Microsoft says to its long-term customers; buy Vista and we'll listen, but we're in this business for new revenue, not supporting you for paying us money in the past.

A Clever Solution in a Validation Message

The Director — Tue, 18 Sep 2007 18:33:48 +0000

The Dr. Pepper Go for More promotion is one of those marketing ploys where when you buy a product, you get a secret code you can enter at the Web site to see if you one deals. Man, I miss the old days where you could find out right away whether you won or not; instead, now we have to opt-in to lose, which seems like a lot of trouble to me and certainly doesn't encourage me to purchase a product where I have a chance to win over one where I do not.

So I go over to the Web site after buying some Dr. Pepper and create an account. However, I exist in the database, so a validation message offers a unique convenience.

Click for full size

You see, when you mouse over the text click here in the validation message, the application displays the name of the link on the page that you should click. Instead of, I don't know, making the text a link.

As an added benefit, here's the error page that displayed when I tried to actually submit the update profile form after clicking the appropriate link:

Click for full size

I will leave it to your imagination, gentle reader, what might have triggered that failure. Was it:

That I triggered the You didn't type your new password/ Confirm Password doesn't match password validation messages because I only wanted to update my address before I bent to the will of poor design that makes me type and retype my password to update my profile even after I logged in?
That whomever developed the sweepstakes did not test existing functionality (such as the update profile) because it works with other programs?

Garbage In, Garbage Out From Databases, Too

The Director — Tue, 18 Sep 2007 19:21:11 +0000

Whenever I bring up performing data validation on information returned from the database to the application, they tell me it's a waste of resources. However, I think Robin Harris might agree with me. From "Data corruption is worse than you know":

CERN found an overall byte error rate of 3 * 10^7, a rate considerably higher than numbers like 10^14 or 10^12 spec’d for components would suggest. This isn’t sinister. It’s the BER of each link in the chain from CPU to disk and back again plus the fact that for some traffic, such as transferring a byte from the network to a disk, requires 6 memory r/w operations. That really pumps up the data volume and with it the likelihood of encountering an error.

From "50 ways to lose your data":

Software: drives contain small computers that run on several hundred thousand lines of code. Is that code bug free? Need you ask? Among the more common bugs - and let’s not get started on the less common ones - are:

New code that fixes a problem and accidently breaks old code

Putting the right data in the wrong place.

Phantom writes that are reported as written but, oops!, aren’t.

Cache management bugs that munge data, or return correct data to the wrong place.

OK, this is less common, but sometimes the on-disk ECC miscorrects the data. ECC is software, right? How do you know it always works correctly? You don’t.

He focuses on the way data can get corrupted on the storage system, for the most part, but data within the databases can get corrupted in additional ways, including:

You let me test it, and I inserted invalid data. Sometimes the dev team would fix the issue, but not clean the data, which meant that the application would bomb out when trying to load the corrupt data from the database. Usually during a demo. By checking the data when reading it from the database, it could have handled my troublesome habits.
Data migrations or batch data loads dump unfiltered or incorrect data in. Because some third shift desk jockey working from a command line is, like developers, LIKE THE GODS! and would never make a mistake or try something he or she did not understand.
Integrated applications aren't as bulletproof as yours; that is, they have 4" mesh data validation where your wonderful project has 2" mesh. Your partners and Web services are allowing crap into your database which can choke your application.

It's a good idea to be suspicious of all data and make sure it's correct. However, that costs time and money, and developers and customer-facing yes men would rather spend their time drinking at lunch. Kind of like QA, actually, in that regard. Because whiskey makes us meaner.

Some “Testers”

The Director — Wed, 19 Sep 2007 19:34:08 +0000

I think Computer World's headline writers need to learn a little more about what the word means when they say "Testers give high marks to new features in SQL Server upgrade":

A conference being held in Denver this week by the Professional Association for SQL Server user group will provide a forum for the biggest public unveiling to date of Microsoft Corp.'s SQL Server 2008 database. But as many as 20,000 users have already been testing the upcoming software, which is scheduled for release by next June. And their reactions, detailed in interviews or via blog postings, have been mostly positive thus far. For instance, David Smith, CIO at ServiceU Corp. in Cordova, Tenn., said that SQL Server 2008 has improvements in "hundreds of areas. It's exactly what I need."

Son, that's executives at customer companies, who are not exactly the sort of people who should be allowed to pass judgment on software, especially basing that judgment on a demo. Real testers who give high marks to anything haven't tested it enough.

Good Enough For Government Work

The Director — Fri, 21 Sep 2007 01:45:59 +0000

You know, it must be easy to work quality assurance on government projects:

Because of a continuing software glitch, the first high-tech "virtual fence" at the nation's borders remains unused, three months after its scheduled debut. Nine 98-foot towers laden with radar, sensors and sophisticated cameras have been built across 28 miles close to the Arizona-Mexico border near Sasabe, southwest of Tucson, in an area heavily trafficked by illegal immigrant and drug smugglers. The towers, each a few miles apart, are intended to deter or detect border-crossers and potential terrorists and to enhance the ability of Border Patrol agents to catch them. Homeland Security Secretary Michael Chertoff said more testing is expected by early October. "We are now looking to begin acceptance testing in about a month - meaning that's the point at which they (contracting officials) will say to us we think you can test this. And we will then kick the tires again," Chertoff told the House Committee on Homeland Security early this month in Washington. But Chertoff also said he's withholding further payment to the prime contractor, Boeing Co., unless and until the pilot project in Arizona works.

Yeah, it must be easy to be a tester on a government project. Unless you care.

In A Stunning Turn Of Events, Hubris In IT

The Director — Mon, 24 Sep 2007 16:29:30 +0000

Forrester Research apparently says that IT departments are overconfident:

Six years after the events of 9/11, many corporate IT operations are overconfident about their ability to handle a disaster, according to a Forrester Research, Inc. report released on Tuesday. The survey of 189 data center decision makers found a severe lack of IT preparation for natural and manmade disasters. For example, the report found that 27% of the respondents' data centers in North America and Europe do not run a failover site to recover data in the event of a disaster. About 23% of respondents said they do not test disaster recovery plans, while 40% test their plans at least once a year. About 33% percent of respondents described their operations as "very prepared" for a manmade or natural disaster while 37% called their sites simply "prepared" for such events.

Overconfident, somewhat; after all, geeks are like the GODS! if you ask them. However, disaster planning, like quality, requires planning and spending. Both of which, unlike confidence, are in short supply in many IT departments.

Problems in VMWare

The Director — Mon, 24 Sep 2007 16:35:38 +0000

As many of you know, I do not care for virtualization as a solution to simulate multiple environments when testing; I'd rather have actual hardware to represent as many possible combinations of user environments as I can. Virtualization software is just another set of excuses that your development staff can use to avoid investigating and fixing their own bugs. Stories like this one give them data points and anecdotes to push you off:

A set of newly discovered flaws in components of VMware Inc.'s virtual machine software has called attention to some of the security risks associated with the practice of running virtual computers on a single system.

It's not just security problems, but performance and whatever else that can fail, invisibly, that can provide erroneous defective behavior in your software. Besides, all kinds of shiny hardware is pretty and keeps the QA lab warm in the winter.

Keeping Your Test Data Clean

The Director — Mon, 24 Sep 2007 17:36:23 +0000

There are many things one can learn from this story:

Symantec Corp.'s early-warning system gave its enterprise customers a brief scare late Friday when it erroneously sent an alert that said an Internet-crippling attack was in progress.

However, the one I want to tease out is this lesson: always keep your junk data clean and readable and clearly defined as test data. In the Symantec alert, the message sort of indicated that it might be a test:

In the body of the e-mailed alert, however, careful readers found the words "Summary: threatcon test threatkhanh otrs" buried among several links.

Whenever you're testing in a live environment or in an environment where databases are copied from dev or test environments into production (not a best practice, I know, but it happens too often), you must structure your test data so that it's marked as test data somehow. For example, on Web data collection forms that use e-mail as a uniqueish identifier, I recommend using a standard domain for your test submissions; in the past, I've used the company's domain. This way, your dev teams can (and should, but whether they will or not depends upon the caliber of your persuasion--I recommend .45) filter out the test records before the records go to the fulfillment house, e-mail vendor, or mere usage reports. On free-text sorts of data entry, it's best to stick with a simple line of "This is a test entry." or something similar. At least Mr. Khan did enter the word "test" in his data entry for Symantec; other times, developers or testers pound on the keyboards with the subtleties of bonobos. If these strings of characters get passed onto actual users, they might look like defective behavior on the part of your application. Worse, if you subversive QA types use gallows humor in your test entries, it might make it past the gatekeepers and onto the end user or client's computer screen. I'm not saying that I ever did anything like that, regularly-scheduled production testing included some feeble humor that the data aggregation and screening person filtered out before passing relevant entries onto the client. When the data aggregation and screening duties passed on to admin staff, one of the almost plausible punchlines made it to the client. The humor wasn't anything that damaged the relationship with the client, but it's better to be safe than sorry. So treat that test data as though your client or end user was going to read it. That way, when your tech team inevitably makes the mistake that presents test data as gospel, at least it will be good spelled. If you use automated testing, make sure that the test data used by your scripts is clean and friendly, too. Because if your client gets to see that information, your client is going to see it a lot. Automated test data that indicates its source in automated testing will explain to the uninformed why he or she is seeing this test message every day at 1am. It's not a bad idea to earmark load testing data as well so that its origin is easily identified; a load test mistakenly sent to a production server should trip some potential Denial of Service alarms with your hosting provider. If you're doing it the right way, that is.

The Dirtiest Trick of All

The Director — Wed, 26 Sep 2007 01:43:53 +0000

You want to stop the heart of your tech team or project managers? Here's how you do it:

Open your crucial, behind, and ultimately doomed Web project in your Web browser.
Type the following into your Web browser's address bar:
javascript:R=0; x1=.1; y1=.05; x2=.25; y2=.24; x3=1.6; y3=.24; x4=300; y4=200; x5=300; y5=200; DI=document.images; DIL=DI.length; function A(){for(i=0; i-DIL; i++){DIS=DI[ i ].style; DIS.position='absolute'; DIS.left=Math.sin(R*x1+i*x2+x3)*x4+x5; DIS.top=Math.cos(R*y1+i*y2+y3)*y4+y5}R++}setInterval('A()',5); void(0);
Press ENTER.
The images on the page will start to swirl. Set focus to the Web browser's address bar.
Retype the URL of the doomed project, but do not press ENTER or click Go (that would reload the page without the JavaScript running.
Walk away from your desk knowing that will display until the screensaver kicks on.

Ah, yes. A wayward project manager wandered over and caught sight of it, almost entering a state of hyperventilation as she summoned the complete tech team to her aid to discover what was going on. I only wish I could have been there to see it, but I was away from my desk. If nothing else, it should teach lessons in shoulder-surfing QA.

The Definition of QA Insanity

The Director — Wed, 26 Sep 2007 19:53:32 +0000

You might have heard that the definition of insanity is doing the same thing over and over again and expecting to get a different result? Software and Web applications are much the same way, in a twisted fashion of their own. You're insane in QA if you do the same thing over and over again and expect to get the same result. That's why QA has to check everything, all the time, over and over again. The simplest and most slam-dunk, we've done this a million times before things. The things everyone else takes for granted. For example, let's look at a simple state combo box/drop-down list. You know it; it's ubiquitous. It's on every opt-in form, every sweepstakes entry, and most registration forms you've ever encountered. But QA must review it every time it appears, even though it's usually part of a simple cast off requirement like "User enters address." Leaving aside the more important question of the actual requirements of what should appear in that drop-down list (Does offer extend to APOs/FPOs--that is, military or foreign service post offices? Are Possessions included, such as Puerto Rico, Guam, and the Marshall Islands? How about our other possession, Canada? How should this control work with the Zip Code/Postal code control for data validation?) The developers, using fingers swollen from playing Halo 3 all night, sometimes introduce variations into the basic spellings of states. And you, gentle QA person, must review all contents of every drop-down list every single time you see one. For example, here are a couple of interesting issues I found while trolling around the Internet just yesterday:

	Alberta, Canada, which is apparently different from Canada - Alberta.
	New Foundland, which was probably named after the original county Foundland in England.
	Special symbols are a bear in drop-down lists; just ask the person who encoded this. No, don't ask him, he doesn't think anything could possibly be wrong; he does state lists all the time.
	After the all night Cary Grant festival, the developer realized he had called it the North-by-Northwest Territories. Fortunately, though, he fixed it before anyone was the wiser.
	Some developer tips off the Canadians to our plans for building a contiguous set of states and an Interstate highway to Alaska.
	Another developer tips of the invasion, but tries to cover the mistake by misspelling "Territory".
	In retaliation, the Canadian Navy takes all but one of the Marshall Islands as a plucky little holdout eludes capture by both Canadian Navy ships by painting itself as a large whale.
	These little islands, Dillon, Kane, and Gerard, off the west coast, should become a state any day now.
	In 2007, there is only one left, and it's very self-conscious of the fact.
	Instead of AA, AP, and AE, we have armed two of our largest states. What could possibly go wrong?
	After a violent intra-state civil war, the state divides into its toothless region to the east and its banjo-pickin' tribes to the west.
	Just for the record, the Canal Zone ceased to exist in 1979. Like 20 years before the Internet became popular. What it's doing on a Web form, I have no idea.
	I should have been a pair of ragged claws Scuttling across the floors of silent Cs.

Well, there's your object lesson for today, friends. Those slam-dunks, appear all the time controls might be tempting to gloss over, but each one has the same chance (85%) for the developer to screw something up. It's not limited to the state lists by any means; sometimes, they put in drop-down lists for birthdays and start the years at 1945, excluding everybody but the boomers and those who follow. Or, Heaven forefend, they expose client data in drop-down lists, allowing "administrators" to insert their own unique variations on reality. That, my friends, is why QA must be ever vigilant, ever meticulous, and ever anal retentive in checking everything, every time.

The Microsoft Excel Bug

The Director — Thu, 27 Sep 2007 14:33:33 +0000

Man, that's a maddening little thing to catch:

Microsoft Corp. yesterday confirmed that Excel 2007, the newest version of its market-leading spreadsheet, returns incorrect calculation results in some cases.

The math bug first surfaced Saturday on Microsoft's own Excel support newsgroup when a user named Molham Serry reported that when he multiplied 850 by 77.1, Excel 2007 returned 100,000 rather than the correct 65,535. Others on the newsgroup quickly took up the standard, eventually posting more than 120 messages to the thread. Among their findings: Other calculations claimed 100,000 was the correct answer. Yesterday, the Excel team offered a mea culpa posting to a Microsoft company blog. "The majority of reports were focused on multiplication, but our testing showed that this really didn't have anything do to with multiplication," said David Gainer, lead project manager for Excel. "It manifested itself with many, but not all calculations in Excel that should have resulted in 65,535. Further testing showed a similar phenomenon with 65,536 as well."

Joel Spolsky, formerly of Microsoft and now of FogBugz, guesses how it happened:

The first thing you have to understand is that Excel keeps numbers, internally, in a binary format, but displays them as strings. For example, when you type 77.1, Excel stores this internally using 64 bits: 0100 0000 0101 0011 0100 0110 0110 0110 0110 0110 0110 0110 0110 0110 0110 0110 The display is showing you four characters: "7", "7", ".", and "1". Somewhere inside Excel is a function that converts binary numbers to strings for displaying. This is the code that has the bug that causes a few numbers which are extremely close to 65,535 to be formatted incorrectly as 100,000.

Spolsky also acknowledges how hard it is to test for this condition:

I'll bet that most of the numeric testing done on the Excel team is done automatically with VBA code. Cells containing this value display as 100,000, but from VBA, they're going to look like 65,535 (since the number would be passed into the Basic runtime in binary, before the display formatting.) I'm sure there's plenty of code to test display formatting, but with a bug like this that only happens on 12 out of 18446744073709551616 possible floating point binary numbers, it's unlikely that any set of black-box tests would cover this case.

Well, with an automation tool and enough time, you could build a script that ran through all possible values; even though this would take a lot of time for something like Excel, you could do it in the matter of weeks that you have for testing. I mean, big companies have actual test cycles, don't they? The trade literature indicates they're not as crazy as my actual experience indicates. The key, in any event, is given a good enough automated tool and time, QA should find these problems. However, QA rarely gets the first and almost never gets the second. It's happened to me before. Once, while testing a sweepstakes entry form over the course of a couple of days among all other emergencies as assigned, our team must not have entered 8 or 9 into the month edit box (!) of the birthday, because when it went into production and people who were actually born in August or September tried to enter, they encountered a JavaScript validation requesting them to enter a valid month. It seems the tech team was converting the numbers into octal or some such before performing the validation, and 8 and 9 were returning invalid because they're not valid octal numbers or something; still, it was an embarrassing oversight. Had we but enough world and time and a decent automated test tool, we would have tried all 365/366 combinations; however, from that time on, we always tried 8 or 9 when encountering an edit box (!) for birthday entries, and encouraged designers to use drop-down lists for birthdays. Given the constraints of time and the business, the best you can do to supplement your regular boundary analysis sorts of tests is to pay attention to variable types and their max values and mess around with them in addition to the maxima and minima for your data entry. And good luck.

Maxim

The Director — Fri, 28 Sep 2007 17:49:01 +0000

If you want it quick and dirty, you'll surely get it dirty.

Thank Goodness Software “Engineers” Aren’t Civil Engineers

The Director — Mon, 01 Oct 2007 11:08:55 +0000

Otherwise, we would see this in the defect tracker:

Defect # 102033
Title:	`Striking bridge support at speed greater than 60 mph causes bridge to collapse`
Severity:	`Critical`
Problem:	If a driver strikes a support beneath the overpass while exceeding approximately 60 miles per hour, the support will buckle and the entire span and bridge will collapse, killing the driver of the car that struck the support, the passengers, and any people passing over the bridge when the support is struck. To recreate: 1. Drive northbound in car at 62 mph. 2. Guide car into support. Support should not buckle nor should the bridge collapse when struck by such a light object at such a low rate of speed.
Status:	`REJECTED`
Developer's Note:	`In a real-world scenario, users would not deviate from the approved workflow by crossing the yellow line that demarcates the edge of the roadway. Also note that posted speed limits are 60 mph, so users would not exceed this posted limit.`
Project Manager's Note:	`Rejection approved. Add to construction notes document.`

Thank goodness we keep these madmen in ill-lit cubicle cells where they can only harm information and not real people.

(Originally posted on Musings from Brian J. Noggle.)

But It’s Web 2.0

The Director — Mon, 01 Oct 2007 17:30:24 +0000

Someone else has been sold a bill of goods, getting the site that its developers/vendors wanted to create rather than what its users needed:

Executives at Martha Stewart Living Omnimedia took great pains earlier this year to make certain the company's redesigned Web site looked flawless before rolling it out to the public. After all, this is a media company whose magazines, books, products and programs feature ideas about attractive and tasteful lifestyles. Why not a beautiful Web site? "That was a big mistake," Wenda Harris Millard, the company's president of media, said this week during a panel discussion at Advertising Week. "We put beauty before utility." She said the front page, with its video player and jazzy graphics, included only about five links to actual content, "so the things people were looking for couldn't be found." The mistake, she said, was in failing to understand that "when the reader or viewer or listener becomes the user, what she's looking for is much different -- at least initially."

Beware any vendor that wants to build you something without understanding your users or audience. Like as not, its rock star developers/designers have some things they want to try out that probably won't benefit you but will look really nice on their resumes.

Trivia Question

The Director — Wed, 03 Oct 2007 19:19:38 +0000

Did you know that Internet Explorer after an IE 6 security fix will display a box around Adobe Flash objects that tells the user to click to activate and use the control? Neither did the "professional" developers of these splash screens: Click for full size Click for full size

Or, worse, a "professional" developer of a Flash screen for a tool that allows you to build Flash movies:

Click for full size

Or this heading banner image:

Click for full size

Or this banner ad presentation:

Click for full size

How would you have liked to spend thousands of dollars on an exquisite Fat Boy or Bad Boy rollover ad with all the fixings, including vocal talent and three days' worth of filming, to present them in such a way that the majority of Internet users won't see them without that extra click. I mean, security-conscious grandmothers have been coached well enough by now that they're not going to click that control and install viruses on their eMachines.

It's not as though this is a fundamental, insurmountable problem with Internet Explorer. With an additional couple keystrokes, you can display these things without the box (see this article on how; it's not buried in some Microsoft vault somewhere, it's on the blooming Internet).

So why are these things all over the place?

Because the hip designers and technocratic ubermensch don't use Internet Explorer because the proletariat do, and they're above using the same browser as the unwashed masses, who don't even know it's okay to activate a safe ActiveX control. And coding for the dirty browser will hinder their prestige in whatever Mason-like secret society developers and the cool kids belong to.

Some Recruiters Are Twits

The Director — Thu, 04 Oct 2007 19:26:57 +0000

My experience with recruiters at staffing companies has never been sterling. When I was looking to move from technical writing to software testing, I interviewed with a recruiter and explained how easy it would be to make the move, how I logged more defects as a technical writer than all but one of the QA staff at my then current employer, and whatnot. He understood, thanked me for interviewing, and then via e-mail presented me with a list of technical writing jobs. Because that's where the company could make the money from me most effectively right away. They weren't really interested in my growth in other areas or trying to place me in a QA position because there was no money in it for them. But one exchange I recently had with a recruiter was particularly disingenuous. I received an e-mail at the public-facing e-mail address from my consulting company:

Brian, I'd like to add you to my professional network on LinkedIn. -

PS: Here is the link: https://www.linkedin.com/ It is free to join and takes less than 60 seconds to sign up. This is an exclusive invitation from to Brian. For security reasons, please do not forward this invitation. © 2007, LinkedIn Corporation

? I'd never heard of before. Gauging from the fact that he was using my public e-mail address and not my personal address, I guessed he had not really heard of me, either. I looked through the referrer logs for the corporate Web site, and he had in fact clicked through the link to my Web site on my LinkedIn profile and read every page on the site. So I responded:

Good day, . Thanks for reviewing my profile on LinkedIn and my company's Web site, but to maintain the integrity of the LinkedIn network, I'd prefer not to join your network as we have no direct business contact or experience together. Thanks.

He said:

You are correct, however I would like to make an introduction and hopefully in the future we would be able to work together. Regardless of connecting via LinkedIn, I would love to set up a time to talk and maybe we would be able to help each other out in some way. Hope you had a great 4th and I look forward to speaking with you soon. Thank you Brian,

And then he sent me another e-mail, including a list of jobs he was looking to fill:

Brian, I was told you have done some interesting work in the world of software test. I tried to get ahold of you today via phone but unfortunately had no luck. I want to share some great positions ( Software Test Analyst, Software Test Manager ) with you in hopes that you could share them with your network. You never know who might be open to a career upgrade. Please remember I only work with the best of the best (and was told that you were one). I'd like to talk further to establish some credibility and to earn your trust. Please call me on my mobile number and I'll call you back. . These are strategic positions with strong technical impact. Please keep me in your network as well. Everyone wants to know a great recruiter! :) How testy are you anyway? (Software test analyst) Our client, a St. Louis based company with an especially complex development environment, is seeking strong software test talent to assist in creating a more process oriented and robust Quality Assurance and Test environment.

. . . .
How testy are you anyway? (Take something good and make it better through Test Management!!) Our client, a St. Louis based company with an especially complex development environment, is seeking strong software test talent to assist in creating a more process oriented and robust Quality Assurance and Test environment. They are looking for a Test Manager to shape the test environment. This person will have the full support of upper management in creating a great test group.

Well, that's odd: he says he tried to reach me via phone and could not. And here I was, sitting at my desk all day, and no phone call. Could it be possible that the fellow stretched the truth, and by "stretched," I mean made it all up? I was interested in having him call me to pin down that he hadn't actually talked to anyone and the "good things" he heard were my single, solitary recommendation on LinkedIn. I responded:

Good day, . You can reach me at . Thanks.

I mean, it's not rocket science; the phone number is on the corporate Web site. Recruiter responds:

Great. Things have been a whirlwind of activity here today. Could you shoot me a copy of your resume so I can take a look while I am in meetings today and I will call you just as soon as I get a chance, thanks,

So I sent him my resume, which kind of, I don't know, looks like the information on my LinkedIn profile, with the single word Attached. Two weeks elapse. I send him a little note because I'm still curious and eager to get him to call me so I can challenge him about how he got my name:

Good day, . Did you receive my resume as requested? Thanks.

To which he responds:

I did, I apologize, I got caught up in a few things. The test position has been placed on hold, and I will be happy to talk to you about the Test Manager, however you may be too senior for this position, but lets talk. Please let me know a good time that will work for you or just give me a call at your earliest convenience. Thanks,

Too senior? Brother, working with the best of the best and QA people who do "interesting" things, and you bother me with a bogus LinkedIn invitation and then discover, to your surprise, that my resume matches my LinkedIn profile, and that I'm probably not interested in entry-level or junior-level testing positions. And he never did call, figuring that I was wasting his time. I was, but he started it.

How’s Your Section 508 Compliance?

The Director — Fri, 05 Oct 2007 15:05:44 +0000

If you're building Web sites or Web applications, you better know what Section 508 Compliance means, because the wild litigants do:

A federal court judge's ruling this week that Target.com, the home page of retailer Target Corp., must be accessible to blind persons under California laws, could extend state and federal disabilities statutes to the Internet. The ruling is part of a memorandum and order issued by Judge Marilyn Patel of the U.S. District Court in San Francisco in connection with a lawsuit filed last year alleging that Target had failed to make its Web site accessible to the blind, and then ignored the issue when confronted with complaints.

What does this mean for QA? A larger number of unspoken and overlooked requirements that you'll have to test for, every thing, every time.

MSN.com Fails Backward Compatibility

The Director — Fri, 05 Oct 2007 15:39:13 +0000

Et tu, MSN.com? Click for full sizeThis error occurs in Internet Explorer 6.0, but not in 7.0, apparently. Some of you, the Web developers amongst you, no doubt, sigh and say, "It's bad enough we have to develop for that Microsoft Proletariat Web browser, but why do you even think we should make our elaborate, elegant, and perfect in Firefox sites work with the old version of IE? Don't you know that Microsoft pushed it out as an automatic download? Only you QA malcreants would even dream of testing in it!"Well, bub, these numbers would indicate otherwise:

Microsoft Internet Explorer 6.0 42.75% Microsoft Internet Explorer 7.0 34.60% Firefox 2.0 13.57% Safari 41 3.91% Opera 9.x 0.86% Firefox 1.5 0.85%

In a lot of cases, large corporations don't push out the latest browsers to their large numbers of desktop users right away, and if your site is broken for IE 6, it's broken for unhip corporate drones. You know, Internet users.

Don’t Forget To Test Your PDFs

The Director — Mon, 08 Oct 2007 22:14:09 +0000

The Adobe Acrobat PDF (Portable Document Format) file has become ubiquitous on the World Wide Web (WWW), and a lot of times Web site and content creators don't give it a thought when they plop files out there on the Web site or generate them dynamically. Since they're out there, you need to make sure that they're correct and proper. So what should you look for when you're reviewing PDFs? For starters, you should check the layout, grammar, and whatnot within the document. Of course, most places don't tend to run the copy and the creative by the QA department; most places suck, and the two might correlate more than most places think. So when you've got a PDF file on your desk, take a look to make sure that the "writers"--who will either be crumb-paid former English majors happy for a job or semi-literate executive types who can't be bothered to communicate effectively while they're paid to lead from the safety of some golf course--haven't misspelled the company name or anything. You should look to the written document as a printed document and proofread accordingly, making sure that fonts are visible, headers and footers say the right things, and that the page numbers line up. That last bit is important, as some applications repaginate base on printer driver settings, and if your content creator used Microsoft Word with old Adobe Acrobat print drivers, it is possible that Microsoft Word automatically reformatted the document before generating the PDF files. Sometimes, this had the effect of "printing" the document to pages shorter than standard size, so that the bottoms of pages would carry over into the next page. This would often "orphan" headings or regenerate page numbers without regenerating the table of contents or indexes so that those two directories didn't line up. Okay, you've looked at the contents. Now, look at the document properties available from the File menu. When your experts generate the PDF files, the process carries forward some metadata information from the source document and from the user who generated it: Click for full sizeYou want to make sure that these properties don't carry with them your company's name instead of your client's name, a template name that doesn't mesh with the document's goals, or your expert's user name on the corporate network. If your company, for some stupidity, transmits Microsoft Word or other source documents, it's doubly important to check those file properties as well and to flush any functionality in macros out before making the documents publicly available, but that's a whole nother story. Okay, now that you've got the document properties cleaned up (after three or four rounds with the content generators, no doubt), you'll also want to check the backward compatibility of the PDF file. Because as a document that requires a user-installed plug-in to appear correct, your "experts" have a later version than most of your readers and they, your experts, no doubt want to include gee-whizzery in the document that will look like crap with older versions or that might not work. So you've got to open the document in a couple versions of Adobe Acrobat Reader to make sure the file is backward compatible. You don't want to subject your reader to things like this: Click for full sizeEven if they wanted to upgrade, some of them cannot due to security practices at work and whatnot. And don't zero in only on this message. Any warning from Adobe Acrobat Reader, such as missing fonts or missing language packs, should be squashed before you subject the world at large to your important and meaningful communique. That's just the basic PDF files, friends. If your organization introduces hyperlinks or other gee-whizzery into its PDF files, you need to test those things, too; if your organization generates dynamic PDF files using customer-entered data, you need to not only test the above, but you need to test to make sure the data is transferred to the PDF correctly and that the document is formatted correctly. For example, one such application I know of failed a backward compatibility test because the line break characters showed up like this in Adobe Acrobat Reader 6.0: That's the result of an unrecognized character, of course; Adobe Acrobat 8 recognized it, but it looked like carp in 6. What, you don't find the center indicative of a mouth and the lines going to the corners are representive of gills? Regardless, the common and often overlooked PDF file represents one more way that your organization can look amateurish and foolish unless QA steps in to save the day. Again.

Dr. Watson Commits Seppuku

The Director — Wed, 10 Oct 2007 01:03:36 +0000

How do you know if you're a good software tester?

Dr. Watson ends it all rather than continue working with you.

Good work!

Optimism from Developers

The Director — Wed, 10 Oct 2007 11:16:29 +0000

This post at Worse Than Failure illustrates, in handy chart format, myriad ways that a development project can fail. However, as the post was written by developers, the number of possible ways for complete and utter disaster are probably underrepresented.

Et Tu, Bug Tracker?

The Director — Wed, 10 Oct 2007 22:30:40 +0000

Following the heels of yesterday's crash, today BugTracker.NET crapped out on me: Click for full sizeWhy, it's almost as though the word is getting out amongst the very tools of the QA trade about me.

Almost An Insult

The Director — Wed, 10 Oct 2007 22:34:38 +0000

If I were an e-mail marketer, I would almost be insulted: Click for full sizeOf course, it's hard to be insulted by an e-mail marketing communication rife with typos and a Web site rife with errors. I realize I single out BizJournals.com a lot, but geez, the quality of its e-mails and its Web site are atrocious and worthy of scorn.

Ask A Silly Question

The Director — Thu, 11 Oct 2007 22:15:46 +0000

In SD Times, columnist David Linthicum asks: Is SOA Quality a Priority?

SOA testing is in the media a bit these days as those who implement SOA have to make sure those new services, abstraction layers and orchestrations are ready for prime time. However, the common approach to SOA deployment is: development now, requirements maybe, and testing if we have the time. You can’t afford to make that mistake; there is too much on the line with this stuff. Indeed, a recent study by Nucleus Research discovered that existing SOA implementations achieved limited success when considering ROI. Only 37 percent of enterprises have achieved a positive return on their investments from SOA deployments. While the root cause of these low ROI numbers can be attributed to many factors, the key issues relate to a lack of planning and a lack of testing. Central to this problem is the fact that quality assurance, in general, is an often overlooked concept to most developers and designers. I mean, you’re admitting that your code and resulting services need to be tested. How can that be?

How long have you been in the industry? This is called business as usual in most sectors. Or, as I like to say (starting today), HIHO: Hubris in, Hopelessness Out.

I Just Clicked A Link

The Director — Fri, 12 Oct 2007 17:02:46 +0000

Some people are born to QA; I must be one of them, since the applications fail at my slightest touch. Click for full size Why do the applications come to me to die? Why do the applications come to me to die?

Sending An Application To Do A Man’s Work

The Director — Tue, 16 Oct 2007 01:16:14 +0000

Throughout much of the IT world, the developers and the people who love them want technology to solve everything for them, to be everything to them. Unfortunately, we in QA spend most of our days steeped in the myriad ways technology fails without remorse on its part and often without remorse on the part of the negligent nabobs who created it. So you can understand why I'm not an early adopter to the latest gee-whizzery that uses faulty algorithms to supplant fallible people. So when I saw several ads for WhiteSmoke, a product that's supposed to review and improve your written work, you might think I would be tempted to go to its Web site and review it for grammar. Brother, you know me too well. So here's the Flash landing page of WhiteSmoke, several screenshots grafted together to present it in one pretty package: Click for full sizeI want to draw your attention to the following items where the grammar and whatnot could be improved:

The bulleted list items Advanced Dictionary & Translator and Artificial Intelligence Technology are not capitalized the same way as the other items in the bulleted list. This is not parallel structure.
The software is designed with for all applications on myriad Windows platforms. I'm not entirely sure why the company included the Apple logo here; are they insinuating that it works with iTunes or QuickTime? Or is it just pretty?
The capitalization on these items is, again, not parallel. Some use title casing, some use sentence casing, and one (All-in-one Proofreading Solution) uses Emily Dickinson casing.
Two things: First, the second series, a legal document, an essay for class, or medical presentation? is not parallel; it should be a legal document, an essay for class, or a medical presentation? (the styles of these vary and should be parallel in both blocks, but internally the others are parallel except where noted). Second, there should not be a comma in a series of two things a short story, or thank you note?. Granted, for stylistic reasons, one could use it in informal writing, but on a Web site that tries to sell something to punch up your business writing, one should adhere to formal writing rules.
The following brand names are incorrect: Microsoft Word, Corel WordPerfect, Microsoft Outlook, Microsoft Outlook Express, Microsoft Works. Etc., the abbreviation, is spelled and punctuated correctly. Also note that these product names are brand names and trademarks owned by other companies, not so you would know it by a footnote at the bottom of the page.
The sentence Also, Explorer version 5.01 and above is required in order to use WhiteSmoke. uses an incorrect brand name, faulty logic, passive voice, and extraneous words. This would read better as Also, you need to use Internet Explorer version 5.01 or laterto use WhiteSmoke. The preceding sentence uses active voice. Did the junior copywriter, responsible for this sentence, fall down on the job? Or did someone consciously choose to break up the repetitiveness of the active voice by inserting passive voice? Perhaps that's "Artificial Intelligence Technology" at work.
The title tag doesn't have spaces after the commas in the series, and it also lacks a serial comma after the last item in the series. Other series in the body use this comma. Sure, it's a "stylistic" choice, but Strunk asserts that one should use it, and I have explained time and again that it makes sense to use it, but above all, one must be consistent in using it or not.
Firefox is capitalized incorrectly.
Copyright and the copyright symbol together are redundant; currently, this reads "copyright copyright 2002-2007." One should be eliminated.

Additionally, here's a frame of the Flash presentation: Notice, again, we're not using the serial comma although it appears in most other places on the page.But, you know, let's not get to into that; perhaps WhiteSmoke outsourced this Web site to someone who didn't use WhiteSmoke. Let's instead focus on the WhiteSmoke demo.Before: After:

The swap of possible for prospective makes sense. Credit where credit is due.
Changing exhibition to technology show adds a word and doesn't really add anything since, within an e-mail, the context is established. I mean, what other sort of exhibition would two co-workers talk about?
Adding innovative before product line just adds market blather to an internal e-mail. Besides, how does WhiteSmoke know that Tim wasn't pitching a line of AS/400 tape backups? It doesn't. It's using its "Artificial Intelligence Technology," and apparently it has the artificial intelligence of a marketing intern.
That's a compound sentence joined by a comma splice. There should be a semicolon before however. Whoops, WhiteSmoke missed that.
WhiteSmoke adds significant? That changes the meaning of the sentence and seems to rely on WhiteSmoke's Artificial Mind Reading technology to scry the nature of the prospect's actual issues.
WhiteSmoke adds generous? Tim might have thought a couple hundred off was a good starting point, but WhiteSmoke has put "60% off" into his mouth.

So it looks like WhiteSmoke not only checks your grammar but also re-imagines your original with less faithfulness than the Gus Van Sant rendition of Psycho. Now, all hijinks aside, I don't think tools like WhiteSmoke are useless, because it can make suggestions that might help a mere mortal writer; however, when it comes to the final review of your written work, whether it's a proposal or a Web site, you cannot beat having a strict grammarian review and improve it.

Gallery of Stack Traces: Number 5 Needs Good Input

The Director — Tue, 16 Oct 2007 21:44:17 +0000

What do you get when you try to put a string where the application wants a double? Well, some developers would call it a training issue, but we here at QA Hates You call it a beautiful, beautiful stack trace: Click for full sizeYeah, you don't see those every day. Because even QA gets Sunday off most weeks.

“Training Issue” Is No Parry Five

The Director — Fri, 19 Oct 2007 15:24:06 +0000

Some developers think the words "Training Issue" are a parry five when it comes to defect resolution. The developers will mark the issue as resolved/won't fix or some such nonsense, brush the crumbs of QA intransigence from their hands, and go back to voting for the Nintendo Wii over the XBox in an Internet poll. I don't know where the developers got the idea of this mythical training upon which they hope to rely to cover their deficient code; none of the places for which I've worked or contracted in this century have had actual training departments. Or much of a documentation department. So the developers who deploy the "training issue" defense really mean that a) This issue will be brought up in the 30 minute demo with the check signer where the check signer decides whether to sign that check or b) They really, really hope that a user doesn't find it or c) "Training Issue" means "customer support issue." No, those developers who deploy the "Training Issue" tactic often use it to defend the fact that they don't build in common-sense data validation, such as allowing the users to enter end dates that are earlier than the entered start dates or allowing users to enter strings where numbers are expected; in any event, the application passes bad data into the database for someone else to deal with somewhere. A couple lines of code and there wouldn't be a "Training Issue" to leave unmentioned, undocumented, and certainly untrained to the poor, unsuspecting users from the deadline day forward forever. Just so the developer can spend more time before lunch talking about the how soon Senjo No Kizuna will come to America instead of writing in boring validation code. So remember, ungentle reader, that "Training Issue" is not a valid response to an issue, as only the last part, "Issue," is true, and the first part, "Training," is a complete and utter fabrication.

You Must Disable Security Features To Use This Web Site

The Director — Mon, 22 Oct 2007 17:16:09 +0000

I am not going to pick on the Royal Caribbean Web site because it lacks a non-flash alternative for its main heading image, prompting users to download a plugin: Click for full size I am not going to pick on the Royal Caribbean Web site because, when you've got Flash installed, it plays an annoying bit of music and the sound control is small and in the corner with text that describes the current state, not what clicking the control will do: Click for full sizeI am not going to pick on the Royal Caribbean Web site, although I should, because it has hanging JavaScript errors waiting for a QAHatesYou.com bashing: Click for full sizeNo, friends, I am going to mock the Royal Caribbean Web site because it requires you to disable your pop-up blocker to work correctly. If you don't, you don't get to see the downloadable brochures for the expensive shore excursions the cruise line wants to bleed you for because they're blocked as unsolicited pop-ups: Click for full sizeThat is, Royal Caribbean has spent enough money on its Web site development. Y, loyal customer who routinely drops thousands of dollars on Royal Caribbean cruises and tours but who probably are not that technically savvy, need to flail around not knowing why it's not working and only when you call the Royal Caribbean technical support line learn that you need to disable the pop-up blocker that your Web browser recommends you turn on, that the giant software nonopoly who wrote your computer software recommends you turn on, and your computer-savvy nephew who comes by every time you're infected with a computer virus recommends you leave on. Just turn it off because Royal Caribbeans Web minions or the bean counters who capped the budget on the project and won't budge on inch to increase the usability of the site so that your downloadable brochures actually appear when most people click the link to see them. I mean, not building your Web site to accommodate the most basic default settings of your consumer Web browsers. How gauche.

That Plays Well In Europe

The Director — Mon, 22 Oct 2007 19:42:25 +0000

I don't know why the designers behind Cheer's current sweepstakes decided to go all continental with the date format: Click for full sizeI guess they wanted to see if I was paying attention. I was not. Thankfully, though, the validation helped me out, but I had to stop and think why my birthday was failing on the birth day of 20; obviously, though, that failed validation for a real month.

Unexpected Server Load = “Malicious Attack”

The Director — Tue, 23 Oct 2007 15:35:31 +0000

As you well know by now, the Colorado Rockies server suffered a meltdown when World Series tickets came on sale. Rockies officials claim it was a malicious attack:

Colorado Rockies officials said Monday their computer system for online-only World Series ticket sales was the target of an "external malicious attack," but online ticket sales were to resume at noon Tuesday. Team spokesperson Jay Alves couldn't immediately provide details of the attack.

You know what the attack was? Everyone wanted World Series tickets, and each customer was multiplied because as a group, they overcame limitations of 1 user 1 computer:

McLeod said he has Internet access from his apartment building but thought the library's computers might be faster. His mother, father, uncle and girlfriend were trying to buy tickets from other computers, he said.

Or some overcame bandwidth exemptions:

Super-fast computers normally used only during emergencies were to be staffed Monday so state employees could buy Rockies World Series tickets online. Until word of the plan got out, that is. "I need volunteers to help push buttons in attempting access," David Holm, recently the acting director of the state Division of Emergency Management, said in an e-mail obtained and released by KUSA-TV on Friday. "You will need to use break time, lunch time or leave time to do this and the only real perk I can offer right now is that if someone does not pay for their tickets within 3 days, you will get first crack at them."

And here's a guess: some ticket brokers or perhaps some developers came up with scripts that submitted orders to the Web site to improve their chances of getting tickets. That's not a "malicious attack." The servers failed from overwhelming demand. Of course, admitting that diminishes the brand, I suppose. But your company got to drink from the fire hose, Colorado Rockies; no one can fault you for failing to predict that your team would ever get to the World Series or what that could do to your systems. Other stories on the breakdown:

And as a reminder, ungentle reader, your friend and mentor The Director would have found a problem here because he doesn't run sissy load tests. However, no doubt the project would move forward even with the failure under extreme load because the stakeholders would be comfortable blaming DDOS.

Tripping Over Your Own Footer Links

The Director — Wed, 24 Oct 2007 19:36:34 +0000

You remember, our unofficial slogan here at QAHatesYou.com is Everything, every time. That's because sometimes the little things that a QA staff doesn't check because they've tested something very similar on a similar page or, heaven forfend, there's no QA team at all and nobody is checking the little things at all. Take, for instance, this contest to win a Chevy Camaro painted like the Transformer Bumblebee. Now, I'm not going to make too much fun of the drop-down list for country and its impact on the entry form, which is namely to change the labels on the state and Zip code edit boxes without fundamentally altering either the choices in them (limiting them to states or provinces, for example). Well, a little mockery is in order: Click for full sizeWhen you select United States as the country, the drop-down list is labeled State, but you can select "Canadian" provinces. Click for full sizeChoosing Canada as the country changes the label, but not the list available in the newly-minted "Province" drop-down list, which includes states. Usability aside, I'm sure that someone argued that the user had to explicitly select a country, which is why that list is available at all (because rules and laws differ from country to country). I mean, selecting a province and entering a Canadian postal code would have sort of implied Canada was the country, but there are hoops that sweepstakers have to jump through. No, I am here to snicker about the footer link that leads to a 404 on the thank you page: Click for full sizeMaybe someone checked the Rules and Regulations link on the entry form page. But no one checked it on the thank you page, where it was broken. I mean, come on; this promotion has a total of like 3 pages (entry, thank you, and rules) and the developing organization managed to make a fundamental mistake on one of them. That's why you have QA check everything, every time. Even the footer links that are on all of the pages.

Deviant Samsung Mini Sweepstakes

The Director — Thu, 25 Oct 2007 14:17:58 +0000

Kudos to the Samsung Mini Sweepstakes for deviating from the norm in ways fashioned to confuse the users. Here's the entry form for the contest: Click for full sizeNotice that Samsung's interactive agency has seen fit to mark, with asterisks, the optional fields; the de facto standard is to mark the fields that are required. The footnote is at the bottom, too, certainly serving to confuse people who rely on text readers who won't hear that until the end. Good work on inserting the e-mail address field in the middle of the street address, which can cause those who type automatically to trigger validation messages.Additionally, there's the obligatory JavaScript error: Click for full sizeOf course, when one completes the form, removing the checkmark from the opt-in checkbox, if you have Internet Explorer's pop-up blocker enabled, you block a pop-up which might be a success message or it might be a warning that the FBI is coming for your attempt to access Samsung's mainframe. Of course, if you don't know what it means and fill out the form again, congratulations! You've voided all of your entries. That's not a bug, that's a feature!Then, one receives an e-mail message whose from address includes a misspelled word that thanks you for opting into the mailings, which, of course, you didn't do and which is a very bad thing. Back when I worked at an interactive agency, I would never have allowed a sweepstakes with these errata onto the Internet. However, the standard seems to be much lower on the World Wide Web at large, but I don't understand how brands, particularly technology companies, think it helps their image to have buggy, ill-designed promotions associated with their names.

J. Deitch Wasn’t Listening

The Director — Thu, 25 Oct 2007 14:24:39 +0000

Remember when I told you how to check your PDFs? Apparently, J. Deitch, who works for someone who does Amazon.com's promotions, wasn't listening. Here are the properties in a PDF file, the official rules for the Amazon.com Heroes Custom Xbox 360 Giveaway Sweepstakes: Click for full sizeJ. Deitch has provided us not only with the title of the file from which he or she copied the current document, but he or she has also provided us with his or her name so we can mock directly. Thanks!

Payout A Software Bug

The Director — Thu, 25 Oct 2007 17:53:09 +0000

You really hate to see stories like this one:

For about an hour last August, Gary Hoffman was a very lucky man. Hoffman was playing the nickel slot machines at the Sandia Resort and Casino on an Indian reservation in New Mexico when he appeared to hit the jackpot: the machine said he won nearly $1.6 million.

"I became ecstatic," he said. But the ecstasy was short-lived. Hoffman says in a lawsuit filed earlier this year that Sandia refused to pay, claiming that the machine malfunctioned. Instead, he said, they gave him about $385 and a few free meals at the casino. "I won money, fair and square, and I've been cheated out of my winnings," Hoffman told ABC News. The casino says it's not responsible for what it describes as a computer error and says it offered Hoffman the maximum payout of $2,500 for that particular slot machine. But, a jury may never decide who is right. Lawyers told ABC News that gamblers like Hoffman may have little legal recourse against Native American casinos, which sometimes operate beyond the reach of U.S. courts.

Because the more this becomes common place ("It's a bug" or "It's a malicious attack!"), the more fault-tolerant our society becomes. Which is okay in the course of online sweepstake, but when all standards for business machinery fall to those low levels, certain non-fungible values (money, altitude, and so on) will be held to the same lax standards.

Developer States The Obvious

The Director — Thu, 25 Oct 2007 18:03:15 +0000

Joel Spolsky provides your team with a handy checklist to ensure that your team collects all the easy ways to fail in software development projects. Count how many apply to your current situation!

Typo, or Something More Sinister?

The Director — Mon, 29 Oct 2007 19:14:28 +0000

Fox News's Suzanne Sena has an all-Flash Web site complete with the Click to Activate Control box. But it's not the interface that's sinister; it's the text within her bio. Click for full sizeJust how many times did Ronald Reagan die, or how did Suzanne Sena cover it more than once? Parallel universe shifting?Probably not; judging by the rest of the copy, her personal PR folk lack complete fluency in written English. But just maybe.....

Evidence-Based Scheduling in FogBugz

The Director — Wed, 31 Oct 2007 19:24:10 +0000

Joel Spolsky of Fog Creek Software explains some of the thinking behind Evidence-Based Scheduling included in the new release of FogBugz.

Over the last year or so at Fog Creek we’ve been developing a system that’s so easy even our grouchiest developers are willing to go along with it. And as far as we can tell, it produces extremely reliable schedules. It’s called Evidence-Based Scheduling, or EBS. You gather evidence, mostly from historical timesheet data, that you feed back into your schedules. What you get is not just one ship date: you get a confidence distribution curve, showing the probability that you will ship on any given date.

Honestly, that's what you ought to be doing if you're taking a scientific approach. However, your organization and its schedule builders aren't scientific, preferring instead to build timelines and effort estimates to fit external constraints, deadlines, or budgets instead of reality. So, carry on with those unwritten tasks of covering your rump when failures occur.

Checkbox Fun

The Director — Wed, 31 Oct 2007 20:56:17 +0000

We at QAHY.com are not the only ones who have fun mocking Web form design; the Loose Wire Blog has found an instance of confusing checkbox, sleazy syndrome. Opted in if you do, opted in if you don't.

If It Needs Third Party Explanation, It’s Poor Design

The Director — Fri, 02 Nov 2007 15:45:47 +0000

We're not talking about a robust, enterprise-level or core business product software application here, we're talking about a sweepstakes entry form:

If you're trying to sign up for the Fred Claus instant win sweepstakes and you're having trouble seeing where the submit button is in the registration form, it might be because the page uses two arrows on the right-hand side for navigation. Those arrows can be easy to overlook, especially since the form looks finished on the first page, but you have to go down through a couple more pages before you see the submit button.

You know, if you're designing a form for general consumers, perhaps you should make it so that general consumers can understand it. By the way, I guess my secret is out; that Web log is where I find a lot of the sweepstakes/promotions that I complain about here.

Amateur Hour at KMOV

The Director — Tue, 06 Nov 2007 01:39:11 +0000

You know, I don't go looking for problems on the general Web. I mean, I'm paid to do that sort of thing, so when someone pays me, then I start applying my dedicated efforts to flush out problems with Web sites and whatnot. However, I am very, very observant and I notice the little things whenever and wherever I go. But some Web sites are just so rife with problems that I can find handfuls of them just trying to use them. For instance, the Web site of KMOV, channel 4 in St. Louis. I was just looking through it trying to find information about one of the stories I'd seen previewed during a football game yesterday. I wanted to post something on my Web log, but the site makes it hard to link to little bits of video. But my goals and the fact that the site was not designed to easily accommodate linking or embedding video in blogs is neither here nor there; what is important are the obvious problems I encountered. Because these are things active news consumers, that is, bloggers will encounter trying to perform the simplest task. For starters, the site uses Flash for its embedded video and because it's amateur hour over there, of course the Click to Activate and Use This Control box displays. However, as an added double plus ungood, it will display over the top of the fancy schmancy dynamic menus: Click to see full size Of course, to access anything that overlays the Flash presentation, you have to click once to activate the control and then click a second time to actually navigate.So I'm trying to find something in particular. So I use the Search function. And the results page displays with a JavaScript error: Click to see full size Regardless of the odds stacked against me, I find what I'm looking for, an online video, and it's what I want. KMOV pops its own wrapper around Windows Media Player and displays the thing in its own browser window. Meanwhile, the main window redirects to a registration form. You can't get away from it by going back to get to your search results, for example.I just click the Become a Member button, and server side validation occurs, but look what happens: a line of code appears as the : <center> <a href="http://www.qahatesyou.com/images/kmovtitle.jpg" target="_new"> <font size="1"><em>Click to see full size</em></font> </a> </center>Dude, your includes are showing!Okay, that's odd. I try to recreate it by copying the URL of the form and pasting it into another browser window, and I get a stack trace: <center> <a href="http://www.qahatesyou.com/images/kmovdirectnav.jpg" target="_new"> <font size="1"><em>Click to see full size</em></font> </a> </center>Never mind, then. I figured it out; the title is okay on the page itself, but it's the server-side validation redisplay of the page that mucks it up.Never mind that. Another thing about the form is the help that displays when you mouse over the little question mark icon. Nice idea, but I notice that it appears in a static place on the page. Which means that if I change my browser window size, I can actually make this help display on a portion of the page that isn't currently displayed, like so: <center> <a href="http://www.qahatesyou.com/images/kmovhelp.jpg" target="_new"> <font size="1"><em>Click to see full size</em></font> </a> </center>So, as I said, I found what I was looking for, but the site isn't designed for ease of use. I mean, I wanted a particular investigative segment's promo. The search result threw open the browser window to play it, but I wanted a single link that would let me link to it directly. So I decided to try the E-mail form that lets you send a link to an e-mail address.It worked. Of course, I noticed that the word e-mail had a hyphen in the form. I wanted to get a screen grab of it for you gentle reader, so I got the form up again. Because I'm naughty, I just clicked send without filling out the required fields.Hey, what does the application care? It showed the confirmation message: <center> <a href="http://www.qahatesyou.com/images/kmovconfirmempty.jpg" target="_new"> <font size="1"><em>Click to see full size</em></font> </a> </center>That's handy.It's almost petty of me, then, to point out spare bullets just hanging out in the dynamic submenu bar, but pettiness makes me succeed: <center> <a href="http://www.qahatesyou.com/images/kmovbullet.jpg" target="_new"> <font size="1"><em>Click to see full size</em></font> </a> </center>Jeez, Louise, in about 20 minutes of tinkering, I've uncovered some pretty glaring problems with the Web site, and all I was trying to do is <em>use</em> it. Well, mostly.So, ungentle reader, do you think that KMOV's Web developer: <p align="left"> </p> <form align="left"> <input type="checkbox" />Used an intern to test it <input type="checkbox" />Didn't test it at all <input type="checkbox" />Ran out of time to test it <input type="checkbox" />Ran out of budget to test it <input type="checkbox" />Didn't think anyone would notice <input type="checkbox" />Thought that only QA would care, thought problem solved by not having QA at all </form>Please note that I selected the checkbox motif so that you could, in fact, select all, even though some of them sort of contradict each other, because in the IT world, the hobgoblin of small minds is adherence to the constraints of reality and logic. </article> <article> <h1>A for Effort, F for Result</h1> <p>The Director — Tue, 06 Nov 2007 19:27:21 +0000</p> <a href="http://www.ff0000.com" target="_blank">Red Interactive Agency</a> relies on a gee-whizzery front page running Adobe Flash and almost handles users who don't have Flash installed elegantly. <center> <a href="http://www.qahatesyou.com/images/contrast.jpg" target="_new"> <font size="1"><em>Click to see full size</em></font> </a> </center>I mean, putting the alternative text in #333333 grey against a #000000 background? Who is that helping? </article> <article> <h1>Remember the Day the World Stopped?</h1> <p>The Director — Wed, 07 Nov 2007 20:39:46 +0000</p> A project manager <a href="http://softwarenerd.blogspot.com/2007/10/guilt-free-software-development.html" target="_blank">who said no</a>. (Link seen on another <a href="http://flibbertigibbet.mu.nu/what_it_means_to_say_no." target="_blank">project manager's site</a>.) </article> <article> <h1>Flash Is AWESOME!!!!!: Counterpoint</h1> <p>The Director — Fri, 09 Nov 2007 05:22:50 +0000</p> The singular of data is anecdote, and <a href="http://www.mcgehee.cc/index.php/zone/comments/i_swear_to_god/" target="_blank">this fellow</a> doesn't care for Flash-heavy Web design: <blockquote> <p class="body"><font face="palatino linotype,serif" size="3">Website designers these days need to be rounded up, dragged off and shot. </font> <font face="palatino linotype,serif" size="3">Who was the genius who decreed that Flash animation on a website is more important than content and navigability? He needs to be dragged off, shot, and then shot again. And then all of his relatives need to be shot. And his little dog, too. </font></blockquote> We here at QAHY are far more lenient than this civilian, as we only wish harm upon developers and designers and do not recommend sanction against their families. </article> <article> <h1>One Fewer Thing To Complain About</h1> <p>The Director — Fri, 09 Nov 2007 21:02:31 +0000</p> Microsoft, apparently, is tired of hearing QAHY.com complain about <a href="http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9046245&source=NLT_PM&nlid=8" target="_blank">this</a>: <blockquote>Microsoft Corp. will strip a "click to activate" warning from Internet Explorer 7 starting next month, a company product manager said yesterday, a side benefit of the settlement that Microsoft struck with Eolas Technologies Inc. in August. Next month, Microsoft will preview the modified Internet Explorer (IE) that eliminates the warning that's been popping up on screens when users select multimedia content, such as clicking on a link to a Flash file or a PDF document. That notice first appeared in IE in April 2006, when Microsoft began requiring users to approve ActiveX controls the first time they were run from the browser.</blockquote> Of course, ungentle reader, you'll be wise to continue testing for it, since the penetration of Internet Explorer 7 is only about half of the total IE market share. So don't let your designers point to this as an excuse to be lazy. Because if there's one thing designers like almost as much as being lazy, it's an excuse in their own minds that being lazy is okay. </article> <article> <h1>Client Side Or Server Side Validation?</h1> <p>The Director — Mon, 12 Nov 2007 21:07:34 +0000</p> As some of you know, I am quite the proponent of client-side validation for Web applications and Web sites. The main reasons are speed and load.  A Web application that expects a pile of use could better be served if its pages are smart enough to capture common problems without sending a server request and making your poor, creaking Web server figure out what's wrong. Basic stuff that client-side logic, generally through JavaScript, can determine includes making sure the user enters required data, enters information in accepted patterns for the expected data (zip codes, e-mail addresses, and so on) and date validation. That way, when your Web server is hammered by a blog linking to your site, you can rest assured your confusing interface will not necessarily bring down your site. On the other hand, there's the speed issue. When users click Submit and send off that form data to your Web site, they'll have to wait again for your wizbang media-rich page to load again or worse, wait for two pages to load if you use separates page for the error message and the form. So what, your cool Web designers and infrastructure people say. Well, friends, I personally have used dial-up AOL in the not too distant past, and I know how slow normal pages are to load. Even if you're not dealing with customers on dial-up AOL at 33k, according to <a href="http://www.websiteoptimization.com/bw/0708/" target="_blank">these numbers</a>, 17% of users don't have broadband. Granted, your tech team and designers are perfectly willing to sacrifice the impatient amongst those slowpokes, but your clients or business users probably wouldn't like to hear that 10,000 of your 100,000 visitors are abandoning forms because they're slow and frustrating. So there are two compelling reasons to use the client-side validation, and they're small potatoes compared to the developers' counter-arguments ("It's too hard" or "It's too expensive" or "If the user has JavaScript disabled, it won't work!"). Good reasons versus poor practices, and you'll be lucky if you win using logic. Now, as a best practice, your organization should actually use both client-side and server-side data validation for the optimal data protection. This will handle any flaws or browser incompatibility in your client-side data validation. What would a QAHatesYou.com post be without pointing out how a Web form has not handled it very well? Ask, and ye shall receive! Newegg.com ran a <a href="http://promotions.newegg.com/NEemail/Nov-0-2007/Promo110107in/Sweepstakes/index.html" target="_blank">Guitar Hero III sweepstakes</a> that didn't follow these best practices. For example, client side validation is in place for the Birthday field: <center><a href="http://www.qahatesyou.com/images/neweggbirthday.jpg" target="_new"> <font size="1"><em>Click for full size</em></font></a></center> We'll ignore the extraneous space in the validation message because.... Well, no, we won't ignore it. Shame, shame!The application confirms the presence of values in the birthday, but not the validity. Notice we've moved on to checking that the user has entered a phone number: <center><a href="http://www.qahatesyou.com/images/neweggphone.jpg" target="_new"> <font size="1"><em>Click for full size</em></font></a></center> The application is perfectly happy to let the bad birthday, February 30, into the database. Even the server-side validation doesn't catch it.However, these two fields are the only ones with client-side validation on them; other things are caught in the server-side validation: <center><a href="http://www.qahatesyou.com/images/neweggserversidevalidation.jpg" target="_new"> <font size="1"><em>Click for full size</em></font></a></center> Not only does this lead to a page separate from the form to display the missing fields, but it also capitalizes the I in verify/verification. Development IDE quirk, no doubt. But do you see the other problem here?That's right; the application is validating on the default instructional text (Enter Email Address, Enter City, and so on). The application should validate to make sure that the user has entered something other than what the designers populated in the edit boxes by default. However, it's unsophisticated and a touch shoddy. So to make a short story long and to give you a set of bullet points you can cut and paste into your best practices document: <ul> <li>Always include client side validation on Web forms/applications to make them faster for users and to remove common data problems from the server load.</li> <li>Use client-side validation to ensure that user has entered information in all required fields.</li> <li>Use client-side validation to ensure that user-entered data matches known patterns for valid data, including Zip codes, dates, and other structured bits of data.</li> <li>Check to make sure user has changed any default instructional text prepopulated in controls.</li> <li>Make sure that validation messages are grammatically correct and spelled correctly.</li> <li>Make sure validation works consistently among browsers.</li> </ul> The Web will be a better place for it. </article> <article> <h1>Said It Before, Say It Again</h1> <p>The Director — Wed, 14 Nov 2007 21:22:30 +0000</p> David S. Linthicum explains in the <em>SD Times </em>why SOA projects in particular <a href="http://www.sdtimes.com/article/column-20071101-03.html" target="_blank">fail</a>: <blockquote>It’s just one of those things. Technical projects fail, and SOA is no exception. So, how do these failures occur? Like they always do: as a result of poor planning, lack of understanding or the inability to execute—and that’s the short list.</blockquote> <p align="left">As you and I know, this is a bright and shiny way of looking at it. The real reasons that projects fail are as follows:</p> <p align="left"></p> <ul> <li><strong>O</strong><strong>verpromise<em>s. </em></strong>Salespeople say that the application or project will do what it cannot. Do they do it willfully? Sometimes. In other cases, they're willfully ignorant of how technology and software development works. Their goal is sometimes happy customers, but always quotas met.</li> <li><strong>Fantastic timelines<em> </em>or<em> </em>budgets<em>. </em></strong>The account people or project managers go to the team and ask how long it will take do a given project or feature. Then, the account people ignore the underestimates given by the developers who want to look like Rock Stars and who think they can build Rome in half a day, minus time for coffee and <em>The Daily Show</em>. Instead, the account people fit estimates and timelines to what the client wants, not what is possible, and as the end of the timeline or budget approaches, the team rallies and takes some shortcuts that ultimately will leave scars. Such as abandoning the QA phase or ignoring the bad news QA presents them about the quality of the product.</li> <li><strong>Egos. </strong>The developers expect that they'll get everything right the first time and that testing their code is not only a waste of energy, but an insult to their manhood (women developers, too). Also, they feel that they're wearing Kevlar trousers and that working by the seats of their pants will be bulletproof. Their flawless improvisational skills will ensure that every bubble-gum-and-baling-wire solution they implement at the last minute without testing will work good enough!</li> <li><strong>Apathy. </strong>If a number of projects fail, the team will become inured to failure and its lack of consequences. Or perhaps the team doesn't take ownership because it's a death march or they're swapped into the project to compensate for others' failures. As the man <a href="http://www.etymonline.com/poems/tramps.htm" target="_blank">said</a>: <blockquote><em>Only where love and need are one, And the work is play for mortal stakes, Is the deed ever really done For Heaven and the future's sakes. </em></blockquote> </li> </ul> Now that's a short list of why projects fail. Drop your own favored technologies into the mix and you'll see the result is the same. </article> <article> <h1>Flout Convention At Your Peril</h1> <p>The Director — Thu, 15 Nov 2007 15:18:33 +0000</p> A year and a half ago, Peter Coffee of eWeek wrote a column exhorting software designers to <a href="http://www.eweek.com/article2/0,1759,1996204,00.asp" target="_blank">stick to convention</a> when designing application interfaces: <blockquote>The Web-browser actions of "back" and "reload," along with a history of visited locations, have become widespread user interface metaphors. Browser-style buttons invite us to explore file systems, review online documents and access digital media—whether or not we're on a network and whether we're viewing resources that are local or remote. It figures that just as we reach the point where everyone knows how these things work, the content creation community is changing the rules. Users have deep-rooted expectations about the semantics of "back" and "reload." "Back" means "show me what I was looking at before I was looking at this." "Reload" means "show me the present state of the content whose past state I'm seeing right now." When designers don't respect these conventions, users become confused and transactions are derailed.</blockquote> He's right, of course, and it goes without saying that in the absence of detailed requirements, testers will test applications to behave like all other applications. So keep your nonconformist genius to the metal sculptures you make in your backyards on the weekends. </article> <article> <h1>Making the Least of Opt Out</h1> <p>The Director — Fri, 16 Nov 2007 16:05:07 +0000</p> I get e-mails because I once signed up for KCStar.com so I could read an article there. I might have opted out once or twice before; I know I've tried to opt-out of the Twin Cities Star several times, but like a drunk-dialing ex, they keep ringing me up with ludicrous offers hundreds of miles from me. So I tried to opt out of the e-mails via the latest offer from KCStar.com, an offer targeted to KU and MU (that's University of Missouri-Columbia, not really an MU like my alma mater). Here's the link in the e-mail: <center><a href="http://www.qahatesyou.com/images/kcstarmail.jpg" target="_new"> <font size="1"><em>Click for full size</em></font></a></center>What would you expect that to take you to? Not where it takes you.  Instead of taking you to a simple opt-out form where you can enter an e-mail address, it redirects you to an error page: <center><a href="http://www.qahatesyou.com/images/kcstarerror.jpg" target="_new"> <font size="1"><em>Click for full size</em></font></a></center>I don't know if the creators of the e-mail gave a bad link or if the e-mail vendor encoded it wrong, but I do know that nobody tried clicking through on a test e-mail. But it's not a bug, <em>it's a feature!</em> So a user then has to go to the log in page, trigger a forgotten password reminder (probably), and then log in to opt out. That's a lot of steps, and most people (like me) will find it easier to continue deleting the mail unread. However, deleted mails are still mails sent, so the metrics still rock! Also, kudos to the KCStar Web team for having a JavaScript error on their error page: <center><a href="http://www.qahatesyou.com/images/kcstarerrorjserror.jpg" target="_new"> <font size="1"><em>Click for full size</em></font></a></center>The <em>pièce de résistance</em>. </article> <article> <h1>In Some Organizations, That’s SOP</h1> <p>The Director — Mon, 19 Nov 2007 18:47:33 +0000</p> An open-source, tech-geek-beloved project set to act like a for-profit, lowest-possible-bidder, fly-by-night, on-some-shore project? <a href="http://www.news.com/8301-10784_3-9818903-7.html?tag=nefd.top" target="_blank">Believe it</a>: <blockquote>Mozilla seems to have forgotten this, with The New York Times reporting that the upcoming Firefox 3.0 set to ship with only 20 percent of its remaining 700 "blocker" (serious enough to justify postponing a release) bugs resolved before it ships. Of course, Mozilla has already fixed over 11,000 bugs, according to Mozilla developer Asa Dotzler. Even so, that doesn't answer the apparent fact that the Firefox development community is planning to ship a product before a wide range of known blocker bugs are resolved.</blockquote> As if millions of developer voices suddenly cried out in terror and were suddenly silenced before anyone cared. Seriously, though, when confronted with critical issues uncovered with most projects, developers are quick to resolve those issues without fixing the problems using the martial art <em>egokido</em>. We in QA are surprised it's taken them this long to begin behaving that way in their avocational pursuits. </article> <article> <h1>Classic Blunder: The 7 Inch Lineman</h1> <p>The Director — Mon, 19 Nov 2007 19:40:13 +0000</p> Back in 2005, Electronic Arts sent out an update for its popular video game <em>Madden 2006</em>. The update included updated roster positions as well as a special "feature": a lineman who was 7 inches tall.  <p align="center"></p> <p align="center"></p> <p align="left">How did it happen? <a href="http://news.zdnet.com/2100-1040_22-5883301.html" target="_blank">Too easily</a>:</p> <blockquote> <p align="left">According to Phil Frazier, a "Madden" producer, the bug was the result of a typo in the spreadsheet that lists player attributes such as weight, height and team affiliation. Frazier explained that the spreadsheet is designed to accept players' heights in inches, and therefore expects a two-digit entry. Madden 2006 Frazier said the roster updates--which are required for players participating in the online version of "Madden" and which can also be downloaded and saved for single-player versions--are used by a relatively small percentage of the game's players. EA offers periodic downloadable updates during the National Football League season that take player trades, injuries and the like into account so players can have the most accurate team rosters. But in the most recent "Madden" roster update, King's height was entered mistakenly as "727," Frazier said. That effectively meant the system thought King was 7 inches tall.</blockquote> <p align="left">Kudos to the game engine for not choking on the "update" and rendering the data as presented, no ludicrous it seemed. However, this occurred because EA did not do two things:</p> <ul> <li>Build the application that imports the updated information from the NFL, in this case a spreadsheet, to include data validation that would have caught things like 60-foot tall linesmen and would have kicked that record out instead of turning 60 feet into 7 inches through the magic of the arbitrary. Remember, gentle reader, we at QA Hates You have identified <a href="http://qahatesyou.com/wordpress/2007/08/01/cheap-shot-your-applications-import-feature/">importers as a weak spot</a>.</li> <li>Visually scanned the 1000 or so records that the spreadsheet probably included to possibly catch the additional digit in this record. The visual scan isn't everything; the importer should handle all real data validation, but a quick check can often catch things before you try to import. The visual scan will probably prove most effective with really dirty data, but every look and touch on something helps.</li> </ul> Otherwise, you're just as well defending your quality with a 7" lineman. <blockquote> <p align="left"> </p> </blockquote> </article> <article> <h1>Merry Christmas from Be kin</h1> <p>The Director — Mon, 19 Nov 2007 19:50:59 +0000</p> An insert in a package of Belkin cables: <p style="text-align: center"></p> <p align="left">Because periods and Ls would have broken the printing budget.</p> </article> <article> <h1>Definition</h1> <p>The Director — Tue, 20 Nov 2007 14:07:00 +0000</p> In Software Quality Assurance, 1.33=C<sub>PK</sub> means you're having a late lunch at California Pizza Kitchen. </article> <article> <h1>Counterintuitive To Whom?</h1> <p>The Director — Tue, 20 Nov 2007 15:59:52 +0000</p> Really, even the journalists of <em>SD Times </em>ignore what <a href="http://www.sdtimes.com/article/LatestNews-20071101-04.html" target="_blank">QA might have been telling them</a>: <blockquote>It seems counterintuitive to think that the biggest time-sink in the application production life cycle would receive the least regard from development managers. However, a survey published by Forrester Consulting has revealed that this conundrum is the cold hard fact for many organizations. The objective of Forrester’s “Problem Resolution Survey Results and Analysis” was to determine where developers and testers are spending their time and to learn what is automated and what is not. The biggest time drain, according to the managers, directors and executives who responded: investigating and resolving application problems. According to Forrester, almost half of the respondents require more than an hour to document a problem, and a problem report uses six types of media on average. “That is interesting to us, given the large number of problems,” remarked Eldad Maniv, vice president of BMC’s Identify Software Business Unit. The respondents spend almost three out of every 10 hours (29 percent) in various stages of troubleshooting: documenting, reproducing or testing. On the average, a problem takes six days or more to resolve, and one in four of the problems reported by a QA or test group are returned as irreproducible.</blockquote> Of course, that's not really news to those of us on the ground in QA. When I'm asked to estimate how long something will take, I have to shrug; if the application works, it will take a couple of hours to run through the test cases. If there's something wrong, it will take until the ultimate deadline to not fix it. The most time consuming part of QA is the defect process; it's trying to figure out exactly what's going wrong and reproducing it as best possible so the developers don't ignore the issue report, and then it's arguing with the developers that it is something wrong, and then it's retesting the "fix" the developers put in, and then it's reopening the issue / rearguing the point, and then it's logging issues broken by the "fix," and then it's testing a fix that accidentally works, and then it's watching for the fixed issue to resurface in a later build because someone overwrote the fix with old code, and then it's .... Well, you get the point. That's what Software Quality Assurance professionals who aren't suck-ups have been saying, but it falls on deaf ears until a highly-priced analyst firm comes to the same conclusions. </article> <article> <h1>Somethings Are Not Better Left Unsaid</h1> <p>The Director — Tue, 20 Nov 2007 16:12:25 +0000</p> Put the CD in and let it automatically run, and you get: <p align="center"></p> <p align="left">That sounds important.</p> <p align="left"></p> <p align="left">Download the latest executable from the Web site, and when you run it, you get:</p> <p align="center"><a href="http://qahatesyou.com/images/setupexe.jpg" target="_blank"> <em>Click for full size</em> </a> <p align="left">What is Symantec's Norton Anti-Virus 2007 trying to tell you? If you poke around enough, you'll find that it's saying that it's not compatible with Windows 2000 machines; however, the installer itself is happy to keep that bit of knowledge to itself.</p> <p align="left">A better set of error handling would have saved me several hours of attempting the impossible and mucking around a Web site to figure out what was wrong.</p> <p align="left">But Symantec reminds me that <em>client time</em> is not worth as much to a profitable company as <em>developer time</em>.</p> </article> <article> <h1>Civilian Mockery</h1> <p>The Director — Tue, 20 Nov 2007 16:24:10 +0000</p> A civilian spots and <a href="http://www.dustbury.com/backlog/2007/11/docomunents_propagan.html" target="_blank">mocks an Adobe banner ad</a>. A civilian, mind you, not a professional who needs to continually find problems for sustenance. Keep that in mind when someone tells you that "only QA notices." </article> <article> <h1>Someone Let The Links Get Stale</h1> <p>The Director — Wed, 21 Nov 2007 19:07:19 +0000</p> If you try to launch the game <em>Sid Meier's Civilization IV: Beyond the Sword</em> without the disc in the optical drive, the application pops open the dialog box that says, essentially, "Stop, ye Pirate, and show me your papers that prove this game is yours!" The dialog box has a message (lacking in a serial comma and a period, I add insult to insultery) and a link: <p align="center"><a href="http://qahatesyou.com/images/clickformoreinformationonthiserror.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">It's a pretty standard link sort of thing, but since it's here on QAHatesYou.com, you can guess the result of clicking the link.</p> <p align="left"></p> <p align="left">That's right, the site rolls a 1 and suffers a critical fumble:</p> <p align="left"> </p> <p style="text-align: center"><a href="http://qahatesyou.com/images/moreinformation.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">Okay, it's not a critical fumble, really, only a high fumble. Instead of information about the error, the link displays the home page for the game's company. The URL indicates that the application has passed in some querystring parameters, but the site doesn't know what to do with them. Better still, a quick glance through the site indicates that the information I'm expecting based on the promises of the desktop application dialog box <em>doesn't even exist.</em></p> <p align="left">I don't know whether the site used to handle those parameters or not (probably not) or if the application developers never talk to the site developers, but I give even odds nobody clicked that link when they built the application and nobody has clicked it since. Which can lead to a frustrating experience for users, but developers consider them second class citizens anyway.</p> <p align="left">So, ungentle reader, what can you do as a QA thug to inconvenience your organization with these sorts of details? Funny you should ask.</p> <p align="left">Of course, you can use an automated link checker to run through your sites, which is good for Web sites and their internal links; however, this won't be enough because you also need to account for some incoming links and external links from within your Web site, and automated link checkers based on spidering principles (you point it at one link and it validates all pages that that page links to, and so on and so forth) might not be the best way to accommodate those considerations.</p> <p align="left">When working for an organization that maintained a fleet of brand sites, many times the brands linked from sites we maintained to other partner or affiliate sites over which we had no control. Because we maintained a very comprehensive set of knowledge about the Web sites we maintained (that is, our team often visited pages within the sites on a frequent basis), we easily identified links to external sites within our pages. We then created a spreadsheet listing those external links and clicked through them every once and again to make sure that a redesign on the target's part didn't leave our site linking to a 404 or some out-of-context information.</p> <p align="left">Even if you're not doing Web sites, but instead test desktop applications, you should ferret out any Web site links included in your Help system or About dialog boxes (as well as various and sundry dialog boxes like the one noted above) and make sure that, over time, they don't break.</p> <p align="left">Now, about the incoming links; it's hard to keep track of where just anyone will link to on your site, but you should keep a running list of landing pages or directories that your e-mails or applications link to and make sure that they handle incoming traffic gracefully after a site redesign or a program or promotion ends. If you put a contest at /contest/ or /contest.cfm, you better make sure that that directory and that page work even after the contest ends and your organization has upgraded to .NET, because that link is out there on one or more contest sites and is sitting in consumer e-mail boxes somewhere, and you don't want users to visit your site through one of your e-mails and get a stack trace or 404.</p> <p align="left">I mean, it's not rocket science, but it is scut work for a developer or designer or organization to implement this as a best practice and to adhere to it.</p> <p align="left">And it's your job to make sure they do.</p> <p align="left">So open up your favorite spreadsheet and start tracking those links. Or I might happen upon your product or project and you might see it right here, mocked before the masses.</p> </article> <article> <h1>Sending An Alert To Do A Confirm’s Job</h1> <p>The Director — Wed, 21 Nov 2007 19:24:56 +0000</p> When you're listening to the online stream for <a href="http://www.sportsradio1250.com/" target="_blank">WSSP-AM</a>, the sports station in Milwaukee, it prompts in a very AOL Idle Message like fashion:  <p align="center"></p> <p align="left">You are, you just don't know it. And you cannot avoid it.</p> <p align="left">Of course, I should have expected as much from an organization whose registration form features the opt-in as a read-only checkbox <em>that is already checked</em>:</p> <p align="center"><a href="http://qahatesyou.com/images/enroll.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">By the way, yes, I know the Confirm box says <em>OK</em> or <em>Cancel</em>, but it made for a better title, and the Confirm box wouldn't work with the current text; but my job isn't to fix issues, it's to point them out and then tell you you're not fixing them right.</p> </article> <article> <h1>Memo To Bottom Liners</h1> <p>The Director — Wed, 21 Nov 2007 19:25:58 +0000</p> Failure is cheap. </article> <article> <h1>Classified Ad Broken Image Link</h1> <p>The Director — Sun, 25 Nov 2007 21:55:23 +0000</p> I'd be very disappointed with my service provider if I paid extra to get a photograph included with my printed classified ad and instead got a placeholder error message: <p align="center"><a href="http://qahatesyou.com/images/ad.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Maybe if you refresh the page by buying another copy of the free newspaper.</p> <p align="left">One must wonder if this set of classifieds is drawn from the actual source used in the online advertisements at the paper's Web site and if this actually drew in a broken image placeholder. I cannot understand it any other way.</p> </article> <article> <h1>It Just So Happens That Your Friend Here Is Only MOSTLY Dead</h1> <p>The Director — Sun, 25 Nov 2007 21:59:30 +0000</p> A <em>Civilization IV: Beyond The Sword </em>warrior gets lucky: <p align="center"><a href="http://qahatesyou.com/images/civiv.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">When my health reaches 0.0, I hope I look that vertical.</p> <p align="left">That being said, I am very glad I don't test games; there are too many variables for me to account for in games. Still, on the whole, they seem to work, don't they?</p> </article> <article> <h1>“Yahoo!” Is What I Said When I Crashed It</h1> <p>The Director — Mon, 26 Nov 2007 20:45:11 +0000</p> <ol> <li>I have multiple machines here in the QAHY lab.</li> <li>I have the Yahoo! Messenger program installed on multiple machines and it's set to automatically log in on a couple.</li> <li>Yahoo! allows a single user to log in only on one machine at a time.</li> <li>I use custom status messages to share my wit, so I often open the dialog box that allows you to enter that text.</li> <li>On patch or installation days, it's not uncommon for my PCs to contend and collide for which one is actually logged into Yahoo! Messenger.</li> </ol> I say this so you'll understand that I wasn't looking for trouble with Yahoo! Instant Messenger. I was just using the software like I normally do.  But when I had that <strong>Enter your status message</strong> dialog box open and another one of my PCs automatically logged into Yahoo!, it crashed the instance of Yahoo! IM that had the dialog box open. In Windows 2000, it looks like this: <p align="center"><a href="http://qahatesyou.com/images/yahoo2000.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left"> In Windows XP, it looks like this:</p> <p align="center"> <em>Click here for full size</em> <p align="left">Because of the business rule that you're only allowed to sign in on one PC at a time, you'd need a whole set of test cases to see what happens when the user has the program in various states when that automatic sign-off occurs. The test case that describes the presence of the <strong>Enter your status message</strong> dialog box, for example, fails in this instance.</p> <p align="left">Testing a rule like this or testing any application where users can work on the same record simultaneously multiplies the number of test cases, as you have to account for what happens on multiple machines or if the users operate on the same record concurrently. Too often, these tests are overlooked, even in robust applications.</p> </article> <article> <h1>Finally, A Contest For Gang Members</h1> <p>The Director — Tue, 27 Nov 2007 01:20:45 +0000</p> Volvo apparently has gang members in mind with <a href="http://volvodrivingschool.onlinepromo.com/" target="_blank">this contest</a>:  <p align="center"><a href="http://www.qahatesyou.com/images/streetname.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">I've never seen a contest that actually asks for your street name before. Incidentally, mine is T-Bone.</p> </article> <article> <h1>It’s Easy To Be A Cool Hipster In Anchorage</h1> <p>The Director — Wed, 28 Nov 2007 20:11:39 +0000</p> Because I signed up for a free subscription for <em>MacWorld</em> when I got my first testing Macintosh and because I've kept up the subscription because co-workers like the magazine, I'm subject to cool, hip advertisements for all the foolish bric-a-mac that the cool and the hip sell to the sheephipsters. The following advertisement always gets my goat, though, because when I worked at an interactive marketing agency, I would have shrieked about the implication. <p style="text-align: center"> </p>  <p align="center"><a href="http://qahatesyou.com/images/livelifeloud.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Take a good look at that clock. Now, take a good look at that sunrise. Now, put the two of them together and realize that the city you see in the distance must be Anchorage, Alaska, because frankly that's about the biggest city you can see in the distance from a place where the sun is only clearing the horizon at a little after 10.</p> <p align="left">Sure, I know why the graphic geniuses did that; they were working from the actual product photograph. If only designers had built-in common sense, though. But they're like developers: in love with their own genius and unable to see the gestalt.</p> </article> <article> <h1>The Poor Forgotten Thank You Page</h1> <p>The Director — Thu, 29 Nov 2007 17:07:06 +0000</p> The poor, lonely confirmation page never gets the attention it needs during the testing cycle of a sweepstakes or promotion. You've seen me identify problems with them before, and you'll see me identify problems with them again, very soon if you click the more link.  <a href="http://www.frontgate.com/text/content/landing/lexussweeps2007/lexus_swpstks.jsp" target="_blank">This sweepstakes</a>, hosted by someone called Frontgate, offers you the chance to win a Lexus. Hey, who wouldn't want to win a Lexus? If you're the type to read the fine print, you click the Terms and Conditions link whose lawyerly information is vetted and fretted by the legal team. Here's the goods: <p align="center"><a href="http://qahatesyou.com/images/lexus1.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">Everything you wanted to know about the contest, including the official name, which I don't see anywhere on the page itself. Okay, now fill out the form.</p> <p align="left">Ignoring, of course, the JavaScript error on form submission:</p> <p align="center"><a href="http://qahatesyou.com/images/lexus1js.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">And the JavaScript error on the page load:</p> <p align="center"><a href="http://qahatesyou.com/images/lexus2js.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">The confirmation page that displays has another link to the terms and conditions. Say you want to check out what you just signed up for. Click it, and, oops:</p> <p align="center"> <em>Click here for full size</em> <p align="left">That's the link to the terms and conditions of a companion program, but a lot of times you'll see cut and paste from previous promotions here that lead to incorrect information or stale links that lead to 404s.</p> <p align="left">Now, someone high on the project's special Rationalization Team will say that....</p> <p align="left">Well, something. When they open their mouths, I tend to hear ambient noise in the room. Go ask your legal team if that's good enough. I doubt it will, and I bet your RT will indicate perhaps the client won't notice.</p> <p align="left">Sure, it's a little bit more time-consuming to check the confirmation pages if you have to submit the form to get to them, but that's what they pay QA badly for.</p> <p align="left">No, the thank you page is indeed something to which the Software QA mantra, <em>Every thing, every time</em> should apply.</p> </article> <article> <h1>That’s Probably Public Domain By Now</h1> <p>The Director — Fri, 30 Nov 2007 17:19:37 +0000</p> Wow, that's an interesting copyright date; the actual forms must have been filed in Rome: <p align="center"><a href="http://qahatesyou.com/images/copyright.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">No, wait, that's not the real copyright date; it's merely that a provider of Web solutions has a product Web site with obvious flaws, such as laying out incorrectly and actually <em>giving incorrect data</em> when viewed in Firefox. But that's not all, of course.</p> <p align="left"> Including a broken link in the navigation:</p> <p align="center"><a href="http://qahatesyou.com/images/reyreybrokenlink.jpg" target="_blank"> <em>Click to view full size</em></a> <p align="left">If you go to the parent company's site, you'll see that it's got a nice JavaScript error right on the front page.</p> <p align="center"><a href="http://qahatesyou.com/images/reyreyjs.jpg" target="_blank"> <em>Click to view full size</em></a> <p align="left">If you click around a bit, you'll find that they're hiring a Quality Analyst. Qualifications: an associate's degree.</p> <p align="left">I was recently speaking to a project manager I know, and she mentioned not understanding how some companies can trust their applications/Web sites to understaffed, underqualified QA departments. Do they not understand that a good QA team can prevent the organization from making a fool of itself in front of clients and users, or do they spend the least amount of money they can and hope for the best?</p> <p align="left">Probably the latter. Again.</p> </article> <article> <h1>The Exact Moment of Cut and Paste Failure</h1> <p>The Director — Tue, 04 Dec 2007 18:47:16 +0000</p> Cut and paste makes everything better, right? Well, no, it gives you a chance to replicate your mistakes multiple times. For example, <a href="http://www.frontgate.com/" target="_blank">Frontgate.com</a> shows us exactly where the designer/developer used cut and paste on the Web site. At the bottom, a set of callouts points you to different sections of the site; the guy who inserted the alt text decided to call the images to the left of the callouts "emblems." Well, mostly. He got this one right: <p align="center"><a href="http://qahatesyou.com/images/frontgate0.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">However, one typo, one CTRL+C, and 2 CTRL+Vs later, we've got a problem:</p> <p align="center"> <a href="http://qahatesyou.com/images/frontgate1.jpg" target="_blank"> <em>Click for full size</em></a> <p align="center"><a href="http://qahatesyou.com/images/frontgate2.jpg" target="_blank"> <em>Click for full size</em></a> <p align="center"> <a href="http://qahatesyou.com/images/frontgate3.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">It's such a common function that we use it without thought, but when you're pasting, you should probably make sure what you're pasting is correct; otherwise, it's retyping, search/replacing, or blushing.</p> </article> <article> <h1>Another Maxim</h1> <p>The Director — Wed, 05 Dec 2007 18:53:23 +0000</p> Sometimes, even if you have a bunch of tools, all problems still look like a nail if you really want to use a hammer. </article> <article> <h1>Don’t Worry, Though; This Only Happens When QA Does It</h1> <p>The Director — Thu, 06 Dec 2007 21:28:12 +0000</p> Altering the URL in the wild? Never happens, the developers and project managers will tell you while subtly waving their hands and hoping you're as weak-minded as they think. Ignore QA's shrieking that the application is behaving inappropriately. Here's the never happening in a <a href="http://www.theglobeandmail.com/servlet/story/RTGAM.20071204.wpassport1204/BNStory/National/home" target="_blank">big way</a>: <blockquote>A security flaw in Passport Canada's website has allowed easy access to the personal information - including social insurance numbers, dates of birth and driver's licence numbers - of people applying for new passports. The breach was discovered last week by an Ontario man completing his own passport application. He found he could easily view the applications of others by altering one character in the Internet address displayed by his Web browser. "I was expecting the site to tell me that I couldn't do that," said Jamie Laning of Huntsville. "I'm just curious about these things so I tried it, and boom, there was somebody else's name and somebody else's data."</blockquote> The site should have told him he couldn't do that. However, since no real user would do this, the developers of the application didn't see fit to account for it. (Link seen on <a href="http://www.techdirt.com/articles/20071205/190901.shtml" target="_blank">Techdirt</a>.) </article> <article> <h1>Don’t Ask, Or You Might Get An Answer</h1> <p>The Director — Thu, 06 Dec 2007 21:33:03 +0000</p> Over at <a href="http://www.techdirt.com/articles/20071204/093451.shtml" target="_blank">TechDirt</a>, <a href="http://www.wsbtv.com/news/14740712/detail.html" target="_blank">this story</a> prompts a question: <blockquote> It’s one thing to bounce a check and it’s another to be so far in the red Bill Gates, Warren Buffet and Donald Trump combined couldn’t come close to bailing you out. A Cobb County man got a letter from his bank with that very shocking news.</blockquote> <blockquote>“And I open up the letter and I look at it and I’m like, ‘No, you’ve got to be kidding me,’ said Joe Martins.Martins said he recently closed an account at Wachovia Bank and made good on an outstanding check. He just got a letter about the closure and his negative balance -- $211,010,028,257,303.00. That’s $211 trillion.</blockquote> TechDirt asks: <blockquote>Furthermore, if an error this size gets through all of the checks and balances, then what other, less noticeable errors are falling through the cracks every day?</blockquote> Let's just put it this way: Your humble director doesn't use online bill pay, okay? </article> <article> <h1>The First Step</h1> <p>The Director — Mon, 10 Dec 2007 18:32:22 +0000</p> The first step in fixing a problem is <a href="http://ask.slashdot.org/article.pl?sid=07/12/10/0744259" target="_blank">admitting you have a problem</a>: <blockquote><em>I am downright embarrassed by the quality of my code. It is buggy, slow, fragile, and a nightmare to maintain. Do you feel the same way? If so, then what is holding you back from realizing your full potential? More importantly, what if anything are you planning to do about it? I enjoy programming and have from a young age (cut my teeth on BASIC on an Apple IIe). I have worked for companies large and small in a variety of languages and platforms. Sadly the one constant in my career is that I am assigned to projects that drift, seemingly aimlessly, from inception to a point where the client runs out of funding. </em></blockquote> Unfortunately, a support group coalesces and tells the developer the equivalent of "I'm OK, You're OK." Which runs counter to reality, unfortunately, and cowboy coding continues unabated. </article> <article> <h1>I’ll Take That Under Advisement</h1> <p>The Director — Mon, 10 Dec 2007 18:38:32 +0000</p> What should you do if your banner ad or banner ad delivery software requires Adobe Flash Player 9.0 and the user only has Adobe Flash Player 7.0 installed? Not this:  <p align="center"><a href="http://qahatesyou.com/images/updatead.jpg" target="_blank"> <em>Click</em> <em>for full size</em></a> The best, or worst, part is that I don't even know what it expects me to upgrade to get the full experience of an intrusive and probably unwanted advertisement. Brothers and sisters, it's good that you're detecting the user's capabilities, but if you detect incompatibility with your marketing genius, you need to throw up a lowest common denominator JPG or GIF version of the ad or a different ad. Not a grammatically incorrect error message telling the user to do something unspecific for something he or she doesn't care. </article> <article> <h1>Chaotic Good or Chaotic Neutral?</h1> <p>The Director — Wed, 12 Dec 2007 21:09:21 +0000</p> Does alignment matter? The people who put together this <a href="http://www.kohls.com/upgrade/sweepstakes/kohlsswp_EntryForm.jsp?swp=Holiday07" target="_blank">Kohl's sweepstakes</a> don't think so.  For starters, all of the data entry fields are aligned flush left, except the unrequired apartment/suite field whose lack of an asterisk makes it flush lefter: <p align="center"><a href="http://qahatesyou.com/images/kohls1.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Then, when you type values into the fields, the text is aligned above the center of the text boxes:</p> <p align="center"><a href="http://qahatesyou.com/images/kohls2.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Perhaps it's from my cranky days as a printer (operating a web press, thank you very much, which all of the recruiters loved about me until I explained that a sheet of paper pulled taut throughout a press doesn't have anything to do with .asp), but I like things to be aligned correctly and completely. When handed a printed resume from a friend that he'd worked on for days and had Kinko's print on good paper, I promptly responded that it was crooked because the right side was 1/32nd of an inch lower than the left. I saw it plain as day.</p> <p align="left">Also, kudos to this rather unorthodox configuration:</p> <p align="center"><a href="http://qahatesyou.com/images/kohls3.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">The form designer has used radio buttons instead of the normal agreement to the rules checkbox; if you click the No radio button, the form triggers validation and won't submit. So not only is this field required, but the <em>correct answer</em> to this field is required as well.</p> <p align="left">Which is why most people go with the checkbox.</p> </article> <article> <h1>Discordant Instructions</h1> <p>The Director — Wed, 12 Dec 2007 21:31:41 +0000</p> Sometimes, it becomes obvious that the copy writers and the form designers are not sitting in the same room and are probably contending with each other for dominance in the creative department internal politics of interactive agencies everywhere. Or maybe it only becomes obvious that nobody is paying attention to the actual interface when writing the instructions or paying attention to the actual text when designing the interface. The following examples show a couple of places where these things don't confluentiate.  For starters, I'm always a fan of the separate thank you page for a successful form submission. When you do that, you have complete control of what's on the screen when you thank the user. If you just swap success text in place of the form, you have the chance (1-5 on a d6, 1-4 on a d6 for elves) that you're going to leave instructions on the screen: <p align="center"><a href="http://qahatesyou.com/images/tyext.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Notice how the form tells the user he or she can enter below, even though the application has removed the form to display the success message. The user <em>cannot</em> enter below. The application is faulty, and if this were an old <em>Star Trek </em>episode, Captain Kirk and the landing party would use this confusion to escape.</p> <p align="left">Here's a bit of legal text for a contest entry that no one involved with the form design has read:</p> <p align="center"><a href="http://qahatesyou.com/images/diablo.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">The instructions tell the user that he or she will have to read the rules and then click the Accept button, but then the form doesn't have an accept button. It doesn't even have an accept checkbox. So legally, I suppose, no one has complied with the terms of the sweepstakes, and the sponsor doesn't have to award the prize. Come to think of it, maybe someone <em>did</em> read it all.</p> <p align="left">Regardless, I find it exceedingly inelegant when the labels of the controls (or their presence) doesn't match the instructions given. In real applications, this often happens because the technical writers are told to document software that's still changing in the interface and/or because the freaking technical writers don't actually bother to run the applications they document. Special note to our technical writer readers: maybe not you, but some actually do that; trust me, I know. True anecdote: one day, after I made the move from technical writing to QA, my replacement came to me to ask me about the layout of a screen as I documented it in RoboHelp (street cred as a tech writer in that word, werd). Apparently, the developers referred to something on the screen that wasn't called the same thing in the documentation, so she decided to ask me about it. I went up there, looked at the doc, and said, "Well, let's open it up and look at it." But you see, my friends, she was actively updating the documentation for a desktop application <em>without actually running the application.</em> So she was flying blind, and no doubt her work showed it.</p> <p align="left">I'm not saying QA should review all technical documentation, although if you can spare the bandwidth, you do have a small case for doing so. However, you should make sure that <em>on screen instructions</em>, whether on a Web page or in a tooltip, make sense and are correct and grammatically sound.</p> <p align="left">If not you, then who? The client, the increasingly dissatisfied client, that's who.</p> </article> <article> <h1>Reminder To Our Client-Facing Co-Workers</h1> <p>The Director — Thu, 13 Dec 2007 17:10:58 +0000</p> Remember, when you try to whitewash a problem, use enough actual whitewash to adequately cover the problem. Otherwise, you're merely applying an antique finish to the problem. </article> <article> <h1>Contending Flash Ads</h1> <p>The Director — Thu, 13 Dec 2007 17:53:37 +0000</p> Kudos to <a href="http://www.computerworld.com/index.jsp" target="_blank">ComputerWorld</a> for the ability to lay banner ads over the top of other banner ads: <p align="center"><a href="http://qahatesyou.com/images/postini.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">You'll note that I have rolled over the expensive PointRollish IBM banner; however, the Postini sub banner ad continues to scroll over it.</p> <p align="left">Brilliant!</p> <p align="left">I'd tell you what the Postini thing is all about, but the link in the little Flash doohickey doesn't seem to resolve.</p> </article> <article> <h1>At Least They’re Not Offal Rules</h1> <p>The Director — Thu, 13 Dec 2007 20:47:55 +0000</p> A common typo on sweepstakes promotion pages, courtesy of Maxim: <p align="center"> <a href="http://qahatesyou.com/images/offcalrules.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left"> I don't know why, but misspelling<em> official</em> as <em>offical</em> in the official rules link happens an awful lot.</p> <p align="left">I know why it's not often caught, though: because <strong>quality hurts.</strong></p> </article> <article> <h1>What To Get Your Team For Christmas</h1> <p>The Director — Mon, 17 Dec 2007 20:27:09 +0000</p> At Christmastime, I like to give my employees a little something extra above and beyond the corporate gift card or gift exchange material. Something to say to my people that they're not just company employees, they're <em>my</em> team. Given that QA is often the us and the everyone else is them, I like to keep that distinction clear. So what do you get your group? Well, I recommend...  <p style="text-align: center"></p> <p style="text-align: center"> </p> Mogen David, preferably Mad Dog 20/20 if you can find it. It establishes the message that even though they treat us like bums, <em>I'll buy the wine</em>. Also, it is affordable on a QA leader's salary. Unfortunately, I've recently moved to a "respectable" suburb where the groceries don't stock Mad Dog, so this year my people got Mogen David. </article> <article> <h1>They Probably Couldn’t Find It In The Dictionary</h1> <p>The Director — Tue, 18 Dec 2007 02:05:40 +0000</p> If they tried to look <em>professional</em> up in the dictionary the way they spelled it in the alt text, GMC wouldn't find it: <p align="center"><a href="http://qahatesyou.com/images/prfessional.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left"> </p> You can guess what professional grade I give them for sloppy quality. </article> <article> <h1>A Key Value, Undefined</h1> <p>The Director — Wed, 19 Dec 2007 04:50:43 +0000</p> I don't know what it means, but <a href="http://mlb.mlb.com/mlb/news/blogs/index.jsp" target="_blank">mlblogs.com</a>, the Major League Baseball compendium of official blogs, has a JavaScript Error on its home page: <p align="center"><a href="http://qahatesyou.com/images/mlblogs.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Something in the variable name, I don't know, maybe <em>key</em>, would indicate someone should have thought to define it.</p> </article> <article> <h1>Wherein Alyssa Milano Teaches Us A Lesson</h1> <p>The Director — Thu, 20 Dec 2007 12:30:08 +0000</p> You might have heard me say <em>Every thing, every time</em> before. That means you check every link on every page, too. Sometimes your leaders, developers, or other scoffqualities will encourage you to shortcut this by only checking the important links or only checking the links that have the same targets once and calling it a pass. But the lovely and talented (and by talented, I mean "lovely" although she's a better actress than Brett Favre) Alyssa Milano helps to illustrate this lesson.  A lot of times, the time shavers in your organization will encourage you to ignore checking the main navigational links on more than one page. The navigation is the same across the site, so the links that work on any page <em>obviously</em> work on any page. For example, here's the <a href="http://alyssa.com/gallery/index.html" target="_blank">gallery page</a> of Alyssa Milano's Web site: <p align="center"><a href="http://qahatesyou.com/images/milanointerior.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Notice that I've moused over the Message Boards link; notice, too, the URL that displays in the footer: http://boards.celebrityloop.com/forumdisplay.php?f=110 <p align="left">This URL is the correct URL and leads to the off-site message boards. Why you put an off-site location in the main navigation, I don't know; I assume it's from a site redesign.</p> <p align="left">I assume that because the same navigational link on the <a href="http://www.alyssa.com/" target="_blank">home page</a> is broken.</p> <p align="center"><a href="http://qahatesyou.com/images/milanohome.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Here, you see that I've also moused over the Message Board link on the home page, and it targets: http://alyssa.com/messageboard/msgboard.html <p align="left">This, friends, returns a 404 right from the home page, which I understand is not a best practice for roughly 40% of Web development companies and interactive agencies (and a full 7% wouldn't think this is an issue since there's a workaround: click another navigational link to a page with the correct target, then click the Message Boards link! See, no problem!).</p> <p align="left">Now, in most cases, you'd find the broken link on the home page because you'd test the navigation on it or because a lot of stake holders would try that combination; however, if the incorrect links were reversed, you might miss that the Message Boards link was broken on one interior page like the Gallery.</p> <p align="left">Unless you test every thing, every time.</p> <p align="left">Even though your development or design team would tell you that the navigational links are inherited, included, or somehow otherwise exempt from the need to test them independently on every page. Your development or design team often misunderestimates its own fallibility and doesn't include in that assessment all the loopholes it builds into the design, such as stray pages developed/designed earlier than the main site, developed/designed later than the main site, special promotional pages, or a host of other places where they have the chance to copy/include old or incorrect code or otherwise bollix it up.</p> <p align="left">So, ungentle reader, you need to ignore the bleatings and check all of those links on pretty much every page to ensure quality because the site's myriad user will find those exceptions to the developer/designer's "rules."</p> <p align="left">This includes the footer links, too, because Ms. Milano also points out that those are independent navigational links, too:</p> <p align="center"><a href="http://qahatesyou.com/images/milanohomefooter.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">That targets: http://www.alyssa.com/messageboard/msgboard.html?f=110 <p align="left">Which is a different broken link than the main navigation at the top, and a quick search and replace to fix the broken link or updated link would have a good chance of bollixing it up again or more.</p> <p align="left">So there you go, ungentle reader. You should check all the links, all the time, when you're testing a Web site. Maybe you've got a good automated tool to help with this, but you need to get it done.</p> <p align="left">That would be the proper concluding paragraph, but Ms. Milano's Web site offers a different lesson in branding that I thought I'd bring up before you're all done here and get back to exploring her galleries. Note the title tag for the site: Alyssa Milano.com.</p> <p align="left">This is a poor decision simply because the site is at alyssa.com, and alyssamilano.com leads not to the worst of all possible worlds, but to a cybersquatter site:</p> <p align="center"><a href="http://qahatesyou.com/images/alyssamilano.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">By intimating that Ms. Milano's Web site is Alyssa Milano.com, the designers and developers are unintentionally feeding traffic to the cybersquatter's Web site. Poor form, Peter.</p> <p align="left"> </p> </article> <article> <h1>Testing With Real Data</h1> <p>The Director — Fri, 21 Dec 2007 16:19:09 +0000</p> An article at Dark Reading explains <a href="http://www.darkreading.com/document.asp?doc_id=141017" target="_blank">Real Data in App Testing Poses Real Risks</a>: <blockquote>If you use real, live customer data in your testing and development of applications, you may want to think twice about the risks of exposing that data. Organizations that use live data in their testing do so basically because it makes the testing more real-world and better puts the app through its paces. Trouble is, it also can expose sensitive data to engineering staff who normally wouldn't have access to that data, as well as to consultants and other outside contractors working with your organization on the testing process. But you don't have to use the real thing in app testing and development: "It needs to be real enough, but it's better if it's not people's confidential information," says Gary McGraw, CTO of Cigital. Still, it's common practice among many organizations today. According to a new study from the Ponemon Institute, which was commissioned by Compuware, 69 percent of the over 800 IT professionals surveyed said they use live data for testing their applications, and 62 percent say they do so in their software development. Over 50 percent outsource their app testing, and of that group, 49 percent of them share live data with the outsourcing organization.</blockquote> The article conflates using <em>real</em> data with using <em>live </em>data, but it's really two different things, both of which comes with its own risks.  I know that the risky behavior that the article talks about, exposing real data for development and testing, occurs frequently; I have worked on a multitude of projects where the technology team gets its verisimilitude by copying a snapshot of the current, real, production database into a development environment or a testing environment. This has a number of risks and problems: <ul> <li>These environments usually aren't as secure as production environments and might not be as well monitored as production environments.</li> <li>The environment isn't usually as robust as a production environment, so the developers will blame performance problems on anemic machines processing enormous databases instead of, perhaps, problems with their impeccable code.</li> <li>Unvetted users can get access to the client data, including project managers, testers, and anyone drafted in the office for that last minute panicked testing push.</li> </ul> As bad as that is, the worse of the two occurs when QA is asked to test not only <em>real</em> data, but also in a <em>live </em>environment. I've worked on projects where the code is promoted hastily directly into production and the powers that is direct QA to test the application in production, where the data is live and any cool things QA could do (such as bringing down the horse) would be very, very bad. Testing an application, particularly testing an <em>untested</em> application when it's live come with its own set of risks: <ul> <li>Any defects found can have an adverse impact on the actual users and the system instead of a nice, safe development environment.</li> <li>Any defects not found but triggered can muck up real data without anyone catching it, which will impact other people's lives.</li> <li>Test data entries will exist in the live system, impacting any functions that aggregate the test data with real data. Someone might find <em>Ace Frehley </em>that lives on <em>1976 Kiss Avenue</em> and come to question the veracity of all data in the system.</li> </ul> Of course, the the risks in testing with <em>real</em> data or <em>live</em> applications and data aren't the only possible combination of disasters waiting to happen. You could get configuration problems where your test environments point to <em>live</em> data (that is, someone has entered the wrong database connection information in the stage environment) or your <em>live </em>applications can point to your test databases (that is, someone pushed the code into production along with connection information to the test data) or a host of myriad other potential problems when dealing with SOA or service-based application where instead of one database and one application, you have many. The moral? Be very attentive to the data and the environment in which you're testing, and when the developers say it's ready, have some way to check to ensure it's ready, whether it's verifying the connections and database names yourself or using only test data you enter yourself. The result? Your humble narrator continues to struggle with the fear that somewhere, somehow, some intern is going to refinance my mortgage at an interest rate of 999.99% while performing a standard boundary analysis sort of test in a live environment. </article> <article> <h1>Don’t Go Halfway With Your Validation Messages</h1> <p>The Director — Thu, 27 Dec 2007 18:09:24 +0000</p> You silly developers and your little halfhearted feints toward usability. Why don't you just go all the way and not put any effort into it at all? Take, for instance, the validation messages that display when the user doesn't enter required data in the Web form on the Washburn Guitars <a href="http://www.washburn.com/dbd/" target="_blank">Death By Decibels</a> contest.  On one hand, the validation is client-side, which is very, very good, but the validation messages use the variable names instead of the labels on the controls: <p align="center"><a href="http://qahatesyou.com/images/fname.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">I mean, it's not the worst case scenario that the variable names are <em>fname</em> and <em>lname</em>, but sometimes these forms trigger validation that say esoterica like <em>q32 is blank</em> or <em>tbSttate is blank</em>.</p> <p align="left">No, if you're not going to provide easy-to-use validation messages, developers, you might as well just put the validation logic in the message box. Because <em>fname=""</em> is only slightly less user friendly for your great aunt Maysie who's just on this Internet thing to enter contests for things she can win but probably wouldn't know what to do with, such as an electric guitar.</p> </article> <article> <h1>Maybe The Developer Was A Snake</h1> <p>The Director — Thu, 27 Dec 2007 20:41:50 +0000</p> The title bar of <a href="http://www.calphalon.com/calphalon/consumer/survey/sweepsSurvey.jhtml" target="_blank">thissss sweepsstakess ssssurvey page</a> indicates either a typo or something more unholy and eldritch: <p align="center"><a href="http://qahatesyou.com/images/calphalontitle.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">Eh, who reads the titles anyway, right?</p> </article> <article> <h1>Creative Validation Message Placement</h1> <p>The Director — Thu, 27 Dec 2007 20:44:06 +0000</p> Well, this <a href="http://www.1800goguard.com/movie/download.php?act=submit" target="_blank">National Guard sign up page</a> does, in fact, perform validation; however, it will place the error messages, one at a time, next to the First Name edit box label. Even if that message does not apply to that control: <p align="center"><a href="http://qahatesyou.com/images/nationalguard.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">Well, good enough for government work, apparently.</p> </article> <article> <h1>GIFT FOR THE DEVELOPER ON YOUR LIST</h1> <p>The Producer — Fri, 28 Dec 2007 18:44:28 +0000</p> While we here at QAHY love what we do, we are not the only ones having fun. Over at <a href="http://www.evilmadscientist.com" target="_blank">Evil Mad Scientist</a>, they have created a <a href="http://evilmadscience.com/else/62-bugs-stickers" target="_blank">sticker</a> which you can purchase in lots of 10 to be sure that all your developer friends have a good supply on hand for any projects they might be currently working on. <p align="center"><a href="http://evilmadscience.com/else/62-bugs-stickers" target="_blank"></a></p> </article> <article> <h1>And I Said, “That’s Good! One Less Thing.”</h1> <p>The Director — Sat, 29 Dec 2007 12:43:59 +0000</p> Web browser compatibility testing <a href="http://www.computerworld.com/action/article.do?command=viewArticleBasic&taxonomyId=16&articleId=9054521&intsrc=hm_topic" target="_blank">gets easier</a>: <blockquote>AOL LLC today pulled the plug on Netscape Navigator, the Web browser that once owned the lion's share of the market and that was the focus of a landmark federal antitrust case against Microsoft Corp. In an announcement posted to AOL's blog for the browser, Tom Drapeau, the director of the company's Netscape brand, said the team is ending development and would cease issuing security updates as of Feb. 1, 2008.</blockquote> It gets easier to justify not testing Netscape, anyway. Because, let's face it, the margin of error still using it will cling to it even though it's not supported. And let's face it, you didn't even know it was to Netscape 9 now, did you? </article> <article> <h1>Trusting Your Explosives To An XML Parser</h1> <p>The Director — Wed, 02 Jan 2008 18:32:30 +0000</p> Actually, I have no idea what technology was involved, but a corrupt file <a href="http://seattlepi.nwsource.com/local/345643_fireworks01.html" target="_blank">caused problems</a> at a Seattle fireworks display: <blockquote>A computer glitch led to a delayed fireworks show at the Seattle Center on Monday night as partiers waited to ring in the new year. The show started on time at 11:59, said Mary Bacarella, spokeswoman for the Space Needle, but in the last minute of 2007 the crew from Pyro Spectaculars realized that the computer file running the show was corrupted.</blockquote> When you're building the software, it's okay to overlook <a href="http://qahatesyou.com/wordpress/2007/08/01/cheap-shot-your-applications-import-feature/" target="_blank">validating and handling corrupt files</a> because only <a href="http://qahatesyou.com/wordpress/2007/07/16/ignoring-the-peril-of-the-badmins/" target="_blank">system administrators</a> would manipulate them, right? Might as well save that little bit of development money; it will help pay for your last paycheck. </article> <article> <h1>Two Out Of Three Ain’t Bad. No, Wait, It Is.</h1> <p>The Director — Thu, 03 Jan 2008 20:20:37 +0000</p> On the new HGTV <a href="http://www.hgtv.com/hgtv/dream_home_2008/text/0,,HGTV_30596_65584,00.html" target="_blank">Dream Home Giveaway sweepstakes</a> thank you page, the site provides you with a standard tell-a-friend sort of form so that you can send a link to one or more e-mail addresses. Easy as pie, right, and everyone does it. Well, HGTV.com loses some style points because it doesn't prepopulate your name and your e-mail address even though you've just provided them. It loses style points and more on the validation.  For starters, although it throws up a quick JavaScript alert when you fail to enter required fields (not marked, by the way, on the form), the copy in the message does not match form labels: <p align="center"><a href="http://qahatesyou.com/images/hgtv.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Hey, wait a minute. Your e-mail address and your name should be required when you fill out a tell-a-friend, but isn't there something else that should be required? Let's enter those two and click the button:</p> <p align="center"><a href="http://qahatesyou.com/images/requiredemail.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Well, two out of three ain't bad if you're a hitter in baseball or a Meatloaf lover, but it's statistically significant if you're developing a Web site. I know it's mathematically incorrect, but I'd just like to say this developer is batting .666.</p> </article> <article> <h1>Single Women Don’t Drive GMC</h1> <p>The Director — Thu, 03 Jan 2008 20:24:49 +0000</p> That's what I infer from the available choices in the title field on <a href="http://web.hgtv.com/webhgtv/dreamHome2008/gmc2008entryform/2008gmcentryform.html" target="_blank">this sweepstakes</a>: <p align="center"><a href="http://qahatesyou.com/images/nomiss.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">No doctors, either, I suppose.</p> </article> <article> <h1>ePrize and Prilosec OTC Channel Henry Ford</h1> <p>The Director — Mon, 07 Jan 2008 02:54:02 +0000</p> You can select any country you like as long as it's US: <p style="text-align: center"><a href="http://qahatesyou.com/images/prilosecotc.jpg" target="_blank"> <em>Click for full size</em></a> I never understood the logic of putting in controls that were read only or allowed only one choice, but the <em>smart ones</em> on the project team would tell you I'm not paid to think, only to say, "Looks good to me, sir!" Which explains why I tend to move from team to team very easily. </article> <article> <h1>Sometimes, Automated Testing Is Folly</h1> <p>The Director — Mon, 07 Jan 2008 23:27:32 +0000</p> In the message from the editor in the November 2006 (<a href="http://stpmag.com/issues/stp-2006-11.pdf" target="_blank">pdf</a>) issue of <em>Software Test & Performance</em>, Edward J. Correia expresses a basic belief in the magical potency of automated testing: <blockquote>Repetitive tasks are bad enough—having to perform them repetitively is insane. If something can be automated, it should be. Even if it takes 10 times longer and costs a hundred times more than the original task itself, it pays in the long run to automate your tests.</blockquote> I cannot tell you how many times I’ve gone into interviews and sales calls for projects where the other person across the table blurts out the benefits of automation and how the company wants to use automation to build a complex suite of automated tests to ensure 100% code coverage to run with nightly builds to ensure project success. But the automated test evangelists, speaking in their tongues, are wrong. Sometimes automated testing is a bane.  In many cases, I’ve found that the people who want the automation see it as a way to make up for a lack of QA resources. Often, in a tight schedule, the organizations hope that the tests can run overnight and supplant the hours that a tester would need to run the same tests. They have 400 hours total to build a Web application and 80 hours for testing, so they think that one person can come along, evaluate a tool, produce the automated test suite, and spend time in meetings. Automation, they hope, will help solve the problem the timeline and budget create for quality assurance. Automated testing is an investment that pays off over time, often a long time. If you’re building an application with a months-long release cycle and have an actual quality assurance team that can dedicate a person to doing automated testing, you will see a great deal of benefit in building an automated test suite. The repetitive tasks will be automated, and a basic set of regression tests can run against every build quickly so long as the subsequent builds don’t overhaul the main functionality nightly. The non-automated testers can attack the new functionality first with manual tests, flushing out the defects before the automated scripts are built—and broken—on defective code. Then, when new functionality stabilizes and the functional tests become regression tests, then the automated tester builds the scripts and begins running them nightly. But in smaller projects where the time or the additional resources aren’t available, automated testing is not the answer. If you’ve got one tester on a project, you should have him spending all of his testing hours on testing, not on building scripts which would account for one-tenth of the number of tests he could run while he’s building the scripts and then have the tester spend fifty percent of his remaining time troubleshooting the scripts and modifying them to make the last minute changes required by the last minute changes made to the code. Short timelines and limited testing resources don’t lend themselves to a good automated testing program; in fact, wasting time on automated testing actually diminishes the efficacy of QA as the poor, beleaguered tester doesn’t get enough time to run the maximum number of distinct test cases against the application and handle all the time-consuming tasks—that is, reproducing and tracking defects—that go along with any test effort. If your main purpose in a short project is quality, you’ll use a lot of manual testing and maybe some automated stuff filled in. But if you’re looking to evangelize or dazzle your clients or internal stakeholders, or if you want to expand your budget or pad your resume with a skill set, I guess automated testing will work for you in any case, regardless of the ten times spent on it or the hundredfold increase in cost. </article> <article> <h1>Whoa: The Rockford Register Star Double Modal</h1> <p>The Director — Mon, 07 Jan 2008 23:35:22 +0000</p> Congratulations to the <em>Rockford Register Star </em>Web site for doing something I had never seen before: it triggered an Internet Explorer error message atop the script problem dialog box in <a href="http://www.rrstar.com/homepage/x469069748" target="_blank">this story</a>: <p align="center"><a href="http://qahatesyou.com/images/doublemodal.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">An error so potent that <em>Internet Explorer couldn't even understand it to display it!</em></p> <p align="left"> Note that if you go to the paper's Web site following the above link, the error doesn't happen that way every time (but it seems to spew JavaSpit all over everything all the time). But if you see it, enjoy it, for you may never see that again.</p> </article> <article> <h1>Wherein The Director Channels Benjamin Franklin</h1> <p>The Director — Tue, 08 Jan 2008 15:29:51 +0000</p> <em>"Those who would give up Essential Quality to earn a little Temporary Profit, will see neither Quality nor Profit."</em> Feel free to use that in your next status meeting, when someone is explaining why the team shouldn't fix a large number of defects. </article> <article> <h1>Because Users Encounter Exceptions Inefficiently</h1> <p>The Director — Thu, 10 Jan 2008 20:02:48 +0000</p> Good to see some outside the box thinking with this Web cast: <a href="http://www.idgconnect.com/informationmanagement/businessprocess/conquering_the_billion_dollar_problem_automate_your_business_process_exceptions_to_save_time_money/index.html" target="_blank">Conquering the Billion Dollar Problem: Automate Your Business Process Exceptions to Save Time & Money</a>. Why wait for the users to find them when you can automate their occurrence? (Actually, that's not what it's about, but that's what I thought when I saw its title.) </article> <article> <h1>That’s A Most Practice</h1> <p>The Director — Thu, 10 Jan 2008 20:18:08 +0000</p> It's not a best practice, but this <a href="http://sharkbait.computerworld.com/?q=node/2048&source=NLT_PM&nlid=8" target="_blank">Shark Bait entry</a> finds a most practice: <blockquote> "These users must learn not to use these special characters." Thus spake my boss, a seasoned professional developer working with databases and web application development for many years. Yet again, one of our 'bonehead' users, who clearly had no attention for detail, had screwed up one of our applications and a database record. Someone had stupidly used an apostrophe. From the looks of the record, they were trying to type in an Irish name. Like I said, boneheads.</blockquote> The aforementioned boss also finds out the proper ways to respond to someone who suggests fixing it. Yeah, we in QA have heard those excuses before. </article> <article> <h1>Business-to-Business Browser Stunner</h1> <p>The Director — Mon, 14 Jan 2008 23:21:00 +0000</p> Apparently, hidden beneath the Firefox-pumping in <a href="http://www.itproductivity.org/PressRelease/press_release20080103Browser.htm" target="_blank">this press release</a>, we find something shocking about the browser market penetration in the business-to-business market: <em><strong>Netscape was third, with almost 9% usage, in January 2008.</strong></em> Keep that in mind when doing your compatibility testing for business application, I guess. You should also infer the continued importance of testing in Internet Explorer (6 and 7) even though your developers swear by Firefox and your designers create their masterpieces using Safari. </article> <article> <h1>But I Like My Solutions Better</h1> <p>The Director — Tue, 15 Jan 2008 16:54:40 +0000</p> This month (<a href="http://stpmag.com/issues/stp-2008-01.pdf">pdf</a>) in <em>Software Test & Performance</em>, editor Edward J. Correia again takes on automated software testing. The intro paragraph led me to believe he might have become a joyous skeptic, like <a href="http://qahatesyou.com/wordpress/2008/01/07/sometimes-automated-testing-is-folly/" target="_blank">us</a>: <blockquote>Why, for instance, do we build software to test other software? This question has never before occurred to me, nor does it parallel such mysteries as people who are financially wealthy but short on values. But it does bear some discussion.</blockquote> Does he then contemplate the possibility that trusting <em>software</em> to test <em>software</em> is something like telling criminals to police themselves? Nah, he just marvels at the beauty of it. As he should, since the automated software companies are the ones buying the ads in his magazine. However, we at QAHatesYou.com disagree with his conclusion: <blockquote>Software is very good at automating things. So when automated testing is the need, why not use the best tool for the job? For the practice of automating software testing, the best tool happens to be more software. Sometimes the best tool is staring you right in the face.</blockquote> <p align="left">Here at QAHatesYou.com, we have found in our experience that the following are sometimes better solutions, especially when tailored to limited budgets:</p> <ul> <li><strong>Zombies</strong>. All you need are a recurring maintenance budget, i.e., <em>brains</em>. You can certainly find some unused brains on your development team anyway. So raise some dead and show them which keys to push, and wallah! Automated software testing using the undead.</li> <li><strong>Steam piston driven software appliances. </strong>All you need is a machine shop, some wrenches, and boiling water to build complex steam-driven keyboard punchers. Mouse-handling and pointing-and-clicking are less accurate, so you'll have to work around that. Also, remember to calibrate the finger-rods correctly, or they will punch right through the keyboard instead of efficiently delivering the keyclick you want.</li> <li><strong>Monkeys.</strong> Just kidding. We use all our monkeys for new functionality testing.</li> </ul> Automated software testing is really only possible through the use of software, which comes with its own hazards which I'll go into some other time. </article> <article> <h1>Focus, Developer, Focus</h1> <p>The Director — Tue, 15 Jan 2008 17:00:11 +0000</p> Is it just me, or is the <a href="https://www.ironclad.com/www/truckGiveaway.jsp" target="_blank">Ironclad Ultimate Work Truck Giveaway sweepstakes form</a> throwing JavaScript errors on setting focus, changing settings, or most other Web browser events? <p align="center"><a href="http://qahatesyou.com/images/ironclad1.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">No, I guess it's not me. However, extra chutzpah points for including an e-mail address at the bottom of the form so that I can send the defects to the developer:</p> <p align="center"><a href="http://qahatesyou.com/images/ironclad2.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">Nothing like coming out and saying directly, <em>"Consumers, you are our quality assurance team."</em></p> </article> <article> <h1>QA Doesn’t Have That Many Friends</h1> <p>The Director — Wed, 16 Jan 2008 20:28:01 +0000</p> The <a href="http://fordvehicles.emipowered.net/bcsbowl2007/tellafriend/" target="_blank">tell-a-friend form</a> for the Built Ford Tough BCS Championship Sweepstakes incorrectly marks all of the Friend's E-mail fields as required: <p align="center"><a href="http://qahatesyou.com/images/5friendsrequired.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">It doesn't look as though the validation holds you to it, but still: someone was playing slops with the copy and paste when building this form, wasn't he (or she)?</p> <p align="left"> </p> </article> <article> <h1>Gallery of Stack Traces: QA From The Fourth Dimension</h1> <p>The Director — Wed, 16 Jan 2008 20:44:47 +0000</p> Developers are from Venus, and QA isn't from Mars; it's from a higher plane of existence than mere code monkeys as demonstrated in this stack trace: <p align="center"> <a href="http://qahatesyou.com/images/stunrepresentablest.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">You can see this bad oscar when the application needs to create a datetime built out of computations instead of simple drop-down lists with day/month/year/time in them and the application doesn't do it right.</p> <p align="left">Or if you try to explain your concept of deadlines to a developer engrossed in finding <a href="http://imdb.com/title/tt1060277/" target="_blank"><em>Cloverfield</em></a> spoilers instead of building a proper calendar widget.</p> </article> <article> <h1>Thanks, Have a JavaScript Error</h1> <p>The Director — Wed, 16 Jan 2008 21:10:14 +0000</p> After I completed the form badly on the <a href="http://promotions.newegg.com/MKTsweepstakes/RosesMemories/index.html" target="_blank">Newegg Roses & Memories Sweepstakes</a>, I got a JavaScript error before the load of the Thank You page.  <p align="center"><a href="http://qahatesyou.com/images/newegg.jpg"> <em>Click here for full size</em></a> <p align="left">Was it because I failed to enter information in a required field, and neither the client side validation (which checked a couple of the fields) and server-side validation (which checked a different couple of fields) caught that some information was missing? Or was this a problem with the product it was trying to load to display?</p> <p align="left">Who cares? It looks bad and should embarrass Newegg either way.</p> </article> <article> <h1>A Designer Tries To Distract QA</h1> <p>The Director — Fri, 18 Jan 2008 15:47:00 +0000</p> A Web designer sends along a flaw with the <em>St. Louis Post-Dispatch</em> Web site. Normally, I don't spend my time critiquing the paper's Web site because it could keep me busy full time. However, I'll show this problem to show that sometimes, after they've been cowed often enough by actual QA, sometimes designers pay attention to small details.  As part of its extensive coverage of the closure of major artery I64 (commonly known as Highway 40 because that's its US Highway numbers, 40-61), the <em>St. Louis Post-Dispatch </em>has gone all Web 2.0 and has grafted on a picture gallery called <a href="http://www.mycapture.com/" target="_blank">MyCapture</a> onto the site so people can upload their images and can vote on them and whatnot. However, as <a href="http://froggiegirl-stl.blogspot.com/" target="_blank">Froggie Girl</a> notes, if the actual slideshow increments incorrectly. If you go to the <a href="http://iwitness.stltoday.com/mycapture/photos/Album.aspx?EventID=400719&CategoryID=35564" target="_blank">gallery page</a>, you'll notice it has a number of images: <p align="center"><a href="http://qahatesyou.com/images/gallery.jpg" target="_blank"> <em>Click here for full size</em></a> <p align="left">Now, note that there are 9 images in the gallery. Click one of the images, and the number at the bottom indicates that there are 8 images in the gallery. Until, of course, you hit image number 9:</p> <p align="center"><a href="http://qahatesyou.com/images/increment.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">It looks like a common enough problem with these simple photo galleries, incrementing these values incorrectly. It would probably be an easy enough fix, too; perhaps understanding 0-based indexes might help.</p> <p align="left">But a little reminder, Ms. Designer, and all of you who would send hints and tips about problems with other people's and other applications' issues and defects: as soon as QA is done counting these poppy seeds, <em>we're still coming for you.</em></p> (Related post <a href="http://qahatesyou.com/wordpress/2007/07/14/what-we-have-here-is-a-failure-to-incrementate/" target="_blank">here</a> dealing with another gallery package with problems.) </article> <article> <h1>Someone Failed At Boundary Testing</h1> <p>The Director — Mon, 21 Jan 2008 18:18:03 +0000</p> In the World of Warcraft, apparently you can have <a href="http://www.wowinsider.com/2008/01/16/apparently-you-can-have-too-much-gold/" target="_blank">too much of a good thing</a>: <blockquote>Today, while skimming over various <em><span style="font-style: italic">WoW</span></em><em> </em>sites, I noticed two forum posts about the same topic: Players have discovered that there's a cap on how much money you can carry in the game. Apparently that amount is 214,748 gold, 36 silver, 48 copper. After you reach that lofty sum, you'll no longer be able to receive money from any source in the game. While some responses to the original posts claim that this exact limit had previously been theorized to exist, there have been no reports of anyone in the game actually achieving this amount via legal means.</blockquote> I feel for game testers. They really must account for all possible combinations and conditions, because some players have nothing better to do than to try to find all the possible flaws in the system. Like doing really well monetarily. (Link seen on <a href="http://games.slashdot.org/article.pl?sid=08/01/19/1321241" target="_blank">Slashdot</a>.) </article> <article> <h1>Tips for Using Automated Link Checking Software</h1> <p>The Director — Wed, 23 Jan 2008 01:35:08 +0000</p> As you expect, gentle reader, even when it comes to checking the links on Web sites, I prefer manual testing, particularly at the onset of a Web site development project. That is, I do want to personally, with my own index finger, click every single link on every single page, including that repeated navigational menu bar that would never, ever change across the pages (the developers and designers say) and don’t tend to change except when they catastrophically fail, for no discernable reason, on a single page. That’s not to say that automated link checking doesn’t have its place, because it does. The remainder of this piece talks about its place. However.  Your automated link checker, regardless of whether it came free from a foreign language site or as part of an elaborate complete testing package costing tens of thousands of dollars, will lack basic context and intelligence to determine if the link returns the right thing. If the Web server returns anything at all, your automated link checker will probably say, “Okay!” and will go on with its automated link checking. Hence, your first pass through the Web site should always make sure that the link text or image leads to an appropriate link target. That is, that the text product information should lead to something like, oh, product information and not the contact page. Your automated software, as I said, will be happy as long as the Web server does not return a 404 error to the Web browser. Obviously, that won’t do. Clicking through the Web site methodically will also make you familiar with all of the pages on the site and its layout. But that’s a positive thing unrelated to the theme of this piece. Once you’ve done that—or have passed over the opportunity to do the right thing to do the quick thing or the sexy technical thing—you can run your automated link checker. Automated link checkers are excellent for finding missing assets. They’ll identify when a page calls for a style sheet, image, or other file that isn’t present at the location where the page expects it. That’s very handy. Before you run your automatic link check, though, you need to know how your Web server handles links to pages that don’t exist. Some Web sites return a handy 404 error, which your standard link checker recognizes as a broken link. Other sites, on the other hand, handle missing pages differently. Some create a separate error page that displays but that doesn’t fire off the 404 flare; an automated link checker that looks specifically for that error won’t know the links are broken. Other sites just display the home page when you request a page that doesn’t exist. If your Web site handles missing Web pages like that, you have to account for it when running an automated link check. Some checkers let you report on the URLs of the returned pages; this will let you look for pages that link to PageNotFound.aspx or whatnot, and that will help you find broken links that your automated link checker doesn’t realize are broken. Next, you have to consider your environment and your Web site promotion process when running your automated link checker. If your organization follows good development process, someone is building it in a development environment and deploying it to a test environment for you to test, and then after you’ve logged all the issues and they ignore them, your tech people will deploy the sites to the production Web servers. Now, some URLs are relative and some are absolute, and you can absolutely count on that some “relative” URLs will point to your development or test environments. That is, instead of www.qahatesyou.com/helloworld.html, the link will target your test environment that’s accessible only through hostfilejitsu or to people inside your corporate firewall. If you run your automated link checker from inside your firewall or with the properly tuned host files, those links will show as good; however, when your user outside the firewall clicks them, the links will fail. Again, if you have good reporting/sorting on your URL targets in your automated link checker, you can check to make sure that your site doesn’t link to the internal environments—but your automated link checker probably won’t find those problems on its own. Those represent a couple of considerations you should make when selecting and using an automated link checker and illustrates a couple of places where the brute force of the software needs the intelligence of a quality assurance professional to help it out. Otherwise, you could just run some software facilely and have a false sense of security that it’s found all of your problems. On the other hand, maybe that’s how you do Quality Assurance in your organization. You wouldn’t be alone. </article> <article> <h1>The Opposite of All Is Some Are Not</h1> <p>The Director — Thu, 24 Jan 2008 20:24:21 +0000</p> Rubbermaid's <a href="http://www.rubbermaid.com/rubbermaid/sweeps/tenGrandSweeps.jhtml?link=rtfma?fmalink=Rubbermaid%20Cas" target="_blank">contest</a> insists that all information is required: <p align="center"><a href="http://qahatesyou.com/images/allissome.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">Which is odd, because the second line of the address usually is not. Come to try it, and it's not required here on this form, which means that Rubbermaid's dev team or designated dev team wasn't paying attention when putting the boilerplate text up.</p> <p align="left">Also, what's up with that State drop-down list? I've always been bothered a touch by the controls that are not sized appropriately to handle the text within them.</p> </article> <article> <h1>Show the Precision and Take It Away</h1> <p>The Director — Thu, 24 Jan 2008 21:07:17 +0000</p> CBS News's video player shows an awful lot of precision when you play with the Play and Pause buttons: <p align="center"><a href="http://qahatesyou.com/images/rounding.jpg" target="_blank"> <em>Click for full size</em></a> <p align="left">The clip length shows to 14 places to the right of the seconds, but it rounds immediately after displaying. If you work it just right, you can get it to display 0 of NaN.</p> <p align="left">Why the developer chose to display the real number before performing the rounding, I don't know. Wait, you're saying it was <em>unplanned</em>? As though the developer just churned out code <em>without thinking</em>? Say it ain't so!</p> <p align="left">But while we're on the subject, let me tell you some of the things I like to do to these Flash media players.</p> <p align="left"></p> <ul> <li>I like to click the Pause button while the the media is still buffering to see if I can actually break the synchronization; that is, whether I can have the Play button display while the clip finishes loading and starts playing. Because then, who knows what will happen when I click Play?</li> <li>Mash those buttons. What happens if I double-click Play? Double-click Pause? Click Play then Pause then All The Way To The End then Stop! You might not know it, but every player has a secret death-move combination that will rip out the spine of whomever is being portrayed and then shake it. I read that on the Internet somewhere; probably the preview mode of this post.</li> <li>Hey, how about that Mute/No Sound icon? Does it show happens when you click it or what it's doing now? Sometimes, the Birkenstock-loving designer likes to show the speaker with the circle and slash when the thing is already muted; this would be like displaying the Play button when the video is already playing, but enforcing that consistency would be logical, and try explaining that to Geoffrey. Also, do the volume controls actually work?</li> <li>What happens if I run that slider bar all the way left or right? What happens if I wiggle it?</li> <li>Does the player reset to a handy state when the video ends, or does the video just end and all the controls remain the same ( the Pause button remains active instead of reverting to the Play button, the last frame hangs in a state where it looks goofy, such as an important person's mouth hangs open instead of fading to black or returning to the first frame?</li> <li>What can I do if I right-click?</li> <li>How does it look in Flash 7?</li> <li>Can I alter or manipulate the text of the display inappropriately?</li> <li>Can I otherwise interact by clicking, double-clicking, clicking and dragging, or otherwise manipulating the objects on the screen?</li> </ul> These things are applications like anything else, and you better make sure they work. It also wouldn't be too much to ask that they not do goofy things. <p align="left"> </p> </article> </main></body></html>