The Skeptical Tester

[article]

On the first project, where we were developing a new bank teller system, the solution architect had left the GL off the system context diagram because it was “not needed.” But our teller system had a direct batch interface to the GL, and it didn’t make sense to me to leave it out. In our first integrated test with the GL, we found that all the credits and debits from our system were reversed—an easy bug to fix, but it hadn’t been detected until we tested with the GL. Through continuing to incorporate the GL in our test, we found several more obscure and complex bugs in our system.

I discovered an even more interesting situation on the second project, which involved ordering, provisioning, and billing for telecom services. Since there would be new financial outcomes, I naturally included the GL system in my end-to-end test scope. But when I reviewed my strategy with the project sponsor, he said it was out of scope—no work being done—and I should remove it. That didn’t feel right to me, and I said I’d like to do a little more investigation before definitely ruling it out. “Well, OK,” he said, “but don’t waste too much time on this. It’s not in scope.”

There was a solution review coming up, so I shelved the question till then. It was a three-day whiteboard session, with each team drawing its systems and data flows, and the integration architects and me asking questions. When the team responsible for the billing and financials had done its diagram, I said, “I must be missing something. I don’t see a line out to the GL, but I thought we were dividing the accounting between two lines of business.”

“That’s right,” their architect replied, “and we’ve created two new sets of GL accounts.”

“Great,” I responded, “but where are we actually dividing the revenues and doing the accounting?”

There was a silence, and then their project manager spoke up. “Um, I think we might have forgotten that. I’ll look into it and get back to this group.” The result was a $250,000 change request. The GL was definitely in scope.

Testers ask questions and then question the answers. It’s our job.

Think about this the next time you start a new project. Or stop for a moment and think about the questions you haven’t asked on your current project. What have you been told that might be wrong or not the whole story? Are the requirements documents you’ve been given the most reliable source of information about the system? Where else should you look? Who else should you talk to?

I took the examples for this article from big projects, but the lessons apply to projects of any size, whether you’re managing an enterprise end-to-end test or you’re the sole tester on your project team.

Never believe everything you’ve been told. Don’t accept that what you’ve been given is everything you need. Your success—and that of your project—depends on your healthy, professional skepticism.

User Comments

13 comments
Fiona Charles's picture

Hi Robert,

You're right to be cautious -- freelancers do occasionally get ripped off. However, I'm not a legal expert and I suggest that if you're really concerned, you start by getting a legal opinion of your contract. You should also find out what "legal hiring" means in your jurisdiction. The laws that govern regular employees may not cover free-lancers.

Here are a couple of general tips that it's a good idea for any freelancer to practice.

Before signing anything, research the reputation of the company proposing to hire you. Ask around if you have any doubts about them.

Read every page of the contract carefully. Don't be tempted to skip over stuff that looks like boilerplate. If you do decide to sign, initial every page even if you haven't been asked to do that.

I don't know any way to assure that you will actually get paid. But don't let your customer snow you. If the money dries up and they miss a scheduled payment, stop working immediately until you get paid. (They might miss the first payment through bureaucratic sloppiness. That's not usually a major concern. But missing a later payment is a danger signal. Don't get trapped into working for free!)

Go out and buy a copy of "Secrets of Consulting" by Gerald M. (Jerry) Weinberg. It has lots of wonderful advice for contractors as well as consultants.

Most of the time, it all goes well and we have good mutual relations with our customers.

Best of luck with your freelance career!

Fiona

August 31, 2012 - 6:30pm
Fiona Charles's picture

Hi Robert,

You're right to be cautious -- freelancers do occasionally get ripped off. However, I'm not a legal expert and I suggest that if you're really concerned, you start by getting a legal opinion of your contract. You should also find out what "legal hiring" means in your jurisdiction. The laws that govern regular employees may not cover free-lancers.

Here are a couple of general tips that it's a good idea for any freelancer to practice.

Before signing anything, research the reputation of the company proposing to hire you. Ask around if you have any doubts about them.

Read every page of the contract carefully. Don't be tempted to skip over stuff that looks like boilerplate. If you do decide to sign, initial every page even if you haven't been asked to do that.

I don't know any way to assure that you will actually get paid. But don't let your customer snow you. If the money dries up and they miss a scheduled payment, stop working immediately until you get paid. (They might miss the first payment through bureaucratic sloppiness. That's not usually a major concern. But missing a later payment is a danger signal. Don't get trapped into working for free!)

Go out and buy a copy of "Secrets of Consulting" by Gerald M. (Jerry) Weinberg. It has lots of wonderful advice for contractors as well as consultants.

Most of the time, it all goes well and we have good mutual relations with our customers.

Best of luck with your freelance career!

Fiona

August 31, 2012 - 6:30pm
Lovett CA's picture
Lovett CA

And people wonder why I ask so many questions. :-) Great article!

May 23, 2012 - 2:54pm

Pages

About the author

Fiona Charles's picture Fiona Charles

Fiona Charles is a Toronto-based test consultant and manager with thirty years of experience in software development and integration projects. Fiona is the editor of The Gift of Time, featuring essays by consultants and managers of various professions about what they've learned from Gerald M. Weinberg. Through her company, Quality Intelligence, Inc., Fiona works with clients in diverse industries to design and implement pragmatic test and test management practices that match their unique business challenges. Her experiential workshops facilitate tester learning by doing, either on the job or at conferences. Contact Fiona via her Web site at www.quality-intelligence.com.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Sep 22
Sep 24
Oct 12
Nov 09