I decided that my programming skills were getting rusty. So, I started
learning Java and Test-Driven Development. I've just finished posting
26 entries about it on my blog (http://blackbox.cs.fit.edu/blog/pat/)
If you look at it, you should probably start with the first entry on 23
June.

I decided to take a program that I had almost working (I swear, really,
it was this close!), written in C using mostly the waterfall model, and
to rewrite it in Java using Test-Driven Development. One part of the
inspiration for this was that I was really getting frustrated with the
program. Every time I wanted to work on it, I had to remember too much
stuff, and I never got started again. Another was that I decided that I
really wanted to learn Java well. Yet another was that, if I'm going to
claim to know something about testing, I should have actually done a
Test-Driven Development project.

Anyway, I've been working on this program for a couple of months now,
and I've learned a lot. I've tried to leave lots of bread crumbs along
my trail so you can follow along. If you decide to leap ahead of me,
please leave me some bread crumbs.

Comments welcome, either on the blog or by email.

Pat

I was at Agile Fusion last week, an event intended to explore the
ways testers could be integrated into XP, and though there was a lot
of good dialogue, there was no resolution. I am not disappointed by
this, as this issue is not something you simply "solve" in one week.
But it has left me thinking as I believe skilled testers can greatly
improve the agile development process.

During a tutorial I attended today about the Lean Development
Toolkit, I found inspiration. Mary Poppendieck talked about
concurrent engineering in the automotive industry and commented that
software development could benefit by adopting the technique. In
concurrent engineering, the product is developed with multiple,
parallel activities aimed at developing different components of the
finished product.

So, I have the following idea, though it is really half baked.
Consider the beginning of an XP project (or any XP iteration for
that matter). During the planning game, testers contribute by asking
questions that help expose areas of risk and necessary
consideration. Though the whole team should be doing this, skilled
testers are particularly adept at this. (Lisa Crispin raised this
idea during Agile Fusion.)

The rest of the planning game proceeds as normal. Once stories are
established and tasks laid out, development begins with normal pairs
of developers, with an occasional third member (a tester) who can
move among pairs, sort of like a bee collecting pollen and cross
pollinating. The tester, at this point is observing more than
participating, but she is gathering information about the internals
of the software, the habits and biases of the programmer, the
architecture decisions, etc.

The tester only does this gathering for s short period of time, at
which point the tester (or testers, though I have though about this
much) break away from the development activities to focus on their
activities. From what I understand of context driven testing, such
testers will formulate theories about how to test specific systems,
effectively developing strategies and plans for testing the system
(though not huge documents of test plans and specifications, but
rather ideas and strategies).

At the end of the iteration, the tester will discuss with the entire
team his theories about how to test the system, the important risks
and any concerns they have. This information is a vital part of the
iterations retrospective and this feedback should be used for the
next planning game. The developers hand off a working system to the
testers and begin their work in the next iteration. The testers have
a working system to test; both for their approaches of how to test
the system and the system itself. They revise their plans, expose
new risks and identify defects (or areas for improvement). The
testers should still join some pairs to constantly learn about the
system internals and the developers should integrate the
contribution of the testers in their automated tests. This workflow
continues for each iteration.

I would characterize this workflow as a weaving in and out of the
tester with the pair programming activities. It allows the testers
to collect information that helps them better formulate their test
strategies, but gives both testers and developers the freedom to
concentrate on the tasks that they are best suited to.

Brian Marick (and other in the context driven school), commented
that one of the core principles of context driven testing is that
testers would rather have working software to test than documents
about how they would test the software. He also noted that context
driven testing develops testing strategies appropriate to the system
they are testing. I think that the workflow I outlined satisfies
both these desires; testers get working software in each iteration
that guides the refining of their strategies. The developers also
get feedback, with each iteration, about what issues they need to
think about and hopefully use this information to improve their
automated tests and overall code.

There is another element that appeals to me in this idea. I think
developers prefer to build something first, then reflect on it, and
integrate their reflections into the software, proceeding in a very
incremental and iterative fashion; developers are anxious to dive
right in. Testers (from what I have observed), prefer to think about
the system, consider it from many perspectives and generally develop
a hypothesis before they actually start. I think both perspectives
are appropriate for their task. A developer's job is to produce
software, so active development directly supports the production of
working software. A tester's job is to inspect software, rather than
produce it. The tester wants to formulate a strategy first to avoid
random "hacking" testing that is undisciplined and immature. There
is conflict if you try to force the developer to spend too much
energy and time before they get to write code and not allow the
tester enough time and thought about how they would test a system.

So, I think the approach I am outlining allows both developer and
testers to work together without getting in each other's way.
The "weaving effect" is just enough collaboration to provide a
benefit without trying to force one to emulate the other's job.

Another benefit is that it provides an iterative and flexible
environment for testing. The testers are integral members of the
process, and with each iteration they have a more functioning system
that they can test and revise their approaches with. I would hope
this increases quality and the efficiency of the software and team.

At this point I am starting to babble and it is getting late. I have
lots more ideas in my head, such as pair debugging (something I have
done with testers), but I want to get feedback.

is there anyone using scenario driven development? are there benefits
related to this method and how difficult is to make the transition
from basic use-case scenarios

am attending a web seminar
http://ad.doubleclick.net/clk;5756968;8195081;d?
http://www.rational.com/events/forms/webinar-reg.jsp?EVENTID=2727

on this topic,, but does anyone have any insights ?

for whomever is interested, an intro to the rapid developer product
is available here -- btw are there any newsgroups focusing on UML and
relatd tools?
http://ad.doubleclick.net/clk;5756991;8195099;i?
http://www.rational.com/events/forms/webinar-reg.jsp?EVENTID=2774

As you know, the IEEE SWEBOK (software engineering body of knowledge)
is under review. They want specific changes based on experience using
the document in the classroom or the field. But they will accept
comments from any practitioner, and In the Box titled, "Description of
Proposed Change", I wrote:

		 I propose that you delete the section (the chapter on testing).
		 Deleting all of the surrounding sections would also be advisable.

In the box titled, "Justification of Proposed Change (Changes justified
by empirical trial usage data will be strongly prioritized)", I wrote a
much longer evaluation of SWEBOK. You can find it on my blog,
http://blackbox.cs.fit.edu/blog/kaner.

You can find more background on SWEBOK in my previous blog entry,
http://blackbox.cs.fit.edu/blog/kaner/archives/000056.html

Please get involved in this review process, which will close on June
30. Go to www.swebok.org to sign up and download swebok, and submit
comments.

Time is short, and you might not be able to read all of SWEBOK in time
to submit detailed comments. That's OK. I recommend that you download
it, skim the parts that are most interesting, realize the extent to
which it excludes modern methods (such as agile development) and, if
this bothers you, you can submit a very simple comment.

You can say something like:

       "I have reviewed SWEBOK. I manage software development staff
        and play a role in their training and supervision. SWEBOK does not
        provide a good basis for the structure or detail of the knowledge
        that I want my staff to have. It emphasizes attitudes and
practices
        that are not helpful on my projects and it downplays or skips
        attitudes and practices that I consider essential. I consider this
        document fundamentally flawed, and if I could vote to disapprove
it,
        I would."

Obviously, you would tailor this to your circumstances.


Cem Kaner, Professor, Department of Computer Sciences,
Florida Institute of Technology, 150 West University Blvd.
Melbourne, FL 32901.
Senior author of
	 Lessons Learned in Software Testing
	 Testing Computer Software, and
	 Bad Software: What to Do When Software Fails.


Cem Kaner, Professor, Department of Computer Sciences,
Florida Institute of Technology, 150 West University Blvd.
Melbourne, FL 32901.
Senior author of
	 Lessons Learned in Software Testing
	 Testing Computer Software, and
	 Bad Software: What to Do When Software Fails.


Cem Kaner, Professor, Department of Computer Sciences,
Florida Institute of Technology, 150 West University Blvd.
Melbourne, FL 32901.
Senior author of
	 Lessons Learned in Software Testing
	 Testing Computer Software, and
	 Bad Software: What to Do When Software Fails.

Just wanted to write up a quick thought mid-deluge here at work . . .

Are testers also customers, in that "developers" put stuff into the
product for the benefit of the testers?

-Dave

--- In agile-testing@yahoogroups.com, "Christian Sepulveda" <cs@a...>
wrote:

<snip>
> So, I have the following idea, though it is really half baked.
> Consider the beginning of an XP project (or any XP iteration for
> that matter). During the planning game, testers contribute by
asking
> questions that help expose areas of risk and necessary
> consideration. Though the whole team should be doing this, skilled
> testers are particularly adept at this. (Lisa Crispin raised this
> idea during Agile Fusion.)
>

I agree that part of a tester's role is to bring up these questions,
expose risks that may not have been thought of, and ask the 'what if
questions". They should be familiar with the customer's needs and
help to solidify those needs. Lisa talks about this in her book.

<snip>
> The rest of the planning game proceeds as normal. Once stories are
> established and tasks laid out, development begins with normal
pairs
> of developers, with an occasional third member (a tester) who can
> move among pairs, sort of like a bee collecting pollen and cross
> pollinating. The tester, at this point is observing more than
> participating, but she is gathering information about the internals
> of the software, the habits and biases of the programmer, the
> architecture decisions, etc.

I also think that ongoing communication is important, but as a
tester, I'm not sure if I need to know too much detail of the
internal workings. I trust that the developer will write unit tests
sufficient to test his/her work. I sometimes pair with the developers
and review unit tests they have written and make suggestions, but my
concentration is more on the business layer. Does the application do
what it is supposed to do?  I don't necessarily need to know the
detailed inner workings of the system.  I guess it may come down to
individual teams. How do they work? What is the role of the tester in
each team?

<snip>
> At the end of the iteration, the tester will discuss with the
entire
> team his theories about how to test the system, the important risks
> and any concerns they have. This information is a vital part of the
> iterations retrospective and this feedback should be used for the
> next planning game. The developers hand off a working system to the
> testers and begin their work in the next iteration. The testers
have
> a working system to test; both for their approaches of how to test
> the system and the system itself. They revise their plans, expose
> new risks and identify defects (or areas for improvement). The
> testers should still join some pairs to constantly learn about the
> system internals and the developers should integrate the
> contribution of the testers in their automated tests. This workflow
> continues for each iteration.
>

As part of our stories, I like to give the developer's high level
acceptance tests (I'm not always successful with this, but I find it
works well), to help them to define what they are building and to
understand the business perspective.  Are you suggesting that those
tests aren't done up front, but only after the development has been
completed?  I flush out my tests after I receive a working copy of
the app, but generally I have a pretty good idea of what is going on
during the iteration so there really isn't any surprises. At the end
of an iteration, if the working software, doesn't pass the acceptance
tests, then it isn't complete.

Some of your ideas challenged my way of thinking. I'm not sure I
agree with everything, but a different perspective is good. I hope my
comments give some food for thought as well.

Janet Gregory

Let me preface my reply by saying that I have never worked in an environment
that declared itself to be agile or XP.  Mine is agile in some ways in order to
survive, but there is no meta-level of understanding in which our people talk
or think in a concise way about how we work, and there is no management or
coaching toward any particular approach.  My understanding of agile methods is
thus based mostly on reading and some very small personal experiments.


--- Christian Sepulveda <cs@...> wrote:
> I was at Agile Fusion last week, an event intended to explore the
> ways testers could be integrated into XP, and though there was a lot
> of good dialogue, there was no resolution. I am not disappointed by
> this, as this issue is not something you simply "solve" in one week.
> But it has left me thinking as I believe skilled testers can greatly
> improve the agile development process.
>
> During a tutorial I attended today about the Lean Development
> Toolkit, I found inspiration. Mary Poppendieck talked about
> concurrent engineering in the automotive industry and commented that
> software development could benefit by adopting the technique. In
> concurrent engineering, the product is developed with multiple,
> parallel activities aimed at developing different components of the
> finished product.
>
> So, I have the following idea, though it is really half baked.
> Consider the beginning of an XP project (or any XP iteration for
> that matter). During the planning game, testers contribute by asking
> questions that help expose areas of risk and necessary
> consideration. Though the whole team should be doing this, skilled
> testers are particularly adept at this. (Lisa Crispin raised this
> idea during Agile Fusion.)
>
> The rest of the planning game proceeds as normal. Once stories are
> established and tasks laid out, development begins with normal pairs
> of developers, with an occasional third member (a tester) who can
> move among pairs, sort of like a bee collecting pollen and cross
> pollinating. The tester, at this point is observing more than
> participating, but she is gathering information about the internals
> of the software, the habits and biases of the programmer, the
> architecture decisions, etc.
>
> The tester only does this gathering for s short period of time, at
> which point the tester (or testers, though I have though about this
> much) break away from the development activities to focus on their
> activities. From what I understand of context driven testing, such
> testers will formulate theories about how to test specific systems,
> effectively developing strategies and plans for testing the system
> (though not huge documents of test plans and specifications, but
> rather ideas and strategies).
>
> At the end of the iteration, the tester will discuss with the entire
> team his theories about how to test the system, the important risks
> and any concerns they have. This information is a vital part of the
> iterations retrospective and this feedback should be used for the
> next planning game. The developers hand off a working system to the
> testers and begin their work in the next iteration.

I thought your idea sounded pretty good until I got to this last sentence,
which made me ask myself, "In an XP environment, why should testers have to
wait until the end of the iteration to either discuss their ideas about the
system or receive software to work with?"  Couldn't a smaller, more iterative
version of what you're describing occur every day during the stand up meeting?
As a tester and test manager, it pained me to envision your scenario, because
it felt like a throwback to traditional test-last methodology after having
built up what sounded like a quite XP-integrated approach.


Al

=====
Albert Davidson Chou

     Get answers to Mac questions at http://www.Mac-Mgrs.org/ .

__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com

I definitely think that testers could and should participate in the
stand up meetings, if so motivated. It would help for the development
team to hear about the testers progress, plans for the day and other
suggestions. It is something that I just didn't consider. (As I
mentioned in the post, my ideas are not complete nor am I trying to
develop a methodology. I have no desire to be a methodologist and I
am just trying to come up with some guidelines for things I might
try.)

As far as the waiting for the end of the iteration, I was imagining
the introduction of bugs and other things that change the
requirements / tasks for an iteration. For example, if the testers
need a tool developed (or want to integrate a tool they developed),
it should be prioritized like any other story. These are the types of
things I would prefer to wait to introduce at iteration boundaries.

Chris

  -----Original Message-----
From: Al Chou [mailto:hotfusionman@...]
Sent: Saturday, June 28, 2003 2:27 PM
To: agile-testing@yahoogroups.com
Subject: Re: [agile-testing] Integrating Testers into XP: Initial
Thoughts



I thought your idea sounded pretty good until I got to this last
sentence,
which made me ask myself, "In an XP environment, why should testers
have to
wait until the end of the iteration to either discuss their ideas
about the
system or receive software to work with?"  Couldn't a smaller, more
iterative
version of what you're describing occur every day during the stand up
meeting?
As a tester and test manager, it pained me to envision your scenario,
because
it felt like a throwback to traditional test-last methodology after
having
built up what sounded like a quite XP-integrated approach.


Al

=====
Albert Davidson Chou

     Get answers to Mac questions at http://www.Mac-Mgrs.org/ .

__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com

Yahoo! Groups Sponsor



To unsubscribe from this group, send an email to:
agile-testing-unsubscribe@yahoogroups.com



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.

-----Original Message-----
From: James McGovern [mailto:james@...]
Sent: Sunday, June 29, 2003 6:52 PM
To: james@...
Subject: [agile-testing] ANN: Agile Outsourcing

The idea of agile outsourcing came to me after finishing up my latest book: Java Web Services Architecture (http://www.amazon.com/exec/obidos/ASIN/1558609008/) and in having discussions with Scott Ambler. Having received large acceptance of this book by outsourcing firms overseas such as Wipro, Covansys, Tata, IBM, EDS, CSC and others, I realized there was a common problem and sought the solution to it.  

 

The industry is going down the path of diverging thoughts. Agile methods (www.agilemanifesto.org) is sometimes viewed as an extreme to the principles practiced by SEI/CMM believers at outsourcers. Being a person that is savage in pursuit of excellence, I thought about solving for the answer.

 

Outsourcing should not be viewed as contrary to agile principles. Instead, the two distinct thoughts should be blended. One of the primary contradictions is the savage pursuit of CMM and the need for comprehensive documentation.

 

Agile methods empower developers to trust their instincts and build software creatively rather than mechanically. Many teams found themselves so burdened by their plan of development that writing any code was almost an afterthought. For example, consider the requirement of keeping detailed on-going code documentation. Developing software is a dynamic process. Developers will often leave the detailed object diagram behind after the first month.

 

Writing software is massively complex in many enterprises. The ability to predict how components will interact as lines of code are added to a project are beyond the masses in the software industry (present company included). At this stage, it is more important to employ an empirical process that uses generous feedback and response. Only then will software in an outsourced model become agile.

 

The feedback stage is crucial. Agile enterprises should not be caught in the vicious deception that documentation at frequent stages in the form of status reports is wise. The only thing that matters is working software. The only proof of working software is working software. Pay close attention to how the software works not what the documentation says how it works.

 

When working with an outsourcer once should mandate frequent releases of software. An agile enterprise will never let themselves not see working software for any more than two weeks. The more frequent one sees working software, the better.

 

To further consider agile outsourcing one should consider relative location of the outsourcer. Countries that provide outsourcing capabilities that are closer to the United States include Barbados, Brazil, Uruguay, Philippines and Ireland. The cost structures in these countries are similar to what can be found in India, China and Russia but have additional advantages including:

 

• Ability to collaborate with offshore resources that are in the same time zone during the business day.
• The ability for offshore developers to travel onsite at a moment's notice without introducing the complexity of securing Visa's.
• The ability for employees to travel to offshore facilities and have zero concerns about climate, potable sources of water or a long regimented set of prescription drugs that must be taken prior to travel. Keeping your own employees healthy is good for business and helps reduce other costs associated with health insurance and other forms of liability.
• India has conflicts with Pakistan, China has conflicts with North Korea and Russia has been known to aid Iraq. Barbados for example, doesn't have any of these concerns and will allow for piece of mind.
• Cultural differences are also more easily bridged if both parties use the same vocabularies for communication. For example, the American culture is not filled with formal English but includes slang and words that have been adjusted over time from their original meaning. In Barbados, and other South American countries, they watch the same Television shows as Americans, so communication is easier.
• The thought of what hit fits the shan is best realized when the country you outsource to is easily reachable by direct flights that can be reached in a single business day from major airports such as JFK, Miami, Chicago's O'Hare, and LAX brings an additional piece of mind.

 

Another form of agile outsourcing is to consider building your own outsourcing solution. Agile companies like Phoenix (Headquartered in Hartford) early on built their own offshore facility in Bangalore India to maintain control of resourcing and make sure they receive all of the benefits a salaried employee in the States would. This allows for them to have an offshore model yet ensure that people working on the project have a vested interest in the success and are not simply interested in increasing the headcount.

 

The Phoenix approach also allows for further cost reduction in that they are keeping the profit margin otherwise given to an outsourcer for themselves. The side advantage of this approach is that they can leverage L1 Visa's in an ethical manner, something in which other outsourcers cannot admit to.

 

The goal of this forum will be to further expand the industry's thoughts of better ways to merge agile methods with outsourcing. I welcome you to join the Agile Outsourcing Yahoo Group at: http://groups.yahoo.com/group/agileoutsourcing.

 

James McGovern

Father of Agile Outsourcing

Co-author of best selling book: Java Web Services Architecture

http://www.amazon.com/exec/obidos/ASIN/1558609008/

 

 

To unsubscribe from this group, send an email to:
agile-testing-unsubscribe@yahoogroups.com



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.

On Friday, June 27, 2003, at 12:10:41 PM, Cem Kaner wrote:

> Time is short, and you might not be able to read all of SWEBOK in time
> to submit detailed comments. That's OK. I recommend that you download
> it, skim the parts that are most interesting, realize the extent to
> which it excludes modern methods (such as agile development) and, if
> this bothers you, you can submit a very simple comment.

I fully agree with Cem here, if not elsewhere. Take a look at this thing
and if you find it flawed, as we do, please comment.

Today is the last day.

Ron Jeffries
www.XProgramming.com
First they ignore you, then they laugh at you, then they fight you, then you
win.
   - Gandhi

On Friday, June 27, 2003, at 1:52:58 PM, Dave Liebreich wrote:

> Are testers also customers, in that "developers" put stuff into the
> product for the benefit of the testers?

Developers put things into the product to aid in development. (Unit tests
are an example from XP.)

Customers put things into the product to provide benefit -- usually direct
benefit -- to end users.

Unless the testers are the end users, putting things in to aid them sounds
more like a development-driven activity than an end-user-driven one.

I'm sure that it's a fuzzy line.

Suppose that the answer were one or the other of (Yes, No). What would you
say next?

Ron Jeffries
www.XProgramming.com
It's easier to act your way into a new way of thinking
than to think your way into a new way of acting.  --Millard Fuller

On Saturday, June 28, 2003, at 5:27:04 PM, Al Chou wrote:

>> At the end of the iteration, the tester will discuss with the entire
>> team his theories about how to test the system, the important risks
>> and any concerns they have. This information is a vital part of the
>> iterations retrospective and this feedback should be used for the
>> next planning game. The developers hand off a working system to the
>> testers and begin their work in the next iteration.

> I thought your idea sounded pretty good until I got to this last sentence,
> which made me ask myself, "In an XP environment, why should testers have to
> wait until the end of the iteration to either discuss their ideas about the
> system or receive software to work with?"  Couldn't a smaller, more iterative
> version of what you're describing occur every day during the stand up meeting?
> As a tester and test manager, it pained me to envision your scenario, because
> it felt like a throwback to traditional test-last methodology after having
> built up what sounded like a quite XP-integrated approach.

You have flagged an important issue. Whenever there is a "hand off", delays
and/or queues are inevitably inserted into the process. I mean inevitably:
it is a mathematical certainty that a hand off will create delays. Feedback
is slowed and the project becomes less agile, though perhaps only
incrementally and perhaps with no real alternative.

As a rule, I would look at every hand off and try to replace it with
something more intimate.

Ron Jeffries
www.XProgramming.com
Do, or do not. There is no try.  --Yoda

Hi,

I've taken part on a review of the SWEBOK some time ago so let me try to
comment a little bit.

As said by sometone else, it may not be perfect but at least it is an
attempt.  As in any incremental activity you can start with something and
move it step-by-step to what it has to become.

Perhaps the whole discussion starts with trying to find an agreement (or
even a disagreement) that software development is an engineering discipline
or not.  Is it art, is it craftsmanship, is it engineering?  We don't have
to redo that discussion again because it's been done many times over
(including special issues of IEEE Software).  At least there are a lot of
parallels with evolution of engineering in other fields.  In my opinion
there are certainly enough elements of engineering in software development.

For those who at least can sympathise with the idea that there are elements
of engineering in software development, it may be a good idea to get
involved in something like the SWEBOK initiative.  For those who cannot
sympathise with that idea, I wouldn't expect to get a lot of attention if
you file your comments.

I think Pierre Bourque and other people are making an honest attempt to find
a body of knowledge in software engineering and making that body of
knowledge available to the world at large.

Is SWEBOK flawed?  Perhaps there are flaws in it but is the idea flawed?
There are probably different camps of opinions but can we reap benefit from
the effort that is being made, I think the answer to that question remains
'Yes'.

All I really want to say is that the idea that software development is an
engineering discipline doesn't disturb me like it seems to disturb some
other people.  And I don't see why it would not be possible to reconcile
SWEBOK with approaches like Agile, Context Driven, Exploratory and other
appraches.  Some poeple seem to be blinded by the differences and cannot see
any commonalities.  To me things are never as black and white as they are
being discussed or presented, even on mailing lists like this.

Regards, Stefan Steurs.
> -----Original Message-----
> From: Ron Jeffries [mailto:ronjeffries@...]
> Sent: Monday, June 30, 2003 1:24 PM
> To: agile-testing@yahoogroups.com
> Subject: Re: [agile-testing] comments on SWEBOK
>
>
> On Friday, June 27, 2003, at 12:10:41 PM, Cem Kaner wrote:
>
> > Time is short, and you might not be able to read all of
> SWEBOK in time
> > to submit detailed comments. That's OK. I recommend that
> you download
> > it, skim the parts that are most interesting, realize the extent to
> > which it excludes modern methods (such as agile
> development) and, if
> > this bothers you, you can submit a very simple comment.
>
> I fully agree with Cem here, if not elsewhere. Take a look at
> this thing
> and if you find it flawed, as we do, please comment.
>
> Today is the last day.
>
> Ron Jeffries
> www.XProgramming.com
> First they ignore you, then they laugh at you, then they
> fight you, then you win.
>   - Gandhi
>
>
> ------------------------ Yahoo! Groups Sponsor
> ---------------------~-->
> Looking for the latest Free IT White Papers?
> Visit SearchMobileComputing.com to access over 500 white papers.
> Get instant access at SearchMobileComputing.com Today
> http://us.click.yahoo.com/ZyjvfD/PLNGAA/uitMAA/03wwlB/TM
> --------------------------------------------------------------
> -------~->
>
> To unsubscribe from this group, send an email to:
> agile-testing-unsubscribe@yahoogroups.com
>
>
>
> Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/



____

This message and any files transmitted with it are legally privileged and
intended for the sole use of the individual(s) or entity to whom they are
addressed. If you are not the intended recipient, please notify the sender by
reply and delete the message and any attachments from your system. Any
unauthorised use or disclosure of the content of this message is strictly
prohibited and may be unlawful.

Nothing in this e-mail message amounts to a contractual or legal commitment on
the part of EUROCONTROL unless it is confirmed by appropriately signed hard
copy.

Any views expressed in this message are those of the sender.

On Friday, June 27, 2003, at 02:26  PM, Janet Gregory wrote:
>
> As part of our stories, I like to give the developer's high level
> acceptance tests (I'm not always successful with this, but I find it
> works well), to help them to define what they are building and to
> understand the business perspective.  Are you suggesting that those
> tests aren't done up front, but only after the development has been
> completed?  I flush out my tests after I receive a working copy of
> the app, but generally I have a pretty good idea of what is going on
> during the iteration so there really isn't any surprises. At the end
> of an iteration, if the working software, doesn't pass the acceptance
> tests, then it isn't complete.
>
> Some of your ideas challenged my way of thinking. I'm not sure I
> agree with everything, but a different perspective is good. I hope my
> comments give some food for thought as well.

I think Christian's proposal is a way of being inclusive (though I
don't know if that was part of his specific intent). Some testers are
more comfortable being vigorous-finders-of-mistakes than
up-front-guiders-of-programming. And those testers often emphasize
coverage (in the broad sense - use of a well-thought-through testing
model of the system/domain/failure space/etc.). That contrasts with the
test-first style, which emphasizes incremental progress and (I think)
makes it harder to avoid gaps in coverage.

Other testers, like you (I think) and me (I know), prefer the up-front
style. And we other testers might make different bets about how often
and how far and in what way a tester needs to step back and think
comprehensively about the problem.

But suppose the team has testers of the first sort? Suppose it *needs*
testers of the first sort? How should they be integrated into the work?
Agile projects have some tricky differences from conventional projects
- an emphasis on steady forward "flow" that batches of bugs can
disrupt, a greater dependence on trust between members of different
interest groups (most notably between programmers and customers, but
also between programmers and testers), a programmer-centricity that a
skeptic would think of as coddling, and so forth. I see in Christian's
proposal ideas for integrating testers of the first sort while
maintaining what's different about agile projects.

-----
Brian Marick
Consulting, training, contracting, and research
Focused on the intersection of testing, programming, and design
marick@..., marick@...
www.testing.com, www.visibleworkings.com

I'm program chair or cohost of these events:
PLoP: <http://jerry.cs.uiuc.edu/~plop/plop2003/>
Analogy Fest: <http://www.visibleworkings.com/analogyfest/>
Please join me.

Ron Jeffries wrote:
>Unless the testers are the end users, putting things in to aid them sounds
>more like a development-driven activity than an end-user-driven one.
>
>I'm sure that it's a fuzzy line.
>
>Suppose that the answer were one or the other of (Yes, No). What would you
>say next?

I wonder if treating some testers as customers could be a viable way of
integrating some testers into XP projects.

If yes, then testability "features" and addressing the findings of the
testing effort could go on to the list of tasks for each iteration.

I'm hoping someone will riff off of this idea, or explain why it won't work.

_Dave

The StickyMinds.com roundtable discussion on SCM and Agility has about two more
weeks to go. There has been some interesting and insightful discussion so far.
Please feel free to join us and participate in the discussion at:
	 http://www.stickyminds.com/roundtable.asp

--
Brad Appleton <brad@...> www.bradapp.net
   Software CM Patterns (www.scmpatterns.com)
    Effective Teamwork, Practical Integration
"And miles to go before I sleep." -- Robert Frost

I respect the intent of the SWEBOK, but my biggest concern is that is
makes too many proclamations with the implication that they can be
generally applied. I feel that the sofware industry is too immature
with respect to having a universal standards guide. I think too much
context needs to be specified in all the practices outlined in the
SWEBOK that they are no longer general practices.

It concerns me that it has passed the "draft" stage and now is under
the "trial" review stage. It doesn't seem implausible that those
behind SWEBOK will want to move to an "adopted" or "enforced" stage,
where software professionals could be held accountable to the
practices of SWEBOK. Though I employ agile methods in my projects, I
don't think XP is appropriate everywhere, for example. I similarly
believe few, if any, of the practices outlined in the SWEBOK are
appropriate everywhere.

Chris




-----Original Message-----
From: STEURS Stefan [mailto:stefan.steurs@...]
Sent: Monday, June 30, 2003 5:52 AM
To: 'agile-testing@yahoogroups.com'
Subject: RE: [agile-testing] comments on SWEBOK


Hi,

I've taken part on a review of the SWEBOK some time ago so let me try
to
comment a little bit.

As said by sometone else, it may not be perfect but at least it is an
attempt.  As in any incremental activity you can start with something
and
move it step-by-step to what it has to become.

Perhaps the whole discussion starts with trying to find an agreement
(or
even a disagreement) that software development is an engineering
discipline
or not.  Is it art, is it craftsmanship, is it engineering?  We don't
have
to redo that discussion again because it's been done many times over
(including special issues of IEEE Software).  At least there are a
lot of
parallels with evolution of engineering in other fields.  In my
opinion
there are certainly enough elements of engineering in software
development.

For those who at least can sympathise with the idea that there are
elements
of engineering in software development, it may be a good idea to get
involved in something like the SWEBOK initiative.  For those who
cannot
sympathise with that idea, I wouldn't expect to get a lot of
attention if
you file your comments.

I think Pierre Bourque and other people are making an honest attempt
to find
a body of knowledge in software engineering and making that body of
knowledge available to the world at large.

Is SWEBOK flawed?  Perhaps there are flaws in it but is the idea
flawed?
There are probably different camps of opinions but can we reap
benefit from
the effort that is being made, I think the answer to that question
remains
'Yes'.

All I really want to say is that the idea that software development
is an
engineering discipline doesn't disturb me like it seems to disturb
some
other people.  And I don't see why it would not be possible to
reconcile
SWEBOK with approaches like Agile, Context Driven, Exploratory and
other
appraches.  Some poeple seem to be blinded by the differences and
cannot see
any commonalities.  To me things are never as black and white as they
are
being discussed or presented, even on mailing lists like this.

Regards, Stefan Steurs.
> -----Original Message-----
> From: Ron Jeffries [mailto:ronjeffries@...]
> Sent: Monday, June 30, 2003 1:24 PM
> To: agile-testing@yahoogroups.com
> Subject: Re: [agile-testing] comments on SWEBOK
>
>
> On Friday, June 27, 2003, at 12:10:41 PM, Cem Kaner wrote:
>
> > Time is short, and you might not be able to read all of
> SWEBOK in time
> > to submit detailed comments. That's OK. I recommend that
> you download
> > it, skim the parts that are most interesting, realize the extent
to
> > which it excludes modern methods (such as agile
> development) and, if
> > this bothers you, you can submit a very simple comment.
>
> I fully agree with Cem here, if not elsewhere. Take a look at
> this thing
> and if you find it flawed, as we do, please comment.
>
> Today is the last day.
>
> Ron Jeffries
> www.XProgramming.com
> First they ignore you, then they laugh at you, then they
> fight you, then you win.
>   - Gandhi
>
>
> ------------------------ Yahoo! Groups Sponsor
> ---------------------~-->
> Looking for the latest Free IT White Papers?
> Visit SearchMobileComputing.com to access over 500 white papers.
> Get instant access at SearchMobileComputing.com Today
> http://us.click.yahoo.com/ZyjvfD/PLNGAA/uitMAA/03wwlB/TM
> --------------------------------------------------------------
> -------~->
>
> To unsubscribe from this group, send an email to:
> agile-testing-unsubscribe@yahoogroups.com
>
>
>
> Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/



____

This message and any files transmitted with it are legally privileged
and intended for the sole use of the individual(s) or entity to whom
they are addressed. If you are not the intended recipient, please
notify the sender by reply and delete the message and any attachments
from your system. Any unauthorised use or disclosure of the content
of this message is strictly prohibited and may be unlawful.

Nothing in this e-mail message amounts to a contractual or legal
commitment on the part of EUROCONTROL unless it is confirmed by
appropriately signed hard copy.

Any views expressed in this message are those of the sender.


Yahoo! Groups Sponsor



To unsubscribe from this group, send an email to:
agile-testing-unsubscribe@yahoogroups.com



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.

On Monday, June 30, 2003, at 8:51:30 AM, STEURS Stefan wrote:

> All I really want to say is that the idea that software development is an
> engineering discipline doesn't disturb me like it seems to disturb some
> other people.  And I don't see why it would not be possible to reconcile
> SWEBOK with approaches like Agile, Context Driven, Exploratory and other
> appraches.  Some poeple seem to be blinded by the differences and cannot see
> any commonalities.  To me things are never as black and white as they are
> being discussed or presented, even on mailing lists like this.

One main issue, as I see it, is that

   a) SWEBOK is intended for use in licensing software professionals;
   b) licensed professionals are subject to suit for malpractice;
   c) malpractice suits typically work by finding anything in the
   profession's BOK that was not done, and claiming that therefore there was
   malpractice.

I am all for people being held responsible for professional behavior.
However, the SWEBOK is not reflective of all forms of good practice, and
therefore would be inappropriate as the basis for licensing and litigation.

Another important issue is that

   a) SWEBOK is based on existing "software engineering" texts;
   b) SWEBOK is used in university course accreditation.

This is a closed loop which makes it difficult, if not impossible, to get
programs containing newer practices accredited.

I'd love to have a "book" that put down everything useful that we know.
SWEBOK does not put down everything useful that we know, and is therefore
not worthy of standing in that place.

Ron Jeffries
www.XProgramming.com
Mixed metaphors are a bright sunny day with no paddle.  -- Phlip

I've been quiet on this because I have some urgent competing deadlines.

We made some interesting progress on framing this issue at the agile
development meeting, in a workshop and in separate discussions. I'll
post a summary of that to my blog within a day or two, and additional
work in a florida tech tech report.

For now, let me suggest some considerations:

-  I propose that it is premature to decide what is the best way to
locate testers in the project. I think that context variables that we
don't yet understand will determine whether the testers are best
integrated into the programming team or best left as a separate group.

- I think that there is an important test-first function, which
can/will be done by many people. Some people who see their work as
primarily testing will/should contribute to this. I think, though, that
most of these people should learn more about programming and help write
the actual tests rather than being a non-programming contributor.
(NOTE: there are contrary successful examples of non-programming
testers contributing a lot to test-first tests.)

- Most of the testing that XP folks talk about is functional testing
(unit tests, scenario-based (aka story-based) testing based on use
cases (aka stories)). This is important stuff. Much of it can be
defined early, but much of it will not emerge from the minds of the
testers, programmers, or customer until the system is being used or
extended.

- Much of testing is parafunctional. Some people call parafunctional
testing "nonfunctional" but that sounds too dismissive and causes too
many confusions. Parafunctional testing involves mitigation of risks
that are not associated with specific functions but are associated with
broader characteristics of the system. Examples are security, stress,
performance, localization, reliability estimation, usability,
accessibility, etc.

It is easy to imagine that the non-unit functional testing role could
live in the customer function and be managed by the customer
organization.  I can imagine some contexts in which this could be
successful.

However, it is harder to imagine success for parafunctional testing
(apart, perhaps, from usability testing) as a customer function task.

Rather than make the arguments here, let me pose the core underlying
puzzle.

Given a task like stress testing:

- who will value the technical skills involved in this work?
- who can mentor / train the testers who will do the work?
- who will benefit from pairing with people who do this work?
- who will benefit from the results of this work?
- who is best suited to supervise this work?

If there is a career path or a personal growth path for a person who is
good at stress testing, what nature of group would best enhance that
career path?

My intuition is that "customer" focus is less technical and more
benefit, less analytical/critical and more results oriented. My
intuition is that "programmer" focus is more technical. My belief is
that parafunctional testing (and much functional testing) is more
technical and needs management that appreciates / defends / funds /
trains technical growth. My intuition is that if there are only two
worlds, customer and programmer, most of the testers that I would want
on XP projects should live in the programmer world rather than the
customer world.

I am hesitant and confused vis-a-vis the existence of a separate test
group.

I am pretty sure that if programmers are trained that the only good
tester is a cheerleader (that is, the opposite of someone who takes
pride and pleasure in discovering bugs), then the steady state for that
company will be a test group that lives outside of the programming team

I don't believe, but could be mistaken, that it is necessary to
externalize that type of tester from the programming team.

In general, I think there are many potential objectives for the testing
function (I refer to a testing function, rather than a testing group,
because I want to stay vague about who does the testing. In some
companies, there is no test group, there are no designated testers, but
there are programmers who sometimes take on the tasks that we would
normally assign to testers). I think that developing a cost-efficient
vehicle for change-detection (for the usual reasons we do regression
testing plus supporting refactoring, which I think is the essential
one) is one important objective. I think finding bugs is another
important objective. I think assessment in support of helping people
make release/no-release decisions is another important objective.

For each objective, we can ask a bunch of questions, like:

     - is it important /valuable?
     - who most benefits from it?
     - who can understand / appreciate the actual work done to achieve it?
     - who is in a best position to train / mentor / defend / fund it?
     - where is the career growth in it?

Answers to questions like these will help us decide where the people
doing this type of work should live, who should pay and supervise them,
how they should be nurtured, how their skills will be extended over
time, etc.

I do think that there are some important risks of creating an
external-to-the-programmers test group:

     - I think it is a mistake to focus testers on end-of-iteration
deliverables unless iterations are very short. If we're talking about
the daily builds, no problem. If we're talking about deliveries t the
customer, I think this is an error.

     - I think that a less technically focused group is more likely to
generate paper, to use paper (physical or electronic) to organize the
testing effort, to track bugs, etc. In general, I think that there is
an inverse relationship between the amount of paper used by a test
group and the skill and contribution of that group to the project.

     - I think that "paper" creates project inertia, as do several types
of less-sophisticated automated test approaches. In general, I think
that inertia harms the project. Some inertia is inevitable in any
project, but when we add any practice, we should appraise its inertial
effect. The more it would cause anyone on the project to resist future
change, the greater the benefit that practice should deliver to the
project (or the practice should be rejected). I suspect that the
further removed from the programming team, the more willing the test
group will be to adopt practices that add inertia, without going
through the cost/benefit analysis that I think is fundamental.

I've been arguing against high-inertia approaches to testing for my
entire technical career. One of the key reasons that XP is exciting to
me is that, at its core, it values low-cost change and therefore
devalues practices that add to inertia.

As we think through the testing work that should be done on XP
projects, I think we should think carefully about ways to introduce
this work with the least additional inertia or in ways that (like
test-first development) actually reduce inertia.

-- Cem Kaner



On Monday, June 30, 2003, at 10:24 AM, Dave Liebreich wrote:

> Ron Jeffries wrote:
>> Unless the testers are the end users, putting things in to aid them
>> sounds
>> more like a development-driven activity than an end-user-driven one.
>>
>> I'm sure that it's a fuzzy line.
>>
>> Suppose that the answer were one or the other of (Yes, No). What
>> would you
>> say next?
>
> I wonder if treating some testers as customers could be a viable way of
> integrating some testers into XP projects.
>
> If yes, then testability "features" and addressing the findings of the
> testing effort could go on to the list of tasks for each iteration.
>
> I'm hoping someone will riff off of this idea, or explain why it won't
> work.
>
> _Dave
>
>
>
> ------------------------ Yahoo! Groups Sponsor
> ---------------------~-->
> Looking for the latest Free IT White Papers?
> Visit SearchNetworking.com to access over 500 white papers.
> Get instant access at SearchNetworking.com Today
> http://us.click.yahoo.com/YyjvfD/OLNGAA/uitMAA/03wwlB/TM
> ---------------------------------------------------------------------
> ~->
>
> To unsubscribe from this group, send an email to:
> agile-testing-unsubscribe@yahoogroups.com
>
>
>
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
>
>
>
Cem Kaner, Professor, Department of Computer Sciences,
Florida Institute of Technology, 150 West University Blvd.
Melbourne, FL 32901.
Senior author of
	 Lessons Learned in Software Testing
	 Testing Computer Software, and
	 Bad Software: What to Do When Software Fails.

On Monday, June 30, 2003, at 2:44:46 PM, Cem Kaner wrote:

> I've been quiet on this because I have some urgent competing deadlines.

> We made some interesting progress on framing this issue at the agile
> development meeting, in a workshop and in separate discussions. I'll
> post a summary of that to my blog within a day or two, and additional
> work in a florida tech tech report.

Indeed we did make some interesting progress. I had many long and enjoyable
conversations with Cem. I'm not sure whether this increases or decreases my
uniqueness ...

I'll chip in here with harmony or counterpoint:

> For now, let me suggest some considerations:

> -       I propose that it is premature to decide what is the best way to
> locate testers in the project. I think that context variables that we
> don't yet understand will determine whether the testers are best
> integrated into the programming team or best left as a separate group.

Surely context will matter. I am having difficulty, however, imagining a
project where external testers are appropriate that would not be aided by
having internal testers as well, or instead.

In fact, if they were free, I would think that internal testers would be of
value to any project. Since they are not, there is some kind of tradeoff
that would operate along the spectrum from non-programming tester through
programming tester to test-savvy programmer. (I do not place non-testing
programmer on the same line, you'll note. ;->)

> -       I think that there is an important test-first function, which
> can/will be done by many people. Some people who see their work as
> primarily testing will/should contribute to this. I think, though, that
> most of these people should learn more about programming and help write
> the actual tests rather than being a non-programming contributor.
> (NOTE: there are contrary successful examples of non-programming
> testers contributing a lot to test-first tests.)

I agree with this, subject only to the observation that I've not had the
pleasure of pairing with an experienced tester, whether code-savvy or not.
I'd like to try that, and expect that it would only increase my agreement,
but I can imagine that it might not.

> -       Most of the testing that XP folks talk about is functional testing
> (unit tests, scenario-based (aka story-based) testing based on use
> cases (aka stories)). This is important stuff. Much of it can be
> defined early, but much of it will not emerge from the minds of the
> testers, programmers, or customer until the system is being used or
> extended.

Yes. And I would add that customers are often not good at inventing the
tests that XP asks them to create, and that they often do not value such
tests at the beginning. This observation stands at the core of my
suggestion that a good place for testers on an XP project to focus their
attention is in support of the customer.

    (As we will come to, this should not be taken to mean that they should
    be part of the customer's home business organization. That's a separate
    topic which Cem and I discussed at length, and I generally share his
    concerns and inclinations, as we'll see below.)

> -       Much of testing is parafunctional. Some people call parafunctional
> testing "nonfunctional" but that sounds too dismissive and causes too
> many confusions. Parafunctional testing involves mitigation of risks
> that are not associated with specific functions but are associated with
> broader characteristics of the system. Examples are security, stress,
> performance, localization, reliability estimation, usability,
> accessibility, etc.

> It is easy to imagine that the non-unit functional testing role could
> live in the customer function and be managed by the customer
> organization.  I can imagine some contexts in which this could be
> successful.

Yes. In fact, I have observed many XP contexts where it /is/ successful,
and expect that, in general, it would be.

> However, it is harder to imagine success for parafunctional testing
> (apart, perhaps, from usability testing) as a customer function task.

I would agree that the skills necessary for this will not likely be found
in the customer organization. However, from the XP perspective, the
responsibility to specify and ensure the delivery of a product meeting the
appropriate characteristics on these dimensions is a customer
responsibility.

It is this interpretation of where responsibility lies that underlies my
suggestion that testers (even of the parafunctional kind) are of special
value in support of the customer. And again, I would agree that the skills
necessary are not likely found in the customer's home organization.

> Rather than make the arguments here, let me pose the core underlying
> puzzle.

> Given a task like stress testing:

> -       who will value the technical skills involved in this work?
> -       who can mentor / train the testers who will do the work?
> -       who will benefit from pairing with people who do this work?
> -       who will benefit from the results of this work?
> -       who is best suited to supervise this work?

> If there is a career path or a personal growth path for a person who is
> good at stress testing, what nature of group would best enhance that
> career path?

> My intuition is that "customer" focus is less technical and more
> benefit, less analytical/critical and more results oriented. My
> intuition is that "programmer" focus is more technical. My belief is
> that parafunctional testing (and much functional testing) is more
> technical and needs management that appreciates / defends / funds /
> trains technical growth. My intuition is that if there are only two
> worlds, customer and programmer, most of the testers that I would want
> on XP projects should live in the programmer world rather than the
> customer world.

I take Cem's points above and largely agree. I would point out, however,
that from the XP perspective (not the only perspective, but /a/
perspective), the customer has responsibility for specifying a suitable
product and for ensuring its delivery. To the extent that a project follows
this pattern, the customer organization may well have a responsibility to
support and nurture some of the necessary testing values.

We see organizations that "delegate" these responsibilities to their
technical organization, sometimes with success and sometimes not. I would
say that even when this is done, the setting of those requirements, and
their verification, is still best characterized, from the XP perspective,
as a "customer" perspective.

> I am hesitant and confused vis-a-vis the existence of a separate test
> group.

I could accept that a separate team might be necessary for some purposes,
and convenient for others. Complex configuration testing or the like,
perhaps. However, in every case, separate group tends to imply delays in
the feedback. Therefore, the development team needs to find ways to
interpret defect reports coming back to them from the separate team, and to
upgrade their internal testing to reduce the number, frequency, and impact
of the defects found externally.

In short: the development team (including its testers, if any) should take
it upon themselves to ship such good software to the external team that the
business value of that team is called into question because they never find
any problems. ;->

> I am pretty sure that if programmers are trained that the only good
> tester is a cheerleader (that is, the opposite of someone who takes
> pride and pleasure in discovering bugs), then the steady state for that
> company will be a test group that lives outside of the programming team

> I don't believe, but could be mistaken, that it is necessary to
> externalize that type of tester from the programming team.

Cem, I'm not following the two paragraphs just above.

> In general, I think there are many potential objectives for the testing
> function (I refer to a testing function, rather than a testing group,
> because I want to stay vague about who does the testing. In some
> companies, there is no test group, there are no designated testers, but
> there are programmers who sometimes take on the tasks that we would
> normally assign to testers). I think that developing a cost-efficient
> vehicle for change-detection (for the usual reasons we do regression
> testing plus supporting refactoring, which I think is the essential
> one) is one important objective.

Yes ...

> I think finding bugs is another
> important objective.

Yes ... though /not/ finding bugs is the way I like to look at this
objective. Finding bugs is bad news. Having them and not finding them is,
of course, usually WORSE news ...

> I think assessment in support of helping people
> make release/no-release decisions is another important objective.

Yes. In XP terms, a "customer" responsibility.

> For each objective, we can ask a bunch of questions, like:

>     -   is it important /valuable?
>     -   who most benefits from it?
>     -   who can understand / appreciate the actual work done to achieve it?
>     -   who is in a best position to train / mentor / defend / fund it?
>     -   where is the career growth in it?

> Answers to questions like these will help us decide where the people
> doing this type of work should live, who should pay and supervise them,
> how they should be nurtured, how their skills will be extended over
> time, etc.

Good starting set of questions. They clarify Cem's concerns with my
simplistic but delightfully charming positions.

> I do think that there are some important risks of creating an
> external-to-the-programmers test group:

>     -   I think it is a mistake to focus testers on end-of-iteration
> deliverables unless iterations are very short. If we're talking about
> the daily builds, no problem. If we're talking about deliveries t the
> customer, I think this is an error.

Yes. On the C3 project, we had some customer tests that ran only overnight.
When these found defects, as they sometimes did, the team found it quite
disruptive as it meant we had to change our intentions for the next day.
Even that much delay is a problem. Weekly or bi-weekly would only be worse.

>     -   I think that a less technically focused group is more likely to
> generate paper, to use paper (physical or electronic) to organize the
> testing effort, to track bugs, etc. In general, I think that there is
> an inverse relationship between the amount of paper used by a test
> group and the skill and contribution of that group to the project.

>     -   I think that "paper" creates project inertia, as do several types
> of less-sophisticated automated test approaches. In general, I think
> that inertia harms the project. Some inertia is inevitable in any
> project, but when we add any practice, we should appraise its inertial
> effect. The more it would cause anyone on the project to resist future
> change, the greater the benefit that practice should deliver to the
> project (or the practice should be rejected). I suspect that the
> further removed from the programming team, the more willing the test
> group will be to adopt practices that add inertia, without going
> through the cost/benefit analysis that I think is fundamental.

Indeed. Compare with the XP Simplicity value.

> I've been arguing against high-inertia approaches to testing for my
> entire technical career. One of the key reasons that XP is exciting to
> me is that, at its core, it values low-cost change and therefore
> devalues practices that add to inertia.

> As we think through the testing work that should be done on XP
> projects, I think we should think carefully about ways to introduce
> this work with the least additional inertia or in ways that (like
> test-first development) actually reduce inertia.

Quite so. I look forward to further fruitful discussions. Given the brevity
for which we are both justly famous, they should serve as a good test of
the Internet, and, I hope, will provide some valuable ideas as well.

Ron Jeffries
www.XProgramming.com
A man hears what he wants to hear, and disregards the rest. -- Paul Simon

> I fully agree with Cem here, if not elsewhere. Take a look at this thing
> and if you find it flawed, as we do, please comment.
>
> Ron Jeffries

I accidentally found myself listening to Ken Burn's American Stories just
now; it was about Susan B. Anthony campaigning. For some reason it reminded
me of our agile "movement".

This really >is< a freedom thing. Hour by hour, day by day, convert by
convert, people all over the world are reducing the odds that my boss will
order me to do something totally stupid.

--
   Phlip
     http://www.c2.com/cgi/wiki?TestFirstUserInterfaces

-----Original Message-----
From: James McGovern [mailto:james@...]
Sent: Sunday, June 29, 2003 6:52 PM
To: james@...
Subject: [agile-testing] ANN: Agile Outsourcing

The idea of agile outsourcing came to me after finishing up my latest book: Java Web Services Architecture (http://www.amazon.com/exec/obidos/ASIN/1558609008/) and in having discussions with Scott Ambler. Having received large acceptance of this book by outsourcing firms overseas such as Wipro, Covansys, Tata, IBM, EDS, CSC and others, I realized there was a common problem and sought the solution to it.  

 

The industry is going down the path of diverging thoughts. Agile methods (www.agilemanifesto.org) is sometimes viewed as an extreme to the principles practiced by SEI/CMM believers at outsourcers. Being a person that is savage in pursuit of excellence, I thought about solving for the answer.

 

Outsourcing should not be viewed as contrary to agile principles. Instead, the two distinct thoughts should be blended. One of the primary contradictions is the savage pursuit of CMM and the need for comprehensive documentation.

 

Agile methods empower developers to trust their instincts and build software creatively rather than mechanically. Many teams found themselves so burdened by their plan of development that writing any code was almost an afterthought. For example, consider the requirement of keeping detailed on-going code documentation. Developing software is a dynamic process. Developers will often leave the detailed object diagram behind after the first month.

 

Writing software is massively complex in many enterprises. The ability to predict how components will interact as lines of code are added to a project are beyond the masses in the software industry (present company included). At this stage, it is more important to employ an empirical process that uses generous feedback and response. Only then will software in an outsourced model become agile.

 

The feedback stage is crucial. Agile enterprises should not be caught in the vicious deception that documentation at frequent stages in the form of status reports is wise. The only thing that matters is working software. The only proof of working software is working software. Pay close attention to how the software works not what the documentation says how it works.

 

When working with an outsourcer once should mandate frequent releases of software. An agile enterprise will never let themselves not see working software for any more than two weeks. The more frequent one sees working software, the better.

 

To further consider agile outsourcing one should consider relative location of the outsourcer. Countries that provide outsourcing capabilities that are closer to the United States include Barbados, Brazil, Uruguay, Philippines and Ireland. The cost structures in these countries are similar to what can be found in India, China and Russia but have additional advantages including:

 

• Ability to collaborate with offshore resources that are in the same time zone during the business day.
• The ability for offshore developers to travel onsite at a moment's notice without introducing the complexity of securing Visa's.
• The ability for employees to travel to offshore facilities and have zero concerns about climate, potable sources of water or a long regimented set of prescription drugs that must be taken prior to travel. Keeping your own employees healthy is good for business and helps reduce other costs associated with health insurance and other forms of liability.
• India has conflicts with Pakistan, China has conflicts with North Korea and Russia has been known to aid Iraq. Barbados for example, doesn't have any of these concerns and will allow for piece of mind.
• Cultural differences are also more easily bridged if both parties use the same vocabularies for communication. For example, the American culture is not filled with formal English but includes slang and words that have been adjusted over time from their original meaning. In Barbados, and other South American countries, they watch the same Television shows as Americans, so communication is easier.
• The thought of what hit fits the shan is best realized when the country you outsource to is easily reachable by direct flights that can be reached in a single business day from major airports such as JFK, Miami, Chicago's O'Hare, and LAX brings an additional piece of mind.

 

Another form of agile outsourcing is to consider building your own outsourcing solution. Agile companies like Phoenix (Headquartered in Hartford) early on built their own offshore facility in Bangalore India to maintain control of resourcing and make sure they receive all of the benefits a salaried employee in the States would. This allows for them to have an offshore model yet ensure that people working on the project have a vested interest in the success and are not simply interested in increasing the headcount.

 

The Phoenix approach also allows for further cost reduction in that they are keeping the profit margin otherwise given to an outsourcer for themselves. The side advantage of this approach is that they can leverage L1 Visa's in an ethical manner, something in which other outsourcers cannot admit to.

 

The goal of this forum will be to further expand the industry's thoughts of better ways to merge agile methods with outsourcing. I welcome you to join the Agile Outsourcing Yahoo Group at: http://groups.yahoo.com/group/agileoutsourcing.

 

James McGovern

Father of Agile Outsourcing

Co-author of best selling book: Java Web Services Architecture

http://www.amazon.com/exec/obidos/ASIN/1558609008/

 

 

To unsubscribe from this group, send an email to:
agile-testing-unsubscribe@yahoogroups.com



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.

> Please elaborate on how the outsourcer's testing team can
> get into working with the agile model?

> How will the testing team know what to test in the absence/scarce
> documentation?

Use both a Wiki and an internal mailing list to communicate.

The testers attend to both.

Whatever's in the air, they write tests for it.

Using a "Test Server" like http://fitnesse.org, everyone can collaborate on
test resources.

Whoever requests features uses FitNesse to document the sample test data and
results.

When FitNesse fails to test, it needs a new fixture. When its tests fail, we
need to fix the code.

Put the address of the failing page into a Bugzilla style bugbase. New items
go to QA; they fix the fixtures, or pass the items to developers.

(We are not talking outsourcing so much as geographic incompatibility here.)

--
   Phlip
    http://www.greencheese.org/EvolutionaryPsychology
   --  A Harley is a musical instrument with tires  --

----- Original Message -----

From: Jayaraman Vasudevan , Tidel Park - Chennai

 

Please elaborate on how the outsourcer's testing team can get into working with the agile model?

 

-----Original Message-----
From: Phlip [mailto:plumlee@...]
Sent: Tuesday, July 01, 2003 11:57 PM
To: agile-testing@yahoogroups.com
Subject: Re: [agile-testing] ANN: Agile Outsourcing

We're forgetting our manners, Jayaraman.

 

Don't post HTML on a technical newsgroup,

 

and post questions about agile outsourcing to the agile outsourcing mailing list.

 

I forwarded my reply there.

----- Original Message -----

From: Jayaraman Vasudevan , Tidel Park - Chennai

 

Please elaborate on how the outsourcer's testing team can get into working with the agile model?

 

  Phlip
          http://www.greencheese.org/ParodyMode
  --  How does Bugs Bunny do it? How does he know when he
      wakes up in the morning to put in his pocket 3 sticks
      of dynamite, a physician costume, and a bicycle pump?  --

To unsubscribe from this group, send an email to:
agile-testing-unsubscribe@yahoogroups.com



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.

> -----Original Message-----
> From: Ron Jeffries [mailto:ronjeffries@...]

There was a very long exchange of ideas on testers, whether they should be
integrated with customers or developers, and I started replying to it but
the reply just become too long and unreadable.

I find the question whether a tester should be integrated with the developer
team or the customer a puzzling question.  Like the long exchange pointed
out (or at least what I understood of it) there are different merits to both
sides.

Although it is questioned that testers could contribute to a test savvy
group of developers, there seems to be a consensus that the customer is in
need of some assistance for non-functional/parafunctional/technical testing.

I see it differently and from experience I think the tester can play a big
role as a kind of bridge between the people who look at the problem they
have to solve (developers) and the people who need a solution (customer).
Because the viewpoint is different, the emphasis placed by both groups will
be different.  I think this is why we have developer testing and customer
testing as 2 separate activities within XP.

I think there is a problem, and I've witnessed it more than once, for the
customer to grasp the intricacies of software development, and also for the
developers to grasp the business/operational needs.  The role I've seen many
testers play successfully is to attempt to unite the views of both camps.
This requires that the tester has knowledge about software development but
also about the business of the customer.

Perhaps this is the reason why Ron says he doesn't believe in testers that
don't have software development experience and I tend to agree upto a level.
A good tester knows about analysis, design, coding, the business the
software has to serve, and specialises in testing.

Why is this needed?  Software Testing is an infinite task because every
software has defects and defects take progressively more time and effort to
be exposed as testing proceeds.  In order to make testing more efficient and
effective, a tester has to look at the software development, the
business/operational needs, the possible testing approaches and using these
parameters indicate where is the highest risk area that needs most of the
attention.  In view of the time available, the effort has to be distributed
in function of the risks identified.

If risks are high because there is a abnormally high complexity to carry out
the tests, the software has to be adapted to increase the testability.  For
me there is no argument about who is responsible, it is both the customer
and the developer together.  The developer because he needs to deliver code
that is of good quality, the customer because he doesn't want to pay for the
cost of failure.  If the customer and developer agree that the tester has
correctly defined the risks of failure, the customer should pay for it and
the developers should implement the necessary testability features.

I see a testers role in trying to figuring out where the risks are and then
deciding how to measure the real risk by running tests to either show that
the risk exists or gain confidence that the necessary has been done to limit
the risk of failure.

Does the role of a tester change depending on the development approach?  Not
a lot.  Testers have to be there from start to finish.  If it is highly
iterative the risk assessment will have to be continuous.  If it is not so
iterative or more waterfall like, the risk assessment happens in bigger
chuncks.  The amount of work for highly iterative is higher I think but it
is more challenging and rewarding at the same time.

So, if I look at software testing as a way to limit the risks of failure,
test driven development becomes risk driven development instead.  You write
more tests for those areas you think are more risky.  In all testing I've
seen this seems to be a common denominator.  There may even be areas with no
risk and I think a tester should be able to decide not to test those areas.

Knowing all this we can return to the question, where do we locate the
testers?  I think this depends on context.  Often the tester will have a
place with development because his test preparation/execution activities
have to interact very intimately with the software development that is going
on.  Sometimes the tester will have a position with the customer because the
customer does not have the technical qualifications to tackle the
implementation of a test approach (infrastructure, tools, etc.).  A tester
can also be independent of both, working on his/her own, making proposals
that either the customer or the developer or both have to implement.

I am tempted to discuss also the comments about inertia.  I will not add too
much but I think it is more fruitful to think in terms of 'ceremony'.  I
don't remember the author but I once read a very good article about needs
for ceremony.  Documentation can be a part of the ceremony.  Humans seem to
be fond of ceremony, milestone events/celebrations, etc.  Waterfal
approaches are very ceremonial with clear milestones to celebrate events.
This gives high visibility.  The drawback is certainly that some people may
get more focused on the ceremony and celebrations than on the actual quality
of what is delivered.
I think however that we must not make the mistake to have milestone-less or
celebration-less software development that is highly iterative but seems
also to be never ending (or alway ending in failure if failure is the moment
it is decided that the software has to be decomissioned).  This has little
to do with the technical merits of either approaches but I think the fact
that humans are involved puts limits both on how little waterfal we can
accept and how much iterative/incremental software development.

Well, there is a lot to say, listen, write, read about all of this.

I look forward to see more discussions like this.  Regards, Stefan Steurs.


____

This message and any files transmitted with it are legally privileged and
intended for the sole use of the individual(s) or entity to whom they are
addressed. If you are not the intended recipient, please notify the sender by
reply and delete the message and any attachments from your system. Any
unauthorised use or disclosure of the content of this message is strictly
prohibited and may be unlawful.

Nothing in this e-mail message amounts to a contractual or legal commitment on
the part of EUROCONTROL unless it is confirmed by appropriately signed hard
copy.

Any views expressed in this message are those of the sender.

> -----Original Message-----
> From: Ron Jeffries [mailto:ronjeffries@...]

> One main issue, as I see it, is that
>
>   a) SWEBOK is intended for use in licensing software professionals;

but isn't that the case for PMBOK as well?  What would be wrong with
licensing.  There is even a licensing program for software testers (ISEB
Foundation and ISEB Practioner's in Software Testing).  You need a driver's
license to drive a car so why don't you need an software engineering license
to write software (not necessarily all software, but in some cases there may
be specific needs).  The License itself doesn't prove everything but if
someone is not capable of getting it, it does prove something to me.

>   b) licensed professionals are subject to suit for malpractice;

Shouldn't all licensed engineers be responsible for the damage that can be
caused by the work that they've done/prepared?  Is there something
inherently different to software development than in other engineering
disciplines?  What happens when bridges collapse, planes crash, ... Software
development is a collaborative exercise, but there is always someone that
has ultimate responsibility.  Since we are more and more dependant on
software throughout our whole life, it better be good.  I think this will
ultimately lead to software certification bodies and indeed responsibility
for software engineers in case of mishaps (again not for all software
everywhere but certainly for some categories).
I can see a positive side in it.  Software engineers can actually subject
their employers to a malpractice suit when they are not allowed to perform
according to accepted standards.
Imagine you are driving a car with drive-by-wire and the software lets you
down, do you want to be able to sue the car manufacturar or should he be
able to get away with 'well you know, it's a bug in the software but you
signed the agreement that we could not be held liable for software bugs'.

>   c) malpractice suits typically work by finding anything in the
>   profession's BOK that was not done, and claiming that
> therefore there was
>   malpractice.

I don't think it is as simple as that.  The BOK is a reference not a
contract.  I think it should be possible to define a contract for a project
that refers to standards whether they be electrical, mechanical, or software
engineering.
>
> I am all for people being held responsible for professional behavior.
> However, the SWEBOK is not reflective of all forms of good
> practice, and
> therefore would be inappropriate as the basis for licensing
> and litigation.

If it is not appropriate, how do you propose to build something that is
appropriate.  Perhaps the agile community on its own could make such a
charter/reference.  Perhaps they could do it in an agile, iterative,
incremental way.  It would certainly help the agile community gain even more
credibility.
>
> Another important issue is that
>
>   a) SWEBOK is based on existing "software engineering" texts;
>   b) SWEBOK is used in university course accreditation.
>
> This is a closed loop which makes it difficult, if not
> impossible, to get
> programs containing newer practices accredited.

I don't know if you have to be that pessimistic.  I think the fundamental
problem is that most universities are not able to attract and pay the savvy
people required to give those kind of courses.  Most of the people who would
be able to deliver such courses are in consultancy, training, etc.  Areas
which are paying a lot more money.  By the way, what have been the
fundamental changes in software engineering (as a discipline) over the past
30 years?  Anything spectacular?  I can see shifts in emphasis but not a lot
of big innovations.
>
> I'd love to have a "book" that put down everything useful
> that we know.
> SWEBOK does not put down everything useful that we know, and
> is therefore
> not worthy of standing in that place.

I think that progress in this area is needed.  How do you think we can make
progress?
>
> Ron Jeffries
> www.XProgramming.com
> Mixed metaphors are a bright sunny day with no paddle.  -- Phlip

Regards, Stefan.


____

This message and any files transmitted with it are legally privileged and
intended for the sole use of the individual(s) or entity to whom they are
addressed. If you are not the intended recipient, please notify the sender by
reply and delete the message and any attachments from your system. Any
unauthorised use or disclosure of the content of this message is strictly
prohibited and may be unlawful.

Nothing in this e-mail message amounts to a contractual or legal commitment on
the part of EUROCONTROL unless it is confirmed by appropriately signed hard
copy.

Any views expressed in this message are those of the sender.

I've let some licensing messages go through because SWEBOK was a topic
with a deadline. Now that the deadline has passed, I fervently hope
that discussions of licensing can go elsewhere, except insofar as
they're narrowly focused on relevance to testing in agile projects.

-----
Brian Marick
Consulting, training, contracting, and research
Focused on the intersection of testing, programming, and design
marick@..., marick@...
www.testing.com, www.visibleworkings.com

I'm program chair or cohost of these events:
PLoP: <http://jerry.cs.uiuc.edu/~plop/plop2003/>
FIT Fest: <http://fitnesse.org/XpFitFest.FrontPage>
Please join me.

Hi all.
I would like to post this topic, as I believe to get a good advise; hope some will be happy to share their experience or opinion on this matter.

As we all know, in Agile development, the developers are to design for testing i.e.. develop with the testing in mind.

It seems logical (at least to me), then, that the developers should be trained in testing - and we in my company are thinking about educating our developers in testing area.

Most of our developers are graduates, or have 1-2 years experience in the industry, usually led by a team leader who is a senior person with many years of experience, and many projects behind.

Therefore our target audience for such a training will be that group of young developers, whose education and experience to date have not provided them with any (in some cases, very little) exposure to testing. They may have done some "gorilla" test execution (being graduate sometimes means that you are to do some testing first...), and some (unstructured) unit testing, but have never been involved in other stages of testing lifecycle  eg. planning, test analysis and/or design.

We have resources and means to organise an internal training course, but not sure about its content.

My questions to the Forum:

*       Do we have to run a "conventional" Testing Concepts course, or will need to design a special "agile testing" training?

*       How would  the two be different?
*       Do they have to be different?

Many thanks and regards,
Julia Rachkovsky
Test Lead