Terms we use

1. A check is one or more lines of test code that seek to verify the validity of a block of functionality of the tested method.
2. A test is a test method which contains one or more checks.

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

Clarence Garraway wrote:
>          1. Break the requirements down to the smallest possible unit
>             that you can see, giving you a requirements hierarchy. Let’s
>             say that there are N levels in the hierarchy, level N being
>             the leaves and 1 the root.
>
> Concerning the test-first story, I have a few questions, which no one
> should take as an affront, but rather my provoking your thoughts so that
> we may learn.
>
>
>    1. If I write one test at a time, then write the functional code to
>       make it pass, it seems to me that the creation of my tested method
>       is segmented, as I must constantly be switching hats - thinking

  From your description I suspect that we are talking about different
things when we use the phrase "test-first".

I have attempted the approach you are describing and I, too, have found
it difficult to context-switch between thinking about a test to verify
the requirements and then writing the code. Fortunately, that's not what
test driven development is about.

I think this is common misunderstanding and I think if you were to read
(and follow along with) Kent Beck's Test Driven Development you might be
quite surprised (hopefully, pleasantly surprised).

>    2. I call my colleague the perfect programmer, because he says that
[snip]
>       'doing' kind of proof, what would you say if you were trying to
>       convince him that test-first is better?
>

If your colleague is the perfect programmer, I would certainly not
attempt to convince him to change.


I have worked with several outstanding programmers who are able to write
perfect code the first time without tests and, in every case, less
perfect programmers like myself have found it difficult to write code
that interfaces with theirs. Programmers often give a great deal of
thought to how their code works from within. TDD teaches us to think
about how the code appears from without. It's a very different approach
which many programmers find unsettling.

In my four or five years of experience with TDD, I have never yet
encountered a programmer who has tried TDD and not liked it and stuck
with it. Most programmers, unfortunately, are not willing to try it
because it does sound so counter-intuitive. Many of them try to adopt a
more ad hoc test-first approach and most of them drop it before long and
go back to their old, familiar ways.

I hope I have provoked your thoughts as much as you have provoked mine.

Kevin

On Friday, July 1, 2005, at 11:39:16 AM, Clarence Garraway wrote:

> I call my colleague the perfect programmer, because he says that
> he writes full documentation on what he wants a method to do,
> thinks deeply about it, modifies the documentation until he is
> satisfied, then writes the method, then tests it, and he says he's
> hardly had to change the interface or the intention of his methods
> over the years of doing several projects. I've worked with him for
> those years and he's right. Therefore, since he's found a
> methodology that's worked for him so well, backed up with that
> 'doing' kind of proof, what would you say if you were trying to
> convince him that test-first is better?

I wouldn't try to convince him. I might tell him the benefits I
receive and invite him to try it and add it to his bag of skills.
I'd trust that if he had the skill, he'd use it appropriately.

Ron Jeffries
www.XProgramming.com
Do I contradict myself? Very well then I contradict myself.
(I am large, I contain multitudes.) --Walt Whitman

Forwarded from "Kal" on news:comp.software.testing :

I am working on a project involving SAP FI-GL, CD, BW, and CM.

The company has an SAP based product that is used to deliver financial
services.  We already have Reqmts Doc, Design Doc and Test Plans
completed.

We have Functional Specs that are sketchy at best, and Tech Specs that
are detailed but written more for the ABAP programmer in mind.

I have to write Test Scripts, but I am having a diffic. time finding
out which SAP screens to test?  Compounding this problem is the fact
that screen button functionaliy does not work in most cases.  I have to
write "negative tests" but how can one do this if virtually none of the
functionaliyt works??

Tks for your help.

--
   Phlip
   http://flea.sourceforge.net/resume.html

----- Original Message -----

From: Phlip

To: agile-testing@yahoogroups.com

Sent: Saturday, July 02, 2005 3:00 AM

Subject: [agile-testing] Testing SAP applications; writing Test Cases, Test Scripts

Forwarded from "Kal" on news:comp.software.testing :

I am working on a project involving SAP FI-GL, CD, BW, and CM.

The company has an SAP based product that is used to deliver financial
services.  We already have Reqmts Doc, Design Doc and Test Plans
completed.

We have Functional Specs that are sketchy at best, and Tech Specs that
are detailed but written more for the ABAP programmer in mind.

I have to write Test Scripts, but I am having a diffic. time finding
out which SAP screens to test?  Compounding this problem is the fact
that screen button functionaliy does not work in most cases.  I have to
write "negative tests" but how can one do this if virtually none of the
functionaliyt works??

Tks for your help.

--
  Phlip
  http://flea.sourceforge.net/resume.html

> Forwarded from "Kal" on news:comp.software.testing :

> I have to write Test Scripts, but I am having a diffic. time finding
> out which SAP screens to test?

I posted because I can't guess this part. Has anyone tested an SAP?
(Obviously without screen scraping and capture/playback?)

If SAP is a kit to build a big interconnected application, shouldn't
the test rig just be another instance of the kit?

--
   Phlip
   http://www.c2.com/cgi/wiki?ZeekLand

I understand that you can't develop low level screen based test cases
right now.
Can you start with high level "scenario like" test cases in this case?

Yury

--- In agile-testing@yahoogroups.com, Phlip <phlip2005@g...> wrote:
> Forwarded from "Kal" on news:comp.software.testing :
....................
> I have to write Test Scripts, but I am having a diffic. time finding
> out which SAP screens to test?  Compounding this problem is the fact
> that screen button functionaliy does not work in most cases.  I have
> to write "negative tests" but how can one do this if virtually none
of
> the functionaliyt works??

lrwrqtp wrote:

> I understand that you can't develop low level screen based test cases
> right now.
> Can you start with high level "scenario like" test cases in this case?

Is there anyone here with a SAP or ERP under test?

(Note the original poster, Kal, is on another newsgroup., so we can't
offer suggestions until we go there, invite him here, or find someone
in a similar situation. I myself am cracking JSUnit night now...)

--
   Phlip
   http://www.c2.com/cgi/wiki?ZeekLand

On Jul 2, 2005, at 12:58 PM, R. Sankara Narayanan wrote:

> Technically speaking, software testing is nothing but validating the
> behaviour of the system in accordance with the explicit and implicit
> requirements of the system/software. If the explicit requirements are
> not clear enough for you to start the testing, you cannot do much of
> the testing. Alternatively, you can sit with the business analysts,
> business users and the developers to understand how the system works
> and test it but you cannot guarantee that you are testing the software
> correctly in the right way. The Software Development process needs to
> be desciplined for you to do effective/ efficient testing.

This seems to me fundamentally at odds with the Agile approach to
software development. Whether and when the Agile approach makes sense
is a discussion some are interested in, but I note that it's not the
topic of this group.

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
www.exampler.com, www.testing.com/cgi-bin/blog
Book in progress: www.exampler.com/book

I performed some tests a few years back, wherein I was testing remote-access
software that was intended to support a bunch of products, including SAP.  I
was startled at how many bugs I found in its GUI.

SAP contains many different modules, serving many different parts of the
business in many different business domains.  The answer whether it's a
green-field project, or whether it's a production update, or whether they're
rolling out a new module, etc., etc. etc.

Meanwhile, when someone says "I have to write test scripts", I believe that
he or she is acting under the close supervision of someone who already has
some ideas about what those test scripts should contain.  If the scripter
wants advice from me, my first question is "What does the supervisor say?"
In this case, there's so little context available that answering the
question otherwise seems foolhardy.

---Michael B.

-----Original Message-----
From: agile-testing@yahoogroups.com [mailto:agile-testing@yahoogroups.com]
On Behalf Of Phlip
Sent: July 2, 2005 11:44 AM
To: agile-testing@yahoogroups.com
Subject: [agile-testing] Re: Testing SAP applications; writing Test Cases,
Test Scripts

> Forwarded from "Kal" on news:comp.software.testing :

> I have to write Test Scripts, but I am having a diffic. time finding
> out which SAP screens to test?

I posted because I can't guess this part. Has anyone tested an SAP?
(Obviously without screen scraping and capture/playback?)

If SAP is a kit to build a big interconnected application, shouldn't the
test rig just be another instance of the kit?

--
   Phlip
   http://www.c2.com/cgi/wiki?ZeekLand



Yahoo! Groups Links

> C. Why test-first?
> If write my method, then write the test(s) for it, won't I end up
> with the same code, both tested and testing?

Typically not, no. The tests are a second client to the class under tests,
and one that has slightly different needs, such as the need to inject a mock
object. Thinking about the tests early helps to get a testable, and thereby
flexible, design.

> Additionally, If I write
> my tests firsts, then, whilst writing the method, change my mind
> about its behaviour,

In my experience, it's much more likely that I change my mind while writing
*the test*. That is because writing the test puts me in the mind of a user
of the code under test.

> Doesn't it make more
> sense to: think carefully about my method;

Writing a test *is* the most powerful technique for thinking carefully about
the code to write I know of. That's exactly why I prefer to do TDD.

Hope this helps, Ilja

Hi all,

I need to know if there is a testing framework for C++ analog to NUnit and
JUnit.
Please your help!

Best regards,
Dan Lupu

********************************************************************************\
*******
The information contained in this communication is proprietary to Israel
Aircraft Industries Ltd. and/or third parties, may contain confidential or
privileged information, and is intended only for the use of the intended
addressee thereof. If you are not the intended addressee, please be aware that
any use, disclosure, distribution and/or copying of this communication is
strictly prohibited. If you receive this communication in error, please notify
the sender immediately and delete it from your computer. Thank you.

Have a peek at cppunit and derivates (10 in total!) at
	 http://www.xprogramming.com/software.htm

BR, Erkki
--
Erkki Pöyhönen, erkki.poyhonen@...
Solution Manager, SW Test Management
Nokia Oyj, Technology Platforms, PCPS
Address: P.O.Box 407, FIN-00045 Nokia Group, Finland
Phone: +358 7180 37595, Fax: +358 7180 36110



>-----Original Message-----
>From: agile-testing@yahoogroups.com
>[mailto:agile-testing@yahoogroups.com] On Behalf Of ext Dan Lupu
>Sent: 04 July, 2005 15:04
>To: agile-testing@yahoogroups.com
>Subject: [agile-testing] Testing framework for C++ ?
>
>Hi all,
>
>I need to know if there is a testing framework for C++ analog
>to NUnit and JUnit.
>Please your help!
>
>Best regards,
>Dan Lupu
>
>***************************************************************
>************************
>The information contained in this communication is proprietary
>to Israel Aircraft Industries Ltd. and/or third parties, may
>contain confidential or privileged information, and is
>intended only for the use of the intended addressee thereof.
>If you are not the intended addressee, please be aware that
>any use, disclosure, distribution and/or copying of this
>communication is strictly prohibited. If you receive this
>communication in error, please notify the sender immediately
>and delete it from your computer. Thank you.
>
>
>
>Yahoo! Groups Links
>
>
>
>
>
>
>

I fount this report very useful:


     Exploring the C++ Unit Testing Framework Jungle


       By Noel Llopis
       28 December 2004

http://www.gamesfromwithin.com/articles/0412/000061.html

I have successfully used cxxunit. The 'using perl' angle puts many
C++ers off but once it's up and running you don't know the perl is
there.  It's the 'least effort to add a new test' framework that I have
used.

Glenn

Dan Lupu wrote:

>Hi all,
>
>I need to know if there is a testing framework for C++ analog to NUnit and
>JUnit.
>Please your help!
>
>Best regards,
>Dan Lupu
>
>*******************************************************************************\
********
>The information contained in this communication is proprietary to Israel
Aircraft Industries Ltd. and/or third parties, may contain confidential or
privileged information, and is intended only for the use of the intended
addressee thereof. If you are not the intended addressee, please be aware that
any use, disclosure, distribution and/or copying of this communication is
strictly prohibited. If you receive this communication in error, please notify
the sender immediately and delete it from your computer. Thank you.
>
>
>
>Yahoo! Groups Links
>
>
>
>
>
>
>
>
>

**********************************************************************
The information contained in this message may be confidential or
legally privileged and is intended for the addressee only, If you have
received this message in error or there are any problems please notify
the originator immediately. The unauthorised use, disclosure, copying
or alteration of this message is strictly forbidden.
**********************************************************************

--- In agile-testing@yahoogroups.com, Clarence Garraway
<stonefishmail76@y...> wrote:
>    I call my colleague the perfect programmer, because he says
that
>    he writes full documentation on what he wants a method to do,
>    thinks deeply about it, modifies the documentation until he is
>    satisfied, then writes the method, then tests it

Perhaps you could think of Test First (or TDD or whatever) as part
of the design process, since it's not actually "testing" anything
until the methods are written.

What it is doing is documenting the behaviour of the method(s) using
an executable notation.  Not far from the model of the "Perfect
programmer".  Maybe he should try TDD!


cheers

Ian

http://www.combisoft.com

1) The name of an official SAP methodology is something like "ASAP".
It sounds "agile" for me.

2) Very often ERP implementation follows iterative approach with an
active involvement of real users.
It also smells "agile" for me.

Yury

P.S. I am not saying that all ERP implementation projects are very
agile :(.


--- In agile-testing@yahoogroups.com, Brian Marick <marick@t...> wrote:
>
> On Jul 2, 2005, at 12:58 PM, R. Sankara Narayanan wrote:
>
> > Technically speaking, software testing is nothing but validating
the
> > behaviour of the system in accordance with the explicit and
implicit
> > requirements of the system/software.
> -------------------------------------
> This seems to me fundamentally at odds with the Agile approach to
> software development. Whether and when the Agile approach makes sense
> is a discussion some are interested in, but I note that it's not the
> topic of this group.

Congratulations, what you have here is a classic 'We are digging a
Hole' TestCase.
Here is how it works. We have a Worker and a Shovel and 8 Holes to dig.
A worker is asked to pick up a shovel and dig 8 holes. The Worker will
parse the shovel's handle to find the appropriate place to grab good
hold of it. Then he will dig 8 holes accordingly.

As you can see this is very easy now to write test cases against the
Worker and test the application.
We only need to know 3 things. What kind of a Worker do we have? Can he
dig holes? How good is he with the shovel? Does he know the sizes of
Holes to dig? Is it raining? What kind of a dirt is it? When will he
know he is done digging the hole? Does he know the dimension of the
hole to dig? did he agree to dig the hole or was he forced to dig the
hole? what motivation does he have to dig the hole? Will he be timed on
this digging? Is he being paid per shovelful or per hole? Why 8 and not
4 holes? What if he successfully digs 2 holes and not 8? What if he
digs only one hole but does a really really really good job, then is it
OK to excuse him from the other 7?

After finding the answer to those 5 things I mentioned earlier you are
now ready to list your test cases and test your application
successfully.
MJ


On Jun 23, 2005, at 7:33 AM, csteraman wrote:

> Hi All,
>
> I'm a new member to this Group. I have a small scenario and I welcome
> your inputs on how I should go about.
>
> I was asked to test the following application:
>
> Objective: To parse an XML file and update the Database tables (8
> tables) accordingly.
>
> To achieve that, we have a small interface having two buttons. 1.
> Browse (to select the XML file to be updated in the database tables)
> 2. Update (after selecting the XML file, clicking this will parse the
> XML file and update the databse tables accordingly.
>
> There are NO SRS or FSD or DDD (detailed design document).
>
> I was asked to do the following:
>
> 1. First write the test cases for the XML Parse application.
> 2. Test the Application.
>
> Pleas advice me in how I should go about this testing.
>
> Thanks much in advance.
>
> -Ram.
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>

> Here is how it works. We have a Worker and a Shovel and 8 Holes to dig.
> A worker is asked to pick up a shovel and dig 8 holes. The Worker will
> parse the shovel's handle to find the appropriate place to grab good
> hold of it. Then he will dig 8 holes accordingly.
>
> As you can see this is very easy now to write test cases against the
> Worker and test the application.

> We only need to know 3 things. What kind of a Worker do we have? Can he
> dig holes? How good is he with the shovel? Does he know the sizes of
> Holes to dig? Is it raining? What kind of a dirt is it? When will he
> know he is done digging the hole? Does he know the dimension of the
> hole to dig? did he agree to dig the hole or was he forced to dig the
> hole? what motivation does he have to dig the hole? Will he be timed on
> this digging? Is he being paid per shovelful or per hole? Why 8 and not
> 4 holes? What if he successfully digs 2 holes and not 8? What if he
> digs only one hole but does a really really really good job, then is it
> OK to excuse him from the other 7?
>
> After finding the answer to those 5 things I mentioned earlier you are
> now ready to list your test cases and test your application
> successfully.

I like your list of seven things.  Here are two more things that it might be
important to know:

How big is the shovel?  How big is the shovel with respect to the hole?  Is
the shovel made of a sufficiently durable material to stand up to the dirt
(rocks, concrete)?  Is the shovel actually available when the worker wants to
begin digging?  Is the shovel full of dirt before he starts, or must he empty
it first?  What happens to the dirt that the worker removes from the hole?  Is
it possible for the worker inadvertently to fill up one hole while he removes
dirt from the next?  How far apart must the holes be in order to keep them
from undermining each other?  Does the worker have sufficient room to stand
while he's digging?  Will he require construction boots and a hard hat so that
we reduce our legal should he injure himself?  What is the purpose of the
hole?  Who will adjudicate the fitness of the hole for the purpose?  To what
precision must the holes fit the size requirements?  Is he going to have to
dig more holes later?  Are there other workers who could dig holes with him?
  Must there be barriers around the holes so children or automobiles won't fall
into them?  Is the digging going on during the day, or at night?  Does the
worker understand English, or do we have to address him in some other manner?
  Do we expect him to be able to report on his progress?  Can we stop him or
slow him down once he's started?  Do we have to feed him?

I agree that writing test cases against the worker is a simple matter.  I hope
that these four items are worthy additions to your list.

---Michael B.

DevelopSense:  Software Testing in Plain English
http://www.developsense.com

On Tuesday, July 5, 2005, at 3:42:03 PM, Marek J wrote:

> Congratulations, what you have here is a classic 'We are digging a
> Hole' TestCase.
> Here is how it works. We have a Worker and a Shovel and 8 Holes to dig.
> A worker is asked to pick up a shovel and dig 8 holes. The Worker will
> parse the shovel's handle to find the appropriate place to grab good
> hold of it. Then he will dig 8 holes accordingly.

> As you can see this is very easy now to write test cases against the
> Worker and test the application.

Nice story! I hope our reader can grab the handle and parse it into
what he needs ... and that he doesn't just dig himself a deeper
hole! ;->

Ron Jeffries
www.XProgramming.com
Find the simple path to what works and follow it,
always looking for a simpler path. -- Patrick D. Smith

On Tuesday, July 5, 2005, at 4:05:11 PM, mb@... wrote:

>> Here is how it works. We have a Worker and a Shovel and 8 Holes to dig.
>> A worker is asked to pick up a shovel and dig 8 holes. The Worker will
>> parse the shovel's handle to find the appropriate place to grab good
>> hold of it. Then he will dig 8 holes accordingly.
>>
>> As you can see this is very easy now to write test cases against the
>> Worker and test the application.

>> We only need to know 3 things. What kind of a Worker do we have? Can he
>> dig holes? How good is he with the shovel? Does he know the sizes of
>> Holes to dig? Is it raining? What kind of a dirt is it? When will he
>> know he is done digging the hole? Does he know the dimension of the
>> hole to dig? did he agree to dig the hole or was he forced to dig the
>> hole? what motivation does he have to dig the hole? Will he be timed on
>> this digging? Is he being paid per shovelful or per hole? Why 8 and not
>> 4 holes? What if he successfully digs 2 holes and not 8? What if he
>> digs only one hole but does a really really really good job, then is it
>> OK to excuse him from the other 7?
>>
>> After finding the answer to those 5 things I mentioned earlier you are
>> now ready to list your test cases and test your application
>> successfully.

> I like your list of seven things.  Here are two more things that it might be
> important to know:

> How big is the shovel?  How big is the shovel with respect to the hole?  Is
> the shovel made of a sufficiently durable material to stand up to the dirt
> (rocks, concrete)?  Is the shovel actually available when the worker wants to
> begin digging?  Is the shovel full of dirt before he starts, or must he empty
> it first?  What happens to the dirt that the worker removes from the hole?  Is
> it possible for the worker inadvertently to fill up one hole while he removes
> dirt from the next?  How far apart must the holes be in order to keep them
> from undermining each other?  Does the worker have sufficient room to stand
> while he's digging?  Will he require construction boots and a hard hat so that
> we reduce our legal should he injure himself?  What is the purpose of the
> hole?  Who will adjudicate the fitness of the hole for the purpose?  To what
> precision must the holes fit the size requirements?  Is he going to have to
> dig more holes later?  Are there other workers who could dig holes with him?
>  Must there be barriers around the holes so children or automobiles won't fall
> into them?  Is the digging going on during the day, or at night?  Does the
> worker understand English, or do we have to address him in some other manner?
>  Do we expect him to be able to report on his progress?  Can we stop him or
> slow him down once he's started?  Do we have to feed him?

> I agree that writing test cases against the worker is a simple matter.  I hope
> that these four items are worthy additions to your list.

... ;-> and people call ME names ... ;->

Ron Jeffries
www.XProgramming.com
Perhaps this Silver Bullet will tell you who I am ...

Dear All,

I would like share with you some ideas that i had tried to write about
test strategy. Unfortunately i never used XP, i just had read about it.
I thought something about use XP and TSP technics.

1 - Unit Test
2 - Interaction Test
     3 - Exploratory Test (Time-Boxed)
4 - Acceptance Test
     5 - Exploratory Test (Time-Boxed)

     * Does XP use some kind of  Interaction Test?
     * Someone has any experience of Exploratory Test usage on Agile
       projects?
     * What do you thing about Explorary Tests?
     * What do you thing about this five steps, related above?

Thanks

Carlos

Carlos,

Is the development team using XP or are you just using it for inspiration?




> -----Original Message-----
> From: agile-testing@yahoogroups.com [mailto:agile-testing@yahoogroups.com]
> On Behalf Of Carlos Antônio Menezes de Albuquerque
> Sent: 05 July 2005 18:21
> To: undisclosed-recipients:
> Subject: [agile-testing] Test Strategy
>
> Dear All,
>
> I would like share with you some ideas that i had tried to write about
> test strategy. Unfortunately i never used XP, i just had read about it.
> I thought something about use XP and TSP technics.
>
> 1 - Unit Test
> 2 - Interaction Test
>     3 - Exploratory Test (Time-Boxed)
> 4 - Acceptance Test
>     5 - Exploratory Test (Time-Boxed)
>
>     * Does XP use some kind of  Interaction Test?
>     * Someone has any experience of Exploratory Test usage on Agile
>       projects?
>     * What do you thing about Explorary Tests?
>     * What do you thing about this five steps, related above?
>
> Thanks
>
> Carlos
>
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>

For some reason, [Antony Marcano] my last e-mail got sent somewhat
prematurely... I wanted to answer your questions directly too...

>     * Does XP use some kind of  Interaction Test?
It depends on what you mean by "Interaction Test". If you mean Interaction
Tests vs. State-based tests... the answer is Yes.

Assuming the developers are using Mocks in their unit tests, then these are
centred around interactions rather than purely state. Mocks aren't to be
confused with stubs.

Acceptance tests that evaluate a user-story can be both state-based as well
as interaction based.

If you mean "Integration Test" then, yes, implicitly in any unit tests where
it is deemed appropriate to not use mocks or stubs for all other objects
that it interacts with. I haven't seen anyone explicitly separate the two
explicitly... perhaps others have.


>     * Someone has any experience of Exploratory Test usage on Agile
>       projects?

Yes. I have personally found that Exploratory Testing is especially useful.
On the project I am working on, a build is deployed once a week for this
purpose. Exploratory Testing is employed for three key purposes:

1. To identify tests that we hadn't previously thought of. Where there is
value in repeating them, these tests can be added to the core test-suite .
These are often first seen as bugs which may later be implemented as tests
or just recorded as bugs.

2. Catch any variances between story and interpretation (i.e. test
implementation) that hadn't been previously spotted.[Antony Marcano]  i.e.
fix incorrect tests

3. Compensate for areas where we may not have had a chance to implement
automated tests for (usually in the UI/Presentation layer)

The outputs feed into what is contained in the next available 1-week
development cycle. (For planning purposes, we use 1 week cycles in a four
week iteration).

>     * What do you thing about Explorary Tests?
If you mean - what do we think about them, I think that this is a very broad
subject and I am sure you will get many views. For me, Exploratory Testing
is a fast and valuable approach that you can apply if it is practical and
effective in your environment. As long as you are avoiding the trap of
confusing "exploratory testing" with "Exploratory Testing" it will  often be
very effective in most cases - to one degree or another.

If you are working in an XP project, using Exploratory Testing as Acceptance
Testing, however, is generally against the ethos of the approach. I haven't
found that it worked because the Acceptance Tests are essentially the team's
measure of progress and completeness. To achieve this, the process somewhat
relies on these tests being automated or at least clearly defined.

I have seen your proposed approach used but it proved to be somewhat
demoralising for the developers. The goal-posts for the project moved way
further and faster during customer feedback than anyone would have
preferred. This was because the Exploratory Testing was really being used by
the customer to compensate for their lack of involvement in determining the
Acceptance Tests before the story was implemented.  The team were
essentially guessing most of the time due to this lack of involvement.

I have found it to work well when Exploratory Testing is used to identify
possible Acceptance Tests that you hadn't previously thought of. These can
then be built into the Acceptance Test suite as appropriate, in the next
iteration.

>     * What do you thing about this five steps, related above?
In lieu of a more comprehensive opinion on this question... (quite difficult
without more context). With the time I have left to spend on this e-mail, my
brief suggestion is: be cautious about trying to use Exploratory Testing as
Acceptance Testing. Try using it to improve your Acceptance Test suite and
identify bugs early.

Best of luck

Antony Marcano
http://testingReflections.com


> -----Original Message-----
> From: Antony Marcano (testingReflections.com)
> [mailto:antony.marcano@...]
> Sent: 06 July 2005 06:37
> To: 'agile-testing@yahoogroups.com'
> Subject: RE: [agile-testing] Test Strategy
>
> Carlos,
>
> Is the development team using XP or are you just using it for inspiration?
>
>
>
>
> > -----Original Message-----
> > From: agile-testing@yahoogroups.com [mailto:agile-
> testing@yahoogroups.com]
> > On Behalf Of Carlos Antônio Menezes de Albuquerque
> > Sent: 05 July 2005 18:21
> > To: undisclosed-recipients:
> > Subject: [agile-testing] Test Strategy
> >
> > Dear All,
> >
> > I would like share with you some ideas that i had tried to write about
> > test strategy. Unfortunately i never used XP, i just had read about it.
> > I thought something about use XP and TSP technics.
> >
> > 1 - Unit Test
> > 2 - Interaction Test
> >     3 - Exploratory Test (Time-Boxed)
> > 4 - Acceptance Test
> >     5 - Exploratory Test (Time-Boxed)
> >
> >     * Does XP use some kind of  Interaction Test?
> >     * Someone has any experience of Exploratory Test usage on Agile
> >       projects?
> >     * What do you thing about Explorary Tests?
> >     * What do you thing about this five steps, related above?
> >
> > Thanks
> >
> > Carlos
> >
> >
> >
> >
> >
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
> >
> >
> >

If the holes are being dug at about the same time, can they start/finish
without interference?  Is digging a continuous activity or can it be put on
hold?  Is there only one or are there multiple shovels?  Can a worker stop
digging and restart later, can he recover the hole or does he have to start
all over?  Can one worker continue where another stopped?  What happens when
the ground is frozen?  What happnes if the worker uses a drill, dynamite,
... instead of a shovel?  Are the 8 holes really required, perhaps 7 will
do, perhaps we need 9?  Does the worker have to be certified for digging?
Does the worker have to be screened?  Is there protection against external
forces (bad weather, earthquakes, ...), do we cover the holes and restore
afterwards?  Does the worker have to pass a gate to get in and get out?
Does he have to take a shower once in a while?  What if there is a worker
that fills holes dug by other workers?  What do we do when there is a cave
underneath the hole being dug?  What is the maximum amount of stuff the
worker can dig per day, what is the average per hour?  What are the
contingency measures if the worker doesn't turn up?  How do we trigger the
worker to start/stop?  Do different workers have different profiles or are
they clones?  Are the properties of a worker variable in function of time?
Is the stuff he has to dig variable in function of time?  Is the amount of
work done proportional to the number of workers digging simultaneously?
Does the worker have to finish one hole before starting the next one or can
he dig the holes little by little in round-robin fashion?  Is there a
specified order in which the holes have to be dug?  Are there timing
requirements between the holes being dug?  Are all holes equal?  Can the
worker get trapped in the hole he is digging?  What does the worker do when
there is an animal in the hole?

...

Nice story.

Stefan

> -----Original Message-----
> From: mb@... [mailto:mb@...]
> Sent: Tuesday, July 05, 2005 10:05 PM
> To: agile-testing@yahoogroups.com
> Subject: Re: [agile-testing] Need Information ..
>
>
> > Here is how it works. We have a Worker and a Shovel and 8
> Holes to dig.
> > A worker is asked to pick up a shovel and dig 8 holes. The
> Worker will
> > parse the shovel's handle to find the appropriate place to
> grab good
> > hold of it. Then he will dig 8 holes accordingly.
> >
> > As you can see this is very easy now to write test cases
> against the
> > Worker and test the application.
>
> > We only need to know 3 things. What kind of a Worker do we
> have? Can he
> > dig holes? How good is he with the shovel? Does he know the
> sizes of
> > Holes to dig? Is it raining? What kind of a dirt is it?
> When will he
> > know he is done digging the hole? Does he know the dimension of the
> > hole to dig? did he agree to dig the hole or was he forced
> to dig the
> > hole? what motivation does he have to dig the hole? Will he
> be timed on
> > this digging? Is he being paid per shovelful or per hole?
> Why 8 and not
> > 4 holes? What if he successfully digs 2 holes and not 8? What if he
> > digs only one hole but does a really really really good
> job, then is it
> > OK to excuse him from the other 7?
> >
> > After finding the answer to those 5 things I mentioned
> earlier you are
> > now ready to list your test cases and test your application
> > successfully.
>
> I like your list of seven things.  Here are two more things
> that it might be
> important to know:
>
> How big is the shovel?  How big is the shovel with respect to
> the hole?  Is
> the shovel made of a sufficiently durable material to stand
> up to the dirt
> (rocks, concrete)?  Is the shovel actually available when the
> worker wants to
> begin digging?  Is the shovel full of dirt before he starts,
> or must he empty
> it first?  What happens to the dirt that the worker removes
> from the hole?  Is
> it possible for the worker inadvertently to fill up one hole
> while he removes
> dirt from the next?  How far apart must the holes be in order
> to keep them
> from undermining each other?  Does the worker have sufficient
> room to stand
> while he's digging?  Will he require construction boots and a
> hard hat so that
> we reduce our legal should he injure himself?  What is the
> purpose of the
> hole?  Who will adjudicate the fitness of the hole for the
> purpose?  To what
> precision must the holes fit the size requirements?  Is he
> going to have to
> dig more holes later?  Are there other workers who could dig
> holes with him?
>  Must there be barriers around the holes so children or
> automobiles won't fall
> into them?  Is the digging going on during the day, or at
> night?  Does the
> worker understand English, or do we have to address him in
> some other manner?
>  Do we expect him to be able to report on his progress?  Can
> we stop him or
> slow him down once he's started?  Do we have to feed him?
>
> I agree that writing test cases against the worker is a
> simple matter.  I hope
> that these four items are worthy additions to your list.
>
> ---Michael B.
>
> DevelopSense:  Software Testing in Plain English
> http://www.developsense.com
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>

____

This message and any files transmitted with it are legally privileged and
intended for the sole use of the individual(s) or entity to whom they are
addressed. If you are not the intended recipient, please notify the sender by
reply and delete the message and any attachments from your system. Any
unauthorised use or disclosure of the content of this message is strictly
prohibited and may be unlawful.

Nothing in this e-mail message amounts to a contractual or legal commitment on
the part of EUROCONTROL unless it is confirmed by appropriately signed hard
copy.

Any views expressed in this message are those of the sender.

---) On Tuesday, July 5, 2005, at 4:05:11 PM, mb@... wrote:

An awful lot of stuff about a worker digging holes with a shovel.

---) And Ron Jeffries replied:

... ;-> and people call ME names ... ;->

Well...

I'm glad you replied because it gave me a chance to say thank you to Marek J
or J Marek or whoever (s)he might be--a great example of thinking like a
tester.

Actually, I found following on to be a really fun exercise.  After doing
exercises like that, I find that it's easier to stretch the extent of the
questions that I can ask about the real stuff that I'm testing.

Does anybody else find those kinds of thinking exercises stimulating and
amusing?  Got any good examples that you might be willing to share?
(Elisabeth Hendrickson, please step forward!)

---Michael B.

Thank you Michael. - Marek J is just s tester currently looking for a
job. He lost his last job by asking too many questions about shovels,
holes and parsability of workers.

Thanks for joining in the fun exercise. I noticed that the majority of
the questions asked by you and then by Stefan S. were all about
establishing Context, The questions we all jumped to ask were maybe a
bit exaggerated (and funny) but they were mostly about testing the
boundaries of context space (which some people might call - problem
space but in fact I don't see it as a problem).

In fact asking questions as those does create the rich context space
which can then be used as MetaData for Test Case creation. Test
Scenarios are not necessarily mechanical steps to be executed. One has
to design scenarios in the first place. Knowing the Context for Test
Case is perhaps the hardest job of a tester, and that context comes
from designing a narrative, a story.

It is quite silly to look at perumtations of all possible inputs and
outputs, cause-effect graphs and step by step instructions tables in
hopes of creating enough data to arrive at some magical 85% Test
Coverage. - I think having the courage to ask stupid questions is the
way to start Use Case creations. Asking 'stupid' questions designs a
narrative space and brings the context into view.

So, back to original question Ram asks about XML file and 8 tables in a
database. It reminds me of one time when a friend was making a cake.
She had three cups filled with flour sitting on a table and I asked her
why she has 3 cups filled with flour standing on a table to which she
replied that the recipe calls for 3 cups of flour. It didn't occur to
her that she could use the same cup 3 times. - In the light of this I
would ask Ram why 8 tables??? Why not 1 table that is updated by the
XML file import? The other tables should be automatically populated,
triggerred by the update.

Anyway, thanks for the fun narrative.
Marek J


On Jul 6, 2005, at 9:11 AM, Michael Bolton wrote:

> ---) On Tuesday, July 5, 2005, at 4:05:11 PM, mb@...
> wrote:
>
> An awful lot of stuff about a worker digging holes with a shovel.
>
> ---) And Ron Jeffries replied:
>
> ... ;-> and people call ME names ... ;->
>
> Well...
>
> I'm glad you replied because it gave me a chance to say thank you to
> Marek J
> or J Marek or whoever (s)he might be--a great example of thinking like
> a
> tester.
>
> Actually, I found following on to be a really fun exercise.  After
> doing
> exercises like that, I find that it's easier to stretch the extent of
> the
> questions that I can ask about the real stuff that I'm testing.
>
> Does anybody else find those kinds of thinking exercises stimulating
> and
> amusing?  Got any good examples that you might be willing to share?
> (Elisabeth Hendrickson, please step forward!)
>
> ---Michael B.
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>

Dave Astels and I will be hosting a workshop at Agile 2005 titled
Test-driven Development and Beyond. The blurb:

      Test Driven Development is becoming a mainstream practice. However,
      it is a step along the way, not a final destination. This workshop
      will explore what the next steps and side paths are, such as
      Behavior Driven Development, Example Driven Development, Story-Test
      Driven Development.

   The idea is that we'll spend up to an hour having participants give
   brief pitches for what they think "lies beyond," then split into
   focus groups to discuss particular topics. The deliverable is a short
   paper outlining the various approaches discussed, together with
   people involved in each.

   You need not submit a position paper to attend. If you'd like to
   present your idea (briefly! which rules out fumbling with a
   projector), send us your topic at <workshop2005@...>.
   Thanks.

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
www.exampler.com, www.testing.com/cgi-bin/blog
Book in progress: www.exampler.com/book

Pl help!