On Oct 8, 2007, at 1:56 PM, Elisabeth Hendrickson wrote:

> the ideal workflow... on an ideal Agile project

Is there an assumption that the ideal workflow will degrade
gracefully on a not-ideal project? Many utopian ideal don't.

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
www.exampler.com, www.exampler.com/blog

Hello!

This was a good idea, count us in :)

Best regards,
Christian and Stein Kaare (who are already in Portland)


--- In aa-ftt@yahoogroups.com, Elisabeth Hendrickson <esh@...> wrote:
>
> Hi all,
>
> Just wondering if anyone would be interested in getting together for
> dinner the night before the workshop, Wednesday Oct 10?
>
> Also, there are some folks who will be in town for the PNSQC
> conference who might be interested in joining in as well.  How would
> you all feel about making the pre-workshop dinner an open the-more-
> the-merrier kind of gathering?
>
> Elisabeth
>

The primary purpose of this workshop is to discuss cutting-edge advancements in and envision possibilities for the future of automated functional testing tools.

Identify the single biggest reason enterprise agile initiatives tend to slide inexorably back toward waterfall, and propose specific initiative to address that challenge

Identify the biggest pain point in introducing agile practices to large organizations and define a study initiative to address it

Hi all,

I agree with Kevin's point that we should add something on  "What
works and what doesn't work with current tools/approaches? Why?"
Though I'm wondering if we could get at this with Lightning Talks.
Anyone want to do a Lightning Talk on "Why Automated Functional
Testing Failed on My Project Even Though Everyone Agreed Automation
Is Super Important?"

Or, does anyone have a story to tell that you'd like to share here?

(Reminder: this list will be opened up to the public after the
workshop.  One of our key goals is to foster open discussions about
functional testing tools.  So please don't tell stories here that
will cause trouble with your employer/clients/whoever.  Same thing
goes for demos/lightning talks in the workshop.  Though if it helps,
you can anonymous-ize stories, as in "Once upon a time at a company
that couldn't possibly have anything to do with any of my past,
present, or future employers or clients, my - um - friend...yeah,
that's it...friend, had this interesting project...")

And it seems appropriate to mention at this point that I just
stumbled across a blog posting that summarizes some challenges with
using FIT that I think is worth reading.  The comments are good too.

The original post:

http://codebetter.com/blogs/jeremy.miller/archive/2007/07/30/is-there-
a-future-for-fit-testing.aspx

And the follow up:

http://codebetter.com/blogs/jeremy.miller/archive/2007/08/12/why-i-m-
suddenly-down-on-fit-fitnesse.aspx

FWIW, my take on the post and comments is that they identify two core
areas of challenges:
- Articulating expectations
- Maintaining tests/fixtures, especially with a large or complex set
of tests

I'm curious about others reactions to Jeremy Miller's perspective -
and what lessons you think his experience suggests for the next
generation of tools/frameworks.

Elisabeth

Hello,

The agenda looks great! Having a session about what has worked and
what has not (both tool and process wise) like Kevin suggested might
be a good addition, though.

I'd like to give a tool-a-palooza demo about the framework we've been
working with.
And I'm already waiting for the beer. =)

Cheers,
     .peke

Elisabeth Hendrickson wrote:
> And, speaking of our meeting, it's time to firm up the agenda.  I'd
> like your feedback on the agenda that your organizers (Jennitta, Ron,
> and I) have been kicking around.
>
> As you read through the agenda, please consider the question: if we
> did these activities - if we structured the meeting in this way -
> would it be the best use of our time?  And if not, what would you like
> to see instead?
I like the agenda, the content and the structure.  There's one topic
that seems to be missing though - "What works and what doesn't work with
current tools/approaches? Why?"

For example, with the approach to story-driven testing that I have
pioneered here at Agitar, the technical problems were relatively easy to
solve compared with the enormous social challenges. But perhaps my
experience is unusual and everyone else has experienced only happiness
and light? If so, I'd like to hear that too!

If mine is a minority opinion and we don't want to darken the agenda
with Horror Stories, I'll be happy to tell a "What didn't go so well"
story in the Lightning Talks session.

Kevin

Hi, I'm Adam Geras, YACA (Yet Another Calgary Attendee) and it's a
thrill to be included in the workshop.  There are people attending that
I've read and followed for years, and I appreciate the opportunity to
listen in and participate.

I've been working to apply agile testing tactics (techniques seems too
formal for what I do) to a variety of project archetypes, including ERP
upgrades, data warehouse testing, custom development, and infrastructure
deployments.  It's the custom development stuff that's more automation
focused, and I guess I'm interested in discussing ways of making the
non-custom development projects more amenable to test automation too.
I'm heavily influenced by Brian's use of a dynamic language for testing,
scripted or exploratory and have sought ways to bring that into my work
where it seems appropriate.

I've been working in IT for 18 years, mostly custom development work and
since 2001, testing.  I wrote a Master's thesis on test-driven
development and trying to understand how/why it feels so good when a
team uses it for both story-driven development and test-first
programming.  I've participated in several agile projects, but probably
none 'pure'.  Enough to experience the successes and pitfalls, and also
enough to want additional healthy experiences.

I've been struck recently by seemingly-unrelated-but-possibly-related
things - first, the importance of language (that I think we all
recognize) in describing software and the use of domain-specific
language in testing.  Second, through work I picked up on Powershell
from Microsoft and how it was intended to support more collaborative
work and knowledge-sharing in the system administration work space.
They chose to achieve that knowledge-sharing by making a dynamic
language built from components called cmdlets (previous to Powershell,
server software was only administered through graphical user interfaces
and had little, if any, scripting capability).  Nothing special there
other than the cmdlets are always named, and enforced by the runtime, in
a verb-noun manner.  The list of verbs is standardized so that whether
you are administering a mail server or a domain, you understand what
'add-thing' means.  The nouns come from the domain, i.e., add-mailbox,
add-useraccount, etc. but the verbs are always selected from a standard
list.  So it's knowledge sharing and learning through use of a
domain-specific language.  I guess I just see a parallel there to the
use of tests for specifying behaviour - that's a knowledge-sharing
activity too.

So I guess I want to acknowledge and understand better how dynamic
languages can help us in testing - are verb-noun combinations
appropriate raw materials for building test cases? If so, how can we get
the domain experts to participate (since a text editor might not be
enough)?  Is it fair to expect a single tool to support the craft-like
aspects of design/specification and the nitty-gritty of
verification/validation at the same time?  Maybe executing the tests can
be done the same way, but there are different ways of working with the
tests, i.e., different editors or workspaces for supporting the
differing work activities?

I'm anxious to see and experience the work of others, see you soon.

Friends --  I'm excited about our upcoming workshop. I feel that we
could easily set direction that could impact a decade. I'm also a
great fan of the LAWST format which I learned from Brian and have now
experienced a half dozen times. It has to be the most effective use
of smart people for the common good I've yet encountered.

I am the original author of Fit who's history is summarized in the
link that follows. This is a history of custom test infrastructure.
Fit was my attempt to offer some standards that were simple and
general enough to unify the practice. I'm happy that it serves to
define a style of test but disappointed that it lacks sticking power.
Brian (again) influenced me with a provocative blog post asking why
people won't keep up Fit tests even when they have them. Why indeed?

	 http://fit.c2.com/wiki.cgi?FrameworkHistory

With Brian's observation in mind, I wrote another test framework,
this time tightly coupled with a small but highly leveraged portal
application for the Eclipse Foundation. With the freedom one has in
one-off code, I sought to explore what further utility one could gain
from agile-style functional tests as the basis of collaboration, an
idea at the heart of Fit. I will be reporting on this work at PNSQC.
You can find my paper and some slides too. The slides, I will warn
you, were written this summer for a research organization that
expected me to talk about wiki. I'm revising them today for the PNSQC
audience.

	 http://c2.com/pnsqc2007/

Elisabeth, more than anyone, has taught me practical techniques of
the exploratory testing. With her in mind I added the capability to
switch between scripted and exploratory testing. This required some
db manipulation code that developers might not be eager to write if
they don't understand the benefits that accrue to both development
and (I hope) testing.

Best regards. -- Ward

Hello,

My name is Pekka Laukkanen and I'm a Finnish test automation
specialist. I started my career as a test engineer year 2000 and found
myself writing simple automation scripts with Perl pretty soon
afterwards. I was studying at Helsinki University of Technology that
time (working while studying is a pretty common in Finland) and had
learned some Java and C there but found Perl more flexible and much
better suited for scripting in general. Around that time I also heard
about Agile processed in the University and we even had a course with
a year long project where we used XP. I liked Agile ideas a lot from
the start but unfortunately it took years before I was in a real Agile
project.

Over the years I've used many different automation tools from xUnit
frameworks to commercial capture-and-replay tools. I've often also
used scripting languages (Perl, Python, Bash, ...) to glue different
tool together. With open source tools I've most of the time been able
to actually get things done but commercial tools have mostly been
waste of time, money, and nerves.

I built my first larger scale test automation framework on top of STAF
(and learned to like Python while doing that). This framework was
data-driven i.e. test cases were created in tabular format and actual
programming code was hidden from "normal" test engineers. The
framework worked pretty well -- at least compared to old automation
experiments in that particular project -- but we didn't get too far
before the company was sold and all the R&D moved to Spain. This
wasn't a disaster for me because I was working for a consultation
company so I didn't lose my job and I had already learned a lot and
made some important contacts.

At that point my studies were finally getting ready and I only had my
Master's Thesis left. The framework I had just worked with had been
pretty promising but I got a feeling that it could've been much
better. To find out more about such frameworks I started studying
large scale automation frameworks in general and data-driven and
keyword-driven frameworks in particular. (People used to FIT
terminology can consider data-driven testing equivalent to using a
column fixture and keyword-driven testing similarly equivalent to do
fixture). The most important finding in my thesis was that if you have
a flexible enough keyword-driven framework you can use it in all those
cases where you'd use a data-driven framework.

I had implemented a simple keyword-driven framework prototype as part
of my Master's Thesis. The thesis itself wasn't even ready when a
manager I had worked earlier (in that project that got terminated)
contacted me and asked was I interested about a new and bigger
project. He had moved to Nokia Networks and was involved with a rather
large project that was moving from waterfall to Agile development.
They had big problems with system/acceptance level automation but
believed that without it moving to Agile is not really possible
especially when there's such a big amount of legacy code.

I was obviously very interested about the project and I was pretty
soon giving a demo about what a flexible keyword-driven framework
could do. Audience was convinced enough for us to get a deal and I
started in that project with my colleague Juha Rantanen (who's
actually joining the workshop too and can introduce himself). This
happened a bit over two years ago and since then we've been
implementing a generic keyword-driven framework, test libraries
(fixtures in FIT terminology) to be used with it and most importantly
helping teams to take the framework into use. Starting from last
spring I've been working in this project as a freelancer consultant.

The project has been fairly successful and we nowadays have hundreds
of users in many different product lines. Different teams are using
the framework to test totally different kinds of systems from network
elements with a Telnet interface to Swing applications and web
systems.

The coolest thing about all this is that we now have a permission to
open source the framework and generic test libraries implemented for
it. We haven't yet decided about the license (it will probably be BSD
or BSD like) and some other things are open too but we can anyway
present the framework and also give out the latest version for those
who are interested.


This mail is getting rather long already and now that the workshop is
so near it probably doesn't make sense for me to write too much about
what our framework can do and how it works. Here's anyway a short list
of the main features in no particular order.

- Implemented with Python so that it runs also on Jython (a Java
implementation of the Python language). Test libraries can be
implemented either using Python or Java.

- Test library API is really simple. For example following code
implements test library MyLibrary with keywords "Greet" and "Should
Start With".

----[MyLibrary.py]----------------------------
def greet(name):
     # This message goes to a log file.
     print "Hello, %s!" % name

def should_start_with(str1, str2):
     if not str1.startswith(str2):
         # If we get here the keyword fails, otherwise it passes
         raise AssertionError("%s doesn't start with %s" % (str1,str2))
----------------------------------------------

- Test are created using tabular format and test data can be either in
HTML or TSV. For example test cases below use above keywords
(formatting ought to be ok when using fixed width font).

| Test Case  | Action            | Argument | Argument |
| First Test | Greet             | AA FTT   |          |
|            | Should Start With | AA FTT   | AA       |
|            |                   |          |          |
| 2nd Test   | Should Start With | Portland | AA       |

- It's possible to create new higher level keywords from existing
keywords using the same tabular format. For example we could create a
keyword "Should Start With A" like below and use it in test cases or
when creating even higher level keywords.

| Keyword             | Action      | Argument  | Argument |
| Should Start With A | [Arguments] | ${string} |          |
|                     | Starts With | ${string} | A        |

- You can easily create ATTD style test cases (a.k.a. story tests)
using keywords like "As an administrator I want to be able to log in
to the system and see all critical alerts immediately on the main
page." There's no limits to keyword names when creating higher level
keywords and it's really easy to map them to lower level ones and
finally to keywords implemented in libraries. (This is something Juha
has specialized.)

- One file can contain multiple test cases and it automatically
creates a test suite from them. Name of a test suite is generated
automatically from the file name (e.g. "my_tests.html" -> "My Tests").
Multiple test case files can be placed into a directory which then
automatically becomes a higher level test suite, it's name again got
from the directory name, and so on. Integrating this with version
control is naturally pretty simple.

- Test cases can be tagged freely. Tags allow from example running
only certain tests and there's automatically statistics per tag.

- Tests are run from command line. Integrating with continuous
integration systems etc. isn't thus too complicated.

- A result of a test run is an XML file and log and report in HTML
format are created automatically from it. Multiple XML output files
can be later combined together to create a higher level report and
other systems can also read the XML if they need details about the
test execution.

- Currently the test data is edited using a normal HTML editor or a
spreadsheet program. The biggest complaint we get from users is the
lack of a real editor and creating one is our biggest task in the near
future.


Written text is never the best way for communication but I hope this
introduction gives some kind of a picture about my background and also
a brief overview of the framework we now are about to open source.
Looking forward to the workshop and all the interesting discussions
there!!

Cheers,
     .peke

Can those of you that attended the Google Test Automation Conference
last year provide a summary of any insights / learnings / direction,
etc that came out of it.  Also any pointers as to how the event or
individual sessions were run would be helpful as we plan our 2 day
workshop.

My flight arrives at 20:35, so I will miss dinner. If anyone wants to
share a ride to the hotel, I can give them an exercise bar on the way.

What's the best way to get to the hotel?

On Oct 4, 2007, at 8:09 PM, Elisabeth Hendrickson wrote:

> Hi all,
>
> Just wondering if anyone would be interested in getting together for
> dinner the night before the workshop, Wednesday Oct 10?
>
> Also, there are some folks who will be in town for the PNSQC
> conference who might be interested in joining in as well. How would
> you all feel about making the pre-workshop dinner an open the-more-
> the-merrier kind of gathering?
>
> Elisabeth
>
>

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
www.exampler.com, www.exampler.com/blog

2007/10/5, Elisabeth Hendrickson <esh@...>:
> Just wondering if anyone would be interested in getting together for
> dinner the night before the workshop, Wednesday Oct 10?

Juha and I will be there already on Tuesday and we are definitely interested.

> Also, there are some folks who will be in town for the PNSQC
> conference who might be interested in joining in as well.  How would
> you all feel about making the pre-workshop dinner an open the-more-
> the-merrier kind of gathering?

This is totally ok.

Cheers,
     .peke

Sounds like an excellent idea.

My plane doesn’t land until 19:40pm, however. So, whether I can come depends
on how long it takes for me to get to the hotel and what time it starts and
how long people are likely to be there.

Antony

________________________________________
From: aa-ftt@yahoogroups.com [mailto:aa-ftt@yahoogroups.com] On Behalf Of
jennitta andrea
Sent: 05 October 2007 05:06
To: aa-ftt@yahoogroups.com
Subject: Re: [aa-ftt] Interested in dinner Wednesday 10/10?

I'm interested in meeting for dinner, and I welcome people from the PNSQC to
join us as well.  Thanks for suggesting this.
On 10/4/07, Elisabeth Hendrickson <esh@...> wrote:
Hi all,

Just wondering if anyone would be interested in getting together for
dinner the night before the workshop, Wednesday Oct 10?

Also, there are some folks who will be in town for the PNSQC
conference who might be interested in joining in as well. How would
you all feel about making the pre-workshop dinner an open the-more-
the-merrier kind of gathering?

Elisabeth

Hi all,

Just wondering if anyone would be interested in getting together for
dinner the night before the workshop, Wednesday Oct 10?

Also, there are some folks who will be in town for the PNSQC
conference who might be interested in joining in as well.  How would
you all feel about making the pre-workshop dinner an open the-more-
the-merrier kind of gathering?

Elisabeth

Hi all, my name is Stephen Michaud and I have been in s/w dev since
the early nineties, predominantly in testing and project management.

My first exposure to agile was as a project manager in Nortel.  I was
working with a small team that wanted to use XP but felt constrained
by the processes in place (ISO 9001, TL 9001, etc...) So I offered
(and did an admirable job I must say :) ) to shield and map their
activities into a corporate acceptable format.  The project was a
success.  There was a long break in between that and my next agile
project (with a stint learning RUP at Rational/IBM in between).

My next exposure to agile was as the head of a larger test
organization when the development team decided we needed to do a
crash conversion to Scrum to meet some insane ... I mean
visionary ... requirements and product deadlines.  So in crash
courses of reading and Scrum master training, the hardest part of
that conversion was figuring out how to accommodate testing (I use to
say fit testing in but it lead to too many groans when I went on to
talk about Fit) in the process.  We had a large legacy application
with about 1200 hours of manual regression tests.  Well through
creative sourcing and reinventing the wheel (we made something almost
exactly like fit but xml driven ... <sigh>) we managed to get our
regression cycle down to about a week and more importantly, get first
pass testing and exploratory testing done in cycle.

Finally, I ended up at Luxoft Canada (part of the largest Eastern
European software solutions provider) where we started the office and
staffed for all contingencies and waiting for the work to roll in.
While waiting we needed to do some work.  Our CEO and I had taken a
serious interest in Fit and FTDD so we thought we should do a project
that would make it easier to adopt Fit.  We focussed our efforts on
the developer (as tools out there had mainly been focussed on the
analyst and tester) and decided to create a plug-in for the eclipse
IDE.  Thus our open source FITpro (http://fitpro.sourceforge.net) was
born.

Besides my experiences with agile and Fit, I have been a student of
the tester career, having led a number of test organizations.  I see
FTDD as a way to integrate and broaden the test role and that is a
perspective I want to bring to the table.  What do I want to get out
of this session?  I want to see what is going to be done to take this
to the next level.  I have some random thoughts that FTDD can aid in
strengthening communication within a distributed team and some
questions on how non-functional requirements might fit into all this.

With no books published, only two speaking engagements under my belt,
and sporadic blog posts at best, I feel like a rookie in amongst you
all.  I plead for patience and hope to contribute my all!

In the spirit of introductions...

I've been working in software development since the mid nineties...
mostly specialising in software testing... My first XP project was in
2000 on an intranet provisioning system for a major data-services
provider to the airline industry. They were working with a US based
consultancy that had already written a framework enabling Silktest to
execute the Acceptance Tests. It had several problems, not least the
fact that Segue Silktest simply wasn't designed for writing
communicative tests and was completely detached from the development
environment.

Since then I've worked on numerous XP/agile projects using a variety
of solutions for writing 'executable specifications' including
'literate APIis'  for writing such tests with NUnit and JUnit. I adopt
either the 'Tester' role or the 'Programmer' role depending on what's
required at any given moment in time... my testing skills and
experience are stronger than my programming skills, however...

In most recent years, I've been working extensively with FIT and
FitNesse, helping Java and Perl teams explore its applicability for
their projects.

What I like about FIT is that it allows you to abstract the
implementation details behind the fixture and focus on the essence of
the business rules that you are trying to extract from the customer.
For example... I've seen many teams use tools like Selenium to write
acceptance tests. These are so tightly coupled to the UI
implementation that the details of specific UI interactions distract
teams from the essence of what the customer is trying to achieve, or
the rule they are trying to express.

On the down-side, FIT tests can be a pain to create, maintain and
refactor compared to writing the tests in code with the productivity
tools of modern IDEs. Frameworks like LiFT and Hamcrest help but can
still be frightening to all but the more technical 'customer' or
'product owner'. I've found it's even more difficult to get customers
to contribute to these tests.

One of the most important things I believe Story Tests/Acceptance
Tests do is focus discussion. Being able to easily express what's
learned during that discussion is important. Ideally, I want to draft
the tests during the discussion without that distracting from the
conversation. It needs to feel natural enough for the customer and
structured enough for the developer.

Another aspect of the tools (that I haven't noticed mentioned so far),
is that the current options don't facilitate organisation of the
tests. You tend to have to organise them by feature or by story. There
are arguments for and against both - this is another area where I want
the best of both worlds.

I'm still optimistic about the value of Story Tests/Acceptance Tests.
Perhaps narrow-mindedly, I don't think there's any other way to
reasonably evaluate completion of a Story (in so far as it is
understood at the time). Other approaches, such as customer testing
and exploratory testing offer feedback or tell you when a feature
doesn't require any further evolution - but the Story Tests are useful
in knowing when a given incremental step in that evolution (i.e. a
Story) is complete. I can't think of any other way... perhaps it's
just me.

I also want to explore whether or not 'backwards-compatibility' with
FIT is important. I'm not sure either way...

It would be interesting to look at how future frameworks might
incorporate the latest 'thinking' in wording Acceptance Tests (e.g.
Given/When/Then) yet remain flexible to evolve as that thinking evolves.

I am hoping that we will establish some sort of vision as to the most
important characteristics of the next-gen Acceptance Test/Story Test
tools... perhaps even end up with a Story List of the most important
features as a starting point for future efforts.

I also hope we agree on the next, most practical steps to making our
vision a reality.

Executive summary: Street cred... typical Marick pseudo-intellectual
gobbledeegook... whine, whine, whine... a list of actions, because I
am incapable of just living with a problem.

---------

I was mostly a programmer in the 80's, mostly a tester in the 90's,
and have been mostly consulting on Agile projects in the 00's. Like a
lot of people, I've been trying to recreate that one great project
and that one great work environment. (In my case, it was implementing
a Lisp system in the 80's. I was the technical lead because I was the
only one who'd ever actually written Lisp code (maybe as much as a
couple of hundred lines). We survived, even did fairly well, because
of lots of testing, because we were too unimportant to meddle with,
and because I spent a lot of effort improving what Dick Gabriel calls
the "habitability" of the code. For example, it was there that I
first adopted the guideline that no type of bug should be hard to
find the second time, which meant it became easy to get the answer to
most any debugging question.)


My schtick with functional tests goes something like this:

1. Two important groups of people who care about the project are the
development team and the business. They have different ways of
speaking, sometimes different ways of thinking, often different
goals. Since I dabble in science studies, I know that various people
have discovered that:
	 (a) you need to find what Eric Evans calls a ubiquitous language,
                which allows the two groups to coordinate their
actions while
                minimizing the need for agreement about what the words
connote,
                what proper goals are, and where value comes from.

	 (b) it's important to have things that both sides can point at
(either literally
                or metaphorically). It's especially helpful if those
things can "move" from
                one group who's identified as the producer or owner,
to another group
                who can use them to accomplish some task of importance
to them.

Tests seem like a good fit. They can be written in the ubiquitous
language, and they let programmers do something they've learned to
care about: break work into small, checked pieces.

2. An important problem when dealing with experts is tacit knowledge:
experts typically do not know how they know something; they simply
act appropriately. In someone's terminology, there are two types of
knowledge: you can "know how" to act, or you can "know that" a fact
is true. Computers deal in the latter; experts deal in the former.
Therefore, creation of the ubiquitous language is (often) not a
matter of translation from "concepts" in the heads of experts to
"classes" in the text of a programming language. A lot of the
language - the way the words connect - has to be invented through
project-long conversations. Examples help a lot in such
conversations. Tests are examples.

3. Since you are inventing a language, you'll be pushing on the
usages of words that have gone unquestioned in the business world.
There's a lot of opportunity for surprise as tacit knowledge becomes
explicit, and then gets questioned. I'd like tests to be a vehicle
for us to surprise ourselves.

So functional tests could be a Big Deal. However, they are in
practice a little deal. I've been wondering why. Some potential reasons:

1. It doesn't seem that they add enough value for the programmers
beyond the tests they're writing for their own reasons (xUnit tests).

2. We've had many years of general-purpose mind-mapping tools,
outliners, organize-your-lifers, and other programs that aim to help
people think and work. The uptake has been disappointing to their
true believers. It's a hard problem. Why should we expect immediate
success in our specialized little domain?

3. Good grief, most of what we all do is hook up yet another web
front end to yet another database to let some new group of people do
CRUD more pleasantly. How much incremental value does doing a superb
job have, over doing a merely mediocre job?

4. We muffed it when Agile "crossed the chasm".  It's not a horrible
oversimplification, I think, to describe a lot of the early impetus
as coming from programmers who had extended their interest to hacking
the social: specifically the small-scale social of the team, but also
with some aspirations to affecting the firm. That latter ran smack
into the dominant tradition that "a good manager can manage anything"
and into the valorization of "leadership" - and folded. So the large-
scale social split off. At the team level, the technical has (to a
lesser extent) split off from the social. In the Rails world, for
example, the "hard" technologies of XP are widely known and somewhat
widely practiced; the "soft" ones are not. (You could pick up most of
the Rails inner circle, drop them into the 70's-era MIT AI lab, and
they'd fit in perfectly.) And the post-Agile people are much less
focused on the team, much more prone to walk away from the team and
optimize their individual work.


So what do I plan to do?

1. Optimize the flexibility and ease of creation of tests, keep the
effort to implement them roughly the same as with Fit today (even
though that effort proves to be too much for many), toss a bone to
programmers by making the tests fit better into their programming
environment (e.g., they must be executable as xUnit or xBehave
tests). My hope for that these days is graphical tests (drawn tests),
which I am prepared to demonstrate.

2. Concede that there are few projects and people who want to reach
for the brass ring. Functional testing is not going to be a mass
movement (like TDD) for quite a while.

3. Realize that we have allies and teachers in the form of the user
experience community (those who are sympathetic to Agile). They're
interested in many of the same things: extracting tacit knowledge,
finding a way to talk about what the problem is and what the solution
is, etc. But they don't have a habit of producing potentially
executable outputs. So what I want to do is sit at Jeff Patton's
feet, learn what it is he does, learn what outputs come naturally
from that kind of work, and then find the way to make those usefully
executable. With luck, that will blow away most of the work in (1).

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
www.exampler.com, www.exampler.com/blog

I've been doing XP since early 2000 (right after the White Book came
out.) Before that we did something that resembles FDD at NorTel all
through the 80's and 90's. In 1996 I "had to" write a small unit test
automation system to test an event notification service I was
building. In the process, I discovered the need for configuring the
framework with "expectations" for each test (essentially Mock Objects.)

I've been working with "customers" trying to apply StoryTest-Driven
Development on a number of projects over the last 5 years. I've
definitely found the tools to be the limiting factor. I've used JUnit
code generators, Fit tables, QTP Recorded Tests, Watir, IeUnit and
even built a domain-specific test recording and playback engine into a
legacy application to allow it to be ported safely from one
environment to another. All of these efforts were beneficial yet all
were way more work than they should be. I'm persistent -- some would
stay stubborn -- and I was determined to make them work but a lot of
people would have given up long before they got any value out of it.
We have to find a way to make it easier for people if we want to get
significant penetration of STDD.

I hope to learn from other people's experiences and to contribute to
advancing the state of the art. I'm looking forward to the workshop.

Gerard

Elisabeth Hendrickson wrote:
> Greetings!
>
> I'll be your facilitator at the FTT workshop in Portland, so I
> thought I'd take a moment to introduce myself
Hi Elisabeth,

I am not sure if you were hoping for everyone to introduce themselves
but, even if you weren't, here goes...

I worked on my first XP project in 2000-2001. It was a tremendous
success that, to my great disappointment, I have never quite been able
to recapture. For the last 15 years I have flitted between development
roles and product management roles. Most recently I am doing both in my
capacity as developer/webmaster/chief-cook-and-bottle-washer of
www.junitfactory.com.

JUnit Factory is based on Agitar's commercial product and, thus, I also
act as an on-site customer (one of three product managers) for AgitarOne
and it is in this role that I come to the workshop - as someone who has
been writing Story Tests for a very complex product for the last 4 years.

I have many experiences to share including
- How to get different parts of the organization - PM, devs, testers -
engaged. And how I failed in this.
- Where the pitfalls are (and how to avoid them).
- How story-testing scales to large, long-running, fast-changing
projects (and how it doesn't).

I have opinions about approaches, technologies and practices to
story-testing but, most of all, I want to know how much my experiences
match or differ from my fellow workshoppers.


I just jotted down quick list of things I might like to talk about at a
workshop like this and I quickly got to 20 bullet points ;-) Did you (or
anyone else) have any thoughts on how we might structure our discussion
so that I might prune my list a little and prepare deeper thoughts on
the part of my list that intersects with yours?



See you in Portland!

Kevin Lawrence
Agitar Software

Greetings!

I'll be your facilitator at the FTT workshop in Portland, so I
thought I'd take a moment to introduce myself.

I worked on my first XP project in 2004 and haven't looked back
since.  I don't do project work with traditional teams anymore.  (I
do, however, still teach classes for traditional teams.  And whenever
I do, I am reminded of *why* I don't do project work with traditional
teams.  Testing on a traditional project is a no-win proposition.)

I've been doing test automation since 1994 when I started working
with SQA Robot, later acquired by Rational.  I moved on from there to
Segue's Silk (now part of Borland).  I became adept with Silk's 4Test
language and used it on a wide variety of projects to do relatively
sophisticated (for the time) data driven testing and model-based
testing.

It took a while for Bret Pettichord and Brian Marick to convince me
to learn Ruby and try Watir, but once I did, I was sold.  From there
I went on to Selenium (and SeleniumRC).  While I've played with FIT
and Fitnesse, I haven't succeeded in using them on real projects
yet.  On the one project where I tried to use FIT, we found that the
cost of keeping the column headers in synch with the code outweighed
the benefits for our particular context.  (In our case I was the only
one creating FIT tables - and I can read/write code.  So there was no
compelling reason to keep the tests in tables.)

Though as your facilitator, I won't be talking about any of that.
Instead, my job in this workshop is create a space where you all will
share your ideas, experiences, successes, wishes, concerns, and
ultimately vision for the next generation of functional testing tools.

I'm excited about this upcoming workshop and look forward to seeing
you all there!

More on the structure of the meeting later...

Elisabeth