--- In agile-testing@yahoogroups.com, Jairo Ernesto Gutierrez Pizarro <jairoegutierrez@...> wrote:
>
> Hi All
>
> I have been working with functional testing automation since a few years ago but now i'm having problems with the scripts maintenance. Somebody Knows how can i automate functional testing with out the scripts use?.
>
> Thanks
>
> Best Regards
>
> Jairo E Gutierrez
>  
>
>
>
> ____________________________________________________________________________________
> ¡Obtén la mejor experiencia en la web!
> Descarga gratis el nuevo Internet Explorer 8.
> http://downloads.yahoo.com/ieak8/?l=e1
>

--- In agile-testing@yahoogroups.com, Lisa Crispin <lisa.crispin@...> wrote:
>
> I worry that terminology might be used to keep newbies on the outside of a
> profession. If they don't learn our secret handshakes, we won't talk to
> them. I personally have never learned all the official testing terminology,
> if there is such a thing. Some people say that means I don't know anything
> about testing.
>
> OTOH, it's helpful to have a common language. I would love it if
> "integration test" meant the same thing to all of us, but I know for sure
> that it means different things to different people - just to give one
> example.
>
> Gerard Meszaros influenced me and Janet when we wrote our book to use
> "standard terminology" (in this case, Gerard's, since we don't know any
> better standard) as much as possible. So I stopped saying "integration
> tests' and started saying "component tests" since that's what I meant, but
> other people may not know that term either. I'm not going to beat them up if
> they don't. I'll just keep trying to communicate as well as I can.
>
> I do like it when people try to come up with clearer terminology. If it
> works for a critical mass of people, it'll stick, and it'll help.
> -- Lisa
>
> On Thu, Nov 12, 2009 at 6:18 PM, Brian Marick marick@... wrote:
>
> >
> >
> >
> > On Nov 12, 2009, at 4:02 AM, erik petersen wrote:
> >
> > > Hi Brian,
> > > OK, how about this alternate explanation. [...] Does that make more
> > > sense?
> >
> > I don't know. How will the world be meaningfully different if we talk
> > like this versus old-fashioned lingo? What does this buy us over
> > longer strings of words like "exploratory testing: needed, no matter
> > how enthusiastic you are about junit, newly-test-infected programmer-
> > type" and "scripted manual tests: ick!" and "talking about examples of
> > executing the app, using business language, before writing code:
> > helpful!" and "rerunning tests frequently: more helpful than Marick
> > thought in 1997, but still risky as the tests come to execute more and
> > more lines of code".
> >
> > The philosopher Richard Rorty said (roughly) in the introduction to /
> > Consequences of Pragmatism/:
> >
> > Long conversations about the definitions of words
> > like "force" and "mass" have turned out to be
> > wonderfully useful. Even longer conversations about
> > the definitions of words like "truth" and "good"
> > haven't, as it turns out. So maybe we philosophers
> > should talk about something else?
> >
> > I think the definitions in our field are closer to "truth" than
> > "mass". Maybe that's because, like "truth", they're words wrapped up
> > with people's self-image and identity.
> >
> >
> > -----
> > Brian Marick, independent consultant
> > Mostly on agile methods with a testing slant
> > Author of _Programming Cocoa with Ruby_
> > www.exampler.com, www.exampler.com/blog, www.twitter.com/marick
> >
> >
> >
>
>
>
> --
> Lisa Crispin
> Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers
> and Agile Teams_ (Addison-Wesley 2009)
> http://lisacrispin.com
>

On Nov 12, 2009, at 6:32 PM, Lisa Crispin wrote:

> I worry that terminology might be used to keep newbies on the
> outside of a profession. If they don't learn our secret handshakes,
> we won't talk to them.

The thing to do when someone offers you a secret handshake is to say,
"Oh. Um. That one. I'm afraid I've forgotten quite how that goes,
because of, you know, the new one. Can you talk me through it?"

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
Author of _Programming Cocoa with Ruby_
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick

On Nov 12, 2009, at 4:02 AM, erik petersen wrote:

> Hi Brian,
> OK, how about this alternate explanation. [...]  Does that make more
> sense?


I don't know. How will the world be meaningfully different if we talk
like this versus old-fashioned lingo? What does this buy us over
longer strings of words like "exploratory testing: needed, no matter
how enthusiastic you are about junit, newly-test-infected programmer-
type" and "scripted manual tests: ick!" and "talking about examples of
executing the app, using business language, before writing code:
helpful!" and "rerunning tests frequently: more helpful than Marick
thought in 1997, but still risky as the tests come to execute more and
more lines of code".

The philosopher Richard Rorty said (roughly) in the introduction to /
Consequences of Pragmatism/:

	 Long conversations about the definitions of words
	 like "force" and "mass" have turned out to be
	 wonderfully useful. Even longer conversations about
	 the definitions of words like "truth" and "good"
	 haven't, as it turns out. So maybe we philosophers
	 should talk about something else?

I think the definitions in our field are closer to "truth" than
"mass". Maybe that's because, like "truth", they're words wrapped up
with people's self-image and identity.



-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
Author of _Programming Cocoa with Ruby_
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick

I love JMeter -- haven't used EC2 yet, but have heard positive things
from "early adopters".  That being said, I've not found a tool to date
that I'd classify as a "one size fits most", let alone a "one size fits
all".  Even the best, most versatile tools range from complicated to
impossible based on what you are trying to accomplish vs. what it supports.

It's my opinion/experience that someone who is a good system's thinker,
with an adequate understanding of the technologies involved (system
under test and load generation system), and a good grasp of what they
are trying to accomplish with their test (i.e. the hypothesis they are
trying to demonstrate/confirm/refute/enhance and/or the theory they are
trying to disprove) can do so rather indifferently with any *viable*
tool.  Conversely, it's my experience that folks who are missing one of
those components often seek a tool fill that gap -- and sometimes are
successful, but almost always end up becoming reliant on finding a tool
to fill that gap instead of learning how to fill it with knowledge and
thus make the tool incidental.

Think carpentry.  My father (a retired industrial arts teacher) can use
pretty much any hammer for pretty much any hammering job -- because he
has the knowledge, skill, strength, and practice to do so.  I, on the
other hand, pretty much need to use "the right hammer for the job"
because I am less knowledgeable, skilled, and practiced at carpentry
than he -- which is probably why dad has 3 hammers & why I have 8.  ;)

--

Scott Barber
Vice President & Executive Director, Association for Software Testing
www.associationforsoftwaretesting.org
executive.director@...

President & Chief Technologist, PerfTestPlus, Inc.
www.perftestplus.com
sbarber@...

"If you can see it in your mind...
      you will find it in your life."

Earlier I claimed that having to rewrite tests because of UI changes
was nevertheless worthwhile because the tests were a checklist of
things I had to continue to make true. Here are some details:
http://bit.ly/b2942

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
Author of _Programming Cocoa with Ruby_
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick

----------------------------------------------------------
http://www.linkedin.com/in/adampknight
http://a-sisyphean-task.blogspot.com
----------------------------------------------------------

--- In agile-testing@yahoogroups.com, "Matthew" <matt.heusser@...> wrote:
>
>
>
> --- In agile-testing@yahoogroups.com, Hubert Matthews hubert@ wrote:
> >One of the things that has stopped me so far
> >is that testing tools in non-native languages
> >(Java, Python, etc) are too slow for our system
> > - they cannot saturate the target systems as
> >the CPU usage is too high per request.
>
> My box tends to tap out at around 20-40 simulataneous users, depending on if the bottleneck is CPU or I/O (bandwidth.) That's why I'm looking at the could - to distribute the load.
>
> --heusser
>

--- In agile-testing@yahoogroups.com, Hubert Matthews <hubert@...> wrote:
>One of the things that has stopped me so far
>is that testing tools in non-native languages
>(Java, Python, etc) are too slow for our system
> - they cannot saturate the target systems as
>the CPU usage is too high per request.

My box tends to tap out at around 20-40 simulataneous users, depending on if the
bottleneck is CPU or I/O (bandwidth.)  That's why I'm looking at the could - to
distribute the load.

--heusser

Matthew wrote:
> Has anyone used Amazon's EC2, or another cloud-based
> infrastructure, to do load testing?  If yes, what did you use it
> with? (JMeter?)  How did it turn out?

I have been looking at using clouds for testing our distributed system
but have yet to do so.  One of the things that has stopped me so far
is that testing tools in non-native languages (Java, Python, etc) are
too slow for our system - they cannot saturate the target systems as
the CPU usage is too high per request.  You will need to understand
whether your constraint is CPU, network bandwidth or latency before
deciding if clouds are the way to go.  You may also find that you get
unreproducible results because of the limited shared network.

The summary (in true consultant speak): It depends.

--
Hubert Matthews         http://www.oxyware.com/
Software Consultant     hubert@...

My company, Socialtext, offers wiki wednesdays, something like googles 20% time.
Today I am experiementing with some load testing alternatives.

Has anyone used Amazon's EC2, or another cloud-based infrastructure, to do load
testing?  If yes, what did you use it with? (JMeter?)  How did it turn out?

regards,


--heusser

Steve-
I think pairing is a great idea. The only problem I can see with it is that our
arcane office setup is going to make it difficult. Still, I could probably make
it work. Thanks!

G



> Gregg,
>
> Have you tried teaching by pairing. In other words, do the work yourself,
> but do it with one of the team members actively participating.  Then next
> time do it with another team member.  Continue until they get it and they
> can start doing the work themselves (perhaps as pairs, too).
>
> Steve

> just read this -- thanks!
>
> Isn't his 'testing' simply 'exploratory testing'?
>

My understanding is that Bach wanted to put more emphasis on exploratory
testing as a task that testers should be dealing with and that
"checking" is something best delegated to a machine. From that
perspective, it makes sense to try to differentiate between the two.

--
gojko adzic
http://gojko.net

On Mon, Nov 09, 2009 at 10:57:38PM -0600, Brian Marick wrote:
>
> On Nov 9, 2009, at 9:30 PM, erik petersen wrote:
>
> > It helps to frame the conversation.  Tests are attempts to
> > investigate software and typically find new bugs; checks are often
> > automated and check that bugs have not been created in code that was
> > working.
>
> That absolutely does not capture what "checks" are. Let me give you an
> example. I'm working on an application that's used to schedule animals
> for teaching labs. An early version of the app had a drag and drop
> interface with three panels. I later changed it to a clicking
> interface with four panels. I had to rewrite the UI code and the tests/
> checks of the UI code. It might seem the checks were worthless during
> the rewrite (and in 1997, I would have thought so). But it turns out
> that all (or almost all) of those checks contained an idea that needed
> to be preserved in the new interface. They were reminders of important
> things to think about, and that (it seemed to me) repaid the cost of
> rewriting them.
>
> That was a big surprise to me.
>
> The, um, ecosystem of programmer tests is subtle and complicated.
> Simple distinctions like "tests" and "checks" don't cut it.

Exactly.  I feel like attempts at precise definitions are Procrustean in nature
[0].

If the discussion highlights an interesting distinction that helps people
gain understanding, great!  But I think that should be the benchmark for
such attempts, not whether or not the definition is useful (because generally
it's not, in the long run).

--titus

[0] http://en.wikipedia.org/wiki/Procrustes -- hey, if YOU'D paid for a liberal
arts education, wouldn't you want to flaunt it?!
--
C. Titus Brown, ctb@...

On Nov 10, 2009, at 7:32 AM, Hubert Matthews wrote:

> What you probably did was put the checking of
> business rules into your UI checking code, thus conflating UI checking
> with acceptance checking. Refactoring the rules into non-UI code
> that is
> tested separately from a wafer-thin UI would be the ideal solution.

No, that is not what I did. It's a model-view-presenter architecture.
The presenters were tested. Changing the controls the presenters
control meant changing the tests. That was not busy-work, though,
because the ideas behind the tests served as a checklist that guided
the code rewrite.

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
Author of _Programming Cocoa with Ruby_
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick

I am in a new job with a new function: I have to find a way to load test /
performance test a new WPF application (with dependency injections though the
MVVM pattern) done through agile-like development. BTW, agile development is
also new to me.

We got a build-server building with MSBuild and we are using Subversion as
source control and Rhino to mock up items in NUnit-tests.

We do _not_ use foundation server (subversion were chosen before I started in
the project). I really do not want to do load testing through foundation server.
That rules out the microsoft tool for load testing, doesn't it?

I was thinking of writing a series of unit tests for the ViewModels' functions
and the organizing the load test around those. The model already has unittests
to test read and updates of single objects so this will  - in a way - be
duplicate tests of communicating with the db in those may instances where the
viewmodel just shows what's in the database . Would that be the right way to do
it?

I've shortly read about Grinder but am in doubt if that is the tool I want. 
Does anyone have any advice for me as for which tool to use?

Thanks,
Inger Marie

What a great thread! As our team's designated test automation engineer, I have
found it difficult to "hand-off" my automated test frameworks to team members
with no development experience. Part of it is my problem, however. I am simply
not a very good teacher. I also fear that as soon as I hand it off, I will have
to spend more time than I would like supporting the framework with new tests and
coverage suffering as a consequence. Still, perhaps having to deal with that
would motivate me to become a better teacher.

Gregg



--- In agile-testing@yahoogroups.com, Scott Schimanski <scott.schimanski@...>
wrote:
>
> I am starting to agree that testers in an agile environment need a technical
> background.  They at least need to know some of the basics.  Otherwise, they
> tend to be stuck in the "over the wall" scenario where they wait for the
> developer to hand them something to test.  They don't become fully "part of
> the team".  This is obviously based on my limited experience.
>
> Does anyone have experience with non-code writing testers on an agile team?
>
> Thanks,
> Scott
>
> On Tue, Nov 3, 2009 at 5:11 PM, Janet Gregory <janet_gregory@...> wrote:
>
> >
> >
> > Agile promotes whole team and blurred roles and test automation as part of
> > everyone's job, so I think over time we will see the trend to have more
> > technical testers. Personally, I like testing way more than I liked
> > programming and really couldn't imagine going back.
> >
> > ~ Janet
> >
> >
>

I am coaching with a large team that is divided into 3 sub-teams for a project.  I have convinced the most junior team to work in this way and they are completing (dev, tested, customer approved) double the stories as the most senior team.    The most senior team works as a set of individuals and only want to perform ‘their role’ saying it only makes since that they do what they are best at.   However, the evidence shows they are very wrong.   We are starting to show them the numbers which show that over 3 months only 1 team (the one working very closely with dev, test, and customer to complete stories) has produced value on a consistent basis. 

Wes

Brian Marick wrote:
> That absolutely does not capture what "checks" are. Let me give you an
> example. I'm working on an application that's used to schedule animals
> for teaching labs. An early version of the app had a drag and drop
> interface with three panels. I later changed it to a clicking
> interface with four panels. I had to rewrite the UI code and the tests/
> checks of the UI code. It might seem the checks were worthless during
> the rewrite (and in 1997, I would have thought so). But it turns out
> that all (or almost all) of those checks contained an idea that needed
> to be preserved in the new interface. They were reminders of important
> things to think about, and that (it seemed to me) repaid the cost of
> rewriting them.
>
> That was a big surprise to me.

I'm not surprised by this at all.  When modelling a system I consider
types, operations and business rules.  People usually focus on the first
of these (types/classes/data), then take a good stab at the second
(operations), followed by some grudging attempt at the third
(rules/constraints).  What you probably did was put the checking of
business rules into your UI checking code, thus conflating UI checking
with acceptance checking.  Refactoring the rules into non-UI code that is
tested separately from a wafer-thin UI would be the ideal solution.

--
Hubert Matthews         http://www.oxyware.com/
Software Consultant     hubert@...

On Nov 9, 2009, at 9:30 PM, erik petersen wrote:

> It helps to frame the conversation.  Tests are attempts to
> investigate software and typically find new bugs; checks are often
> automated and check that bugs have not been created in code that was
> working.

That absolutely does not capture what "checks" are. Let me give you an
example. I'm working on an application that's used to schedule animals
for teaching labs. An early version of the app had a drag and drop
interface with three panels. I later changed it to a clicking
interface with four panels. I had to rewrite the UI code and the tests/
checks of the UI code. It might seem the checks were worthless during
the rewrite (and in 1997, I would have thought so). But it turns out
that all (or almost all) of those checks contained an idea that needed
to be preserved in the new interface. They were reminders of important
things to think about, and that (it seemed to me) repaid the cost of
rewriting them.

That was a big surprise to me.

The, um, ecosystem of programmer tests is subtle and complicated.
Simple distinctions like "tests" and "checks" don't cut it.

-----
Brian Marick, independent consultant
Mostly on agile methods with a testing slant
Author of _Programming Cocoa with Ruby_
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick

On Fri, Nov 06, 2009 at 11:21:03AM +0000, Gojko Adzic wrote:
> James Bach talked today at Oredev, i especially liked the part on
> differences between testing and checking. here's a short write-up:
>
> http://gojko.net/2009/11/06/checking-is-not-testing-testing-is-not-checking/

Hi Gojko,

just read this -- thanks!

Isn't his 'testing' simply 'exploratory testing'?

(I always have trouble figuring out whether such attempts to redefine
terminology are serious; they seem like a waste of time to me.   There will
always be people who use the right words in the right way but don't have a
deeper understanding, and there will always be people who use the wrong words
but do have a deeper understanding.  I aspire to have a deeper understanding &
I could care less what things are called; the understanding is the point.)

--titus
--
C. Titus Brown, ctb@...

------------------