Can anybody give me their estimates of a comparison for tests running times
against Selenium RC vs. Selenium WebDriver ?

E.g.  If 150 UAT tests take 4 minutes using RobotFramework (Selenium RC), what
improvement can I expect using:
a) GEB or SpecFlow calling Selenium WebDriver (Firefox)
b) GEB or SpecFlow calling Selenium WebDriver (HtmlUnit)

This is an Asp.Net MVC website I'm trying to test.

PS: Any or your wild guesses are better than my complete ignorance, provided
they're based on experience.

Mark Pawelek wrote:
> Can anybody give me their estimates of a comparison for tests running
> times against Selenium RC vs. Selenium WebDriver ?
>
> E.g.  If 150 UAT tests take 4 minutes using RobotFramework (Selenium RC),
> what improvement can I expect using:
> a) GEB or SpecFlow calling Selenium WebDriver (Firefox)
> b) GEB or SpecFlow calling Selenium WebDriver (HtmlUnit)
>
> This is an Asp.Net MVC website I'm trying to test.
>
> PS: Any or your wild guesses are better than my complete ignorance,
> provided they're based on experience.

You probably need to know that IE is very much slower than Firefox when
running Selenium test by about an order of magnitude, based on one of my
client's experience.

--
Hubert Matthews         http://www.oxyware.com/
Software Consultant     hubert@...

I am the Manager of Product Quality at Ingenium Technologies, a mid-sized
software development organization in Vancouver, BC Canada. We are looking to
expand our development team with one more test automation developer.

Details are below and you can apply at our website
http://www.ingenium-tech.com/html/careers.html

Software Developer: Test Automation

Responsibilities:
     * Design, develop, implement and execute tests for various applications
     * Develop and enhance test tools and support their use by the development
team
     * Collaborate with the product owner and development team on software design
to increase product testability and support more extensive test automation
     * Contribute to our continuous improvement process
     * Champion product issues with product owners and developers to help improve
customer experience

Requirements:
     * Exceptional communication skills: providing feedback to your team,
assertiveness to stand up for quality
     * Inquisitive, analytical, creative, technical and fun to work with
     * Experience developing in C#(preferred) or VB.Net
     * Browser automation experience: Selenium/WebDriver or WatiN/WatiR
     * A hands-on tester who is comfortable across a whole range of exploratory
and functional testing (both manual and automated)
     * Enjoy working closely with developers and clients in a highly
collaborative environment
     * Able to build strong relationships with developers, client services & our
users
     * Excellent problem solving skills
     * Experience participating in the full SDLC from requirements gathering to
deployment
     * Experience on teams using agile development processes (Scrum, XP, etc)
     * Track-record of innovative approaches to software QA
     * Experience with relational databases (MS SQL 2005/2008 preferred)
     * Experience with BDD, executable specifications or automated acceptance
testing
     * Experience creating test frameworks & strategies
     * Computing science degree or equivalent experience

I would support Lisa's post - and suggest you read her book!

I also had the same thoughts and anxieties moving to Agile, and it is
understandable. The key points to raise in favour of Agile for testers is that
User Stories are easier to understand, breaking things down into a short
statement that business and testers can relate to rather than a bis document
thats out of date before its finished. Also the fact that everyone is
responsible for quality, not just the testers, so we get away from having code
thrown over the wall to us riddled with bugs.

What I would say is that it sounds odd that there is a separate QA test stream
at the end of the iterations, so maybe the concept of regression testing in
sprint is being missed, or possibly there are concerns about integration tests.
Either way, testing shouldnt happen again in a sprint of its own.

It seems as though some people are concerned about having to do their jobs
differently, feel a little threatened by this and possibly see a loss of control
over their daily workload. Having a team come in to help (such as Thoughtworks
or Valtech) can really make a difference, as they use a blended team approach in
order to pass on their best practices.

There is no magic wand, you just have to find a way to sell the benefits and
help iron out any problems.

Let us know how you get on,

Steve Watson

--- In agile-testing@yahoogroups.com, Rick Webster <rickw1206@...> wrote:
>
> I had this problem as well in a previous job.  What we ended up doing was
> having a two week sprint after dev was done for regression and bug fixes.
>  We then introduced the idea of a "targeted regression"; one that only
> tested the areas that were touched during previous sprints.
>
> It was a little dangerous at first, but started working out to where we did
> it on a quarterly basis after the first few times.
>

Hi, Todd

I'm sure this overview will be helpful, especially the chart.

One omission: there's no mention of the Product Owner role in this process. But
surely one of the key benefits of ATDD is the opportunity for more effective
collaboration with PO to clarify acceptance criteria.

The "Test Cycle" is a red flag. As you probably understand, this is something
you really want to avoid. A common breakdown occurs when most the integration
testing piles up at the end and then gets skipped. For testers especially, you
ought to be selling one of the key benefits of ATDD -- doing test thinking and
test execution earlier and more often and avoiding the bogus "test bottleneck"
at the end of the sprint.

It sounds like your job is not really explanation -- it's persuasion. In which
case, you need to find your audience's pain and show them how ATDD will make a
difference.

Regards,
Kerry

From the intro blurb...

Proofpoint, a visionary Internet-based Messaging company, is looking for a
world-class Quality Assurance Architect. As we deliver the next generation of
exceptional services to our customers, you will play a critical role in the
various phases--from architecture all the way through to deployment!

http://newton.newtonondemand.com/career/JobIntroduction.action?clientId=4028f88c\
1ff935ca0120011b2ffe0181&id=4028f88b2d94341d012da4d058825d61&specialization=4028\
f88c29254ff8012928541778061e&source=

Todd,

The latest results of the ATDD Patterns worshop are  here:
http://aaftt.agilealliance.org:8080/display/AAFTT/The+Pattern+Language+Project

Hope you find something useful to clarify your situation.

I have also had a challenging time selling ATDD at my company, but we
are slowly making progress. I think the key is to explain all of the
benefits and not worry so much about a prescribed flow. For example in
some cases, devs may not even write unit tests and AT will be the only
automation. As any consultant will tell you - it all depends on the
context.

Regards,
David
http://testdriven.com

Researchers at the Federal University of Lavras (UFLA) would like to invite you
to participate in an online survey about a project aims to analyze the
performance of agile methods and practices, in order to develop a framework
personalized with best practices, based on basic characteristics of the business
segments studied in the project.

To participate in our research please visit the link below:

https://www.surveymonkey.com/s.aspx?sm=3KTyPsPYqp6CQWa_2fSCnpAevzfb9TfIBAFT_2buv\
KIuoCg_3d

The study will last for 10 minutes and you will be entered into a raffle to win
raffle the book "Agile Project Management: Creating Innovative Products (2nd
Edition)" http://amzn.to/fJjaCL. Important: to compete for the book, the
respondent shall inform the name or email survey.

The raffle will be held on Wednesday March 30 and the winner will be notified
and sent the certificate by email.

If you have questions about this survey or the research project, please contact
us by email mariana@....

Thanks in advance for your cooperation.

Best regards,


Mariana de Azevedo Santos (Researcher, CNPq / DCC - UFLA),
Paulo Henrique de Souza Bermejo (Project Supervisor / DCC-UFLA),
Marcelo Silva de Oliveira (Co-supervisor of the project / DEX - UFLA),
Adriano Olimpio Tonelli (Co-supervisor / DCC - UFLA).

How do testers add value to an Agile project? What
needs to happen to realize that value fully? And how
is their work different if the developers use TDD
heavily?

If you have questions like these, you could ask me
this Wednesday on February's "Q & Agile" podcast. Our
topic is technical agility, which is what team members
do in practice to support the rapid cycles.

Sign up here (and ensure that you'll get the recording):

http://www.3pvantage.com/qnagile/opt-in.php?ver=YAT#reg

You'll also be able to ask me a question live at the
end during open mic.

Yours sincerely,
Gil

http://www.3pvantage.com/

Hi Todd,

you may want to take a look at Bridging the Communication gap from Gojko Adzic
or the upcoming Specification by Example. The books cover stories from teams
that successfully implemented ATDD and which approaches they used to get it
working for them. I wrote a blog summary of one of his seminars, which you can
find here:
http://www.shino.de/2011/02/05/specification-by-example-the-big-win/

One of the things I kept in mind - amd nase upon my previous experience - is to
try to get a common understanding of the stories that you're pulling in for the
iteration. You said you use Scrum for this. How do you estimate the stories
currently? How does your product owner prioritize the stories, and when is the
first time your team has a discussion about the underlying implications with
your product owner? I would try to sort out the stories that need further
discussions during estimation, and get a common understanding of the story in
some examples on the back of the story card before the sprint starts. These
examples are then subject to refinement during work on the story within the
sprint or iteration.

But, this is of course based on my narrow view on Scrum, and some assumptions I
made for your team. So, any input how you currently do it could give you a
better answer for you.

Best
Markus Gärtner
http://www.it-agile.de
http://www.shino.de/blog


--- In agile-testing@yahoogroups.com, Todd Alexander <todd_alexander@...> wrote:
>
> Hi,
>
> We've had difficulty at my company getting buy-in from non QA (and even some
> QA!) on the utility of ATDD.  One way that I've tried to explain how dev and
QA
> work together in the process is to correlate the ATDD 'work flow' to the
> standard waterfall approach.  We've 'implemented' Scrum here, but it really
> hasn't got past mini waterfalls and so I'm still trying to describe concepts
in
> that paradigm.  I'd appreciate feedback on the attached outline and 'flow'
> diagram as a method of explaining ATDD to a 'mini waterfall' audience.
>  Thank you,
> Todd Alexander
>

Hi Jean,

which question are you trying to answer by the test coverage?

There is one lightweight approach to make your coverage transparent. Brian
Marick wrote down the idea of the testing quadrants some years back:
http://www.exampler.com/old-blog/2004/05/26/#directions-toc

Lisa and Janet discuss this topic largely in their book as well as in her
classes. Recently I used the approach to make not only coverage but also time
spent in testing transparent to a team I visited in January. I got a write-up on
my blog about it:
http://www.shino.de/2011/01/22/teaching-from-the-back-of-the-room/

Best
Markus Gärtner
http://www.it-agile.de
http://www.shino.de/blog

--- In agile-testing@yahoogroups.com, "Jean Richardson" <jean@...> wrote:
>
> Can anyone shed light on how to calculate or strategize for test coverage in
> a Scrum model?
>
>
>
> n  Jean
>

Hi John,

thank you very much for your compliments! It's a great encouragement to keep
writing in the future!

Best regards,
Andreas

--- In agile-testing@yahoogroups.com, "Smith, John H" <John.Smith2@...> wrote:
>
> Andres,
>
>
>
> Great pack and excellent for training - I am sure this will be used by
> many.
>
>
>
> Well done.
>
> John
>
>
>
> From: agile-testing@yahoogroups.com
> [mailto:agile-testing@yahoogroups.com] On Behalf Of Pepijn van de Vorst
> Sent: 20 February 2011 21:23
> To: agile-testing@yahoogroups.com
> Subject: Re: [agile-testing] My First Blog Article: Photographic
> Software Testing: Lessons Learned from Belgium Testing Days 2011
>
>
>
>
>
> Hi Andres,
>
>
>
> I like your blog that compares photography and testing very much.
>
>
>
> Last half year I've experienced the power of metaphors.
>
> The Photography metaphor is a very powerful one regarding testing.
>
> And the photo's are very beautiful.
>
>
>
> Thank you for adding a new metaphor to my repertoire!
>
>
>
> Regards,
>
> Pepijn
>
> 2011/2/20 Andreas Simon <andreas.simon@...>
>
>
>
> Hello,
>
>
> I'm quite proud of releasing my first blog post yesterday. After the BTD
> 2011 a little parable about testing and photography came to my mind
> which I would like to share with you. I would be happy if some of you
> took the time to have a look at it.
> http://andreas-simon.blogspot.com/2011/02/photographic-testing-lessons-l
> earned.html
> Needless to say that a bit of feedback would be awesome for me.
>
>
> Best regards,
> Andreas
>
> http://andreas-simon.blogspot.com <http://andreas-simon.blogspot.com/>
> https://www.xing.com/profile/Andreas_Simon16
> http://de.linkedin.com/pub/andreas-simon/25/5a7/45a
>
> Twitter: @redspot357
>

--- In agile-testing@yahoogroups.com, "Jean Richardson" <jean@...> wrote:
>
> Can anyone shed light on how to calculate or strategize for test coverage in
> a Scrum model?
>
>
>
> --Jean
>

Hello Jean.  James Bach suggests one way to visualize quality in his
presentation "A low tech testing dashboard" --

http://www.satisfice.com/presentations/dashboard.pdf

We have used some of his ideas with some success at Socialtext.

good luck!

--heusser

Hi Todd,

you seem to be working around the real problem and trying to use ATDD to make a
change. So far, I have not seen this succeed.

How often do you run retrospectives after the Sprint review meetings? I'd expect
problems like handovers and dev/test separation to be topics within the
retrospectives.

How come six months of Scrum adoption regression have gone unnoticed? There is
something wrong about your process, which ATDD can help you with, to some
extent, but I doubt that it provides the extent that you will need.

So, keep me updated on your progress.

Best
Markus Gärtner
http://www.it-agile.de
http://www.shino.de/blog
Twitter: @mgaertne

--- In agile-testing@yahoogroups.com, Todd Alexander <todd_alexander@...> wrote:
>
> Janet,
>
> Thanks for the reply.  My company is in an odd state at this point, regarding
> Scrum adoption.  We had been making progress toward a working Scrum
methodology,
> but the two main roadblocks traditionally have been differing standard across
> the multiple (and unstable) teams and the insistence on the separate of dev
and
> QA tasks into the traditional waterfall perspectives.  Unfortunately, the last
6
> months have seen a regression in Scrum adoption rather than progress.   As
such,
> when I say dev cycle I mean the 'feature complete' or 'story
> complete' hand-off from dev to the QA.  Iterations are held to sprint-level
> timeboxing (on the disciplined teams) rather than story-level handoffs within
a
> sprint.  Given that structure, and a lack of useful automation, it's my
proposal
> that ATDD principles of Acceptance testing at each phase of the project cycle
> offers many of the solutions of ATDD to typical pain points, such as
ambiguity,
> dependencies and team throughput capacity.  I'm looking at a manual,
procedural
> approach to ATDD cycles in the short-term rather than waiting for the
automation
> and continuous build infrastructure to be in place.  As such, I'm using the
> following definitions:
> Acceptance tests - high-level tests to verify the completeness of a user story
> by defining the business value of the story
> Functional tests - verification of the features' behavior outside of the user
> story requirements
> Regression tests - Acceptance-level tests for features previously released and
> expected to remain functional in the current feature set.
>
> Ideally, we could get dev to hand off iteratively as they complete acceptance
> tests in their sandbox, but in the meantime I'm trying to push for acceptance
> tests being part of the 'feature complete', QA complete and Gold regression
> phases' defintions of done.
>
> Thank you,
> Todd Alexander
> 206-913-8764
>
>
>
>
> ________________________________
> From: Janet Gregory <janet_gregory@...>
> To: agile-testing@yahoogroups.com
> Sent: Sun, February 20, 2011 9:28:06 PM
> Subject: RE: [agile-testing] ATDD process correlated to 'standard' project
cycle
>
>
> Todd,
>
> I’m a bit confused with your timeline, but it could be my misunderstanding
of
> your wording. I like your explanation of defining acceptance tests for the
user
> story, but I get confused on the Dev and QA cycles.  When you talk about the
dev
> cycle, do you mean an iteration? Or the whole project. I guess my confusion
> comes in where you say AT’s must pass to declare the project feature
complete.
> I also don’t quite understand the functional tests in the QA cycle.
>
>
> Also, are you automating your acceptance tests so that they can be added to
the
> daily regression test run?
>
>
> Perhaps an alternative timeline could look like this (assuming features have
> multiple stories):
>
>
> ·         Each iteration â€" write AT â€" code â€" write and execute extended
tests,
> including exploratory tests, etc. â€" run AT â€" accept story as done
> ·         For each feature â€" write AT â€" when all stories are done for a
feature,
> run exploratory tests, run AT â€" accept feature as done
>
> ·         For each project â€" run regression tests daily, integrate daily.
> During the end game, run final tests as needed.
>
>
> All tests including load, performance, security, etc. should be run as soon as
> possible for each story, and feature.
>
> Does something like this make sense?
>
>
> Janet Gregory
> Co-author with Lisa Crispin,  Agile Testing: A Practical Guide for Testers and
> Agile Teams, Addison-Wesley 2009
>
> DragonFire Inc.
> www.janetgregory.ca
>
> janet@...
> Twitter: janetgregoryca
>
>
>
>
> From:agile-testing@yahoogroups.com [mailto:agile-testing@yahoogroups.com] On
> Behalf Of Todd Alexander
> Sent: February-18-11 11:43 AM
> To: agile-testing@yahoogroups.com
> Subject: [agile-testing] ATDD process correlated to 'standard' project cycle
[1
> Attachment]
>
>
> [Attachment(s) from Todd Alexander included below]
> Hi,
>
> We've had difficulty at my company getting buy-in from non QA (and even some
> QA!) on the utility of ATDD.  One way that I've tried to explain how dev and
QA
> work together in the process is to correlate the ATDD 'work flow' to the
> standard waterfall approach.  We've 'implemented' Scrum here, but it really
> hasn't got past mini waterfalls and so I'm still trying to describe concepts
in
> that paradigm.  I'd appreciate feedback on the attached outline and 'flow'
> diagram as a method of explaining ATDD to a 'mini waterfall' audience.
>
> Thank you,
> Todd Alexander
>