So if I understand the view you're suggesting … devs "check" (i.e. anything that
can be reduced to a boolean outcome) while testers "test" (i.e. any
non-functional testing + exploratory testing). Reference:
http://www.developsense.com/blog/2009/08/testing-vs-checking/

Is that correct?

--- In aa-ftt@yahoogroups.com, Jeff Morgan <kuzman@...> wrote:
>
> Dave,
>
> You are making a very good point.  Each time you move to a higher
> level language your productivity increases significantly.  That is the
> reason a few of us are trying to suggest a language that is even higher level
and
> simpler than Java.
>

I took your email as push-back against Java /in general/, and suggesting that
test ('check' if you prefer) automation might be the first, least painful place
to start.

If that's what you are saying, Jeff, I think we have substantive agreement! :-)

regards,


--
Matthew Heusser,
Consulting Software Tester/Writer
http://flavors.me/mheusser#150/linkedin

--- In aa-ftt@yahoogroups.com, Jeff Morgan <kuzman@...> wrote:
>I take it a step further and condone ATDD as a practice to
> foster closer collaboration between the testers, developers and product
owners.
> Instead of having the developers write the code and _then_ the testers
> check it to ensure correctness I have the developers and testers collaborate
> while the automation and codebase are being developed (at the same time)
> to try to produce a defect free application.

Sure.  At my current client, we have testers involved with programmers before
the developers write a line of code, defining acceptance criteria in fitnesse. 
That's all pretty standard stuff.

I don't mean to argue with you man; in fact I think we have substantive
agreement.  I suspect you do not mean the words you are typing the same way some
dude did a few years ago that is in the back of my mind that is setting me off,
and email isn't the best medium.  (Terms like "defect free", etc.  You could
talk to Levi Wilson about me. :-)

Let's have a beer and talk about it at test coach camp.

Oh, hey, yeah.  Test Coach Camp.  It's going to be awesome, I hope some of you
folks will consider applying to attend:

http://www.associationforsoftwaretesting.org/conference/cast-2012/test-coach-cam\
p/

regards,


--
Matthew Heusser,
Consulting Software Tester/Writer
http://flavors.me/mheusser#150/linkedin

In my second post on this thread you'll notice that I state that I've already
created a framework for our regression testing using Selenium 2.0 and Java. 
That framework covers a wide range of tests across the essential parts of our
application, and I used Java because it was the one language *I* was familiar
with.  There might be easier languages to use, and I certainly won't rule out
using another language going forward, but I have invested enough time already in
this Java project that I'm not going to go back and re-architect it in another
language.  Not to mention the fact that I'm pretty satisfied with what I have
already created.

The idea on my part was that I would do the heavy lifting of creating the
automated regression tests for all our existing core functionality.  Once I had
completed that task (as best I could) then the whole QA would join me in
automating NEW functionality as it was added, without having to go back and
reinvent the wheel on existing tests.  So say for instance we add a new field on
a page; I've already created tests to interact with all existing fields on that
page, via java methods, so anything new means just writing that new method.

The hope was that with this framework in place the pain of my QA team learning
to code could be smoothed over by having the framework in place.  So the
framework would serve as it's own training tool if you will, plus establish
coding standards in the hopes of keeping it clean.  That said, I know that my
team will need at least some basic Java training to understand objects, classes,
variables, methods...etc before even getting started on my framework.  Hence why
I posted my question in the first place.

Does my QA team want to learn to code?  That is the million dollar question
isn't it.  I have told all of them that they should look at this as great
opportunity since being a QA resource that knows how to code and write
automation is a great skill to have, maybe even essential in the modern QA
world.  It is yet to be determined if they will all embrace this...at this point
some certainly seem more eager then others.

Anyway, long winded response to many of the questions and comments to this
thread I have started.  Thanks everyone for your input and suggestions.



--- In aa-ftt@yahoogroups.com, Charley Baker <charley.baker@...> wrote:
>
> Why does the team need to know Java for their tests. Familiarity is great
> and knowing the basics of programming is fine. I'm slightly biased in
> teaching Ruby to devs, qa and anyone who listens. That's been on Java, .Net
> and other teams.
>
>
> Cheers,
>
> Charley
>

--- In aa-ftt@yahoogroups.com, "g_ziebold" <ggz@...> wrote:
>That is the million dollar question isn't it. I have told all of them
>that they should look at this as great opportunity since being a
>QA resource that knows how to code and write automation is a
>great skill to have,

(off-topic)

When you wrote "resource" above, you meant "person", right?

Just checking.

--heusser

 

What a fantastic thread to close the year with! Lots of good stuff here. I don’t feel comfortable rendering advice to Geoff because I don’t know the context of his shop, but let me share my findings from the last 2.5 years working on “Agile” large enterprise projects.

Here is what I have learned so far:

Before even getting in the arguments of what language to use, make sure your testers can get comfortable with version control. Apparently most have had years of experience with Excel and shared drives, so using version control was a much bigger issue than any of us “programmers” anticipated and we had to backtrack and provide training and support. In the “Lean Startup” parlance this was the first pivot.

Even with a simple framework and coaching testers had problems writing tests from scratch -- even when their resume said Masters in CS! if they wrote tests from scratch the code was not maintainable without severe refactoring -- there is a huge jump in skill from simple procedural programming to proper use of OO.

So the way forward is collaboration and the next pivot is to break the distinction between production and test code in terms of ownership. Developers own the code! Its that simple. If a test breaks developers have to jump on it right away. If the code is difficult to maintain, its developers’ problem. The goal is to the the right number of automated tests that the team needs in order to ship at a level of quality that satisfied business objectives. At the same time developers need to write test code in a way that is accessible to testers so they can take advantage of grow this resource -- the trick is to separate the “what” from the “how”. Actually this is how “clean” code should be written, its just that programmers can muddle through poorly written code and testers usually can’t.

How do we get testers involved in the code? I think that sending them to a typical training course designed for programmers is a bad idea. It can just reinforce their fears of programming. Its much better to start with developer/tester pairing (of course a coach can really help with this part). Lets take a look at a plausible interaction:

David is a senior developer. Lisa is a senior tester. Both are veterans of several enterprise projects. The project today is an insurance policy management system.



David: “Lisa here is the code I wrote to test adding a new car to the policy”


//use Jane and Buick sample data for this test
  PersonalPolicy policy =
    PersonalPolicyBuilder.ownedBy(Jane_Smith() );
  policy.addCar( BUICK_2000().withComprehensiveCoverage()
    .withDeductibleOf( 500 )
    .withMedicalLimit( 10000));
  assertThat( policy.totalPremium(), equals( 765.00));

They run the test using Selenium/WebDriver/IE and all looks good. Lisa asks: “I need to see how the system looks when there is more cars that can fit on one page (only 3 cars can fit on the first page). David: “No problem” David modifies the code by adding 3 more cars to the policy.

..
policy.addCar( Chevy_2010().withComprehensiveCoverage()
    .withDeductibleOf( 500 )
    .withMedicalLimit( 10000));
policy.addCar( Totoya_2010().withComprehensiveCoverage()
    .withDeductibleOf( 500 )
    .withMedicalLimit( 10000));
policy.addCar( Mazda_2010().withComprehensiveCoverage()
    .withDeductibleOf( 500 )
    .withMedicalLimit( 10000));

This time we run the test without the UI for speed because we want to examine the end result. Sure enough when we press the “Next Page” link we get a NullPointerException. Hooray for pair exploratory testing! Lisa checks in the the new test as “addFourCarsToPlocy()”

The pairing session is over. Lisa leaves to file the bug that is now easily reproducible. After the bug is filed, Lisa checks out the latest version of the code and starts playing with the deductible amount. She makes a copy of the previous test and changes the deductible to ‘-500’

policy.addCar( Mazda_2010().withComprehensiveCoverage()
    .withDeductibleOf( -500 )
    .withMedicalLimit( 10000));

Another exception! The system has not been tested for negative deductible. Lisa files another bug with tag NEEDS_UNIT_TEST. Next she runs the working test using Firefox and Chrome....

As you can see from the above example Lisa has been programming in Java to speed up the testing process. She had no problem writing in Java because it has been designed and factored in a way that separates the “what” from the “how”. Lisa is not worried that at this point she is just beginning her journey into Java programming because she feels supported by developers and she can learn programming at an appropriate pace while constantly adding value to the project as a tester.

David is happy with the setup as well. He has great respect for Lisa’s domain knowledge and testing skills. He feels she ‘has his back’ and he can worry about deep technical issues on the project. David like that the tests are written in Java, not because its the best language, but because switching languages is painful, the Java IDEs are world class and he knows how to ‘milk’ their powerful refactoring features to effortlessly refactor the code towards an optimal design.

In conclusion, I will say that the biggest problem I observed (at least in the Bay Area) is the difficulty of hiring testers who can cut it in the enterprise space in terms of becoming recognised experts in the domain and having decent testing skills. If you can find and hire good testers, getting them to collaborate with developers is typically not a problem using the approach I described above whatever the language.

David

http://testdriven.com

On Sat, Dec 31, 2011 at 1:11 AM, Mohinder

What a fantastic thread to close the year with! Lots of good stuff here. I don’t feel comfortable rendering advice to Geoff because I don’t know the context of his shop, but let me share my findings from the last 2.5 years working on “Agile” large enterprise projects.

Here is what I have learned so far:

Before even getting in the arguments of what language to use, make sure your testers can get comfortable with version control. Apparently most have had years of experience with Excel and shared drives, so using version control was a much bigger issue than any of us “programmers” anticipated and we had to backtrack and provide training and support. In the “Lean Startup” parlance this was the first pivot.

Even with a simple framework and coaching testers had problems writing tests from scratch -- even when their resume said Masters in CS! if they wrote tests from scratch the code was not maintainable without severe refactoring -- there is a huge jump in skill from simple procedural programming to proper use of OO.

So the way forward is collaboration and the next pivot is to break the distinction between production and test code in terms of ownership. Developers own the code! Its that simple. If a test breaks developers have to jump on it right away. If the code is difficult to maintain, its developers’ problem. The goal is to the the right number of automated tests that the team needs in order to ship at a level of quality that satisfied business objectives. At the same time developers need to write test code in a way that is accessible to testers so they can take advantage of grow this resource -- the trick is to separate the “what” from the “how”. Actually this is how “clean” code should be written, its just that programmers can muddle through poorly written code and testers usually can’t.

How do we get testers involved in the code? I think that sending them to a typical training course designed for programmers is a bad idea. It can just reinforce their fears of programming. Its much better to start with developer/tester pairing (of course a coach can really help with this part). Lets take a look at a plausible interaction:

David is a senior developer. Lisa is a senior tester. Both are veterans of several enterprise projects. The project today is an insurance policy management system.



David: “Lisa here is the code I wrote to test adding a new car to the policy”


//use Jane and Buick sample data for this test
  PersonalPolicy policy =
    PersonalPolicyBuilder.ownedBy(Jane_Smith() );
  policy.addCar( BUICK_2000().withComprehensiveCoverage()
    .withDeductibleOf( 500 )
    .withMedicalLimit( 10000));
  assertThat( policy.totalPremium(), equals( 765.00));

They run the test using Selenium/WebDriver/IE and all looks good. Lisa asks: “I need to see how the system looks when there is more cars that can fit on one page (only 3 cars can fit on the first page). David: “No problem” David modifies the code by adding 3 more cars to the policy.

..
policy.addCar( Chevy_2010().withComprehensiveCoverage()
    .withDeductibleOf( 500 )
    .withMedicalLimit( 10000));
policy.addCar( Totoya_2010().withComprehensiveCoverage()
    .withDeductibleOf( 500 )
    .withMedicalLimit( 10000));
policy.addCar( Mazda_2010().withComprehensiveCoverage()
    .withDeductibleOf( 500 )
    .withMedicalLimit( 10000));

This time we run the test without the UI for speed because we want to examine the end result. Sure enough when we press the “Next Page” link we get a NullPointerException. Hooray for pair exploratory testing! Lisa checks in the the new test as “addFourCarsToPlocy()”

The pairing session is over. Lisa leaves to file the bug that is now easily reproducible. After the bug is filed, Lisa checks out the latest version of the code and starts playing with the deductible amount. She makes a copy of the previous test and changes the deductible to ‘-500’

policy.addCar( Mazda_2010().withComprehensiveCoverage()
    .withDeductibleOf( -500 )
    .withMedicalLimit( 10000));

Another exception! The system has not been tested for negative deductible. Lisa files another bug with tag NEEDS_UNIT_TEST. Next she runs the working test using Firefox and Chrome....

As you can see from the above example Lisa has been programming in Java to speed up the testing process. She had no problem writing in Java because it has been designed and factored in a way that separates the “what” from the “how”. Lisa is not worried that at this point she is just beginning her journey into Java programming because she feels supported by developers and she can learn programming at an appropriate pace while constantly adding value to the project as a tester.

David is happy with the setup as well. He has great respect for Lisa’s domain knowledge and testing skills. He feels she ‘has his back’ and he can worry about deep technical issues on the project. David like that the tests are written in Java, not because its the best language, but because switching languages is painful, the Java IDEs are world class and he knows how to ‘milk’ their powerful refactoring features to effortlessly refactor the code towards an optimal design.

In conclusion, I will say that the biggest problem I observed (at least in the Bay Area) is the difficulty of hiring testers who can cut it in the enterprise space in terms of becoming recognised experts in the domain and having decent testing skills. If you can find and hire good testers, getting them to collaborate with developers is typically not a problem using the approach I described above whatever the language.

David

http://testdriven.com

On Sat, Dec 31, 2011 at 1:11 AM, Mohinder <mohinder.khosla@...> wrote:

 

If you are looking for inspiration then read the article entitled Teach Tourself Programming in 10 Years http://www.readability.com/read?url=http://norvig.com/21-days.html

 

Thanks

 

With kind regards,

Mohinder Khosla, PhD

--
http://www.testdriven.com
http://twitter.com/vydra

I'm afraid I don't know of one.  My suggestion is you come t test coach camp and
help us build it, tho. :-)

http://www.associationforsoftwaretesting.org/conference/cast-2012/test-coach-cam\
p/


We are organizing the camp itself on a wiki, so, there's that. :-)


--heusser

--- In aa-ftt@yahoogroups.com, Al Snow <jasnow1@...> wrote:
>
>
> Hi,      Do we have a wiki page that shows our favorite Test Coaching
resources (books, podcasts, workshops, conferences, etc)?If not appropriate
here, where?
> Thanks,Al Snow
> Happy New Year
>

Hi Everyone,

We have a web based product built in combination of C# and AJAX. I am having a
cross browser testing dilemma right now. I just joined this company and before i
joined they used to test the website on Ie9,8,7,6 and Firefox then safari then
iPad all separately. I can understand testing separately on iPad. However to
repeat your tests on different browser makes no sense to me.
I am looking for either a tool or a different approach to test this. I looked
browserlab.adobe.com but the problem is its not interactive that is it shows you
screen only. Ideally there would be some tool where you can test; and see your
screen in different browsers simultaneously(perhaps too far fetched)

Any ideas? as i am pretty sure someone must have done it in the past.

thanks

Hassan

--- In aa-ftt@yahoogroups.com, "hassanmurtaza10" <hassan.murtaza@...> wrote:
>
> Hi Everyone,
>
> We have a web based product built in combination of C# and AJAX. I am having a
cross browser testing dilemma right now. I just joined this company and before i
joined they used to test the website on Ie9,8,7,6 and Firefox then safari then
iPad all separately. I can understand testing separately on iPad. However to
repeat your tests on different browser makes no sense to me.
> I am looking for either a tool or a different approach to test this. I looked
browserlab.adobe.com but the problem is its not interactive that is it shows you
screen only. Ideally there would be some tool where you can test; and see your
screen in different browsers simultaneously(perhaps too far fetched)
>
> Any ideas? as i am pretty sure someone must have done it in the past.

We deal with a couple of tools for this permutation explosion problem:
Testswarm  is good for testing JavaScript in multiple browsers.
Another tool is written by a colleage is Terminus:
http://terminus.jcoglan.com/
Which sounds similar to what you are looking for. This is run through Ruby and
Capybara.

HTH
Joseph Wilk
http://blog.josephwilk.net
>
> thanks
>
> Hassan
>

--- In aa-ftt@yahoogroups.com, "thebouncyjoe" <joe@...> wrote:
>
> --- In aa-ftt@yahoogroups.com, "hassanmurtaza10" <hassan.murtaza@> wrote:
> >
> > Hi Everyone,
> >
> > We have a web based product built in combination of C# and AJAX. I am having
a cross browser testing dilemma right now. I just joined this company and before
i joined they used to test the website on Ie9,8,7,6 and Firefox then safari then
iPad all separately. I can understand testing separately on iPad. However to
repeat your tests on different browser makes no sense to me.
> > I am looking for either a tool or a different approach to test this. I
looked browserlab.adobe.com but the problem is its not interactive that is it
shows you screen only. Ideally there would be some tool where you can test; and
see your screen in different browsers simultaneously(perhaps too far fetched)
> >
> > Any ideas? as i am pretty sure someone must have done it in the past.
>
> We deal with a couple of tools for this permutation explosion problem:
> Testswarm  is good for testing JavaScript in multiple browsers.
> Another tool is written by a colleage is Terminus:
> http://terminus.jcoglan.com/
> Which sounds similar to what you are looking for. This is run through Ruby and
Capybara.
>


There is also Selenium Grid that google uses for multi browser testing:

http://selenium-grid.seleniumhq.org/

> HTH
> Joseph Wilk
> http://blog.josephwilk.net
> >
> > thanks
> >
> > Hassan
> >
>

It sounds like all the responses were about tools to automate the tests. But if
I understood you correctly, you want to run the test manually in one browser
have several other browsers execute the exact same test at the same time so you
can eyeball any differences. Is this a correct interpretation?

If so, is anyone aware of any tools that would do this?

Gerard

On 1/24/2012 9:13 PM, hassanmurtaza10 wrote:
> Hi Everyone,
>
> We have a web based product built in combination of C# and AJAX. I am having a
> cross browser testing dilemma right now. I just joined this company and before
i
> joined they used to test the website on Ie9,8,7,6 and Firefox then safari then
> iPad all separately. I can understand testing separately on iPad. However to
> repeat your tests on different browser makes no sense to me.
> I am looking for either a tool or a different approach to test this. I looked
> browserlab.adobe.com but the problem is its not interactive that is it shows
you
> screen only. Ideally there would be some tool where you can test; and see your
> screen in different browsers simultaneously(perhaps too far fetched)
>
> Any ideas? as i am pretty sure someone must have done it in the past.
>
> thanks
>
> Hassan
>
>

--
Gerard Meszaros
Lean/Agile Coach/Mentor/Trainer
http://www.gerardmeszaros.com
1-403-827-2967

Author of the Jolt Productivity Award winning book "xUnit Test Patterns -
Refactoring Test Code" and winner of the "Programming with the Stars"
competition at Agile 2009. Learn more at http://xunitpatterns.com/index.html