I got it.  Thanks!`

On Thu, Sep 8, 2011 at 5:08 PM, sschroev <stephan202@...> wrote:
> David,
>
> I just sent you an email on the address I found on your homepage. It contains
an attachment with a maven project that fails if the JUnit JAR is shipped with
hamcrest-core-1.1 classes. It's not optimal, but it's a start, I guess.
>
> Regards,
> Stephan
>
> --- In junit@yahoogroups.com, David Saff <david@...> wrote:
>>
>> Not really that easy.  In this case, the bug was that I was creating a
>> jar named junit-dep-XXX.jar, with the correct contents, but due to an
>> obscurity in ant, junit-XXX.jar was being uploaded to maven under the
>> artifact id junit-dep.  So I'd like to automatically make sure that
>> what's in the central repo has the right contents.
>>
>>    David Saff
>>
>> On Thu, Sep 8, 2011 at 3:43 PM, Malte Finsterwalder
>> <malte@...> wrote:
>> > Maybe just look into the jar and see th?
>> >
>> > On 8 September 2011 19:51, David Saff <david@...> wrote:
>> >> I'm trying to write a test so this doesn't happen again.  Stephan, or
>> >> anyone else, do you have an example of something that fails when
>> >> junit-dep ships with the hamcrest classes, and passes when the jar
>> >> contents are correct?  Thanks,
>> >>
>> >>   David Saff
>> >>
>> >> On Thu, Aug 25, 2011 at 10:45 AM, sschroev <stephan202@...> wrote:
>> >>> Hi David,
>> >>>
>> >>> W.r.t. your last question: as far as I could tell the beta versions were
indeed not available in Maven central; we had to manually deploy them to our
corporate Nexus instance.
>> >>>
>> >>> Anyway, I just updated to JUnit 4.9 and noticed that the maven JAR
contains org.hamcrest.* classes, similar to the issue described in
http://jira.codehaus.org/browse/MAVENUPLOAD-1651.
>> >>>
>> >>> This is not the end of the world, but it requires people that want to use
a more recent version of Hamcrest (e.g. 1.2.1) to declare the Hamcrest
dependency in front of JUnit; not very nice. I see that you did define the
hamcrest-core dependency in the POM, but since the class files are also shipped
this has no effect.
>> >>>
>> >>> I wonder: is this bundling on purpose, and if not, will it be corrected?
>> >>>
>> >>> Kind regards,
>> >>> Stephan
>> >>>
>> >>>
>> >>>
>> >>> --- In junit@yahoogroups.com, David Saff <david@> wrote:
>> >>>>
>> >>>> So, one minute--does this mean that the beta staging repositories that
>> >>>> I closed, but didn't release, were not available to users for beta
>> >>>> testing?
>> >>>>
>> >>>>    David Saff
>> >>>>
>> >>>> On Wed, Aug 24, 2011 at 11:40 AM, Max Bowsher <maxb@> wrote:
>> >>>> > Yes, that looks fine.
>> >>>> >
>> >>>> > You probably also want to either promote or drop - probably drop - the
>> >>>> > remaining active staging repositories containing 4.9 beta releases
from
>> >>>> > oss.sonatype.org.
>> >>>> >
>> >>>> > Max.
>> >>>> >
>> >>>> > On 24/08/11 12:35, David Saff wrote:
>> >>>> >> I just had to click the button, it turns out.  It should now be
>> >>>> >> promoted.  Please let me know if it looks that way to you (since I'm
>> >>>> >> still, obviously, new at this).
>> >>>> >>
>> >>>> >>    David Saff
>> >>>> >>
>> >>>> >> On Wed, Aug 24, 2011 at 7:29 AM, Max Bowsher <maxb@> wrote:
>> >>>> >>> On 23/08/11 16:44, David Saff wrote:
>> >>>> >>>> Hello, everyone.  We're happy to announce the release of JUnit 4.9.
>> >>>> >>>
>> >>>> >>> Great!
>> >>>> >>>
>> >>>> >>> I see you've already staged it at oss.sonatype.org - what still
needs to
>> >>>> >>> happen before that staging repository gets promoted into Maven
Central?
>> >>>> >>>
>> >>>> >>> Max.
>> >>>> >>>
>> >>>> >>>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> ------------------------------------
>> >>>
>> >>> Yahoo! Groups Links
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >> ------------------------------------
>> >>
>> >> Yahoo! Groups Links
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

mdankan,

For JUnit 3.x, there was the Cook's Tour:
http://junit.sourceforge.net/doc/cookstour/cookstour.htm

For JUnit 4.x, no such document exists.  I'd love to write one, but
I'm not sure if it's high enough priority to put managing the code on
hold...

    David

On Fri, Aug 26, 2011 at 4:34 AM, mdankan <mdankan@...> wrote:
> Hi Everyone,
>
> I have a simple question:
> I have downloaded the source of JUnit and I would love to learn how the
structure was designed and how the execution works.
>
> I tried to read all the code but sometimes I get confused and frustrated,
therefore here is my question:
> Where can I find documentation about the *structure* of Junit?
>
> Thank you all for your time
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

All,

To date, we've kept two pieces of documentary metadata along with
JUnit: acknowledgements.txt and ReleaseNotes files.  In order:

1) acknowledgements.txt is an attempt at a log of everyone outside of
the primary maintainers who has contributed in some way to JUnit.  It
was, I believe, originally intended to serve two purposes: (a) for
traceability, should any legal questions arise about the provenance of
a piece of code, (b) before github, we often ended up massaging (or
replacing) patches ourselves before committing them to CVS, and we
wanted a way to genuinely thank and remember the original authors.

GitHub now helps with both of these issues: since we can discuss pull
requests before they're accepted, we can ask contributors to fix minor
issues, and github keeps excellent track of what came from where.  In
addition, distributed maintenance of the acknowledgements.txt file has
become a pain, since as a linear document, there is often a race
condition between appended items.

2) I've been asking committers to include updates to ReleaseNotes
files as part of check-ins.  This has some of the same issues as
above: it's no longer strictly necessary given GitHub support, and
also leads to race conditions.

So, my proposal is:

1) Drop further appends to acknowedgements.txt
2) Adopt a more formal approach to git commit messages, following
along http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html,
along release notes to be largely auto-generated at release time.

Thoughts?

David Saff wrote on 9.9.2011 23:26:
> 1) Drop further appends to acknowedgements.txt
> 2) Adopt a more formal approach to git commit messages, following
> along http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html,
> along release notes to be largely auto-generated at release time.
>
> Thoughts?

Good idea. That would reduce the work of both contributors and the one
who makes the release.

Auto-generating release notes would probably require some manual
filtering, because the addition of one feature easily comprises of
multiple commits. Or then there would need to be a very formal approach.
Some options:
1. Each new feature must be rebased into one commit, whose message is
used in release notes.
2. Each feature is developed in a topic branch and merged with the
--no-ff option, and then the merge's commit message is used for release
notes.

--
Esko Luontola
www.orfjackal.net

+1 for both

--- In junit@yahoogroups.com, David Saff <david@...> wrote:
>
> All,
>
> To date, we've kept two pieces of documentary metadata along with
> JUnit: acknowledgements.txt and ReleaseNotes files.  In order:
>
> 1) acknowledgements.txt is an attempt at a log of everyone outside of
> the primary maintainers who has contributed in some way to JUnit.  It
> was, I believe, originally intended to serve two purposes: (a) for
> traceability, should any legal questions arise about the provenance of
> a piece of code, (b) before github, we often ended up massaging (or
> replacing) patches ourselves before committing them to CVS, and we
> wanted a way to genuinely thank and remember the original authors.
>
> GitHub now helps with both of these issues: since we can discuss pull
> requests before they're accepted, we can ask contributors to fix minor
> issues, and github keeps excellent track of what came from where.  In
> addition, distributed maintenance of the acknowledgements.txt file has
> become a pain, since as a linear document, there is often a race
> condition between appended items.
>
> 2) I've been asking committers to include updates to ReleaseNotes
> files as part of check-ins.  This has some of the same issues as
> above: it's no longer strictly necessary given GitHub support, and
> also leads to race conditions.
>
> So, my proposal is:
>
> 1) Drop further appends to acknowedgements.txt
> 2) Adopt a more formal approach to git commit messages, following
> along http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html,
> along release notes to be largely auto-generated at release time.
>
> Thoughts?
>

Indeed.

For those not initiated, here's an example of using Github to host a
whatever static content you like

- http://static.fuzzydb.org/site/ is contained in:
https://github.com/whirlwind-match/whirlwind-match.github.com

So as well as that Maven generated info, there is other static content
such as a public XSD I want available.


  From past emails, I think this is how junit.org is being hosted too.







On 09/09/2011 13:09, jens_schauder wrote:
> +1
>
> I think junit.org should be that site
>
> it should have links to 'other resources' clearly definining what these
resources are good for (eg. source code repository ...)
>
> All these other resources should link back in a very prominent way to
junit.org. Something like<if you aren't looking for the SOURCECODE REPOSITORY,
you'll find more information at the main site junit.org
>
> Jens
>
> --- In junit@yahoogroups.com, "Stefan Birkner"<mail@...>  wrote:
>> The information about JUnit is across three different places: junit.org,
junit.sourceforge.net and github. Additionally some information is missing or
very hard to find (new features like rules, current version, how to contribute,
...)
>>
>> I would like to have one site with all the information. This makes it easier
to find the information and JUnit would look more professional. IMHO the github
pages are a good place for the JUnit documentation, because we can use all the
git(hub) features for creating the documentation.
>>
>> What do you think?
>>
>> Stefan Birkner
>>
>
>
>

On Fri, Sep 9, 2011 at 8:33 AM, Neale Upstone
<neale.upstone@...> wrote:
> Indeed.
>
> For those not initiated, here's an example of using Github to host a
> whatever static content you like
>
> - http://static.fuzzydb.org/site/ is contained in:
> https://github.com/whirlwind-match/whirlwind-match.github.com
>
> So as well as that Maven generated info, there is other static content
> such as a public XSD I want available.
>
>
>  From past emails, I think this is how junit.org is being hosted too.

That would be news to me!

    David

>
>
>
>
>
>
>
> On 09/09/2011 13:09, jens_schauder wrote:
>> +1
>>
>> I think junit.org should be that site
>>
>> it should have links to 'other resources' clearly definining what these
resources are good for (eg. source code repository ...)
>>
>> All these other resources should link back in a very prominent way to
junit.org. Something like<if you aren't looking for the SOURCECODE REPOSITORY,
you'll find more information at the main site junit.org
>>
>> Jens
>>
>> --- In junit@yahoogroups.com, "Stefan Birkner"<mail@...>  wrote:
>>> The information about JUnit is across three different places: junit.org,
junit.sourceforge.net and github. Additionally some information is missing or
very hard to find (new features like rules, current version, how to contribute,
...)
>>>
>>> I would like to have one site with all the information. This makes it easier
to find the information and JUnit would look more professional. IMHO the github
pages are a good place for the JUnit documentation, because we can use all the
git(hub) features for creating the documentation.
>>>
>>> What do you think?
>>>
>>> Stefan Birkner
>>>
>>
>>
>>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

On Fri, Sep 9, 2011 at 9:32 AM, David Saff <david@...> wrote:
> Yes.  This is a pain for me, as well.
>
> Sourceforge.net is purely vestigial.  The moment there's a great,
> authoritative master location to link to, I'll do what it takes to
> kill sourceforge and point it there.
>
> junit.org is an oddity.  For reasons that predate me and I only dimly
> understand, junit.org was registered back in 2000 by Object Mentor.
> To my knowledge, it has never been actively maintained by any of the
> active maintainers of the JUnit source.  The site currently there
> (developed by Ben Rady and a merry crew roughly four years ago) is one
> of the better incarnations of the site.  However, its design
> presupposed that the junit.org site itself would serve as an active
> community, and I think that never fully materialized, with this
> mailing list and, now, github, being the main points of vibrancy.
>
> To me, I think the ultimate solution would be a site accessible at
> http://junit.org, backed by github pages at
> http://junit-team.github.com/, making available a superset of the
> information available at the current junit.org site.  I'd have to talk
> to a couple of stakeholders to see if this can be made to happen.

I believe everyone concerned is on-board.  Once we have most of the
content currently at junit.org copied over to
https://github.com/KentBeck/junit/tree/gh-pages, the owner of
junit.org has agreed to flip the CNAME, and we'll be serving junit.org
from github.

So now, any volunteers to help port the docs?  :-)

    David Saff

>
>   David Saff
>
> On Thu, Sep 8, 2011 at 5:43 PM, Stefan Birkner <mail@...> wrote:
>> The information about JUnit is across three different places: junit.org,
junit.sourceforge.net and github. Additionally some information is missing or
very hard to find (new features like rules, current version, how to contribute,
...)
>>
>> I would like to have one site with all the information. This makes it easier
to find the information and JUnit would look more professional. IMHO the github
pages are a good place for the JUnit documentation, because we can use all the
git(hub) features for creating the documentation.
>>
>> What do you think?
>>
>> Stefan Birkner
>>
>>
>>
>> ------------------------------------
>>
>> Yahoo! Groups Links
>>
>>
>>
>>
>

I believe I have a fixed version of junit-dep-4.9.1, and I'd like to
test the whole round trip (but I'm not ready to release an official
4.9.1 jar yet).  Am I right that what I probably want to do is upload
a snapshot?  Does anyone know how to upload a snapshot to sonatype?
Their documentation seems to not be organized in a way most helpful to
me.

    David Saff

On Thu, Sep 8, 2011 at 1:51 PM, David Saff <david@...> wrote:
> I'm trying to write a test so this doesn't happen again.  Stephan, or
> anyone else, do you have an example of something that fails when
> junit-dep ships with the hamcrest classes, and passes when the jar
> contents are correct?  Thanks,
>
>   David Saff
>
> On Thu, Aug 25, 2011 at 10:45 AM, sschroev <stephan202@...> wrote:
>> Hi David,
>>
>> W.r.t. your last question: as far as I could tell the beta versions were
indeed not available in Maven central; we had to manually deploy them to our
corporate Nexus instance.
>>
>> Anyway, I just updated to JUnit 4.9 and noticed that the maven JAR contains
org.hamcrest.* classes, similar to the issue described in
http://jira.codehaus.org/browse/MAVENUPLOAD-1651.
>>
>> This is not the end of the world, but it requires people that want to use a
more recent version of Hamcrest (e.g. 1.2.1) to declare the Hamcrest dependency
in front of JUnit; not very nice. I see that you did define the hamcrest-core
dependency in the POM, but since the class files are also shipped this has no
effect.
>>
>> I wonder: is this bundling on purpose, and if not, will it be corrected?
>>
>> Kind regards,
>> Stephan
>>
>>
>>
>> --- In junit@yahoogroups.com, David Saff <david@...> wrote:
>>>
>>> So, one minute--does this mean that the beta staging repositories that
>>> I closed, but didn't release, were not available to users for beta
>>> testing?
>>>
>>>    David Saff
>>>
>>> On Wed, Aug 24, 2011 at 11:40 AM, Max Bowsher <maxb@...> wrote:
>>> > Yes, that looks fine.
>>> >
>>> > You probably also want to either promote or drop - probably drop - the
>>> > remaining active staging repositories containing 4.9 beta releases from
>>> > oss.sonatype.org.
>>> >
>>> > Max.
>>> >
>>> > On 24/08/11 12:35, David Saff wrote:
>>> >> I just had to click the button, it turns out.  It should now be
>>> >> promoted.  Please let me know if it looks that way to you (since I'm
>>> >> still, obviously, new at this).
>>> >>
>>> >>    David Saff
>>> >>
>>> >> On Wed, Aug 24, 2011 at 7:29 AM, Max Bowsher <maxb@...> wrote:
>>> >>> On 23/08/11 16:44, David Saff wrote:
>>> >>>> Hello, everyone.  We're happy to announce the release of JUnit 4.9.
>>> >>>
>>> >>> Great!
>>> >>>
>>> >>> I see you've already staged it at oss.sonatype.org - what still needs to
>>> >>> happen before that staging repository gets promoted into Maven Central?
>>> >>>
>>> >>> Max.
>>> >>>
>>> >>>
>>> >
>>> >
>>> >
>>>
>>
>>
>>
>>
>> ------------------------------------
>>
>> Yahoo! Groups Links
>>
>>
>>
>>
>

As promised by mail I consulted a colleague. He says you can use the following
repository: http://repository.apache.org/snapshots/

(Note that you may have to update your setting.xml or pom.xml in order to access
that repo. My colleague suggests asking for support in irc.codehaus.org.)

Good luck,
Stephan

--- In junit@yahoogroups.com, David Saff <david@...> wrote:
>
> I believe I have a fixed version of junit-dep-4.9.1, and I'd like to
> test the whole round trip (but I'm not ready to release an official
> 4.9.1 jar yet).  Am I right that what I probably want to do is upload
> a snapshot?  Does anyone know how to upload a snapshot to sonatype?
> Their documentation seems to not be organized in a way most helpful to
> me.
>
>    David Saff
>
> On Thu, Sep 8, 2011 at 1:51 PM, David Saff <david@...> wrote:
> > I'm trying to write a test so this doesn't happen again.  Stephan, or
> > anyone else, do you have an example of something that fails when
> > junit-dep ships with the hamcrest classes, and passes when the jar
> > contents are correct?  Thanks,
> >
> >   David Saff
> >
> > On Thu, Aug 25, 2011 at 10:45 AM, sschroev <stephan202@...> wrote:
> >> Hi David,
> >>
> >> W.r.t. your last question: as far as I could tell the beta versions were
indeed not available in Maven central; we had to manually deploy them to our
corporate Nexus instance.
> >>
> >> Anyway, I just updated to JUnit 4.9 and noticed that the maven JAR contains
org.hamcrest.* classes, similar to the issue described in
http://jira.codehaus.org/browse/MAVENUPLOAD-1651.
> >>
> >> This is not the end of the world, but it requires people that want to use a
more recent version of Hamcrest (e.g. 1.2.1) to declare the Hamcrest dependency
in front of JUnit; not very nice. I see that you did define the hamcrest-core
dependency in the POM, but since the class files are also shipped this has no
effect.
> >>
> >> I wonder: is this bundling on purpose, and if not, will it be corrected?
> >>
> >> Kind regards,
> >> Stephan
> >>
> >>
> >>
> >> --- In junit@yahoogroups.com, David Saff <david@> wrote:
> >>>
> >>> So, one minute--does this mean that the beta staging repositories that
> >>> I closed, but didn't release, were not available to users for beta
> >>> testing?
> >>>
> >>>    David Saff
> >>>
> >>> On Wed, Aug 24, 2011 at 11:40 AM, Max Bowsher <maxb@> wrote:
> >>> > Yes, that looks fine.
> >>> >
> >>> > You probably also want to either promote or drop - probably drop - the
> >>> > remaining active staging repositories containing 4.9 beta releases from
> >>> > oss.sonatype.org.
> >>> >
> >>> > Max.
> >>> >
> >>> > On 24/08/11 12:35, David Saff wrote:
> >>> >> I just had to click the button, it turns out.  It should now be
> >>> >> promoted.  Please let me know if it looks that way to you (since I'm
> >>> >> still, obviously, new at this).
> >>> >>
> >>> >>    David Saff
> >>> >>
> >>> >> On Wed, Aug 24, 2011 at 7:29 AM, Max Bowsher <maxb@> wrote:
> >>> >>> On 23/08/11 16:44, David Saff wrote:
> >>> >>>> Hello, everyone.  We're happy to announce the release of JUnit 4.9.
> >>> >>>
> >>> >>> Great!
> >>> >>>
> >>> >>> I see you've already staged it at oss.sonatype.org - what still needs
to
> >>> >>> happen before that staging repository gets promoted into Maven
Central?
> >>> >>>
> >>> >>> Max.
> >>> >>>
> >>> >>>
> >>> >
> >>> >
> >>> >
> >>>
> >>
> >>
> >>
> >>
> >> ------------------------------------
> >>
> >> Yahoo! Groups Links
> >>
> >>
> >>
> >>
> >
>

There, I fixed it...

https://github.com/KentBeck/junit/pull/310

This pull request updates the build machinery such that:

- we can now push maven snapshots to the sonatype snapshot repository
- junit-dep is uploaded correctly
- We now have a test that will fail if junit-dep is ever uploaded
incorrectly again

I also uploaded a snapshot of the fixed junit-dep as 4.9.1-SNAPSHOT.
Can someone (Stephan?)  point a pom at
https://oss.sonatype.org/content/groups/public, with junit-dep version
set to LATEST, and make sure that the included classes are now
correct?  Thanks,

    David Saff

On Mon, Sep 12, 2011 at 5:43 PM, sschroev <stephan202@...> wrote:
> As promised by mail I consulted a colleague. He says you can use the following
repository: http://repository.apache.org/snapshots/
>
> (Note that you may have to update your setting.xml or pom.xml in order to
access that repo. My colleague suggests asking for support in irc.codehaus.org.)
>
> Good luck,
> Stephan
>
> --- In junit@yahoogroups.com, David Saff <david@...> wrote:
>>
>> I believe I have a fixed version of junit-dep-4.9.1, and I'd like to
>> test the whole round trip (but I'm not ready to release an official
>> 4.9.1 jar yet).  Am I right that what I probably want to do is upload
>> a snapshot?  Does anyone know how to upload a snapshot to sonatype?
>> Their documentation seems to not be organized in a way most helpful to
>> me.
>>
>>    David Saff
>>
>> On Thu, Sep 8, 2011 at 1:51 PM, David Saff <david@...> wrote:
>> > I'm trying to write a test so this doesn't happen again.  Stephan, or
>> > anyone else, do you have an example of something that fails when
>> > junit-dep ships with the hamcrest classes, and passes when the jar
>> > contents are correct?  Thanks,
>> >
>> >   David Saff
>> >
>> > On Thu, Aug 25, 2011 at 10:45 AM, sschroev <stephan202@...> wrote:
>> >> Hi David,
>> >>
>> >> W.r.t. your last question: as far as I could tell the beta versions were
indeed not available in Maven central; we had to manually deploy them to our
corporate Nexus instance.
>> >>
>> >> Anyway, I just updated to JUnit 4.9 and noticed that the maven JAR
contains org.hamcrest.* classes, similar to the issue described in
http://jira.codehaus.org/browse/MAVENUPLOAD-1651.
>> >>
>> >> This is not the end of the world, but it requires people that want to use
a more recent version of Hamcrest (e.g. 1.2.1) to declare the Hamcrest
dependency in front of JUnit; not very nice. I see that you did define the
hamcrest-core dependency in the POM, but since the class files are also shipped
this has no effect.
>> >>
>> >> I wonder: is this bundling on purpose, and if not, will it be corrected?
>> >>
>> >> Kind regards,
>> >> Stephan
>> >>
>> >>
>> >>
>> >> --- In junit@yahoogroups.com, David Saff <david@> wrote:
>> >>>
>> >>> So, one minute--does this mean that the beta staging repositories that
>> >>> I closed, but didn't release, were not available to users for beta
>> >>> testing?
>> >>>
>> >>>    David Saff
>> >>>
>> >>> On Wed, Aug 24, 2011 at 11:40 AM, Max Bowsher <maxb@> wrote:
>> >>> > Yes, that looks fine.
>> >>> >
>> >>> > You probably also want to either promote or drop - probably drop - the
>> >>> > remaining active staging repositories containing 4.9 beta releases from
>> >>> > oss.sonatype.org.
>> >>> >
>> >>> > Max.
>> >>> >
>> >>> > On 24/08/11 12:35, David Saff wrote:
>> >>> >> I just had to click the button, it turns out.  It should now be
>> >>> >> promoted.  Please let me know if it looks that way to you (since I'm
>> >>> >> still, obviously, new at this).
>> >>> >>
>> >>> >>    David Saff
>> >>> >>
>> >>> >> On Wed, Aug 24, 2011 at 7:29 AM, Max Bowsher <maxb@> wrote:
>> >>> >>> On 23/08/11 16:44, David Saff wrote:
>> >>> >>>> Hello, everyone.  We're happy to announce the release of JUnit 4.9.
>> >>> >>>
>> >>> >>> Great!
>> >>> >>>
>> >>> >>> I see you've already staged it at oss.sonatype.org - what still needs
to
>> >>> >>> happen before that staging repository gets promoted into Maven
Central?
>> >>> >>>
>> >>> >>> Max.
>> >>> >>>
>> >>> >>>
>> >>> >
>> >>> >
>> >>> >
>> >>>
>> >>
>> >>
>> >>
>> >>
>> >> ------------------------------------
>> >>
>> >> Yahoo! Groups Links
>> >>
>> >>
>> >>
>> >>
>> >
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

All,

For a brief time, I was trying to maintain two different code
branches, 4.9.1 with bug fixes, and 4.10 with new feature work.  I now
believe that was mistakenly complex, given the size and agility that
we should be able to maintain.  So, I have simplified matters.  All
new pull requests, in general should go directly against
KentBeck/master.

When it comes time to cut a release branch, I will introduce a new
branch with the release version number (i.e., 4.10).  The only pull
requests that should go against the release branch should be for
regression bugs found during beta testing.

Thanks,

    David Saff

Hi David, I confirmed that the most recent junit-dep snapshot does not contain
hamcrest (whereas the most recent junit snapshot does).

Thanks for taking care of this! I guess it should somewhere be documented that
junit-dep is the preferred package for Maven users?

Stephan

--- In junit@yahoogroups.com, David Saff <david@...> wrote:
>
> There, I fixed it...
>
> https://github.com/KentBeck/junit/pull/310
>
> This pull request updates the build machinery such that:
>
> - we can now push maven snapshots to the sonatype snapshot repository
> - junit-dep is uploaded correctly
> - We now have a test that will fail if junit-dep is ever uploaded
> incorrectly again
>
> I also uploaded a snapshot of the fixed junit-dep as 4.9.1-SNAPSHOT.
> Can someone (Stephan?)  point a pom at
> https://oss.sonatype.org/content/groups/public, with junit-dep version
> set to LATEST, and make sure that the included classes are now
> correct?  Thanks,
>
>    David Saff
>
> On Mon, Sep 12, 2011 at 5:43 PM, sschroev <stephan202@...> wrote:
> > As promised by mail I consulted a colleague. He says you can use the
following repository: http://repository.apache.org/snapshots/
> >
> > (Note that you may have to update your setting.xml or pom.xml in order to
access that repo. My colleague suggests asking for support in irc.codehaus.org.)
> >
> > Good luck,
> > Stephan
> >
> > --- In junit@yahoogroups.com, David Saff <david@> wrote:
> >>
> >> I believe I have a fixed version of junit-dep-4.9.1, and I'd like to
> >> test the whole round trip (but I'm not ready to release an official
> >> 4.9.1 jar yet).  Am I right that what I probably want to do is upload
> >> a snapshot?  Does anyone know how to upload a snapshot to sonatype?
> >> Their documentation seems to not be organized in a way most helpful to
> >> me.
> >>
> >>    David Saff
> >>
> >> On Thu, Sep 8, 2011 at 1:51 PM, David Saff <david@> wrote:
> >> > I'm trying to write a test so this doesn't happen again.  Stephan, or
> >> > anyone else, do you have an example of something that fails when
> >> > junit-dep ships with the hamcrest classes, and passes when the jar
> >> > contents are correct?  Thanks,
> >> >
> >> >   David Saff
> >> >
> >> > On Thu, Aug 25, 2011 at 10:45 AM, sschroev <stephan202@> wrote:
> >> >> Hi David,
> >> >>
> >> >> W.r.t. your last question: as far as I could tell the beta versions were
indeed not available in Maven central; we had to manually deploy them to our
corporate Nexus instance.
> >> >>
> >> >> Anyway, I just updated to JUnit 4.9 and noticed that the maven JAR
contains org.hamcrest.* classes, similar to the issue described in
http://jira.codehaus.org/browse/MAVENUPLOAD-1651.
> >> >>
> >> >> This is not the end of the world, but it requires people that want to
use a more recent version of Hamcrest (e.g. 1.2.1) to declare the Hamcrest
dependency in front of JUnit; not very nice. I see that you did define the
hamcrest-core dependency in the POM, but since the class files are also shipped
this has no effect.
> >> >>
> >> >> I wonder: is this bundling on purpose, and if not, will it be corrected?
> >> >>
> >> >> Kind regards,
> >> >> Stephan
> >> >>
> >> >>
> >> >>
> >> >> --- In junit@yahoogroups.com, David Saff <david@> wrote:
> >> >>>
> >> >>> So, one minute--does this mean that the beta staging repositories that
> >> >>> I closed, but didn't release, were not available to users for beta
> >> >>> testing?
> >> >>>
> >> >>>    David Saff
> >> >>>
> >> >>> On Wed, Aug 24, 2011 at 11:40 AM, Max Bowsher <maxb@> wrote:
> >> >>> > Yes, that looks fine.
> >> >>> >
> >> >>> > You probably also want to either promote or drop - probably drop -
the
> >> >>> > remaining active staging repositories containing 4.9 beta releases
from
> >> >>> > oss.sonatype.org.
> >> >>> >
> >> >>> > Max.
> >> >>> >
> >> >>> > On 24/08/11 12:35, David Saff wrote:
> >> >>> >> I just had to click the button, it turns out.  It should now be
> >> >>> >> promoted.  Please let me know if it looks that way to you (since I'm
> >> >>> >> still, obviously, new at this).
> >> >>> >>
> >> >>> >>    David Saff
> >> >>> >>
> >> >>> >> On Wed, Aug 24, 2011 at 7:29 AM, Max Bowsher <maxb@> wrote:
> >> >>> >>> On 23/08/11 16:44, David Saff wrote:
> >> >>> >>>> Hello, everyone.  We're happy to announce the release of JUnit
4.9.
> >> >>> >>>
> >> >>> >>> Great!
> >> >>> >>>
> >> >>> >>> I see you've already staged it at oss.sonatype.org - what still
needs to
> >> >>> >>> happen before that staging repository gets promoted into Maven
Central?
> >> >>> >>>
> >> >>> >>> Max.
> >> >>> >>>
> >> >>> >>>
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> ------------------------------------
> >> >>
> >> >> Yahoo! Groups Links
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >>
> >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>

Note: I have also posted this to junit-devel@... but
I think that wider input could be beneficial

Consider the case where you are testing a List class...

we have

public class ListTest {

 @Test
 public void newListIsEmpty() {
   assertThat(new List().isEmpty(), is(true);
 }

 @Test
 public void newListHasSizeZero() {
   assertThat(new List().size(), is(0));
 }

 @Test
 public void addPutsAnElementIntoAnEmptyList() {
   List l = new List();
   l.add(new Object());
   assertThat(l.isEmpty(), is(false));
 }

 @Test
 public void addIncreasesSizeOfPopulatedListByOne() {
   List l = new List();
   l.add(new Object());
   int s = l.size();
   l.add(new Object());
   assertThat(l.size(), is(s + 1));
 }

}

We now want to add some tests of the delete functionality... but the
reality is that until/unless some of the preceding tests are passing,
the tests for delete are meaningless. We could have a perfectly
functional List.delete() method but until such time as the above tests
are passing, there is no way to tell that the method does not work.

Now I could code my tests like such

 @Test
 public void deleteIsANoOpOnEmptyList() {
   List l = new List();
   assumeThat(l.isEmpty(), is(true));
   l.delete(new Object());
 }

But all that I am doing is repeating code from the preceding tests,
having changed all those tests' assertThat(...)s into assumeThat(...)s

That does not seem agile to me, copy & paste & search & replace... ban
code smell there

I would much rather be able to annotate the tests with an @Assumes
annotation that indicates that the test assumes that the specified
tests are passing, e.g.

 @Test
 @Assumes("newListIsEmpty")
 public void deleteIsANoOpOnEmptyList() {
   List l = new List();
   l.delete(new Object());
 }

 @Test
 @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
 public void deleteRemovesAnElement() {
   List l = new List();
   Object o = new Object();
   l.add(o);
   l.delete(o);
   assertThat(l.isEmpty(), is(true));
 }

In fact in my initial example of tests, there are some additional
assumptions that I didn't make explicit


 @Test
 @Assumes("newListIsEmpty")
 public void addPutsAnElementIntoAnEmptyList() {
   ...
 }

and

 @Test
 @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
 public void addIncreasesSizeOfPopulatedListByOne() {
   ...
 }

Now you could get some of this functionality via a TestRule...

You could watch tests to see if they pass, and skip tests annotated
with the annotation if assumed functionality is failing, but that
would result in sporadic failures of, e.g. deleteRemovesAnElement
because of the failing newListIsEmpty being executed _after_
deleteRemovesAnElement rather than before.

The simple point is that the test result of deleteRemovesAnElement is
meaningless until its assumptions are true, and while I could code the
assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.

Another alternative to @Assumes would be to invoke the assumed
method(s) at the start of the test, e.g.

 @Test
 public void deleteRemovesAnElement() {
   newListIsEmpty(); // verify assumed functionality
   addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
   ...
 }

That gets rid of the C&P&S&R, but there are two issues with that:

 1. We have to manually invoke any setup/tearDown methods, including
all those of the rules that the test class has... very messy

 2. The test fails when the assumed test fails. In actuality we can
say nothing at all about whether deleteRemovesAnElement if a
newListIsEmpty is not passing... yes we could code the test
differently, but that is just moving our assumptions somewhere else.

I am sure that there are others out there who feel there is a point 3...

 3. We already ran those tests why waste time running them again?

Well the answer to 3 is that these are UNIT tests which should be very
fast, so what is the harm...

So, in my view, best practice unit testing needs the ability to mark
tests as assuming that other tests are passing, so that those tests
can be skipped when the assumptions are known to be failing or
skipped. [This is a deliberately loaded criteria... if the
org.junit.runner.Request does not include the assumed test, then that
test is neither known failing or known skipped, so we can run the test
and output a warning that the failure may be because of assumed
functionality... the use case of executing one and only one test
repeatedly until you get that test passing]

The annotation would have implications on test sorting, as any assumed
tests would have to always happen before the assuming tests (as long
as the assumed tests are in the org.junit.runner.Request)

Also might have to be two annotations, e.g.

@Assumes(methodNames)
@AssumesClasses(classes)

though in my view the @AssumesClasses is less critical, as these are
UNIT tests and each test class should be independent to a large
extent. However I am willing to consider that some people may have
many test classes for one class under test, one test class containing
all the tests of the constructors, another testing the Add methods,
etc. in which case an @AssumesClasses annotation makes sense.

Where tests contain a circular dependency, fail/error both tests

Ok, let the critique begin!

-Stephen

P.S.

I pinged Kent with an earlier version of this idea... but I think that
he missed the point about eliminating C&P&S&R that this feature would
provide because I didn't frame the idea correctly...

---------- Forwarded message ----------
From: "Kent Beck"
Date: 13 Sep 2011 17:11
Subject: Re: JUnit and test dependencies
To: "Stephen Connolly"

Stephen,

Thank you for articulating your idea so clearly. The short answer is that
no, we don't plan to support dependencies. If I have tests that are slow
enough that I care about dependencies, my most productive option is
generally to work on the design of the software until the tests are fast
enough that I no longer care. That said, my voice is only one of many. The
longer answer is that I encourage you to post your idea on the JUnit mailing
list for community discussion.

Regards,

Kent

On Sep 13, 2011, at 8:32 AM, Stephen Connolly wrote:

> Kent,
>
> Are there any plans for JUnit to support some test dependencies, such as:
>
> public class OnlyRunTestsThatMakeSenseTest {
>
>  @Test
>  public void basicFunctionalityWorks() {
>    ...
>  }
>
>  @Test
>  @AssumesPasses("basicFunctionalityWorks")
>  public void advancedFunctionalityWorks() {
>    ...
>  }
>
>  @Test
>  @AssumesPasses("basicFunctionalityWorks")
>  public void basicFunctionalityWorksWithBevel() {
>    ...
>  }
>
>  @Test
>
 @AssumesPasses({"basicFunctionalityWorksWithBevel","advancedFunctionalityWorks"\
})
>  public void advancedFunctionalityWorksWithBevel() {
>    ...
>  }
>
> }
>
> In the above example, no matter what sorting is applied,
> basicFunctionalityWorks will always be run first, and the other three
> tests will only be run if basicFunctionalityWorks passed.
>
> I see the above being completely in the spirit of unit testing, the
> point with the above is that the @Before and @After's will be run
> around each method, you are just saying that there is no point even
> trying to test the advanced functionality when the basic functionality
> is broken, skip those tests which we know cannot pass. That allows the
> person writing advancedFunctionalityWorks to power through the setup
> that depends on the basic functionality and not have to litter their
> advanced test with asserts that are redundant because of the basic
> functionality. Those people who are relying on side-effects should
> really, for unit tests at least, be invoking the method who's
> side-effects they depend on directly within their test method, rather
> than relying on accidental ordering.
>
> Having said that, a second feature that I think would be good is
> something like a @RunAfter and/or @RunBefore which would ensure that
> the test method is run in sequence even if the before or after tests
> fail/are skipped. with @RunAfter and @RunBefore I still think the
> @Before and @After methods should be invoked in-between, this would be
> moving towards more of a general purpose testing framework as opposed
> to being unit-testing focused, but JUnit is just too good ;-)
>
> Thoughts?
>
> -Stephen

On Wed, Sep 14, 2011 at 3:41 AM, sschroev <stephan202@...> wrote:
> Hi David, I confirmed that the most recent junit-dep snapshot does not contain
hamcrest (whereas the most recent junit snapshot does).
>
> Thanks for taking care of this! I guess it should somewhere be documented that
junit-dep is the preferred package for Maven users?

Yes.  I assume this would go in a "getting started" guide somewhere on
the new website.

    David Saff

>
> Stephan
>
> --- In junit@yahoogroups.com, David Saff <david@...> wrote:
>>
>> There, I fixed it...
>>
>> https://github.com/KentBeck/junit/pull/310
>>
>> This pull request updates the build machinery such that:
>>
>> - we can now push maven snapshots to the sonatype snapshot repository
>> - junit-dep is uploaded correctly
>> - We now have a test that will fail if junit-dep is ever uploaded
>> incorrectly again
>>
>> I also uploaded a snapshot of the fixed junit-dep as 4.9.1-SNAPSHOT.
>> Can someone (Stephan?)  point a pom at
>> https://oss.sonatype.org/content/groups/public, with junit-dep version
>> set to LATEST, and make sure that the included classes are now
>> correct?  Thanks,
>>
>>    David Saff
>>
>> On Mon, Sep 12, 2011 at 5:43 PM, sschroev <stephan202@...> wrote:
>> > As promised by mail I consulted a colleague. He says you can use the
following repository: http://repository.apache.org/snapshots/
>> >
>> > (Note that you may have to update your setting.xml or pom.xml in order to
access that repo. My colleague suggests asking for support in irc.codehaus.org.)
>> >
>> > Good luck,
>> > Stephan
>> >
>> > --- In junit@yahoogroups.com, David Saff <david@> wrote:
>> >>
>> >> I believe I have a fixed version of junit-dep-4.9.1, and I'd like to
>> >> test the whole round trip (but I'm not ready to release an official
>> >> 4.9.1 jar yet).  Am I right that what I probably want to do is upload
>> >> a snapshot?  Does anyone know how to upload a snapshot to sonatype?
>> >> Their documentation seems to not be organized in a way most helpful to
>> >> me.
>> >>
>> >>    David Saff
>> >>
>> >> On Thu, Sep 8, 2011 at 1:51 PM, David Saff <david@> wrote:
>> >> > I'm trying to write a test so this doesn't happen again.  Stephan, or
>> >> > anyone else, do you have an example of something that fails when
>> >> > junit-dep ships with the hamcrest classes, and passes when the jar
>> >> > contents are correct?  Thanks,
>> >> >
>> >> >   David Saff
>> >> >
>> >> > On Thu, Aug 25, 2011 at 10:45 AM, sschroev <stephan202@> wrote:
>> >> >> Hi David,
>> >> >>
>> >> >> W.r.t. your last question: as far as I could tell the beta versions
were indeed not available in Maven central; we had to manually deploy them to
our corporate Nexus instance.
>> >> >>
>> >> >> Anyway, I just updated to JUnit 4.9 and noticed that the maven JAR
contains org.hamcrest.* classes, similar to the issue described in
http://jira.codehaus.org/browse/MAVENUPLOAD-1651.
>> >> >>
>> >> >> This is not the end of the world, but it requires people that want to
use a more recent version of Hamcrest (e.g. 1.2.1) to declare the Hamcrest
dependency in front of JUnit; not very nice. I see that you did define the
hamcrest-core dependency in the POM, but since the class files are also shipped
this has no effect.
>> >> >>
>> >> >> I wonder: is this bundling on purpose, and if not, will it be
corrected?
>> >> >>
>> >> >> Kind regards,
>> >> >> Stephan
>> >> >>
>> >> >>
>> >> >>
>> >> >> --- In junit@yahoogroups.com, David Saff <david@> wrote:
>> >> >>>
>> >> >>> So, one minute--does this mean that the beta staging repositories that
>> >> >>> I closed, but didn't release, were not available to users for beta
>> >> >>> testing?
>> >> >>>
>> >> >>>    David Saff
>> >> >>>
>> >> >>> On Wed, Aug 24, 2011 at 11:40 AM, Max Bowsher <maxb@> wrote:
>> >> >>> > Yes, that looks fine.
>> >> >>> >
>> >> >>> > You probably also want to either promote or drop - probably drop -
the
>> >> >>> > remaining active staging repositories containing 4.9 beta releases
from
>> >> >>> > oss.sonatype.org.
>> >> >>> >
>> >> >>> > Max.
>> >> >>> >
>> >> >>> > On 24/08/11 12:35, David Saff wrote:
>> >> >>> >> I just had to click the button, it turns out.  It should now be
>> >> >>> >> promoted.  Please let me know if it looks that way to you (since
I'm
>> >> >>> >> still, obviously, new at this).
>> >> >>> >>
>> >> >>> >>    David Saff
>> >> >>> >>
>> >> >>> >> On Wed, Aug 24, 2011 at 7:29 AM, Max Bowsher <maxb@> wrote:
>> >> >>> >>> On 23/08/11 16:44, David Saff wrote:
>> >> >>> >>>> Hello, everyone.  We're happy to announce the release of JUnit
4.9.
>> >> >>> >>>
>> >> >>> >>> Great!
>> >> >>> >>>
>> >> >>> >>> I see you've already staged it at oss.sonatype.org - what still
needs to
>> >> >>> >>> happen before that staging repository gets promoted into Maven
Central?
>> >> >>> >>>
>> >> >>> >>> Max.
>> >> >>> >>>
>> >> >>> >>>
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------
>> >> >>
>> >> >> Yahoo! Groups Links
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >
>> >>
>> >
>> >
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

It sound like a mock, isn't it?

On Wed, Sep 14, 2011 at 6:02 PM, Stephen Connolly <
stephen.alan.connolly@...> wrote:

> Note: I have also posted this to junit-devel@... but
> I think that wider input could be beneficial
>
> Consider the case where you are testing a List class...
>
> we have
>
> public class ListTest {
>
>  @Test
>  public void newListIsEmpty() {
>    assertThat(new List().isEmpty(), is(true);
>  }
>
>  @Test
>  public void newListHasSizeZero() {
>    assertThat(new List().size(), is(0));
>  }
>
>  @Test
>  public void addPutsAnElementIntoAnEmptyList() {
>    List l = new List();
>    l.add(new Object());
>    assertThat(l.isEmpty(), is(false));
>  }
>
>  @Test
>  public void addIncreasesSizeOfPopulatedListByOne() {
>    List l = new List();
>    l.add(new Object());
>    int s = l.size();
>    l.add(new Object());
>    assertThat(l.size(), is(s + 1));
>  }
>
> }
>
> We now want to add some tests of the delete functionality... but the
> reality is that until/unless some of the preceding tests are passing,
> the tests for delete are meaningless. We could have a perfectly
> functional List.delete() method but until such time as the above tests
> are passing, there is no way to tell that the method does not work.
>
> Now I could code my tests like such
>
>  @Test
>  public void deleteIsANoOpOnEmptyList() {
>    List l = new List();
>    assumeThat(l.isEmpty(), is(true));
>    l.delete(new Object());
>  }
>
> But all that I am doing is repeating code from the preceding tests,
> having changed all those tests' assertThat(...)s into assumeThat(...)s
>
> That does not seem agile to me, copy & paste & search & replace... ban
> code smell there
>
> I would much rather be able to annotate the tests with an @Assumes
> annotation that indicates that the test assumes that the specified
> tests are passing, e.g.
>
>  @Test
>  @Assumes("newListIsEmpty")
>  public void deleteIsANoOpOnEmptyList() {
>    List l = new List();
>    l.delete(new Object());
>  }
>
>  @Test
>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>  public void deleteRemovesAnElement() {
>    List l = new List();
>    Object o = new Object();
>    l.add(o);
>    l.delete(o);
>    assertThat(l.isEmpty(), is(true));
>  }
>
> In fact in my initial example of tests, there are some additional
> assumptions that I didn't make explicit
>
>
>  @Test
>  @Assumes("newListIsEmpty")
>  public void addPutsAnElementIntoAnEmptyList() {
>    ...
>  }
>
> and
>
>  @Test
>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>  public void addIncreasesSizeOfPopulatedListByOne() {
>    ...
>  }
>
> Now you could get some of this functionality via a TestRule...
>
> You could watch tests to see if they pass, and skip tests annotated
> with the annotation if assumed functionality is failing, but that
> would result in sporadic failures of, e.g. deleteRemovesAnElement
> because of the failing newListIsEmpty being executed _after_
> deleteRemovesAnElement rather than before.
>
> The simple point is that the test result of deleteRemovesAnElement is
> meaningless until its assumptions are true, and while I could code the
> assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.
>
> Another alternative to @Assumes would be to invoke the assumed
> method(s) at the start of the test, e.g.
>
>  @Test
>  public void deleteRemovesAnElement() {
>    newListIsEmpty(); // verify assumed functionality
>    addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
>    ...
>  }
>
> That gets rid of the C&P&S&R, but there are two issues with that:
>
>  1. We have to manually invoke any setup/tearDown methods, including
> all those of the rules that the test class has... very messy
>
>  2. The test fails when the assumed test fails. In actuality we can
> say nothing at all about whether deleteRemovesAnElement if a
> newListIsEmpty is not passing... yes we could code the test
> differently, but that is just moving our assumptions somewhere else.
>
> I am sure that there are others out there who feel there is a point 3...
>
>  3. We already ran those tests why waste time running them again?
>
> Well the answer to 3 is that these are UNIT tests which should be very
> fast, so what is the harm...
>
> So, in my view, best practice unit testing needs the ability to mark
> tests as assuming that other tests are passing, so that those tests
> can be skipped when the assumptions are known to be failing or
> skipped. [This is a deliberately loaded criteria... if the
> org.junit.runner.Request does not include the assumed test, then that
> test is neither known failing or known skipped, so we can run the test
> and output a warning that the failure may be because of assumed
> functionality... the use case of executing one and only one test
> repeatedly until you get that test passing]
>
> The annotation would have implications on test sorting, as any assumed
> tests would have to always happen before the assuming tests (as long
> as the assumed tests are in the org.junit.runner.Request)
>
> Also might have to be two annotations, e.g.
>
> @Assumes(methodNames)
> @AssumesClasses(classes)
>
> though in my view the @AssumesClasses is less critical, as these are
> UNIT tests and each test class should be independent to a large
> extent. However I am willing to consider that some people may have
> many test classes for one class under test, one test class containing
> all the tests of the constructors, another testing the Add methods,
> etc. in which case an @AssumesClasses annotation makes sense.
>
> Where tests contain a circular dependency, fail/error both tests
>
> Ok, let the critique begin!
>
> -Stephen
>
> P.S.
>
> I pinged Kent with an earlier version of this idea... but I think that
> he missed the point about eliminating C&P&S&R that this feature would
> provide because I didn't frame the idea correctly...
>
> ---------- Forwarded message ----------
> From: "Kent Beck"
> Date: 13 Sep 2011 17:11
> Subject: Re: JUnit and test dependencies
> To: "Stephen Connolly"
>
> Stephen,
>
> Thank you for articulating your idea so clearly. The short answer is that
> no, we don't plan to support dependencies. If I have tests that are slow
> enough that I care about dependencies, my most productive option is
> generally to work on the design of the software until the tests are fast
> enough that I no longer care. That said, my voice is only one of many. The
> longer answer is that I encourage you to post your idea on the JUnit
> mailing
> list for community discussion.
>
> Regards,
>
> Kent
>
> On Sep 13, 2011, at 8:32 AM, Stephen Connolly wrote:
>
> > Kent,
> >
> > Are there any plans for JUnit to support some test dependencies, such as:
> >
> > public class OnlyRunTestsThatMakeSenseTest {
> >
> >  @Test
> >  public void basicFunctionalityWorks() {
> >    ...
> >  }
> >
> >  @Test
> >  @AssumesPasses("basicFunctionalityWorks")
> >  public void advancedFunctionalityWorks() {
> >    ...
> >  }
> >
> >  @Test
> >  @AssumesPasses("basicFunctionalityWorks")
> >  public void basicFunctionalityWorksWithBevel() {
> >    ...
> >  }
> >
> >  @Test
> >
>
> 
@AssumesPasses({"basicFunctionalityWorksWithBevel","advancedFunctionalityWorks"}\
)
> >  public void advancedFunctionalityWorksWithBevel() {
> >    ...
> >  }
> >
> > }
> >
> > In the above example, no matter what sorting is applied,
> > basicFunctionalityWorks will always be run first, and the other three
> > tests will only be run if basicFunctionalityWorks passed.
> >
> > I see the above being completely in the spirit of unit testing, the
> > point with the above is that the @Before and @After's will be run
> > around each method, you are just saying that there is no point even
> > trying to test the advanced functionality when the basic functionality
> > is broken, skip those tests which we know cannot pass. That allows the
> > person writing advancedFunctionalityWorks to power through the setup
> > that depends on the basic functionality and not have to litter their
> > advanced test with asserts that are redundant because of the basic
> > functionality. Those people who are relying on side-effects should
> > really, for unit tests at least, be invoking the method who's
> > side-effects they depend on directly within their test method, rather
> > than relying on accidental ordering.
> >
> > Having said that, a second feature that I think would be good is
> > something like a @RunAfter and/or @RunBefore which would ensure that
> > the test method is run in sequence even if the before or after tests
> > fail/are skipped. with @RunAfter and @RunBefore I still think the
> > @Before and @After methods should be invoked in-between, this would be
> > moving towards more of a general purpose testing framework as opposed
> > to being unit-testing focused, but JUnit is just too good ;-)
> >
> > Thoughts?
> >
> > -Stephen
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>


[Non-text portions of this message have been removed]

Nope not a mock at all.

Say you have a class List (which is not java.util.List but looks a lot
like it in terms of interface)

This list class is something you are developing via TDD.

You write a whole lot of unit tests for it.

Now suppose it happens to be a linked list, and you decide to change
from a null for end of list marker to a sentinel instead. All your
methods are working now, except for the isEmpty() method, but yet 200+
test cases are failing because they all rely on isEmpty working. Which
unit test do I try to fix first?

with @Assumes, you would have 1 failing test and 199 skipped tests...

No mocks involved at all

On 14 September 2011 16:34, Carfield Yim <carfield@...> wrote:
> It sound like a mock, isn't it?
>
> On Wed, Sep 14, 2011 at 6:02 PM, Stephen Connolly <
> stephen.alan.connolly@...> wrote:
>
>> Note: I have also posted this to junit-devel@... but
>> I think that wider input could be beneficial
>>
>> Consider the case where you are testing a List class...
>>
>> we have
>>
>> public class ListTest {
>>
>>  @Test
>>  public void newListIsEmpty() {
>>    assertThat(new List().isEmpty(), is(true);
>>  }
>>
>>  @Test
>>  public void newListHasSizeZero() {
>>    assertThat(new List().size(), is(0));
>>  }
>>
>>  @Test
>>  public void addPutsAnElementIntoAnEmptyList() {
>>    List l = new List();
>>    l.add(new Object());
>>    assertThat(l.isEmpty(), is(false));
>>  }
>>
>>  @Test
>>  public void addIncreasesSizeOfPopulatedListByOne() {
>>    List l = new List();
>>    l.add(new Object());
>>    int s = l.size();
>>    l.add(new Object());
>>    assertThat(l.size(), is(s + 1));
>>  }
>>
>> }
>>
>> We now want to add some tests of the delete functionality... but the
>> reality is that until/unless some of the preceding tests are passing,
>> the tests for delete are meaningless. We could have a perfectly
>> functional List.delete() method but until such time as the above tests
>> are passing, there is no way to tell that the method does not work.
>>
>> Now I could code my tests like such
>>
>>  @Test
>>  public void deleteIsANoOpOnEmptyList() {
>>    List l = new List();
>>    assumeThat(l.isEmpty(), is(true));
>>    l.delete(new Object());
>>  }
>>
>> But all that I am doing is repeating code from the preceding tests,
>> having changed all those tests' assertThat(...)s into assumeThat(...)s
>>
>> That does not seem agile to me, copy & paste & search & replace... ban
>> code smell there
>>
>> I would much rather be able to annotate the tests with an @Assumes
>> annotation that indicates that the test assumes that the specified
>> tests are passing, e.g.
>>
>>  @Test
>>  @Assumes("newListIsEmpty")
>>  public void deleteIsANoOpOnEmptyList() {
>>    List l = new List();
>>    l.delete(new Object());
>>  }
>>
>>  @Test
>>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>>  public void deleteRemovesAnElement() {
>>    List l = new List();
>>    Object o = new Object();
>>    l.add(o);
>>    l.delete(o);
>>    assertThat(l.isEmpty(), is(true));
>>  }
>>
>> In fact in my initial example of tests, there are some additional
>> assumptions that I didn't make explicit
>>
>>
>>  @Test
>>  @Assumes("newListIsEmpty")
>>  public void addPutsAnElementIntoAnEmptyList() {
>>    ...
>>  }
>>
>> and
>>
>>  @Test
>>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>>  public void addIncreasesSizeOfPopulatedListByOne() {
>>    ...
>>  }
>>
>> Now you could get some of this functionality via a TestRule...
>>
>> You could watch tests to see if they pass, and skip tests annotated
>> with the annotation if assumed functionality is failing, but that
>> would result in sporadic failures of, e.g. deleteRemovesAnElement
>> because of the failing newListIsEmpty being executed _after_
>> deleteRemovesAnElement rather than before.
>>
>> The simple point is that the test result of deleteRemovesAnElement is
>> meaningless until its assumptions are true, and while I could code the
>> assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.
>>
>> Another alternative to @Assumes would be to invoke the assumed
>> method(s) at the start of the test, e.g.
>>
>>  @Test
>>  public void deleteRemovesAnElement() {
>>    newListIsEmpty(); // verify assumed functionality
>>    addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
>>    ...
>>  }
>>
>> That gets rid of the C&P&S&R, but there are two issues with that:
>>
>>  1. We have to manually invoke any setup/tearDown methods, including
>> all those of the rules that the test class has... very messy
>>
>>  2. The test fails when the assumed test fails. In actuality we can
>> say nothing at all about whether deleteRemovesAnElement if a
>> newListIsEmpty is not passing... yes we could code the test
>> differently, but that is just moving our assumptions somewhere else.
>>
>> I am sure that there are others out there who feel there is a point 3...
>>
>>  3. We already ran those tests why waste time running them again?
>>
>> Well the answer to 3 is that these are UNIT tests which should be very
>> fast, so what is the harm...
>>
>> So, in my view, best practice unit testing needs the ability to mark
>> tests as assuming that other tests are passing, so that those tests
>> can be skipped when the assumptions are known to be failing or
>> skipped. [This is a deliberately loaded criteria... if the
>> org.junit.runner.Request does not include the assumed test, then that
>> test is neither known failing or known skipped, so we can run the test
>> and output a warning that the failure may be because of assumed
>> functionality... the use case of executing one and only one test
>> repeatedly until you get that test passing]
>>
>> The annotation would have implications on test sorting, as any assumed
>> tests would have to always happen before the assuming tests (as long
>> as the assumed tests are in the org.junit.runner.Request)
>>
>> Also might have to be two annotations, e.g.
>>
>> @Assumes(methodNames)
>> @AssumesClasses(classes)
>>
>> though in my view the @AssumesClasses is less critical, as these are
>> UNIT tests and each test class should be independent to a large
>> extent. However I am willing to consider that some people may have
>> many test classes for one class under test, one test class containing
>> all the tests of the constructors, another testing the Add methods,
>> etc. in which case an @AssumesClasses annotation makes sense.
>>
>> Where tests contain a circular dependency, fail/error both tests
>>
>> Ok, let the critique begin!
>>
>> -Stephen
>>
>> P.S.
>>
>> I pinged Kent with an earlier version of this idea... but I think that
>> he missed the point about eliminating C&P&S&R that this feature would
>> provide because I didn't frame the idea correctly...
>>
>> ---------- Forwarded message ----------
>> From: "Kent Beck"
>> Date: 13 Sep 2011 17:11
>> Subject: Re: JUnit and test dependencies
>> To: "Stephen Connolly"
>>
>> Stephen,
>>
>> Thank you for articulating your idea so clearly. The short answer is that
>> no, we don't plan to support dependencies. If I have tests that are slow
>> enough that I care about dependencies, my most productive option is
>> generally to work on the design of the software until the tests are fast
>> enough that I no longer care. That said, my voice is only one of many. The
>> longer answer is that I encourage you to post your idea on the JUnit
>> mailing
>> list for community discussion.
>>
>> Regards,
>>
>> Kent
>>
>> On Sep 13, 2011, at 8:32 AM, Stephen Connolly wrote:
>>
>> > Kent,
>> >
>> > Are there any plans for JUnit to support some test dependencies, such as:
>> >
>> > public class OnlyRunTestsThatMakeSenseTest {
>> >
>> >  @Test
>> >  public void basicFunctionalityWorks() {
>> >    ...
>> >  }
>> >
>> >  @Test
>> >  @AssumesPasses("basicFunctionalityWorks")
>> >  public void advancedFunctionalityWorks() {
>> >    ...
>> >  }
>> >
>> >  @Test
>> >  @AssumesPasses("basicFunctionalityWorks")
>> >  public void basicFunctionalityWorksWithBevel() {
>> >    ...
>> >  }
>> >
>> >  @Test
>> >
>>
>>
 @AssumesPasses({"basicFunctionalityWorksWithBevel","advancedFunctionalityWorks"\
})
>> >  public void advancedFunctionalityWorksWithBevel() {
>> >    ...
>> >  }
>> >
>> > }
>> >
>> > In the above example, no matter what sorting is applied,
>> > basicFunctionalityWorks will always be run first, and the other three
>> > tests will only be run if basicFunctionalityWorks passed.
>> >
>> > I see the above being completely in the spirit of unit testing, the
>> > point with the above is that the @Before and @After's will be run
>> > around each method, you are just saying that there is no point even
>> > trying to test the advanced functionality when the basic functionality
>> > is broken, skip those tests which we know cannot pass. That allows the
>> > person writing advancedFunctionalityWorks to power through the setup
>> > that depends on the basic functionality and not have to litter their
>> > advanced test with asserts that are redundant because of the basic
>> > functionality. Those people who are relying on side-effects should
>> > really, for unit tests at least, be invoking the method who's
>> > side-effects they depend on directly within their test method, rather
>> > than relying on accidental ordering.
>> >
>> > Having said that, a second feature that I think would be good is
>> > something like a @RunAfter and/or @RunBefore which would ensure that
>> > the test method is run in sequence even if the before or after tests
>> > fail/are skipped. with @RunAfter and @RunBefore I still think the
>> > @Before and @After methods should be invoked in-between, this would be
>> > moving towards more of a general purpose testing framework as opposed
>> > to being unit-testing focused, but JUnit is just too good ;-)
>> >
>> > Thoughts?
>> >
>> > -Stephen
>>
>>
>> ------------------------------------
>>
>> Yahoo! Groups Links
>>
>>
>>
>>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Stephen,

I can see how this could be useful.  It shouldn't be too hard to try
it out as a custom runner:
- subclass BlockJUnit4ClassRunner
- override computeTestMethods to order the methods based on assumptions
- override methodBlock to notice when tests fail, and use that to mark
assumption failures on later tests.

Would you be interested in contributing something like this to junit.contrib?

    David Saff

On Wed, Sep 14, 2011 at 6:02 AM, Stephen Connolly
<stephen.alan.connolly@...> wrote:
> Note: I have also posted this to junit-devel@... but
> I think that wider input could be beneficial
>
> Consider the case where you are testing a List class...
>
> we have
>
> public class ListTest {
>
>  @Test
>  public void newListIsEmpty() {
>    assertThat(new List().isEmpty(), is(true);
>  }
>
>  @Test
>  public void newListHasSizeZero() {
>    assertThat(new List().size(), is(0));
>  }
>
>  @Test
>  public void addPutsAnElementIntoAnEmptyList() {
>    List l = new List();
>    l.add(new Object());
>    assertThat(l.isEmpty(), is(false));
>  }
>
>  @Test
>  public void addIncreasesSizeOfPopulatedListByOne() {
>    List l = new List();
>    l.add(new Object());
>    int s = l.size();
>    l.add(new Object());
>    assertThat(l.size(), is(s + 1));
>  }
>
> }
>
> We now want to add some tests of the delete functionality... but the
> reality is that until/unless some of the preceding tests are passing,
> the tests for delete are meaningless. We could have a perfectly
> functional List.delete() method but until such time as the above tests
> are passing, there is no way to tell that the method does not work.
>
> Now I could code my tests like such
>
>  @Test
>  public void deleteIsANoOpOnEmptyList() {
>    List l = new List();
>    assumeThat(l.isEmpty(), is(true));
>    l.delete(new Object());
>  }
>
> But all that I am doing is repeating code from the preceding tests,
> having changed all those tests' assertThat(...)s into assumeThat(...)s
>
> That does not seem agile to me, copy & paste & search & replace... ban
> code smell there
>
> I would much rather be able to annotate the tests with an @Assumes
> annotation that indicates that the test assumes that the specified
> tests are passing, e.g.
>
>  @Test
>  @Assumes("newListIsEmpty")
>  public void deleteIsANoOpOnEmptyList() {
>    List l = new List();
>    l.delete(new Object());
>  }
>
>  @Test
>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>  public void deleteRemovesAnElement() {
>    List l = new List();
>    Object o = new Object();
>    l.add(o);
>    l.delete(o);
>    assertThat(l.isEmpty(), is(true));
>  }
>
> In fact in my initial example of tests, there are some additional
> assumptions that I didn't make explicit
>
>
>  @Test
>  @Assumes("newListIsEmpty")
>  public void addPutsAnElementIntoAnEmptyList() {
>    ...
>  }
>
> and
>
>  @Test
>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>  public void addIncreasesSizeOfPopulatedListByOne() {
>    ...
>  }
>
> Now you could get some of this functionality via a TestRule...
>
> You could watch tests to see if they pass, and skip tests annotated
> with the annotation if assumed functionality is failing, but that
> would result in sporadic failures of, e.g. deleteRemovesAnElement
> because of the failing newListIsEmpty being executed _after_
> deleteRemovesAnElement rather than before.
>
> The simple point is that the test result of deleteRemovesAnElement is
> meaningless until its assumptions are true, and while I could code the
> assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.
>
> Another alternative to @Assumes would be to invoke the assumed
> method(s) at the start of the test, e.g.
>
>  @Test
>  public void deleteRemovesAnElement() {
>    newListIsEmpty(); // verify assumed functionality
>    addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
>    ...
>  }
>
> That gets rid of the C&P&S&R, but there are two issues with that:
>
>  1. We have to manually invoke any setup/tearDown methods, including
> all those of the rules that the test class has... very messy
>
>  2. The test fails when the assumed test fails. In actuality we can
> say nothing at all about whether deleteRemovesAnElement if a
> newListIsEmpty is not passing... yes we could code the test
> differently, but that is just moving our assumptions somewhere else.
>
> I am sure that there are others out there who feel there is a point 3...
>
>  3. We already ran those tests why waste time running them again?
>
> Well the answer to 3 is that these are UNIT tests which should be very
> fast, so what is the harm...
>
> So, in my view, best practice unit testing needs the ability to mark
> tests as assuming that other tests are passing, so that those tests
> can be skipped when the assumptions are known to be failing or
> skipped. [This is a deliberately loaded criteria... if the
> org.junit.runner.Request does not include the assumed test, then that
> test is neither known failing or known skipped, so we can run the test
> and output a warning that the failure may be because of assumed
> functionality... the use case of executing one and only one test
> repeatedly until you get that test passing]
>
> The annotation would have implications on test sorting, as any assumed
> tests would have to always happen before the assuming tests (as long
> as the assumed tests are in the org.junit.runner.Request)
>
> Also might have to be two annotations, e.g.
>
> @Assumes(methodNames)
> @AssumesClasses(classes)
>
> though in my view the @AssumesClasses is less critical, as these are
> UNIT tests and each test class should be independent to a large
> extent. However I am willing to consider that some people may have
> many test classes for one class under test, one test class containing
> all the tests of the constructors, another testing the Add methods,
> etc. in which case an @AssumesClasses annotation makes sense.
>
> Where tests contain a circular dependency, fail/error both tests
>
> Ok, let the critique begin!
>
> -Stephen
>
> P.S.
>
> I pinged Kent with an earlier version of this idea... but I think that
> he missed the point about eliminating C&P&S&R that this feature would
> provide because I didn't frame the idea correctly...
>
> ---------- Forwarded message ----------
> From: "Kent Beck"
> Date: 13 Sep 2011 17:11
> Subject: Re: JUnit and test dependencies
> To: "Stephen Connolly"
>
> Stephen,
>
> Thank you for articulating your idea so clearly. The short answer is that
> no, we don't plan to support dependencies. If I have tests that are slow
> enough that I care about dependencies, my most productive option is
> generally to work on the design of the software until the tests are fast
> enough that I no longer care. That said, my voice is only one of many. The
> longer answer is that I encourage you to post your idea on the JUnit mailing
> list for community discussion.
>
> Regards,
>
> Kent
>
> On Sep 13, 2011, at 8:32 AM, Stephen Connolly wrote:
>
>> Kent,
>>
>> Are there any plans for JUnit to support some test dependencies, such as:
>>
>> public class OnlyRunTestsThatMakeSenseTest {
>>
>>  @Test
>>  public void basicFunctionalityWorks() {
>>    ...
>>  }
>>
>>  @Test
>>  @AssumesPasses("basicFunctionalityWorks")
>>  public void advancedFunctionalityWorks() {
>>    ...
>>  }
>>
>>  @Test
>>  @AssumesPasses("basicFunctionalityWorks")
>>  public void basicFunctionalityWorksWithBevel() {
>>    ...
>>  }
>>
>>  @Test
>>
>
 @AssumesPasses({"basicFunctionalityWorksWithBevel","advancedFunctionalityWorks"\
})
>>  public void advancedFunctionalityWorksWithBevel() {
>>    ...
>>  }
>>
>> }
>>
>> In the above example, no matter what sorting is applied,
>> basicFunctionalityWorks will always be run first, and the other three
>> tests will only be run if basicFunctionalityWorks passed.
>>
>> I see the above being completely in the spirit of unit testing, the
>> point with the above is that the @Before and @After's will be run
>> around each method, you are just saying that there is no point even
>> trying to test the advanced functionality when the basic functionality
>> is broken, skip those tests which we know cannot pass. That allows the
>> person writing advancedFunctionalityWorks to power through the setup
>> that depends on the basic functionality and not have to litter their
>> advanced test with asserts that are redundant because of the basic
>> functionality. Those people who are relying on side-effects should
>> really, for unit tests at least, be invoking the method who's
>> side-effects they depend on directly within their test method, rather
>> than relying on accidental ordering.
>>
>> Having said that, a second feature that I think would be good is
>> something like a @RunAfter and/or @RunBefore which would ensure that
>> the test method is run in sequence even if the before or after tests
>> fail/are skipped. with @RunAfter and @RunBefore I still think the
>> @Before and @After methods should be invoked in-between, this would be
>> moving towards more of a general purpose testing framework as opposed
>> to being unit-testing focused, but JUnit is just too good ;-)
>>
>> Thoughts?
>>
>> -Stephen
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Am Montag, den 12.09.2011, 11:40 -0400 schrieb David Saff:
>
> On Fri, Sep 9, 2011 at 9:32 AM, David Saff <david@...> wrote:
> > Yes.  This is a pain for me, as well.
> >
> > Sourceforge.net is purely vestigial.  The moment there's a great,
> > authoritative master location to link to, I'll do what it takes to
> > kill sourceforge and point it there.
> >
> > junit.org is an oddity.  For reasons that predate me and I only
> dimly
> > understand, junit.org was registered back in 2000 by Object Mentor.
> > To my knowledge, it has never been actively maintained by any of the
> > active maintainers of the JUnit source.  The site currently there
> > (developed by Ben Rady and a merry crew roughly four years ago) is
> one
> > of the better incarnations of the site.  However, its design
> > presupposed that the junit.org site itself would serve as an active
> > community, and I think that never fully materialized, with this
> > mailing list and, now, github, being the main points of vibrancy.
> >
> > To me, I think the ultimate solution would be a site accessible at
> > http://junit.org, backed by github pages at
> > http://junit-team.github.com/, making available a superset of the
> > information available at the current junit.org site.  I'd have to
> talk
> > to a couple of stakeholders to see if this can be made to happen.
>
> I believe everyone concerned is on-board. Once we have most of the
> content currently at junit.org copied over to
> https://github.com/KentBeck/junit/tree/gh-pages, the owner of
> junit.org has agreed to flip the CNAME, and we'll be serving junit.org
> from github.
>
> So now, any volunteers to help port the docs? :-)
I started to move the content of junit.org to github. Will need one or
two days.

> David Saff
>
> >
> >   David Saff
> >
> > On Thu, Sep 8, 2011 at 5:43 PM, Stefan Birkner
> <mail@...> wrote:
> >> The information about JUnit is across three different places:
> junit.org, junit.sourceforge.net and github. Additionally some
> information is missing or very hard to find (new features like rules,
> current version, how to contribute, ...)
> >>
> >> I would like to have one site with all the information. This makes
> it easier to find the information and JUnit would look more
> professional. IMHO the github pages are a good place for the JUnit
> documentation, because we can use all the git(hub) features for
> creating the documentation.
> >>
> >> What do you think?
> >>
> >> Stefan Birkner
> >>
> >>
> >>
> >> ------------------------------------
> >>
> >> Yahoo! Groups Links
> >>
> >>
> >>
> >>
> >
>
>
>
>

On Wednesday, 14 September 2011, David Saff wrote:

> Stephen,
>
> I can see how this could be useful.  It shouldn't be too hard to try
> it out as a custom runner:
> - subclass BlockJUnit4ClassRunner
> - override computeTestMethods to order the methods based on assumptions
> - override methodBlock to notice when tests fail, and use that to mark
> assumption failures on later tests.
>
> Would you be interested in contributing something like this to
> junit.contrib?
>
>
As a proof of concept, implementing as a custom runner is fine, but
ultimately it would need some hooks that can feed back to Request.sort(...)
as, IIUC, that applies after the Runner's sorting and this would really be
enforcing restrictions on the final order.

Where do I git clone and send the pull request for junit.contrib or is that
just a package space in the standard junit git repo?

-Stephen


>   David Saff
>
> On Wed, Sep 14, 2011 at 6:02 AM, Stephen Connolly
> <stephen.alan.connolly@... <javascript:;>> wrote:
> > Note: I have also posted this to junit-devel@... but
> > I think that wider input could be beneficial
> >
> > Consider the case where you are testing a List class...
> >
> > we have
> >
> > public class ListTest {
> >
> >  @Test
> >  public void newListIsEmpty() {
> >    assertThat(new List().isEmpty(), is(true);
> >  }
> >
> >  @Test
> >  public void newListHasSizeZero() {
> >    assertThat(new List().size(), is(0));
> >  }
> >
> >  @Test
> >  public void addPutsAnElementIntoAnEmptyList() {
> >    List l = new List();
> >    l.add(new Object());
> >    assertThat(l.isEmpty(), is(false));
> >  }
> >
> >  @Test
> >  public void addIncreasesSizeOfPopulatedListByOne() {
> >    List l = new List();
> >    l.add(new Object());
> >    int s = l.size();
> >    l.add(new Object());
> >    assertThat(l.size(), is(s + 1));
> >  }
> >
> > }
> >
> > We now want to add some tests of the delete functionality... but the
> > reality is that until/unless some of the preceding tests are passing,
> > the tests for delete are meaningless. We could have a perfectly
> > functional List.delete() method but until such time as the above tests
> > are passing, there is no way to tell that the method does not work.
> >
> > Now I could code my tests like such
> >
> >  @Test
> >  public void deleteIsANoOpOnEmptyList() {
> >    List l = new List();
> >    assumeThat(l.isEmpty(), is(true));
> >    l.delete(new Object());
> >  }
> >
> > But all that I am doing is repeating code from the preceding tests,
> > having changed all those tests' assertThat(...)s into assumeThat(...)s
> >
> > That does not seem agile to me, copy & paste & search & replace... ban
> > code smell there
> >
> > I would much rather be able to annotate the tests with an @Assumes
> > annotation that indicates that the test assumes that the specified
> > tests are passing, e.g.
> >
> >  @Test
> >  @Assumes("newListIsEmpty")
> >  public void deleteIsANoOpOnEmptyList() {
> >    List l = new List();
> >    l.delete(new Object());
> >  }
> >
> >  @Test
> >  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
> >  public void deleteRemovesAnElement() {
> >    List l = new List();
> >    Object o = new Object();
> >    l.add(o);
> >    l.delete(o);
> >    assertThat(l.isEmpty(), is(true));
> >  }
> >
> > In fact in my initial example of tests, there are some additional
> > assumptions that I didn't make explicit
> >
> >
> >  @Test
> >  @Assumes("newListIsEmpty")
> >  public void addPutsAnElementIntoAnEmptyList() {
> >    ...
> >  }
> >
> > and
> >
> >  @Test
> >  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
> >  public void addIncreasesSizeOfPopulatedListByOne() {
> >    ...
> >  }
> >
> > Now you could get some of this functionality via a TestRule...
> >
> > You could watch tests to see if they pass, and skip tests annotated
> > with the annotation if assumed functionality is failing, but that
> > would result in sporadic failures of, e.g. deleteRemovesAnElement
> > because of the failing newListIsEmpty being executed _after_
> > deleteRemovesAnElement rather than before.
> >
> > The simple point is that the test result of deleteRemovesAnElement is
> > meaningless until its assumptions are true, and while I could code the
> > assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.
> >
> > Another alternative to @Assumes would be to invoke the assumed
> > method(s) at the start of the test, e.g.
> >
> >  @Test
> >  public void deleteRemovesAnElement() {
> >    newListIsEmpty(); // verify assumed functionality
> >    addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
> >  > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>


[Non-text portions of this message have been removed]

https://github.com/junit-team/junit.contrib/pull/5

On Wednesday, 14 September 2011, Stephen Connolly wrote:

> On Wednesday, 14 September 2011, David Saff wrote:
>
>> Stephen,
>>
>> I can see how this could be useful.  It shouldn't be too hard to try
>> it out as a custom runner:
>> - subclass BlockJUnit4ClassRunner
>> - override computeTestMethods to order the methods based on assumptions
>> - override methodBlock to notice when tests fail, and use that to mark
>> assumption failures on later tests.
>>
>> Would you be interested in contributing something like this to
>> junit.contrib?
>>
>>
> As a proof of concept, implementing as a custom runner is fine, but
> ultimately it would need some hooks that can feed back to Request.sort(...)
> as, IIUC, that applies after the Runner's sorting and this would really be
> enforcing restrictions on the final order.
>
> Where do I git clone and send the pull request for junit.contrib or is that
> just a package space in the standard junit git repo?
>
> -Stephen
>
>
>>   David Saff
>>
>> On Wed, Sep 14, 2011 at 6:02 AM, Stephen Connolly
>> <stephen.alan.connolly@...> wrote:
>> > Note: I have also posted this to junit-devel@... but
>> > I think that wider input could be beneficial
>> >
>> > Consider the case where you are testing a List class...
>> >
>> > we have
>> >
>> > public class ListTest {
>> >
>> >  @Test
>> >  public void newListIsEmpty() {
>> >    assertThat(new List().isEmpty(), is(true);
>> >  }
>> >
>> >  @Test
>> >  public void newListHasSizeZero() {
>> >    assertThat(new List().size(), is(0));
>> >  }
>> >
>> >  @Test
>> >  public void addPutsAnElementIntoAnEmptyList() {
>> >    List l = new List();
>> >    l.add(new Object());
>> >    assertThat(l.isEmpty(), is(false));
>> >  }
>> >
>> >  @Test
>> >  public void addIncreasesSizeOfPopulatedListByOne() {
>> >    List l = new List();
>> >    l.add(new Object());
>> >    int s = l.size();
>> >    l.add(new Object());
>> >    assertThat(l.size(), is(s + 1));
>> >  }
>> >
>> > }
>> >
>> > We now want to add some tests of the delete functionality... but the
>> > reality is that until/unless some of the preceding tests are passing,
>> > the tests for delete are meaningless. We could have a perfectly
>> > functional List.delete() method but until such time as the above tests
>> > are passing, there is no way to tell that the method does not work.
>> >
>> > Now I could code my tests like such
>> >
>> >  @Test
>> >  public void deleteIsANoOpOnEmptyList() {
>> >    List l = new List();
>> >    assumeThat(l.isEmpty(), is(true));
>> >    l.delete(new Object());
>> >  }
>> >
>> > But all that I am doing is repeating code from the preceding tests,
>> > having changed all those tests' assertThat(...)s into assumeThat(...)s
>> >
>> > That does not seem agile to me, copy & paste & search & replace... ban
>> > code smell there
>> >
>> > I would much rather be able to annotate the tests with an @Assumes
>> > annotation that indicates that the test assumes that the specified
>> > tests are passing, e.g.
>> >
>> >  @Test
>> >  @Assumes("newListIsEmpty")
>> >  public void deleteIsANoOpOnEmptyList() {
>> >    List l = new List();
>> >    l.delete(new Object());
>> >  }
>> >
>> >  @Test
>> >  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>> >  public void deleteRemovesAnElement() {
>> >    List l = new List();
>> >    Object o = new Object();
>> >    l.add(o);
>> >    l.delete(o);
>> >    assertThat(l.isEmpty(), is(true));
>> >  }
>> >
>> > In fact in my initial example of tests, there are some additional
>> > assumptions that I didn't make explicit
>> >
>> >
>> >  @Test
>> >  @Assumes("newListIsEmpty")
>> >  public void addPutsAnElementIntoAnEmptyList() {
>> >    ...
>> >  }
>> >
>> > and
>> >
>> >  @Test
>> >  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>> >  public void addIncreasesSizeOfPopulatedListByOne() {
>> >    ...
>> >  }
>> >
>> > Now you could get some of this functionality via a TestRule...
>> >
>> > You could watch tests to see if they pass, and skip tests annotated
>> > with the annotation if assumed functionality is failing, but that
>> > would result in sporadic failures of, e.g. deleteRemovesAnElement
>> > because of the failing newListIsEmpty being executed _after_
>> > deleteRemovesAnElement rather than before.
>> >
>> > The simple point is that the test result of deleteRemovesAnElement is
>> > meaningless until its assumptions are true, and while I could code the
>> > assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.
>> >
>> > Another alternative to @Assumes would be to invoke the assumed
>> > method(s) at the start of the test, e.g.
>> >
>> >  @Test
>> >  public void deleteRemovesAnElement() {
>> >    newListIsEmpty(); // verify assumed functionality
>> >    addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
>> >  > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>>
>> ------------------------------------
>>
>> Yahoo! Groups Links
>>
>>
>>
>>


[Non-text portions of this message have been removed]

Does anyone know if there is a copyright statement for JUnit?  I was not aware
of any and have been unable to find one, but I need to double check.  Thanks!

Hi Dale,

Thanks for your reply. Ok so I wrote a simple test that shows a JFrame and uses
Java Robot to click on it. Works fine run in Eclipse on Mac as an application or
JUnit test but fails when run as JUnit plugin test. Frame would show 1 out of 5
or so times. Removing all unneeded plugins resulted in frame always being shown
but the click never seems to happen.

I also tried Window Licker and the included Calculator test(s) - same thing... 
runs as a JUnit Test but fails as a JUnit Plugin Test. One time the frame was
shown, but nothing happened (so the test failed). The other times, the frame
didn't even show. Tried removing all VM args and unnecessary plugins.

Is this related to the args that Eclipse uses to run plugin tests on Mac? Or the
core plugins included to run the tests? Or...?

Shaun.

--- In junit@yahoogroups.com, Dale Emery <dale@...> wrote:
>
> Hi whoever you are,
>
> > Hi - thanks for your reply. I'm actually using Alex Ruiz's FEST for our
testing and it works quite well on Windows. With or without FEST on the Mac, I
can't seem to get any UI tests to run. The initial frame/window comes up in some
cases but I can't do much more than that. When running JUnit tests (again with
or without FEST), I don't even see the frame.
> >
> > I've gone through what docs there are for FEST and have read the
blog/article but I can't figure it out.
>
>
> I've used FEST to test Swing code on a toy project:
> https://github.com/dhemery/towers.dead
>
> Later I restarted the project and used WindowLicker instead:
> https://github.com/dhemery/Towers
>
> It's been a while since I visited either project, but I think the (few) UI
tests run. Neither version of the code does much, so there aren't many UI tests.
The older one (which uses FEST) has more features and tests.
>
> I wrote all of this code on Macs running Snow Leopard and Eclipse Ganymede. I
haven't tried it on Lion or Helios.
>
> Dale
>
> --
> Dale Emery
> Consultant to software teams and leaders
> http://dhemery.com
>
>
>
>
> [Non-text portions of this message have been removed]
>

Hi Stephen and all,

> Nope not a mock at all.
>
More like TestNG's "dependsOnMethods" feature.

I understand your desire for this. I also understand the JUnit guys' reluctance
to implement it. My intermediate-ish experience with TestNG leaves me with the
conclusion that inter-test dependencies lead to madness. To be fair, I had
already concluded that before working with clients who insisted on specifying
inter-test dependencies.

My first half-baked thought is to use a Rule to prevent running methods that
depend on another method that has already failed. But that would work only if
the tests were run in the appropriate order.

So my next half-baked thought is to write a custom Runner that builds the
dependency graph and sequences the tests accordingly. Then either the runner or
a rule could decide whether to run a test. I'm confident that a runner could do
what you want, but that requires delving into the arcane bits of JUnit.

A final half-baked thought: What if you ran all the tests, then used the
dependency graph only to mark the results somehow, so as to highlight the tests
on which others depend. If you see a swarm of failures, and the display
indicates that some common prerequisite test also failed, that would aid your
investigation.

I've seen situations where some prerequisite test fails, but some of the
dependent tests pass. Sometimes the prerequisite test is lying to me (and I need
to fix that). Sometimes the dependency is over-specified--the dependent test
doesn't depend on everything that the prerequisite test tests (which means I
need to refactor the prerequisite test and the dependency specifications).
Sometimes it means that the prerequisite test fails intermittently (which means
I need to track down the uncontrolled factors that affect the test).

Skipping the dependent tests hides that information. Running them and marking
them would highlight that

Dale

--
Dale Emery
Consultant to software teams and leaders
http://dhemery.com



[Non-text portions of this message have been removed]

Sound like TestNG dependsOn feature. But from what I know, JUnit is against
any kind of test dependencies (there was a recent discussion on this mailing
list, see http://tech.groups.yahoo.com/group/junit/message/23568). A short
quote from this discussion: "It has always been a deliberate design decision
in JUnit that we would *not* guarantee the order of test execution."

--
Regards / Pozdrawiam
Tomek Kaczanowski
http://kaczanowscy.pl/tomek

2011/9/14 Stephen Connolly <stephen.alan.connolly@...>

>
>
> Nope not a mock at all.
>
> Say you have a class List (which is not java.util.List but looks a lot
> like it in terms of interface)
>
> This list class is something you are developing via TDD.
>
> You write a whole lot of unit tests for it.
>
> Now suppose it happens to be a linked list, and you decide to change
> from a null for end of list marker to a sentinel instead. All your
> methods are working now, except for the isEmpty() method, but yet 200+
> test cases are failing because they all rely on isEmpty working. Which
> unit test do I try to fix first?
>
> with @Assumes, you would have 1 failing test and 199 skipped tests...
>
> No mocks involved at all
>
>
> On 14 September 2011 16:34, Carfield Yim <carfield@...> wrote:
> > It sound like a mock, isn't it?
> >
> > On Wed, Sep 14, 2011 at 6:02 PM, Stephen Connolly <
> > stephen.alan.connolly@...> wrote:
> >
> >> Note: I have also posted this to junit-devel@... but
> >> I think that wider input could be beneficial
> >>
> >> Consider the case where you are testing a List class...
> >>
> >> we have
> >>
> >> public class ListTest {
> >>
> >>  @Test
> >>  public void newListIsEmpty() {
> >>    assertThat(new List().isEmpty(), is(true);
> >>  }
> >>
> >>  @Test
> >>  public void newListHasSizeZero() {
> >>    assertThat(new List().size(), is(0));
> >>  }
> >>
> >>  @Test
> >>  public void addPutsAnElementIntoAnEmptyList() {
> >>    List l = new List();
> >>    l.add(new Object());
> >>    assertThat(l.isEmpty(), is(false));
> >>  }
> >>
> >>  @Test
> >>  public void addIncreasesSizeOfPopulatedListByOne() {
> >>    List l = new List();
> >>    l.add(new Object());
> >>    int s = l.size();
> >>    l.add(new Object());
> >>    assertThat(l.size(), is(s + 1));
> >>  }
> >>
> >> }
> >>
> >> We now want to add some tests of the delete functionality... but the
> >> reality is that until/unless some of the preceding tests are passing,
> >> the tests for delete are meaningless. We could have a perfectly
> >> functional List.delete() method but until such time as the above tests
> >> are passing, there is no way to tell that the method does not work.
> >>
> >> Now I could code my tests like such
> >>
> >>  @Test
> >>  public void deleteIsANoOpOnEmptyList() {
> >>    List l = new List();
> >>    assumeThat(l.isEmpty(), is(true));
> >>    l.delete(new Object());
> >>  }
> >>
> >> But all that I am doing is repeating code from the preceding tests,
> >> having changed all those tests' assertThat(...)s into assumeThat(...)s
> >>
> >> That does not seem agile to me, copy & paste & search & replace... ban
> >> code smell there
> >>
> >> I would much rather be able to annotate the tests with an @Assumes
> >> annotation that indicates that the test assumes that the specified
> >> tests are passing, e.g.
> >>
> >>  @Test
> >>  @Assumes("newListIsEmpty")
> >>  public void deleteIsANoOpOnEmptyList() {
> >>    List l = new List();
> >>    l.delete(new Object());
> >>  }
> >>
> >>  @Test
> >>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
> >>  public void deleteRemovesAnElement() {
> >>    List l = new List();
> >>    Object o = new Object();
> >>    l.add(o);
> >>    l.delete(o);
> >>    assertThat(l.isEmpty(), is(true));
> >>  }
> >>
> >> In fact in my initial example of tests, there are some additional
> >> assumptions that I didn't make explicit
> >>
> >>
> >>  @Test
> >>  @Assumes("newListIsEmpty")
> >>  public void addPutsAnElementIntoAnEmptyList() {
> >>    ...
> >>  }
> >>
> >> and
> >>
> >>  @Test
> >>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
> >>  public void addIncreasesSizeOfPopulatedListByOne() {
> >>    ...
> >>  }
> >>
> >> Now you could get some of this functionality via a TestRule...
> >>
> >> You could watch tests to see if they pass, and skip tests annotated
> >> with the annotation if assumed functionality is failing, but that
> >> would result in sporadic failures of, e.g. deleteRemovesAnElement
> >> because of the failing newListIsEmpty being executed _after_
> >> deleteRemovesAnElement rather than before.
> >>
> >> The simple point is that the test result of deleteRemovesAnElement is
> >> meaningless until its assumptions are true, and while I could code the
> >> assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.
> >>
> >> Another alternative to @Assumes would be to invoke the assumed
> >> method(s) at the start of the test, e.g.
> >>
> >>  @Test
> >>  public void deleteRemovesAnElement() {
> >>    newListIsEmpty(); // verify assumed functionality
> >>    addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
> >>    ...
> >>  }
> >>
> >> That gets rid of the C&P&S&R, but there are two issues with that:
> >>
> >>  1. We have to manually invoke any setup/tearDown methods, including
> >> all those of the rules that the test class has... very messy
> >>
> >>  2. The test fails when the assumed test fails. In actuality we can
> >> say nothing at all about whether deleteRemovesAnElement if a
> >> newListIsEmpty is not passing... yes we could code the test
> >> differently, but that is just moving our assumptions somewhere else.
> >>
> >> I am sure that there are others out there who feel there is a point 3...
> >>
> >>  3. We already ran those tests why waste time running them again?
> >>
> >> Well the answer to 3 is that these are UNIT tests which should be very
> >> fast, so what is the harm...
> >>
> >> So, in my view, best practice unit testing needs the ability to mark
> >> tests as assuming that other tests are passing, so that those tests
> >> can be skipped when the assumptions are known to be failing or
> >> skipped. [This is a deliberately loaded criteria... if the
> >> org.junit.runner.Request does not include the assumed test, then that
> >> test is neither known failing or known skipped, so we can run the test
> >> and output a warning that the failure may be because of assumed
> >> functionality... the use case of executing one and only one test
> >> repeatedly until you get that test passing]
> >>
> >> The annotation would have implications on test sorting, as any assumed
> >> tests would have to always happen before the assuming tests (as long
> >> as the assumed tests are in the org.junit.runner.Request)
> >>
> >> Also might have to be two annotations, e.g.
> >>
> >> @Assumes(methodNames)
> >> @AssumesClasses(classes)
> >>
> >> though in my view the @AssumesClasses is less critical, as these are
> >> UNIT tests and each test class should be independent to a large
> >> extent. However I am willing to consider that some people may have
> >> many test classes for one class under test, one test class containing
> >> all the tests of the constructors, another testing the Add methods,
> >> etc. in which case an @AssumesClasses annotation makes sense.
> >>
> >> Where tests contain a circular dependency, fail/error both tests
> >>
> >> Ok, let the critique begin!
> >>
> >> -Stephen
> >>
> >> P.S.
> >>
> >> I pinged Kent with an earlier version of this idea... but I think that
> >> he missed the point about eliminating C&P&S&R that this feature would
> >> provide because I didn't frame the idea correctly...
> >>
> >> ---------- Forwarded message ----------
> >> From: "Kent Beck"
> >> Date: 13 Sep 2011 17:11
> >> Subject: Re: JUnit and test dependencies
> >> To: "Stephen Connolly"
> >>
> >> Stephen,
> >>
> >> Thank you for articulating your idea so clearly. The short answer is
> that
> >> no, we don't plan to support dependencies. If I have tests that are slow
> >> enough that I care about dependencies, my most productive option is
> >> generally to work on the design of the software until the tests are fast
> >> enough that I no longer care. That said, my voice is only one of many.
> The
> >> longer answer is that I encourage you to post your idea on the JUnit
> >> mailing
> >> list for community discussion.
> >>
> >> Regards,
> >>
> >> Kent
> >>
> >> On Sep 13, 2011, at 8:32 AM, Stephen Connolly wrote:
> >>
> >> > Kent,
> >> >
> >> > Are there any plans for JUnit to support some test dependencies, such
> as:
> >> >
> >> > public class OnlyRunTestsThatMakeSenseTest {
> >> >
> >> >  @Test
> >> >  public void basicFunctionalityWorks() {
> >> >    ...
> >> >  }
> >> >
> >> >  @Test
> >> >  @AssumesPasses("basicFunctionalityWorks")
> >> >  public void advancedFunctionalityWorks() {
> >> >    ...
> >> >  }
> >> >
> >> >  @Test
> >> >  @AssumesPasses("basicFunctionalityWorks")
> >> >  public void basicFunctionalityWorksWithBevel() {
> >> >    ...
> >> >  }
> >> >
> >> >  @Test
> >> >
> >>
> >>
> 
@AssumesPasses({"basicFunctionalityWorksWithBevel","advancedFunctionalityWorks"}\
)
> >> >  public void advancedFunctionalityWorksWithBevel() {
> >> >    ...
> >> >  }
> >> >
> >> > }
> >> >
> >> > In the above example, no matter what sorting is applied,
> >> > basicFunctionalityWorks will always be run first, and the other three
> >> > tests will only be run if basicFunctionalityWorks passed.
> >> >
> >> > I see the above being completely in the spirit of unit testing, the
> >> > point with the above is that the @Before and @After's will be run
> >> > around each method, you are just saying that there is no point even
> >> > trying to test the advanced functionality when the basic functionality
> >> > is broken, skip those tests which we know cannot pass. That allows the
> >> > person writing advancedFunctionalityWorks to power through the setup
> >> > that depends on the basic functionality and not have to litter their
> >> > advanced test with asserts that are redundant because of the basic
> >> > functionality. Those people who are relying on side-effects should
> >> > really, for unit tests at least, be invoking the method who's
> >> > side-effects they depend on directly within their test method, rather
> >> > than relying on accidental ordering.
> >> >
> >> > Having said that, a second feature that I think would be good is
> >> > something like a @RunAfter and/or @RunBefore which would ensure that
> >> > the test method is run in sequence even if the before or after tests
> >> > fail/are skipped. with @RunAfter and @RunBefore I still think the
> >> > @Before and @After methods should be invoked in-between, this would be
> >> > moving towards more of a general purpose testing framework as opposed
> >> > to being unit-testing focused, but JUnit is just too good ;-)
> >> >
> >> > Thoughts?
> >> >
> >> > -Stephen
> >>
> >>
> >> ------------------------------------
> >>
> >> Yahoo! Groups Links
> >>
> >>
> >>
> >>
> >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>


[Non-text portions of this message have been removed]

On Wed, Sep 14, 2011 at 10:41 AM, Dale Emery <dale@...> wrote:

>
> A final half-baked thought: What if you ran all the tests, then used the
> dependency graph only to mark the results somehow, so as to highlight the
> tests on which others depend. If you see a swarm of failures, and the
> display indicates that some common prerequisite test also failed, that would
> aid your investigation.
>

This is actually one of the main values of dependent tests: accurate
reporting.

Without dependencies, one test failing might result in "100 Failures", but
when the testing framework knows your dependency graph, it will be "1
Failure, 99 Skips", which helps narrow down what's going on much faster.

Another value is obviously with functional testing, so you avoid using mocks
completely but you also don't spend your time recreating expensive state
before each test method. And you're also testing the state created by your
production code in the previous step, which is much more representative of
what's actually going on.

From what I can gather, dependent tests are also particularly useful for
frameworks like Selenium, which need to test heavily dependent pathways
("Don't test this page if logging in failed", etc...).

Anyway, different purposes. JUnit is for unit testing and you should most
likely not use dependencies for that, but they are very useful for all the
other kinds of tests.

--
Cédric


[Non-text portions of this message have been removed]

Hi Dale,

I think it's an overinterpretation to call this a dependency. The syntax
merely says
that the test makes an assumption. Given such an assumption, the framework
might do a number of things, only one of which is to treat the connection as
a dependency.

In my own work - which of course is not junit-related - I'm looking ways to
specify relations like this *without* calling for any specific behavior on
the
part of the framework.

Of course, the "obvious" and perhaps simplest implementation is to run
the "assumed" test first and skip the test using the assume in the case
where it fails. But I like the idea that the user might express known or
believed-to-be-true logical relations among tests while the framework
would be free to make use of that information in ways not necessarily
predictable by the user.

As an example, imagine a framework that gave a message like
"You stated that TestA assumes TestB but TestA passed even
though TestB failed."

Charlie
On Wed, Sep 14, 2011 at 10:41 AM, Dale Emery <dale@...> wrote:

> **
>
>
> Hi Stephen and all,
>
> > Nope not a mock at all.
> >
> More like TestNG's "dependsOnMethods" feature.
>
> I understand your desire for this. I also understand the JUnit guys'
> reluctance to implement it. My intermediate-ish experience with TestNG
> leaves me with the conclusion that inter-test dependencies lead to madness.
> To be fair, I had already concluded that before working with clients who
> insisted on specifying inter-test dependencies.
>
> My first half-baked thought is to use a Rule to prevent running methods
> that depend on another method that has already failed. But that would work
> only if the tests were run in the appropriate order.
>
> So my next half-baked thought is to write a custom Runner that builds the
> dependency graph and sequences the tests accordingly. Then either the runner
> or a rule could decide whether to run a test. I'm confident that a runner
> could do what you want, but that requires delving into the arcane bits of
> JUnit.
>
> A final half-baked thought: What if you ran all the tests, then used the
> dependency graph only to mark the results somehow, so as to highlight the
> tests on which others depend. If you see a swarm of failures, and the
> display indicates that some common prerequisite test also failed, that would
> aid your investigation.
>
> I've seen situations where some prerequisite test fails, but some of the
> dependent tests pass. Sometimes the prerequisite test is lying to me (and I
> need to fix that). Sometimes the dependency is over-specified--the dependent
> test doesn't depend on everything that the prerequisite test tests (which
> means I need to refactor the prerequisite test and the dependency
> specifications). Sometimes it means that the prerequisite test fails
> intermittently (which means I need to track down the uncontrolled factors
> that affect the test).
>
> Skipping the dependent tests hides that information. Running them and
> marking them would highlight that
>
> Dale
>
> --
> Dale Emery
> Consultant to software teams and leaders
> http://dhemery.com
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

On 14 September 2011 18:25, Tomek Kaczanowski
<kaczanowski.tomek@...> wrote:
> Sound like TestNG dependsOn feature. But from what I know, JUnit is against
> any kind of test dependencies (there was a recent discussion on this mailing
> list, see http://tech.groups.yahoo.com/group/junit/message/23568). A short
> quote from this discussion: "It has always been a deliberate design decision
> in JUnit that we would *not* guarantee the order of test execution."

Ahem...

*caugh*
http://junit.sourceforge.net/javadoc/org/junit/runner/manipulation/Sortable.html
*caugh*

However as that relies on Collections.sort you are not guaranteed to
have all the N*N/2 comparisons made that would be required for a true
sort... hence https://github.com/KentBeck/junit/pull/316

>
> --
> Regards / Pozdrawiam
> Tomek Kaczanowski
> http://kaczanowscy.pl/tomek
>
> 2011/9/14 Stephen Connolly <stephen.alan.connolly@...>
>
>>
>>
>> Nope not a mock at all.
>>
>> Say you have a class List (which is not java.util.List but looks a lot
>> like it in terms of interface)
>>
>> This list class is something you are developing via TDD.
>>
>> You write a whole lot of unit tests for it.
>>
>> Now suppose it happens to be a linked list, and you decide to change
>> from a null for end of list marker to a sentinel instead. All your
>> methods are working now, except for the isEmpty() method, but yet 200+
>> test cases are failing because they all rely on isEmpty working. Which
>> unit test do I try to fix first?
>>
>> with @Assumes, you would have 1 failing test and 199 skipped tests...
>>
>> No mocks involved at all
>>
>>
>> On 14 September 2011 16:34, Carfield Yim <carfield@...> wrote:
>> > It sound like a mock, isn't it?
>> >
>> > On Wed, Sep 14, 2011 at 6:02 PM, Stephen Connolly <
>> > stephen.alan.connolly@...> wrote:
>> >
>> >> Note: I have also posted this to junit-devel@... but
>> >> I think that wider input could be beneficial
>> >>
>> >> Consider the case where you are testing a List class...
>> >>
>> >> we have
>> >>
>> >> public class ListTest {
>> >>
>> >>  @Test
>> >>  public void newListIsEmpty() {
>> >>    assertThat(new List().isEmpty(), is(true);
>> >>  }
>> >>
>> >>  @Test
>> >>  public void newListHasSizeZero() {
>> >>    assertThat(new List().size(), is(0));
>> >>  }
>> >>
>> >>  @Test
>> >>  public void addPutsAnElementIntoAnEmptyList() {
>> >>    List l = new List();
>> >>    l.add(new Object());
>> >>    assertThat(l.isEmpty(), is(false));
>> >>  }
>> >>
>> >>  @Test
>> >>  public void addIncreasesSizeOfPopulatedListByOne() {
>> >>    List l = new List();
>> >>    l.add(new Object());
>> >>    int s = l.size();
>> >>    l.add(new Object());
>> >>    assertThat(l.size(), is(s + 1));
>> >>  }
>> >>
>> >> }
>> >>
>> >> We now want to add some tests of the delete functionality... but the
>> >> reality is that until/unless some of the preceding tests are passing,
>> >> the tests for delete are meaningless. We could have a perfectly
>> >> functional List.delete() method but until such time as the above tests
>> >> are passing, there is no way to tell that the method does not work.
>> >>
>> >> Now I could code my tests like such
>> >>
>> >>  @Test
>> >>  public void deleteIsANoOpOnEmptyList() {
>> >>    List l = new List();
>> >>    assumeThat(l.isEmpty(), is(true));
>> >>    l.delete(new Object());
>> >>  }
>> >>
>> >> But all that I am doing is repeating code from the preceding tests,
>> >> having changed all those tests' assertThat(...)s into assumeThat(...)s
>> >>
>> >> That does not seem agile to me, copy & paste & search & replace... ban
>> >> code smell there
>> >>
>> >> I would much rather be able to annotate the tests with an @Assumes
>> >> annotation that indicates that the test assumes that the specified
>> >> tests are passing, e.g.
>> >>
>> >>  @Test
>> >>  @Assumes("newListIsEmpty")
>> >>  public void deleteIsANoOpOnEmptyList() {
>> >>    List l = new List();
>> >>    l.delete(new Object());
>> >>  }
>> >>
>> >>  @Test
>> >>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>> >>  public void deleteRemovesAnElement() {
>> >>    List l = new List();
>> >>    Object o = new Object();
>> >>    l.add(o);
>> >>    l.delete(o);
>> >>    assertThat(l.isEmpty(), is(true));
>> >>  }
>> >>
>> >> In fact in my initial example of tests, there are some additional
>> >> assumptions that I didn't make explicit
>> >>
>> >>
>> >>  @Test
>> >>  @Assumes("newListIsEmpty")
>> >>  public void addPutsAnElementIntoAnEmptyList() {
>> >>    ...
>> >>  }
>> >>
>> >> and
>> >>
>> >>  @Test
>> >>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>> >>  public void addIncreasesSizeOfPopulatedListByOne() {
>> >>    ...
>> >>  }
>> >>
>> >> Now you could get some of this functionality via a TestRule...
>> >>
>> >> You could watch tests to see if they pass, and skip tests annotated
>> >> with the annotation if assumed functionality is failing, but that
>> >> would result in sporadic failures of, e.g. deleteRemovesAnElement
>> >> because of the failing newListIsEmpty being executed _after_
>> >> deleteRemovesAnElement rather than before.
>> >>
>> >> The simple point is that the test result of deleteRemovesAnElement is
>> >> meaningless until its assumptions are true, and while I could code the
>> >> assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.
>> >>
>> >> Another alternative to @Assumes would be to invoke the assumed
>> >> method(s) at the start of the test, e.g.
>> >>
>> >>  @Test
>> >>  public void deleteRemovesAnElement() {
>> >>    newListIsEmpty(); // verify assumed functionality
>> >>    addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
>> >>    ...
>> >>  }
>> >>
>> >> That gets rid of the C&P&S&R, but there are two issues with that:
>> >>
>> >>  1. We have to manually invoke any setup/tearDown methods, including
>> >> all those of the rules that the test class has... very messy
>> >>
>> >>  2. The test fails when the assumed test fails. In actuality we can
>> >> say nothing at all about whether deleteRemovesAnElement if a
>> >> newListIsEmpty is not passing... yes we could code the test
>> >> differently, but that is just moving our assumptions somewhere else.
>> >>
>> >> I am sure that there are others out there who feel there is a point 3...
>> >>
>> >>  3. We already ran those tests why waste time running them again?
>> >>
>> >> Well the answer to 3 is that these are UNIT tests which should be very
>> >> fast, so what is the harm...
>> >>
>> >> So, in my view, best practice unit testing needs the ability to mark
>> >> tests as assuming that other tests are passing, so that those tests
>> >> can be skipped when the assumptions are known to be failing or
>> >> skipped. [This is a deliberately loaded criteria... if the
>> >> org.junit.runner.Request does not include the assumed test, then that
>> >> test is neither known failing or known skipped, so we can run the test
>> >> and output a warning that the failure may be because of assumed
>> >> functionality... the use case of executing one and only one test
>> >> repeatedly until you get that test passing]
>> >>
>> >> The annotation would have implications on test sorting, as any assumed
>> >> tests would have to always happen before the assuming tests (as long
>> >> as the assumed tests are in the org.junit.runner.Request)
>> >>
>> >> Also might have to be two annotations, e.g.
>> >>
>> >> @Assumes(methodNames)
>> >> @AssumesClasses(classes)
>> >>
>> >> though in my view the @AssumesClasses is less critical, as these are
>> >> UNIT tests and each test class should be independent to a large
>> >> extent. However I am willing to consider that some people may have
>> >> many test classes for one class under test, one test class containing
>> >> all the tests of the constructors, another testing the Add methods,
>> >> etc. in which case an @AssumesClasses annotation makes sense.
>> >>
>> >> Where tests contain a circular dependency, fail/error both tests
>> >>
>> >> Ok, let the critique begin!
>> >>
>> >> -Stephen
>> >>
>> >> P.S.
>> >>
>> >> I pinged Kent with an earlier version of this idea... but I think that
>> >> he missed the point about eliminating C&P&S&R that this feature would
>> >> provide because I didn't frame the idea correctly...
>> >>
>> >> ---------- Forwarded message ----------
>> >> From: "Kent Beck"
>> >> Date: 13 Sep 2011 17:11
>> >> Subject: Re: JUnit and test dependencies
>> >> To: "Stephen Connolly"
>> >>
>> >> Stephen,
>> >>
>> >> Thank you for articulating your idea so clearly. The short answer is
>> that
>> >> no, we don't plan to support dependencies. If I have tests that are slow
>> >> enough that I care about dependencies, my most productive option is
>> >> generally to work on the design of the software until the tests are fast
>> >> enough that I no longer care. That said, my voice is only one of many.
>> The
>> >> longer answer is that I encourage you to post your idea on the JUnit
>> >> mailing
>> >> list for community discussion.
>> >>
>> >> Regards,
>> >>
>> >> Kent
>> >>
>> >> On Sep 13, 2011, at 8:32 AM, Stephen Connolly wrote:
>> >>
>> >> > Kent,
>> >> >
>> >> > Are there any plans for JUnit to support some test dependencies, such
>> as:
>> >> >
>> >> > public class OnlyRunTestsThatMakeSenseTest {
>> >> >
>> >> >  @Test
>> >> >  public void basicFunctionalityWorks() {
>> >> >    ...
>> >> >  }
>> >> >
>> >> >  @Test
>> >> >  @AssumesPasses("basicFunctionalityWorks")
>> >> >  public void advancedFunctionalityWorks() {
>> >> >    ...
>> >> >  }
>> >> >
>> >> >  @Test
>> >> >  @AssumesPasses("basicFunctionalityWorks")
>> >> >  public void basicFunctionalityWorksWithBevel() {
>> >> >    ...
>> >> >  }
>> >> >
>> >> >  @Test
>> >> >
>> >>
>> >>
>>
 @AssumesPasses({"basicFunctionalityWorksWithBevel","advancedFunctionalityWorks"\
})
>> >> >  public void advancedFunctionalityWorksWithBevel() {
>> >> >    ...
>> >> >  }
>> >> >
>> >> > }
>> >> >
>> >> > In the above example, no matter what sorting is applied,
>> >> > basicFunctionalityWorks will always be run first, and the other three
>> >> > tests will only be run if basicFunctionalityWorks passed.
>> >> >
>> >> > I see the above being completely in the spirit of unit testing, the
>> >> > point with the above is that the @Before and @After's will be run
>> >> > around each method, you are just saying that there is no point even
>> >> > trying to test the advanced functionality when the basic functionality
>> >> > is broken, skip those tests which we know cannot pass. That allows the
>> >> > person writing advancedFunctionalityWorks to power through the setup
>> >> > that depends on the basic functionality and not have to litter their
>> >> > advanced test with asserts that are redundant because of the basic
>> >> > functionality. Those people who are relying on side-effects should
>> >> > really, for unit tests at least, be invoking the method who's
>> >> > side-effects they depend on directly within their test method, rather
>> >> > than relying on accidental ordering.
>> >> >
>> >> > Having said that, a second feature that I think would be good is
>> >> > something like a @RunAfter and/or @RunBefore which would ensure that
>> >> > the test method is run in sequence even if the before or after tests
>> >> > fail/are skipped. with @RunAfter and @RunBefore I still think the
>> >> > @Before and @After methods should be invoked in-between, this would be
>> >> > moving towards more of a general purpose testing framework as opposed
>> >> > to being unit-testing focused, but JUnit is just too good ;-)
>> >> >
>> >> > Thoughts?
>> >> >
>> >> > -Stephen
>> >>
>> >>
>> >> ------------------------------------
>> >>
>> >> Yahoo! Groups Links
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > [Non-text portions of this message have been removed]
>> >
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

On 14 September 2011 18:41, Dale Emery <dale@...> wrote:
> Hi Stephen and all,
>
>> Nope not a mock at all.
>>
> More like TestNG's "dependsOnMethods" feature.

A bit, but not completely.

I see dependsOnMethods as being a hard restriction, i.e. stating that
you must run the depended on method first.

Assumes acts of more of a hint that you should run the assumed method first.
also that a failure of the assumed method should imply a failure of
the assuming method (so Assumes does not say that you must not run the
assuming method is the assumed method is failing... in fact you could
if the method with the Assumes passes when it's assumed methods fail,
then that turns the pass into an InvalidAssumptionError ;-) )

>
> I understand your desire for this. I also understand the JUnit guys'
reluctance to implement it. My intermediate-ish experience with TestNG leaves me
with the conclusion that inter-test dependencies lead to madness. To be fair, I
had already concluded that before working with clients who insisted on
specifying inter-test dependencies.
>

I agree, which is why Assumes is a "should" and not a "must"

> My first half-baked thought is to use a Rule to prevent running methods that
depend on another method that has already failed. But that would work only if
the tests were run in the appropriate order.
>
> So my next half-baked thought is to write a custom Runner that builds the
dependency graph and sequences the tests accordingly. Then either the runner or
a rule could decide whether to run a test. I'm confident that a runner could do
what you want, but that requires delving into the arcane bits of JUnit.
>

Already implemented: https://github.com/junit-team/junit.contrib/pull/5

Some notes about the above, the sorting is not perfect, perfect
sorting would require a full bubble sort (i.e. resort until no changes
are made) the sorting just makes a "best effort" to get tests that
Assume after the tests the assume. If you assume a test that makes
other assumptions, there are cases where the assuming test might run
first... I see nothing wrong with that other than I currently don't
see a mechanism to catch the failure and mark it as less critical if
the later run assumed test also fails.

> A final half-baked thought: What if you ran all the tests, then used the
dependency graph only to mark the results somehow, so as to highlight the tests
on which others depend. If you see a swarm of failures, and the display
indicates that some common prerequisite test also failed, that would aid your
investigation.
>

I see merit in that too, like I said, Assumes is a weaker contract than Depends

> I've seen situations where some prerequisite test fails, but some of the
dependent tests pass. Sometimes the prerequisite test is lying to me (and I need
to fix that). Sometimes the dependency is over-specified--the dependent test
doesn't depend on everything that the prerequisite test tests (which means I
need to refactor the prerequisite test and the dependency specifications).
Sometimes it means that the prerequisite test fails intermittently (which means
I need to track down the uncontrolled factors that affect the test).
>
> Skipping the dependent tests hides that information. Running them and marking
them would highlight that
>
> Dale
>
> --
> Dale Emery
> Consultant to software teams and leaders
> http://dhemery.com
>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Thank you, Stefan!

    David

On Wed, Sep 14, 2011 at 4:22 PM, Stefan Birkner <mail@...> wrote:
> Am Montag, den 12.09.2011, 11:40 -0400 schrieb David Saff:
>>
>> On Fri, Sep 9, 2011 at 9:32 AM, David Saff <david@...> wrote:
>> > Yes.  This is a pain for me, as well.
>> >
>> > Sourceforge.net is purely vestigial.  The moment there's a great,
>> > authoritative master location to link to, I'll do what it takes to
>> > kill sourceforge and point it there.
>> >
>> > junit.org is an oddity.  For reasons that predate me and I only
>> dimly
>> > understand, junit.org was registered back in 2000 by Object Mentor.
>> > To my knowledge, it has never been actively maintained by any of the
>> > active maintainers of the JUnit source.  The site currently there
>> > (developed by Ben Rady and a merry crew roughly four years ago) is
>> one
>> > of the better incarnations of the site.  However, its design
>> > presupposed that the junit.org site itself would serve as an active
>> > community, and I think that never fully materialized, with this
>> > mailing list and, now, github, being the main points of vibrancy.
>> >
>> > To me, I think the ultimate solution would be a site accessible at
>> > http://junit.org, backed by github pages at
>> > http://junit-team.github.com/, making available a superset of the
>> > information available at the current junit.org site.  I'd have to
>> talk
>> > to a couple of stakeholders to see if this can be made to happen.
>>
>> I believe everyone concerned is on-board. Once we have most of the
>> content currently at junit.org copied over to
>> https://github.com/KentBeck/junit/tree/gh-pages, the owner of
>> junit.org has agreed to flip the CNAME, and we'll be serving junit.org
>> from github.
>>
>> So now, any volunteers to help port the docs? :-)
> I started to move the content of junit.org to github. Will need one or
> two days.
>
>> David Saff
>>
>> >
>> >   David Saff
>> >
>> > On Thu, Sep 8, 2011 at 5:43 PM, Stefan Birkner
>> <mail@...> wrote:
>> >> The information about JUnit is across three different places:
>> junit.org, junit.sourceforge.net and github. Additionally some
>> information is missing or very hard to find (new features like rules,
>> current version, how to contribute, ...)
>> >>
>> >> I would like to have one site with all the information. This makes
>> it easier to find the information and JUnit would look more
>> professional. IMHO the github pages are a good place for the JUnit
>> documentation, because we can use all the git(hub) features for
>> creating the documentation.
>> >>
>> >> What do you think?
>> >>
>> >> Stefan Birkner
>> >>
>> >>
>> >>
>> >> ------------------------------------
>> >>
>> >> Yahoo! Groups Links
>> >>
>> >>
>> >>
>> >>
>> >
>>
>>
>>
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>