Skip to search.

Breaking News Visit Yahoo! News for the latest.

×Close this window

junit · JUnit, the Java unit testing framework written by Kent Beck and Erich Gamma.

The Yahoo! Groups Product Blog

Check it out!

Group Information

  • Members: 31224
  • Category: Java
  • Founded: Nov 6, 2000
  • Language: English
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Real people. Real stories. See how Yahoo! Groups impacts members worldwide.

Messages

Advanced
Messages Help
Messages 23749 - 23778 of 24387   Oldest  |  < Older  |  Newer >  |  Newest
Messages: Show Message Summaries Sort by Date ^  
#23749 From: "sinu" <srijesh_a@...>
Date: Mon Oct 31, 2011 11:00 pm
Subject: Re: Better parameterised tests
srijesh_a
Send Email Send Email
 
Hello,

Tye link does not work. Could you please also post the correct link.

I am having suite filter runner with classes that fall into that suite. One of
the classes I am using Extended Runner to run a specific test case for multiple
times.  Please let me know how can we over ride the runners for a specific test
case.

Thanks
Rajesh
--- In junit@yahoogroups.com, "p.lipinski" <p.lipinski@...> wrote:
>
> Hi
> Some time ago I've written a runner for JUnit which is a replacement for the
standard Parameterized runner - is easier & more flexible to use, gives nicer
output in IDEs. I've made it available at http://code.google.com/p/junitparams.
The website has a description of all the features.
>
> JUnitParams now updated to work properly with JUnit 4.10.
>
> Pawel
>

#23750 From: Dawid Weiss <dawid.weiss@...>
Date: Mon Oct 31, 2011 5:58 pm
Subject: Re: Improvements to Assume (issue 355).
dawid.weiss@...
Send Email Send Email
 
Thanks David!
Dawid

On Mon, Oct 31, 2011 at 6:00 PM, David Saff <david@...> wrote:

> **
>
>
> All pull requests have their own issue numbers: it's awkward, but you
> get used to it. If the name of the pull request (or possibly the
> description) contains the original issue number in one of the formats
> that GitHub expects (My favorite is, for example, "Fixes gh-355:
> explict messages in Assume"), then a link to the pull request will
> automatically get added to the original issue.
>
> Thanks!
>
> David
>
>
> On Sun, Oct 30, 2011 at 4:17 PM, Dawid Weiss <dawid.weiss@...>
> wrote:
> > Hi,
> >
> > I've created an issue to extend Assume to contain explicit messages/
> > Throwable cause if needed. The issue includes a patch:
> > https://github.com/KentBeck/junit/issues/355
> >
> > Can anybody enlighten me if it's possible to attach a pull request to
> > an _existing_ issue in github? I'm used to Jira and the way I usually
> > work is create an issue, create a git branch with the issue's number,
> > then fold in. With github pull requests seem to create a new issue so
> > there's no way to have the issue number before (?).
> >
> > Dawid
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>


[Non-text portions of this message have been removed]

#23751 From: Dawid Weiss <dawid.weiss@...>
Date: Tue Nov 1, 2011 4:53 pm
Subject: Re: Re: Better parameterised tests
dawid.weiss@...
Send Email Send Email
 
It does work if you omit the final dot...

http://code.google.com/p/junitparams

Dawid

On Tue, Nov 1, 2011 at 12:00 AM, sinu <srijesh_a@...> wrote:

> **
>
>
>
> Hello,
>
> Tye link does not work. Could you please also post the correct link.
>
> I am having suite filter runner with classes that fall into that suite.
> One of the classes I am using Extended Runner to run a specific test case
> for multiple times. Please let me know how can we over ride the runners for
> a specific test case.
>
> Thanks
> Rajesh
>
> --- In junit@yahoogroups.com, "p.lipinski" <p.lipinski@...> wrote:
> >
> > Hi
> > Some time ago I've written a runner for JUnit which is a replacement for
> the standard Parameterized runner - is easier & more flexible to use, gives
> nicer output in IDEs. I've made it available at
> http://code.google.com/p/junitparams. The website has a description of
> all the features.
> >
> > JUnitParams now updated to work properly with JUnit 4.10.
> >
> > Pawel
> >
>
>
>


[Non-text portions of this message have been removed]

#23752 From: "sschroev" <stephan202@...>
Date: Tue Nov 1, 2011 5:57 pm
Subject: Re: Better parameterised tests
sschroev
Send Email Send Email
 
The URL was auto-linked incorrectly; the trailing dot was added to the href. The
link is: http://code.google.com/p/junitparams/

--- In junit@yahoogroups.com, "sinu" <srijesh_a@...> wrote:
>
>
> Hello,
>
> Tye link does not work. Could you please also post the correct link.
>
> I am having suite filter runner with classes that fall into that suite. One of
the classes I am using Extended Runner to run a specific test case for multiple
times.  Please let me know how can we over ride the runners for a specific test
case.
>
> Thanks
> Rajesh
> --- In junit@yahoogroups.com, "p.lipinski" <p.lipinski@> wrote:
> >
> > Hi
> > Some time ago I've written a runner for JUnit which is a replacement for the
standard Parameterized runner - is easier & more flexible to use, gives nicer
output in IDEs. I've made it available at http://code.google.com/p/junitparams.
The website has a description of all the features.
> >
> > JUnitParams now updated to work properly with JUnit 4.10.
> >
> > Pawel
> >
>

#23753 From: Matthew Farwell <matthew@...>
Date: Thu Nov 3, 2011 9:50 am
Subject: Re: Open up Description class to allow arbitrary test hierarchies?
mjfarwell
Send Email Send Email
 
Hi,

I'm interested in this, though for slightly different reasons, maybe we
could exchange ideas on this. I haven't exactly thought this through in
detail, but I think my requirements come down to being able to store
arbitrary information in a Description, a key.

I'm interesting in testing with Scala, and there are (at least) two generic
problems:

1) Tests which are not linked to a specific method, but are run as separate
tests. Scalatest for example allows the following

@RunWith(classOf[JUnitRunner])
class FileSpec extends FlatSpec with ShouldMatchers {

   "A Stack" should "pop values in last-in-first-out order" in {
   }

   it should "throw NoSuchElementException if an empty stack is popped" in {
   }
}

This creates a single method with the two tests within. However, the
JUnitRunner runs the single method but records the two tests separately.
This can be seen in Eclipse, see attached scalatest1.png. Now if I double
click on the test name in the junit view, it tries to find a method of that
name, and fails. Obviously, to correct this would require a fix to Eclipse,
but if I could store the file & line number instead of the method name, I
could jump to the appropriate line.

2) Scala allows spaces and other characters in method names, but these
names are munged on compilation. For instance, space gets translated to
$0020.

class FileTest extends AssertionsForJUnit {
@Test def `foo bar`() = {

}
}

Here the jump to in Eclipse works correctly, but the displayed description
in Eclipse isn't nice, see scalatest2.png.

So in both cases, the string I would like to display (in Eclipse, surefire
whatever) doesn't match the method name, either because there is no method
name or because It's not the same thing. I think that I could work around
most issues is Eclipse etc if the test runner could add some arbitrary
information to the description, which it could subsequently use. A string
would be fine, because I could define the contents of that String, but
arbitrary data would be even better. String would be simpler for everyone I
think.

Please note that I'm just thinking aloud at the minute, and I haven't
actually done any proof of concepts as yet, but if you have any ideas,
please feel free to take this to email.

Matthew Farwell.


[Non-text portions of this message have been removed]

#23754 From: Bill Venners <bill@...>
Date: Fri Nov 4, 2011 10:01 pm
Subject: Re: Open up Description class to allow arbitrary test hierarchies?
billvenners
Send Email Send Email
 
Hi Matthew,

Actually we are planning to "fix Eclipse" very soon. The next release of
ScalaTest, version 2.0, will include enhancements to support IDE integration,
and before releasing 2.0 we're going to help do the actual integration work for
Eclipse, IDEA, and NetBeans. That way we can be sure we have it right before
releasing the ScalaTest enhancements. I would like your feedback on it. I'll
post announcements of preview snapshots on the scalatest-users mailing list, so
please join that if you'd like to give input.

Thanks.

Bill
----
Bill Venners
Artima, Inc.
http://www.artima.com

On Nov 3, 2011, at 5:50 AM, Matthew Farwell <matthew@...> wrote:

> Hi,
>
> I'm interested in this, though for slightly different reasons, maybe we
> could exchange ideas on this. I haven't exactly thought this through in
> detail, but I think my requirements come down to being able to store
> arbitrary information in a Description, a key.
>
> I'm interesting in testing with Scala, and there are (at least) two generic
> problems:
>
> 1) Tests which are not linked to a specific method, but are run as separate
> tests. Scalatest for example allows the following
>
> @RunWith(classOf[JUnitRunner])
> class FileSpec extends FlatSpec with ShouldMatchers {
>
> "A Stack" should "pop values in last-in-first-out order" in {
> }
>
> it should "throw NoSuchElementException if an empty stack is popped" in {
> }
> }
>
> This creates a single method with the two tests within. However, the
> JUnitRunner runs the single method but records the two tests separately.
> This can be seen in Eclipse, see attached scalatest1.png. Now if I double
> click on the test name in the junit view, it tries to find a method of that
> name, and fails. Obviously, to correct this would require a fix to Eclipse,
> but if I could store the file & line number instead of the method name, I
> could jump to the appropriate line.
>
> 2) Scala allows spaces and other characters in method names, but these
> names are munged on compilation. For instance, space gets translated to
> $0020.
>
> class FileTest extends AssertionsForJUnit {
> @Test def `foo bar`() = {
>
> }
> }
>
> Here the jump to in Eclipse works correctly, but the displayed description
> in Eclipse isn't nice, see scalatest2.png.
>
> So in both cases, the string I would like to display (in Eclipse, surefire
> whatever) doesn't match the method name, either because there is no method
> name or because It's not the same thing. I think that I could work around
> most issues is Eclipse etc if the test runner could add some arbitrary
> information to the description, which it could subsequently use. A string
> would be fine, because I could define the contents of that String, but
> arbitrary data would be even better. String would be simpler for everyone I
> think.
>
> Please note that I'm just thinking aloud at the minute, and I haven't
> actually done any proof of concepts as yet, but if you have any ideas,
> please feel free to take this to email.
>
> Matthew Farwell.
>
> [Non-text portions of this message have been removed]
>
>


[Non-text portions of this message have been removed]

#23755 From: Dawid Weiss <dawid.weiss@...>
Date: Sat Nov 5, 2011 10:13 am
Subject: Re: Open up Description class to allow arbitrary test hierarchies?
dawid.weiss@...
Send Email Send Email
 
I think your reasons are aligned with mine -- the problem is that a test's
visual description for the user interface and where it "points to" should
be separate things. My thoughts on the subject so far are that Description
should provide a String with the human description and some sort of an
(empty) interface, let's call it a TestSource which would provide
additional hints which resource to open and where. So something like this
(ideas, not implementation):

Description {
   String getDescription();
   List<TestSource> getSource();
}

I can think of numerous advantages of such an approach, for example a
single parameterized test could open the test case's source code by
returning:

JavaTestSource {
   String className();
   String methodName(); // full signature?
}

but it could also point to the file with parameterized test data and open
_that_ as well, as in:

FileTestSource {
   File file();
   int line();
}

Not giving much thought about the implementation, but you get the idea.
Obviously this would be a major pain for all tools that use JUnit because
this would pretty much break backwards compatibility... but I'm all for it
if a new major revision of JUnit is to ever see the light of day.

Temporarily, and interestingly, Eclipse (and Ant and other tools I think)
can be cheated a bit -- they all use a JUnit version provided by the user's
classpath via reflective access typically (they don't have fixed version
dependency). If you "classpath-shadow" your JUnit's Description (by
prepending classpath with your own Description class, keeping the same API)
you can do anything you with with the descriptions... This is a dirty hack
though :(

Dawid

On Thu, Nov 3, 2011 at 10:50 AM, Matthew Farwell <matthew@...>wrote:

> **
>
>
> Hi,
>
> I'm interested in this, though for slightly different reasons, maybe we
> could exchange ideas on this. I haven't exactly thought this through in
> detail, but I think my requirements come down to being able to store
> arbitrary information in a Description, a key.
>
> I'm interesting in testing with Scala, and there are (at least) two generic
> problems:
>
> 1) Tests which are not linked to a specific method, but are run as separate
> tests. Scalatest for example allows the following
>
> @RunWith(classOf[JUnitRunner])
> class FileSpec extends FlatSpec with ShouldMatchers {
>
> "A Stack" should "pop values in last-in-first-out order" in {
> }
>
> it should "throw NoSuchElementException if an empty stack is popped" in {
> }
> }
>
> This creates a single method with the two tests within. However, the
> JUnitRunner runs the single method but records the two tests separately.
> This can be seen in Eclipse, see attached scalatest1.png. Now if I double
> click on the test name in the junit view, it tries to find a method of that
> name, and fails. Obviously, to correct this would require a fix to Eclipse,
> but if I could store the file & line number instead of the method name, I
> could jump to the appropriate line.
>
> 2) Scala allows spaces and other characters in method names, but these
> names are munged on compilation. For instance, space gets translated to
> $0020.
>
> class FileTest extends AssertionsForJUnit {
> @Test def `foo bar`() = {
>
> }
> }
>
> Here the jump to in Eclipse works correctly, but the displayed description
> in Eclipse isn't nice, see scalatest2.png.
>
> So in both cases, the string I would like to display (in Eclipse, surefire
> whatever) doesn't match the method name, either because there is no method
> name or because It's not the same thing. I think that I could work around
> most issues is Eclipse etc if the test runner could add some arbitrary
> information to the description, which it could subsequently use. A string
> would be fine, because I could define the contents of that String, but
> arbitrary data would be even better. String would be simpler for everyone I
> think.
>
> Please note that I'm just thinking aloud at the minute, and I haven't
> actually done any proof of concepts as yet, but if you have any ideas,
> please feel free to take this to email.
>
> Matthew Farwell.
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

#23756 From: Matthew Farwell <matthew@...>
Date: Sat Nov 5, 2011 6:43 am
Subject: Re: Open up Description class to allow arbitrary test hierarchies?
mjfarwell
Send Email Send Email
 
Hi Bill,

I did the JUnit integration for Scala-IDE, so yes I would like to be
involved.

Currently Scala-IDE uses the Junit plugin for running and displaying the
test results, and it would obviously be better if we could avoid forking
that. But for instance the jump to method doesn't work for FlatSpec stuff,
and will never work because it's trying to find a method called xxx where
xxx is the first word in the description of the test. Which obviously will
never work. For Scala-IDE we can work around this using aspects, but it
would be nice it this worked nicely. Hence my interest in changing
Description.

Matthew.

Le 4 novembre 2011 23:01, Bill Venners <bill@...> a écrit :

> **
>
>
> Hi Matthew,
>
> Actually we are planning to "fix Eclipse" very soon. The next release of
> ScalaTest, version 2.0, will include enhancements to support IDE
> integration, and before releasing 2.0 we're going to help do the actual
> integration work for Eclipse, IDEA, and NetBeans. That way we can be sure
> we have it right before releasing the ScalaTest enhancements. I would like
> your feedback on it. I'll post announcements of preview snapshots on the
> scalatest-users mailing list, so please join that if you'd like to give
> input.
>
> Thanks.
>
> Bill
> ----
> Bill Venners
> Artima, Inc.
> http://www.artima.com
>
>
> On Nov 3, 2011, at 5:50 AM, Matthew Farwell <matthew@...> wrote:
>
> > Hi,
> >
> > I'm interested in this, though for slightly different reasons, maybe we
> > could exchange ideas on this. I haven't exactly thought this through in
> > detail, but I think my requirements come down to being able to store
> > arbitrary information in a Description, a key.
> >
> > I'm interesting in testing with Scala, and there are (at least) two
> generic
> > problems:
> >
> > 1) Tests which are not linked to a specific method, but are run as
> separate
> > tests. Scalatest for example allows the following
> >
> > @RunWith(classOf[JUnitRunner])
> > class FileSpec extends FlatSpec with ShouldMatchers {
> >
> > "A Stack" should "pop values in last-in-first-out order" in {
> > }
> >
> > it should "throw NoSuchElementException if an empty stack is popped" in {
> > }
> > }
> >
> > This creates a single method with the two tests within. However, the
> > JUnitRunner runs the single method but records the two tests separately.
> > This can be seen in Eclipse, see attached scalatest1.png. Now if I double
> > click on the test name in the junit view, it tries to find a method of
> that
> > name, and fails. Obviously, to correct this would require a fix to
> Eclipse,
> > but if I could store the file & line number instead of the method name, I
> > could jump to the appropriate line.
> >
> > 2) Scala allows spaces and other characters in method names, but these
> > names are munged on compilation. For instance, space gets translated to
> > $0020.
> >
> > class FileTest extends AssertionsForJUnit {
> > @Test def `foo bar`() = {
> >
> > }
> > }
> >
> > Here the jump to in Eclipse works correctly, but the displayed
> description
> > in Eclipse isn't nice, see scalatest2.png.
> >
> > So in both cases, the string I would like to display (in Eclipse,
> surefire
> > whatever) doesn't match the method name, either because there is no
> method
> > name or because It's not the same thing. I think that I could work around
> > most issues is Eclipse etc if the test runner could add some arbitrary
> > information to the description, which it could subsequently use. A string
> > would be fine, because I could define the contents of that String, but
> > arbitrary data would be even better. String would be simpler for
> everyone I
> > think.
> >
> > Please note that I'm just thinking aloud at the minute, and I haven't
> > actually done any proof of concepts as yet, but if you have any ideas,
> > please feel free to take this to email.
> >
> > Matthew Farwell.
> >
> > [Non-text portions of this message have been removed]
> >
> >
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

#23757 From: Kevin Cooney <kcooney@...>
Date: Mon Nov 7, 2011 8:22 pm
Subject: Re: Open up Description class to allow arbitrary test hierarchies?
kevincooney2000
Send Email Send Email
 
I think that the reason why Description is final is because it is a value
class, and there is code (especially in filters) that expects
Description.equals() to obey the general contract of Object.equals().
Unfortunately, there is "no way to extend an instantiable class and add a
value component while preserving the equals contract" (quoted from
Effective Java, second edition, page 38).

It's perfectly fine to add optional properties to the existing Description
class, as long as Description.equals() and Description.hashCode() are
updated, and all of the properties that are added have equals() methods
that obey the general contract of Object.equals().

-- Kevin

On Fri, Nov 4, 2011 at 11:43 PM, Matthew Farwell <matthew@...>wrote:

> Hi Bill,
>
> I did the JUnit integration for Scala-IDE, so yes I would like to be
> involved.
>
> Currently Scala-IDE uses the Junit plugin for running and displaying the
> test results, and it would obviously be better if we could avoid forking
> that. But for instance the jump to method doesn't work for FlatSpec stuff,
> and will never work because it's trying to find a method called xxx where
> xxx is the first word in the description of the test. Which obviously will
> never work. For Scala-IDE we can work around this using aspects, but it
> would be nice it this worked nicely. Hence my interest in changing
> Description.
>
> Matthew.
>
> Le 4 novembre 2011 23:01, Bill Venners <bill@...> a écrit :
>
> > **
> >
> >
> > Hi Matthew,
> >
> > Actually we are planning to "fix Eclipse" very soon. The next release of
> > ScalaTest, version 2.0, will include enhancements to support IDE
> > integration, and before releasing 2.0 we're going to help do the actual
> > integration work for Eclipse, IDEA, and NetBeans. That way we can be sure
> > we have it right before releasing the ScalaTest enhancements. I would
> like
> > your feedback on it. I'll post announcements of preview snapshots on the
> > scalatest-users mailing list, so please join that if you'd like to give
> > input.
> >
> > Thanks.
> >
> > Bill
> > ----
> > Bill Venners
> > Artima, Inc.
> > http://www.artima.com
> >
> >
> > On Nov 3, 2011, at 5:50 AM, Matthew Farwell <matthew@...>
> wrote:
> >
> > > Hi,
> > >
> > > I'm interested in this, though for slightly different reasons, maybe we
> > > could exchange ideas on this. I haven't exactly thought this through in
> > > detail, but I think my requirements come down to being able to store
> > > arbitrary information in a Description, a key.
> > >
> > > I'm interesting in testing with Scala, and there are (at least) two
> > generic
> > > problems:
> > >
> > > 1) Tests which are not linked to a specific method, but are run as
> > separate
> > > tests. Scalatest for example allows the following
> > >
> > > @RunWith(classOf[JUnitRunner])
> > > class FileSpec extends FlatSpec with ShouldMatchers {
> > >
> > > "A Stack" should "pop values in last-in-first-out order" in {
> > > }
> > >
> > > it should "throw NoSuchElementException if an empty stack is popped"
> in {
> > > }
> > > }
> > >
> > > This creates a single method with the two tests within. However, the
> > > JUnitRunner runs the single method but records the two tests
> separately.
> > > This can be seen in Eclipse, see attached scalatest1.png. Now if I
> double
> > > click on the test name in the junit view, it tries to find a method of
> > that
> > > name, and fails. Obviously, to correct this would require a fix to
> > Eclipse,
> > > but if I could store the file & line number instead of the method
> name, I
> > > could jump to the appropriate line.
> > >
> > > 2) Scala allows spaces and other characters in method names, but these
> > > names are munged on compilation. For instance, space gets translated to
> > > $0020.
> > >
> > > class FileTest extends AssertionsForJUnit {
> > > @Test def `foo bar`() = {
> > >
> > > }
> > > }
> > >
> > > Here the jump to in Eclipse works correctly, but the displayed
> > description
> > > in Eclipse isn't nice, see scalatest2.png.
> > >
> > > So in both cases, the string I would like to display (in Eclipse,
> > surefire
> > > whatever) doesn't match the method name, either because there is no
> > method
> > > name or because It's not the same thing. I think that I could work
> around
> > > most issues is Eclipse etc if the test runner could add some arbitrary
> > > information to the description, which it could subsequently use. A
> string
> > > would be fine, because I could define the contents of that String, but
> > > arbitrary data would be even better. String would be simpler for
> > everyone I
> > > think.
> > >
> > > Please note that I'm just thinking aloud at the minute, and I haven't
> > > actually done any proof of concepts as yet, but if you have any ideas,
> > > please feel free to take this to email.
> > >
> > > Matthew Farwell.
> > >
> > > [Non-text portions of this message have been removed]
> > >
> > >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>


[Non-text portions of this message have been removed]

#23758 From: "Phil A" <padamstx@...>
Date: Mon Nov 7, 2011 3:24 pm
Subject: junit report formatting question
padamstx
Send Email Send Email
 
I'm using a test suite class to collect up a bunch of individual testcase
classes.   I also use the "junitreport" ant task to create an html report of the
test results.   Currently, my test results are formatted such that all the
individual test methods are organized only the test suite classname in the
report, and I lose visibility as to which testcase class the test methods belong
to.

Instead, I would like the test results to be organized such that the individual
testcase classnames appear in the report with the associated results for each of
their test methods.

Does anyone know if it's possible to configure the "junitreport" ant task to do
this?

Thanks

#23759 From: David Saff <david@...>
Date: Wed Nov 9, 2011 12:48 am
Subject: Re: junit report formatting question
dsaff
Send Email Send Email
 
Phil,

Hopefully someone here knows.  However, the junitreport task was
written by people on the ant project, so you may want to ask on their
lists, as well.

Good luck,

    David

On Mon, Nov 7, 2011 at 10:24 AM, Phil A <padamstx@...> wrote:
> I'm using a test suite class to collect up a bunch of individual testcase
classes.   I also use the "junitreport" ant task to create an html report of the
test results.   Currently, my test results are formatted such that all the
individual test methods are organized only the test suite classname in the
report, and I lose visibility as to which testcase class the test methods belong
to.
>
> Instead, I would like the test results to be organized such that the
individual testcase classnames appear in the report with the associated results
for each of their test methods.
>
> Does anyone know if it's possible to configure the "junitreport" ant task to
do this?
>
> Thanks
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

#23760 From: Dawid Weiss <dawid.weiss@...>
Date: Wed Nov 9, 2011 7:35 am
Subject: Re: junit report formatting question
dawid.weiss@...
Send Email Send Email
 
Hi Phil,

ANT's junit task can include multiple <formatter> sub-elements. For your
use-case I would suggest:

1) to use the default XML formatter and post-process the output using XSLT
to emit a HTML with individual test cases,
2) write a custome formatter class (very simple) that emits individual
plain-text or HTML files for each test case or suite, then aggregate.
3) write a formatter that System.println's directly, without creating any
files. Your "report" is then the build file's output.

We are using option (3) along with a custom runner, but you can peek here
at what it looks like:
http://builds.carrot2.org/download/C2HEAD-CORE-JOB1/build_logs/C2HEAD-CORE-JOB1-\
975.log

Dawid

On Wed, Nov 9, 2011 at 1:48 AM, David Saff <david@...> wrote:

> **
>
>
> Phil,
>
> Hopefully someone here knows. However, the junitreport task was
> written by people on the ant project, so you may want to ask on their
> lists, as well.
>
> Good luck,
>
> David
>
>
> On Mon, Nov 7, 2011 at 10:24 AM, Phil A <padamstx@...> wrote:
> > I'm using a test suite class to collect up a bunch of individual
> testcase classes.   I also use the "junitreport" ant task to create an html
> report of the test results.   Currently, my test results are formatted such
> that all the individual test methods are organized only the test suite
> classname in the report, and I lose visibility as to which testcase class
> the test methods belong to.
> >
> > Instead, I would like the test results to be organized such that the
> individual testcase classnames appear in the report with the associated
> results for each of their test methods.
> >
> > Does anyone know if it's possible to configure the "junitreport" ant
> task to do this?
> >
> > Thanks
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>


[Non-text portions of this message have been removed]

#23761 From: Neale Upstone <neale.upstone@...>
Date: Thu Nov 10, 2011 10:40 am
Subject: Re: Open up Description class to allow arbitrary test hierarchies?
nealeu
Send Email Send Email
 
So does this mean that adding a Map<String,Object> is being considered
(i.e. having Description implement Map) ?

On 7 November 2011 20:22, Kevin Cooney <kcooney@...> wrote:

>
> It's perfectly fine to add optional properties to the existing Description
> class, as long as Description.equals() and Description.hashCode() are
> updated, and all of the properties that are added have equals() methods
> that obey the general contract of Object.equals().
>


[Non-text portions of this message have been removed]

#23762 From: "Jason_Bodnar" <jason@...>
Date: Thu Nov 10, 2011 5:21 pm
Subject: Customizing the "got ..." message for custom Matchers
Jason_Bodnar
Send Email Send Email
 
Hi,

Is it possible to customize the "got ..." message when writing custom Matchers?
I found this message in the Archives:

http://groups.yahoo.com/group/junit/message/21939

which talks about a describeMismatch method but I don't see in the javadocs for
the Matcher or SelfDescribing interfaces.

Am I missing something?

Thanks,

Jason

#23763 From: Kevin Cooney <kcooney@...>
Date: Thu Nov 10, 2011 5:18 pm
Subject: Re: Open up Description class to allow arbitrary test hierarchies?
kevincooney2000
Send Email Send Email
 
I'm not sure what David is considering. I'm just pointing out why making
Description final is a good idea.

-- Kevin


On Thu, Nov 10, 2011 at 2:40 AM, Neale Upstone
<neale.upstone@...>wrote:

> **
>
>
> So does this mean that adding a Map<String,Object> is being considered
> (i.e. having Description implement Map) ?
>
>
> On 7 November 2011 20:22, Kevin Cooney <kcooney@...> wrote:
>
> >
> > It's perfectly fine to add optional properties to the existing
> Description
> > class, as long as Description.equals() and Description.hashCode() are
> > updated, and all of the properties that are added have equals() methods
> > that obey the general contract of Object.equals().
> >
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

#23764 From: Esko Luontola <esko.luontola@...>
Date: Fri Nov 11, 2011 6:31 am
Subject: Re: Customizing the "got ..." message for custom Matchers
egeluontola
Send Email Send Email
 
JUnit uses version 1.1 of the Hamcrest library and it doesn't have that
method. Hamcrest 1.2 and newer have it.


Jason_Bodnar wrote on 10.11.2011 19:21:
> Hi,
>
> Is it possible to customize the "got ..." message when writing custom
> Matchers? I found this message in the Archives:
>
> http://groups.yahoo.com/group/junit/message/21939
>
> which talks about a describeMismatch method but I don't see in the
> javadocs for the Matcher or SelfDescribing interfaces.
>
> Am I missing something?
>
> Thanks,
>
> Jason
>

--
Esko Luontola
www.orfjackal.net

#23765 From: "tibord47" <tibord47@...>
Date: Sat Nov 19, 2011 12:28 pm
Subject: How to announce all to features acceptance. The aceeptance process takes long.
tibord47
Send Email Send Email
 
We are all developing new features and bug fixes in to JUnit. You had to already
see that this is not short time to accept a new design.

When I submit something, then only Kent and David are announced. I would expect
that much more developers are announced as well, or even the entire group. This
may speed up.

What are you suggesting to speed up the acceptance?

#23766 From: "tibord47" <tibord47@...>
Date: Mon Nov 21, 2011 6:32 pm
Subject: JUnit Developers Group on the LinkedIn
tibord47
Send Email Send Email
 
Hi All,

I have right now created an open group on the LinkedIn for the JUnit which we
can enjoy.

You are All Welcome to join!

It is a good style that the professionals use in order to share their knowledge,
code, asking questions, sending announcements, or even posting jobs, and
discussing and clarifying detailed issues regarding the technologies -JUnit in
our case.

http://www.linkedin.com/groups?home=&gid=4184979&trk=anet_ug_hm

With Best Regards,
Tibor

#23767 From: "foo_bar_baz_qux" <dimitar.dimitrov@...>
Date: Sun Dec 11, 2011 2:10 pm
Subject: Re: Better parameterised tests
foo_bar_baz_qux
Send Email Send Email
 
I was going to start a new thread, but I'll plug in here, to keep the discussion
in one place.

I have had the same problem, different solution and the patch is in github:
https://github.com/KentBeck/junit/pull/145

Florian has same problem, different approach:
https://github.com/KentBeck/junit/pull/358

May the Powers That Be realize that this is a very annoying aspect of JUnit
(forcing us to count lines when a test fails) and consider including any of
these patches in the JUnit tree.

Best regards,
Dimitar

#23768 From: "jfuerth" <jfuerth@...>
Date: Wed Dec 14, 2011 10:01 pm
Subject: Reporting on test runs from a custom Runner
jfuerth
Send Email Send Email
 
Hi everyone,

I've created a custom Runner meant to help with performance and scalability
testing. It lets you create tests like this:

@RunWith(PerfRunner.class)
public class MultiThreadUsageExampleTest {

   @Test
   public void demonstrationTest(
       @Varying(from=1, to=10) int threadCount,
       @Varying(from=100000, to=1000000, step=100000) int entryCount) {

       // code here that relies on threadCount and entryCount
   }

}

In the above example, my runner calls demonstrationTest() 100 times, with
parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10, 1000000).

What I want to do now is produce charts of how long each call took. There are a
lot of approaches, but I don't know which is "right."

1. Register a TestListener and write my own reports to disk from it (but is
there a generic way to register listeners?)
2. Create my own main program that runs the test suite, and have it create the
report when the suite is done (maybe this is the only way to register a
TestListener anyhow?)
3. Transform the XML output of the ant junit task (but then it's bound to Ant)
4. Transform the XML output of the maven-surefure-plugin (but then it's bound to
Maven)
5. Write performance results to disk directly from my TestRunner (should work in
any environment, but feels like a dirty approach)

Other ideas are welcome too, of course. My test runner is open source and
published on GitHub; I want a solution that can work for everyone. So I think
ideas 3 and 4 are out.

Has anyone else thought about getting data from custom test runners into
human-readable reports?

-Jonathan

#23769 From: "Georg" <le_garcon_enerve@...>
Date: Thu Dec 15, 2011 10:45 am
Subject: Re: Reporting on test runs from a custom Runner
le_garcon_en...
Send Email Send Email
 
Hi!

Just my 2 cents:
  - an ant-script could still be used in maven (it has plugins for that purpose).
It may need a bit of tinkering - I'm not very good in maven and ant.
  - Code injection via AspectJ could work well. Place a cutpoint on the method
looping over the test again and again and measure the time before and after.

Cheers,
  Georg

--- In junit@yahoogroups.com, "jfuerth" <jfuerth@...> wrote:
>
> Hi everyone,
>
> I've created a custom Runner meant to help with performance and scalability
testing. It lets you create tests like this:
>
> @RunWith(PerfRunner.class)
> public class MultiThreadUsageExampleTest {
>
>   @Test
>   public void demonstrationTest(
>       @Varying(from=1, to=10) int threadCount,
>       @Varying(from=100000, to=1000000, step=100000) int entryCount) {
>
>       // code here that relies on threadCount and entryCount
>   }
>
> }
>
> In the above example, my runner calls demonstrationTest() 100 times, with
parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10, 1000000).
>
> What I want to do now is produce charts of how long each call took. There are
a lot of approaches, but I don't know which is "right."
>
> 1. Register a TestListener and write my own reports to disk from it (but is
there a generic way to register listeners?)
> 2. Create my own main program that runs the test suite, and have it create the
report when the suite is done (maybe this is the only way to register a
TestListener anyhow?)
> 3. Transform the XML output of the ant junit task (but then it's bound to Ant)
> 4. Transform the XML output of the maven-surefure-plugin (but then it's bound
to Maven)
> 5. Write performance results to disk directly from my TestRunner (should work
in any environment, but feels like a dirty approach)
>
> Other ideas are welcome too, of course. My test runner is open source and
published on GitHub; I want a solution that can work for everyone. So I think
ideas 3 and 4 are out.
>
> Has anyone else thought about getting data from custom test runners into
human-readable reports?
>
> -Jonathan
>

#23770 From: Matthew Farwell <matthew@...>
Date: Thu Dec 15, 2011 12:40 pm
Subject: Re: Reporting on test runs from a custom Runner
mjfarwell
Send Email Send Email
 
Salut,

The best way would be to create your own RunListener (btw TestListener is
JUnit3, RunListener is Junit4). Then, you can plug this into standard
tools. For instance, in maven surefire, you can specify:

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <version>2.10</version>
   <configuration>
     <properties>
       <property>
         <name>listener</name>

<value>com.mycompany.MyResultListener,com.mycompany.MyResultListener2</value>
       </property>
   </configuration>
</plugin>

I think you can do similar things with ant. So if you do 1, 3 and 4 aren't
a problem.

Cordialement,

Matthew Farwell.

Le 14 décembre 2011 23:01, jfuerth <jfuerth@...> a écrit :

> **
>
> What I want to do now is produce charts of how long each call took. There
> are a lot of approaches, but I don't know which is "right."
>
> 1. Register a TestListener and write my own reports to disk from it (but
> is there a generic way to register listeners?)
> 2. Create my own main program that runs the test suite, and have it create
> the report when the suite is done (maybe this is the only way to register a
> TestListener anyhow?)
> 3. Transform the XML output of the ant junit task (but then it's bound to
> Ant)
> 4. Transform the XML output of the maven-surefure-plugin (but then it's
> bound to Maven)
> 5. Write performance results to disk directly from my TestRunner (should
> work in any environment, but feels like a dirty approach)
>
> Other ideas are welcome too, of course. My test runner is open source and
> published on GitHub; I want a solution that can work for everyone. So I
> think ideas 3 and 4 are out.
>
> Has anyone else thought about getting data from custom test runners into
> human-readable reports?
>
> -Jonathan
>
>
>


[Non-text portions of this message have been removed]

#23771 From: Dawid Weiss <dawid.weiss@...>
Date: Thu Dec 15, 2011 10:54 am
Subject: Re: Re: Reporting on test runs from a custom Runner
dawid.weiss@...
Send Email Send Email
 
I should probably also note for completeness that Google Caliper is
specifically designed for micro-benchmarking (with or without
parameterization):

http://code.google.com/p/caliper/

Dawid

On Thu, Dec 15, 2011 at 11:45 AM, Georg <le_garcon_enerve@...> wrote:

> **
>
>
>
> Hi!
>
> Just my 2 cents:
> - an ant-script could still be used in maven (it has plugins for that
> purpose). It may need a bit of tinkering - I'm not very good in maven and
> ant.
> - Code injection via AspectJ could work well. Place a cutpoint on the
> method looping over the test again and again and measure the time before
> and after.
>
> Cheers,
> Georg
>
>
> --- In junit@yahoogroups.com, "jfuerth" <jfuerth@...> wrote:
> >
> > Hi everyone,
> >
> > I've created a custom Runner meant to help with performance and
> scalability testing. It lets you create tests like this:
> >
> > @RunWith(PerfRunner.class)
> > public class MultiThreadUsageExampleTest {
> >
> > @Test
> > public void demonstrationTest(
> > @Varying(from=1, to=10) int threadCount,
> > @Varying(from=100000, to=1000000, step=100000) int entryCount) {
> >
> > // code here that relies on threadCount and entryCount
> > }
> >
> > }
> >
> > In the above example, my runner calls demonstrationTest() 100 times,
> with parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10,
> 1000000).
> >
> > What I want to do now is produce charts of how long each call took.
> There are a lot of approaches, but I don't know which is "right."
> >
> > 1. Register a TestListener and write my own reports to disk from it (but
> is there a generic way to register listeners?)
> > 2. Create my own main program that runs the test suite, and have it
> create the report when the suite is done (maybe this is the only way to
> register a TestListener anyhow?)
> > 3. Transform the XML output of the ant junit task (but then it's bound
> to Ant)
> > 4. Transform the XML output of the maven-surefure-plugin (but then it's
> bound to Maven)
> > 5. Write performance results to disk directly from my TestRunner (should
> work in any environment, but feels like a dirty approach)
> >
> > Other ideas are welcome too, of course. My test runner is open source
> and published on GitHub; I want a solution that can work for everyone. So I
> think ideas 3 and 4 are out.
> >
> > Has anyone else thought about getting data from custom test runners into
> human-readable reports?
> >
> > -Jonathan
> >
>
>
>


[Non-text portions of this message have been removed]

#23772 From: Dawid Weiss <dawid.weiss@...>
Date: Thu Dec 15, 2011 8:47 am
Subject: Re: Reporting on test runs from a custom Runner
dawid.weiss@...
Send Email Send Email
 
You may want to check out JUnitBenchmarks -- it does what I think you want
to achieve:
http://labs.carrotsearch.com/junit-benchmarks.html

Dawid

On Wed, Dec 14, 2011 at 11:01 PM, jfuerth <jfuerth@...> wrote:

> **
>
>
> Hi everyone,
>
> I've created a custom Runner meant to help with performance and
> scalability testing. It lets you create tests like this:
>
> @RunWith(PerfRunner.class)
> public class MultiThreadUsageExampleTest {
>
> @Test
> public void demonstrationTest(
> @Varying(from=1, to=10) int threadCount,
> @Varying(from=100000, to=1000000, step=100000) int entryCount) {
>
> // code here that relies on threadCount and entryCount
> }
>
> }
>
> In the above example, my runner calls demonstrationTest() 100 times, with
> parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10,
> 1000000).
>
> What I want to do now is produce charts of how long each call took. There
> are a lot of approaches, but I don't know which is "right."
>
> 1. Register a TestListener and write my own reports to disk from it (but
> is there a generic way to register listeners?)
> 2. Create my own main program that runs the test suite, and have it create
> the report when the suite is done (maybe this is the only way to register a
> TestListener anyhow?)
> 3. Transform the XML output of the ant junit task (but then it's bound to
> Ant)
> 4. Transform the XML output of the maven-surefure-plugin (but then it's
> bound to Maven)
> 5. Write performance results to disk directly from my TestRunner (should
> work in any environment, but feels like a dirty approach)
>
> Other ideas are welcome too, of course. My test runner is open source and
> published on GitHub; I want a solution that can work for everyone. So I
> think ideas 3 and 4 are out.
>
> Has anyone else thought about getting data from custom test runners into
> human-readable reports?
>
> -Jonathan
>
>
>


[Non-text portions of this message have been removed]

#23773 From: Jonathan Fuerth <jfuerth@...>
Date: Thu Dec 15, 2011 4:40 pm
Subject: Re: Reporting on test runs from a custom Runner
jfuerth
Send Email Send Email
 
Thanks for the pointer, Dawid. I have to admit I didn't find
JUnitBenchmarks in my searches. I went looking for something back in
September and again earier this week and came up empty.

As far as what I'm trying to accomplish, it's kind of two things.

First, I want to microbenchmark the performance of some internal data
structures we're experimenting with for a web framework. I need to vary
more than one parameter for these tests: number of threads allowed, number
of messages to send through it, size of those messages, timing of the
messages, and more. A single test method is likely to run thousands of
times and produce multiple 2D charts worth of data. It looks like Google
Caliper (thanks for that pointer too!) may be well suited to this task.

Second, I want to run end-to-end scalability tests in various Java EE 6
containers. These tests will have many of the same variables. For these
tests, I think Caliper is out. We're going to need Arquillian to wrangle
all the server setup and teardown work. Arquillian is a JUnit 4 test runner
too--I haven't thought through how to reconcile my test runner with
Arquillian's, but I work on the same team as the Arquillian guys, and I'm
hoping we can figure something out. It will probably involve pointing
Arquillian at my test runner, or vice-versa.

Thank you once again for the responses. I really appreciate it!

Cheers,

-Jonathan

On Thu, Dec 15, 2011 at 3:47 AM, Dawid Weiss <dawid.weiss@...> wrote:

> **
>
>
> You may want to check out JUnitBenchmarks -- it does what I think you want
> to achieve:
> http://labs.carrotsearch.com/junit-benchmarks.html
>
> Dawid
>
> On Wed, Dec 14, 2011 at 11:01 PM, jfuerth <jfuerth@...> wrote:
>
> > **
>
> >
> >
> > Hi everyone,
> >
> > I've created a custom Runner meant to help with performance and
> > scalability testing. It lets you create tests like this:
> >
> > @RunWith(PerfRunner.class)
> > public class MultiThreadUsageExampleTest {
> >
> > @Test
> > public void demonstrationTest(
> > @Varying(from=1, to=10) int threadCount,
> > @Varying(from=100000, to=1000000, step=100000) int entryCount) {
> >
> > // code here that relies on threadCount and entryCount
> > }
> >
> > }
> >
> > In the above example, my runner calls demonstrationTest() 100 times, with
> > parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10,
> > 1000000).
> >
> > What I want to do now is produce charts of how long each call took. There
> > are a lot of approaches, but I don't know which is "right."
> >
> > 1. Register a TestListener and write my own reports to disk from it (but
> > is there a generic way to register listeners?)
> > 2. Create my own main program that runs the test suite, and have it
> create
> > the report when the suite is done (maybe this is the only way to
> register a
> > TestListener anyhow?)
> > 3. Transform the XML output of the ant junit task (but then it's bound to
> > Ant)
> > 4. Transform the XML output of the maven-surefure-plugin (but then it's
> > bound to Maven)
> > 5. Write performance results to disk directly from my TestRunner (should
> > work in any environment, but feels like a dirty approach)
> >
> > Other ideas are welcome too, of course. My test runner is open source and
> > published on GitHub; I want a solution that can work for everyone. So I
> > think ideas 3 and 4 are out.
> >
> > Has anyone else thought about getting data from custom test runners into
> > human-readable reports?
> >
> > -Jonathan
> >
> >
> >
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

#23774 From: Jonathan Fuerth <jfuerth@...>
Date: Thu Dec 15, 2011 4:23 pm
Subject: Re: Reporting on test runs from a custom Runner
jfuerth
Send Email Send Email
 
Thanks Matthew.

I will try writing out the reports from a RunListener. If they can be
registered by both the Ant and Maven plugins, that's pretty good.

Eclipse is another story: after a cursory search, I don't see a way to
register custom RunListeners in Eclipse's JUnit launcher. There is a
method in org.eclipse.jdt.junit.JUnitCore for registering
RunListeners, but nothing I can find in the UI for specifying one.
Worst case, I suppose I could just register the RunListener from
within my Runner, perhaps only when enabled by a system property.
That's not as bad as my original picture of option 5.

-Jonathan

On Thu, Dec 15, 2011 at 7:40 AM, Matthew Farwell <matthew@...> wrote:
> Salut,
>
> The best way would be to create your own RunListener (btw TestListener is
> JUnit3, RunListener is Junit4). Then, you can plug this into standard
> tools. For instance, in maven surefire, you can specify:
>
> <plugin>
>  <groupId>org.apache.maven.plugins</groupId>
>  <artifactId>maven-surefire-plugin</artifactId>
>  <version>2.10</version>
>  <configuration>
>    <properties>
>      <property>
>        <name>listener</name>
>
> <value>com.mycompany.MyResultListener,com.mycompany.MyResultListener2</value>
>      </property>
>  </configuration>
> </plugin>
>
> I think you can do similar things with ant. So if you do 1, 3 and 4 aren't
> a problem.
>
> Cordialement,
>
> Matthew Farwell.
>
> Le 14 décembre 2011 23:01, jfuerth <jfuerth@...> a écrit :
>
>> **
>>
>> What I want to do now is produce charts of how long each call took. There
>> are a lot of approaches, but I don't know which is "right."
>>
>> 1. Register a TestListener and write my own reports to disk from it (but
>> is there a generic way to register listeners?)
>> 2. Create my own main program that runs the test suite, and have it create
>> the report when the suite is done (maybe this is the only way to register a
>> TestListener anyhow?)
>> 3. Transform the XML output of the ant junit task (but then it's bound to
>> Ant)
>> 4. Transform the XML output of the maven-surefure-plugin (but then it's
>> bound to Maven)
>> 5. Write performance results to disk directly from my TestRunner (should
>> work in any environment, but feels like a dirty approach)
>>
>> Other ideas are welcome too, of course. My test runner is open source and
>> published on GitHub; I want a solution that can work for everyone. So I
>> think ideas 3 and 4 are out.
>>
>> Has anyone else thought about getting data from custom test runners into
>> human-readable reports?
>>
>> -Jonathan
>>
>>
>>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

#23775 From: Marc Philipp <mphilipp82@...>
Date: Thu Dec 15, 2011 7:38 pm
Subject: Re: Reporting on test runs from a custom Runner
mphilipp1982
Send Email Send Email
 
Am 15.12.2011 um 17:23 schrieb Jonathan Fuerth:
> Eclipse is another story: after a cursory search, I don't see a way to
> register custom RunListeners in Eclipse's JUnit launcher. There is a
> method in org.eclipse.jdt.junit.JUnitCore for registering
> RunListeners, but nothing I can find in the UI for specifying one.
>

I think this method is intended to be used by other Eclipse plugins, I don't
think there's an UI for that. You could wrap your custom listener in an Eclipse
plugin, though. Of course, that would only be worth the effort if you intend to
use it a lot.

Cheers,

Marc



[Non-text portions of this message have been removed]

#23776 From: Jonathan Fuerth <jfuerth@...>
Date: Thu Dec 15, 2011 11:08 pm
Subject: Re: Reporting on test runs from a custom Runner
jfuerth
Send Email Send Email
 
On Thu, Dec 15, 2011 at 2:38 PM, Marc Philipp <mphilipp82@...> wrote:

> **
>
>
> Am 15.12.2011 um 17:23 schrieb Jonathan Fuerth:
>
> > Eclipse is another story: after a cursory search, I don't see a way to
> > register custom RunListeners in Eclipse's JUnit launcher. There is a
> > method in org.eclipse.jdt.junit.JUnitCore for registering
> > RunListeners, but nothing I can find in the UI for specifying one.
> >
>
> I think this method is intended to be used by other Eclipse plugins, I
> don't think there's an UI for that. You could wrap your custom listener in
> an Eclipse plugin, though. Of course, that would only be worth the effort
> if you intend to use it a lot.
>
>
Yes, I think I'd probably cheat and register the listener directly from
within the runner. If every JUnit launcher needs its own custom
configuration or plugin, that's not a very good experience for users of the
framework.

-Jonathan


[Non-text portions of this message have been removed]

#23777 From: "sschroev" <stephan202@...>
Date: Sun Dec 18, 2011 1:21 pm
Subject: Re: Reporting on test runs from a custom Runner
sschroev
Send Email Send Email
 
Hi Jonathan,

Out of curiosity: have you considered implementing your framework extension as a
TestRule instead of a Runner?

Cheers,
Stephan

--- In junit@yahoogroups.com, "jfuerth" <jfuerth@...> wrote:
>
> Hi everyone,
>
> I've created a custom Runner meant to help with performance and scalability
testing. It lets you create tests like this:
>
> @RunWith(PerfRunner.class)
> public class MultiThreadUsageExampleTest {
>
>   @Test
>   public void demonstrationTest(
>       @Varying(from=1, to=10) int threadCount,
>       @Varying(from=100000, to=1000000, step=100000) int entryCount) {
>
>       // code here that relies on threadCount and entryCount
>   }
>
> }
>
> In the above example, my runner calls demonstrationTest() 100 times, with
parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10, 1000000).
>
> What I want to do now is produce charts of how long each call took. There are
a lot of approaches, but I don't know which is "right."
>
> 1. Register a TestListener and write my own reports to disk from it (but is
there a generic way to register listeners?)
> 2. Create my own main program that runs the test suite, and have it create the
report when the suite is done (maybe this is the only way to register a
TestListener anyhow?)
> 3. Transform the XML output of the ant junit task (but then it's bound to Ant)
> 4. Transform the XML output of the maven-surefure-plugin (but then it's bound
to Maven)
> 5. Write performance results to disk directly from my TestRunner (should work
in any environment, but feels like a dirty approach)
>
> Other ideas are welcome too, of course. My test runner is open source and
published on GitHub; I want a solution that can work for everyone. So I think
ideas 3 and 4 are out.
>
> Has anyone else thought about getting data from custom test runners into
human-readable reports?
>
> -Jonathan
>

#23778 From: Noel Yap <noel.yap@...>
Date: Tue Dec 20, 2011 1:55 am
Subject: Categories throws InitializationError
yap_noel
Send Email Send Email
 
When there are no tests matching the Include/Exclude filters, Categories
throws an InitializationError.  Can this behavior be changed?  IMO, if
nothing matches the criteria, the test execution should pass.

Currently, we're using a hack around this problem:

public class NullTest {
     @Test
     public void test() {
     }
}

Thanks,
Noel


[Non-text portions of this message have been removed]

Messages 23749 - 23778 of 24387   Oldest  |  < Older  |  Newer >  |  Newest
Add to My Yahoo!      XML What's This?

Copyright © 2010 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines NEW - Help