Hello,

Tye link does not work. Could you please also post the correct link.

I am having suite filter runner with classes that fall into that suite. One of
the classes I am using Extended Runner to run a specific test case for multiple
times.  Please let me know how can we over ride the runners for a specific test
case.

Thanks
Rajesh
--- In junit@yahoogroups.com, "p.lipinski" <p.lipinski@...> wrote:
>
> Hi
> Some time ago I've written a runner for JUnit which is a replacement for the
standard Parameterized runner - is easier & more flexible to use, gives nicer
output in IDEs. I've made it available at http://code.google.com/p/junitparams.
The website has a description of all the features.
>
> JUnitParams now updated to work properly with JUnit 4.10.
>
> Pawel
>

Thanks David!
Dawid

On Mon, Oct 31, 2011 at 6:00 PM, David Saff <david@...> wrote:

> **
>
>
> All pull requests have their own issue numbers: it's awkward, but you
> get used to it. If the name of the pull request (or possibly the
> description) contains the original issue number in one of the formats
> that GitHub expects (My favorite is, for example, "Fixes gh-355:
> explict messages in Assume"), then a link to the pull request will
> automatically get added to the original issue.
>
> Thanks!
>
> David
>
>
> On Sun, Oct 30, 2011 at 4:17 PM, Dawid Weiss <dawid.weiss@...>
> wrote:
> > Hi,
> >
> > I've created an issue to extend Assume to contain explicit messages/
> > Throwable cause if needed. The issue includes a patch:
> > https://github.com/KentBeck/junit/issues/355
> >
> > Can anybody enlighten me if it's possible to attach a pull request to
> > an _existing_ issue in github? I'm used to Jira and the way I usually
> > work is create an issue, create a git branch with the issue's number,
> > then fold in. With github pull requests seem to create a new issue so
> > there's no way to have the issue number before (?).
> >
> > Dawid
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>


[Non-text portions of this message have been removed]

It does work if you omit the final dot...

http://code.google.com/p/junitparams

Dawid

On Tue, Nov 1, 2011 at 12:00 AM, sinu <srijesh_a@...> wrote:

> **
>
>
>
> Hello,
>
> Tye link does not work. Could you please also post the correct link.
>
> I am having suite filter runner with classes that fall into that suite.
> One of the classes I am using Extended Runner to run a specific test case
> for multiple times. Please let me know how can we over ride the runners for
> a specific test case.
>
> Thanks
> Rajesh
>
> --- In junit@yahoogroups.com, "p.lipinski" <p.lipinski@...> wrote:
> >
> > Hi
> > Some time ago I've written a runner for JUnit which is a replacement for
> the standard Parameterized runner - is easier & more flexible to use, gives
> nicer output in IDEs. I've made it available at
> http://code.google.com/p/junitparams. The website has a description of
> all the features.
> >
> > JUnitParams now updated to work properly with JUnit 4.10.
> >
> > Pawel
> >
>
>
>


[Non-text portions of this message have been removed]

The URL was auto-linked incorrectly; the trailing dot was added to the href. The
link is: http://code.google.com/p/junitparams/

--- In junit@yahoogroups.com, "sinu" <srijesh_a@...> wrote:
>
>
> Hello,
>
> Tye link does not work. Could you please also post the correct link.
>
> I am having suite filter runner with classes that fall into that suite. One of
the classes I am using Extended Runner to run a specific test case for multiple
times.  Please let me know how can we over ride the runners for a specific test
case.
>
> Thanks
> Rajesh
> --- In junit@yahoogroups.com, "p.lipinski" <p.lipinski@> wrote:
> >
> > Hi
> > Some time ago I've written a runner for JUnit which is a replacement for the
standard Parameterized runner - is easier & more flexible to use, gives nicer
output in IDEs. I've made it available at http://code.google.com/p/junitparams.
The website has a description of all the features.
> >
> > JUnitParams now updated to work properly with JUnit 4.10.
> >
> > Pawel
> >
>

Hi,

I'm interested in this, though for slightly different reasons, maybe we
could exchange ideas on this. I haven't exactly thought this through in
detail, but I think my requirements come down to being able to store
arbitrary information in a Description, a key.

I'm interesting in testing with Scala, and there are (at least) two generic
problems:

1) Tests which are not linked to a specific method, but are run as separate
tests. Scalatest for example allows the following

@RunWith(classOf[JUnitRunner])
class FileSpec extends FlatSpec with ShouldMatchers {

   "A Stack" should "pop values in last-in-first-out order" in {
   }

   it should "throw NoSuchElementException if an empty stack is popped" in {
   }
}

This creates a single method with the two tests within. However, the
JUnitRunner runs the single method but records the two tests separately.
This can be seen in Eclipse, see attached scalatest1.png. Now if I double
click on the test name in the junit view, it tries to find a method of that
name, and fails. Obviously, to correct this would require a fix to Eclipse,
but if I could store the file & line number instead of the method name, I
could jump to the appropriate line.

2) Scala allows spaces and other characters in method names, but these
names are munged on compilation. For instance, space gets translated to
$0020.

class FileTest extends AssertionsForJUnit {
@Test def `foo bar`() = {

}
}

Here the jump to in Eclipse works correctly, but the displayed description
in Eclipse isn't nice, see scalatest2.png.

So in both cases, the string I would like to display (in Eclipse, surefire
whatever) doesn't match the method name, either because there is no method
name or because It's not the same thing. I think that I could work around
most issues is Eclipse etc if the test runner could add some arbitrary
information to the description, which it could subsequently use. A string
would be fine, because I could define the contents of that String, but
arbitrary data would be even better. String would be simpler for everyone I
think.

Please note that I'm just thinking aloud at the minute, and I haven't
actually done any proof of concepts as yet, but if you have any ideas,
please feel free to take this to email.

Matthew Farwell.


[Non-text portions of this message have been removed]

Hi Matthew,

Actually we are planning to "fix Eclipse" very soon. The next release of
ScalaTest, version 2.0, will include enhancements to support IDE integration,
and before releasing 2.0 we're going to help do the actual integration work for
Eclipse, IDEA, and NetBeans. That way we can be sure we have it right before
releasing the ScalaTest enhancements. I would like your feedback on it. I'll
post announcements of preview snapshots on the scalatest-users mailing list, so
please join that if you'd like to give input.

Thanks.

Bill
----
Bill Venners
Artima, Inc.
http://www.artima.com

On Nov 3, 2011, at 5:50 AM, Matthew Farwell <matthew@...> wrote:

> Hi,
>
> I'm interested in this, though for slightly different reasons, maybe we
> could exchange ideas on this. I haven't exactly thought this through in
> detail, but I think my requirements come down to being able to store
> arbitrary information in a Description, a key.
>
> I'm interesting in testing with Scala, and there are (at least) two generic
> problems:
>
> 1) Tests which are not linked to a specific method, but are run as separate
> tests. Scalatest for example allows the following
>
> @RunWith(classOf[JUnitRunner])
> class FileSpec extends FlatSpec with ShouldMatchers {
>
> "A Stack" should "pop values in last-in-first-out order" in {
> }
>
> it should "throw NoSuchElementException if an empty stack is popped" in {
> }
> }
>
> This creates a single method with the two tests within. However, the
> JUnitRunner runs the single method but records the two tests separately.
> This can be seen in Eclipse, see attached scalatest1.png. Now if I double
> click on the test name in the junit view, it tries to find a method of that
> name, and fails. Obviously, to correct this would require a fix to Eclipse,
> but if I could store the file & line number instead of the method name, I
> could jump to the appropriate line.
>
> 2) Scala allows spaces and other characters in method names, but these
> names are munged on compilation. For instance, space gets translated to
> $0020.
>
> class FileTest extends AssertionsForJUnit {
> @Test def `foo bar`() = {
>
> }
> }
>
> Here the jump to in Eclipse works correctly, but the displayed description
> in Eclipse isn't nice, see scalatest2.png.
>
> So in both cases, the string I would like to display (in Eclipse, surefire
> whatever) doesn't match the method name, either because there is no method
> name or because It's not the same thing. I think that I could work around
> most issues is Eclipse etc if the test runner could add some arbitrary
> information to the description, which it could subsequently use. A string
> would be fine, because I could define the contents of that String, but
> arbitrary data would be even better. String would be simpler for everyone I
> think.
>
> Please note that I'm just thinking aloud at the minute, and I haven't
> actually done any proof of concepts as yet, but if you have any ideas,
> please feel free to take this to email.
>
> Matthew Farwell.
>
> [Non-text portions of this message have been removed]
>
>


[Non-text portions of this message have been removed]

I think your reasons are aligned with mine -- the problem is that a test's
visual description for the user interface and where it "points to" should
be separate things. My thoughts on the subject so far are that Description
should provide a String with the human description and some sort of an
(empty) interface, let's call it a TestSource which would provide
additional hints which resource to open and where. So something like this
(ideas, not implementation):

Description {
   String getDescription();
   List<TestSource> getSource();
}

I can think of numerous advantages of such an approach, for example a
single parameterized test could open the test case's source code by
returning:

JavaTestSource {
   String className();
   String methodName(); // full signature?
}

but it could also point to the file with parameterized test data and open
_that_ as well, as in:

FileTestSource {
   File file();
   int line();
}

Not giving much thought about the implementation, but you get the idea.
Obviously this would be a major pain for all tools that use JUnit because
this would pretty much break backwards compatibility... but I'm all for it
if a new major revision of JUnit is to ever see the light of day.

Temporarily, and interestingly, Eclipse (and Ant and other tools I think)
can be cheated a bit -- they all use a JUnit version provided by the user's
classpath via reflective access typically (they don't have fixed version
dependency). If you "classpath-shadow" your JUnit's Description (by
prepending classpath with your own Description class, keeping the same API)
you can do anything you with with the descriptions... This is a dirty hack
though :(

Dawid

On Thu, Nov 3, 2011 at 10:50 AM, Matthew Farwell <matthew@...>wrote:

> **
>
>
> Hi,
>
> I'm interested in this, though for slightly different reasons, maybe we
> could exchange ideas on this. I haven't exactly thought this through in
> detail, but I think my requirements come down to being able to store
> arbitrary information in a Description, a key.
>
> I'm interesting in testing with Scala, and there are (at least) two generic
> problems:
>
> 1) Tests which are not linked to a specific method, but are run as separate
> tests. Scalatest for example allows the following
>
> @RunWith(classOf[JUnitRunner])
> class FileSpec extends FlatSpec with ShouldMatchers {
>
> "A Stack" should "pop values in last-in-first-out order" in {
> }
>
> it should "throw NoSuchElementException if an empty stack is popped" in {
> }
> }
>
> This creates a single method with the two tests within. However, the
> JUnitRunner runs the single method but records the two tests separately.
> This can be seen in Eclipse, see attached scalatest1.png. Now if I double
> click on the test name in the junit view, it tries to find a method of that
> name, and fails. Obviously, to correct this would require a fix to Eclipse,
> but if I could store the file & line number instead of the method name, I
> could jump to the appropriate line.
>
> 2) Scala allows spaces and other characters in method names, but these
> names are munged on compilation. For instance, space gets translated to
> $0020.
>
> class FileTest extends AssertionsForJUnit {
> @Test def `foo bar`() = {
>
> }
> }
>
> Here the jump to in Eclipse works correctly, but the displayed description
> in Eclipse isn't nice, see scalatest2.png.
>
> So in both cases, the string I would like to display (in Eclipse, surefire
> whatever) doesn't match the method name, either because there is no method
> name or because It's not the same thing. I think that I could work around
> most issues is Eclipse etc if the test runner could add some arbitrary
> information to the description, which it could subsequently use. A string
> would be fine, because I could define the contents of that String, but
> arbitrary data would be even better. String would be simpler for everyone I
> think.
>
> Please note that I'm just thinking aloud at the minute, and I haven't
> actually done any proof of concepts as yet, but if you have any ideas,
> please feel free to take this to email.
>
> Matthew Farwell.
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

Hi Bill,

I did the JUnit integration for Scala-IDE, so yes I would like to be
involved.

Currently Scala-IDE uses the Junit plugin for running and displaying the
test results, and it would obviously be better if we could avoid forking
that. But for instance the jump to method doesn't work for FlatSpec stuff,
and will never work because it's trying to find a method called xxx where
xxx is the first word in the description of the test. Which obviously will
never work. For Scala-IDE we can work around this using aspects, but it
would be nice it this worked nicely. Hence my interest in changing
Description.

Matthew.

Le 4 novembre 2011 23:01, Bill Venners <bill@...> a écrit :

> **
>
>
> Hi Matthew,
>
> Actually we are planning to "fix Eclipse" very soon. The next release of
> ScalaTest, version 2.0, will include enhancements to support IDE
> integration, and before releasing 2.0 we're going to help do the actual
> integration work for Eclipse, IDEA, and NetBeans. That way we can be sure
> we have it right before releasing the ScalaTest enhancements. I would like
> your feedback on it. I'll post announcements of preview snapshots on the
> scalatest-users mailing list, so please join that if you'd like to give
> input.
>
> Thanks.
>
> Bill
> ----
> Bill Venners
> Artima, Inc.
> http://www.artima.com
>
>
> On Nov 3, 2011, at 5:50 AM, Matthew Farwell <matthew@...> wrote:
>
> > Hi,
> >
> > I'm interested in this, though for slightly different reasons, maybe we
> > could exchange ideas on this. I haven't exactly thought this through in
> > detail, but I think my requirements come down to being able to store
> > arbitrary information in a Description, a key.
> >
> > I'm interesting in testing with Scala, and there are (at least) two
> generic
> > problems:
> >
> > 1) Tests which are not linked to a specific method, but are run as
> separate
> > tests. Scalatest for example allows the following
> >
> > @RunWith(classOf[JUnitRunner])
> > class FileSpec extends FlatSpec with ShouldMatchers {
> >
> > "A Stack" should "pop values in last-in-first-out order" in {
> > }
> >
> > it should "throw NoSuchElementException if an empty stack is popped" in {
> > }
> > }
> >
> > This creates a single method with the two tests within. However, the
> > JUnitRunner runs the single method but records the two tests separately.
> > This can be seen in Eclipse, see attached scalatest1.png. Now if I double
> > click on the test name in the junit view, it tries to find a method of
> that
> > name, and fails. Obviously, to correct this would require a fix to
> Eclipse,
> > but if I could store the file & line number instead of the method name, I
> > could jump to the appropriate line.
> >
> > 2) Scala allows spaces and other characters in method names, but these
> > names are munged on compilation. For instance, space gets translated to
> > $0020.
> >
> > class FileTest extends AssertionsForJUnit {
> > @Test def `foo bar`() = {
> >
> > }
> > }
> >
> > Here the jump to in Eclipse works correctly, but the displayed
> description
> > in Eclipse isn't nice, see scalatest2.png.
> >
> > So in both cases, the string I would like to display (in Eclipse,
> surefire
> > whatever) doesn't match the method name, either because there is no
> method
> > name or because It's not the same thing. I think that I could work around
> > most issues is Eclipse etc if the test runner could add some arbitrary
> > information to the description, which it could subsequently use. A string
> > would be fine, because I could define the contents of that String, but
> > arbitrary data would be even better. String would be simpler for
> everyone I
> > think.
> >
> > Please note that I'm just thinking aloud at the minute, and I haven't
> > actually done any proof of concepts as yet, but if you have any ideas,
> > please feel free to take this to email.
> >
> > Matthew Farwell.
> >
> > [Non-text portions of this message have been removed]
> >
> >
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

I think that the reason why Description is final is because it is a value
class, and there is code (especially in filters) that expects
Description.equals() to obey the general contract of Object.equals().
Unfortunately, there is "no way to extend an instantiable class and add a
value component while preserving the equals contract" (quoted from
Effective Java, second edition, page 38).

It's perfectly fine to add optional properties to the existing Description
class, as long as Description.equals() and Description.hashCode() are
updated, and all of the properties that are added have equals() methods
that obey the general contract of Object.equals().

-- Kevin

On Fri, Nov 4, 2011 at 11:43 PM, Matthew Farwell <matthew@...>wrote:

> Hi Bill,
>
> I did the JUnit integration for Scala-IDE, so yes I would like to be
> involved.
>
> Currently Scala-IDE uses the Junit plugin for running and displaying the
> test results, and it would obviously be better if we could avoid forking
> that. But for instance the jump to method doesn't work for FlatSpec stuff,
> and will never work because it's trying to find a method called xxx where
> xxx is the first word in the description of the test. Which obviously will
> never work. For Scala-IDE we can work around this using aspects, but it
> would be nice it this worked nicely. Hence my interest in changing
> Description.
>
> Matthew.
>
> Le 4 novembre 2011 23:01, Bill Venners <bill@...> a écrit :
>
> > **
> >
> >
> > Hi Matthew,
> >
> > Actually we are planning to "fix Eclipse" very soon. The next release of
> > ScalaTest, version 2.0, will include enhancements to support IDE
> > integration, and before releasing 2.0 we're going to help do the actual
> > integration work for Eclipse, IDEA, and NetBeans. That way we can be sure
> > we have it right before releasing the ScalaTest enhancements. I would
> like
> > your feedback on it. I'll post announcements of preview snapshots on the
> > scalatest-users mailing list, so please join that if you'd like to give
> > input.
> >
> > Thanks.
> >
> > Bill
> > ----
> > Bill Venners
> > Artima, Inc.
> > http://www.artima.com
> >
> >
> > On Nov 3, 2011, at 5:50 AM, Matthew Farwell <matthew@...>
> wrote:
> >
> > > Hi,
> > >
> > > I'm interested in this, though for slightly different reasons, maybe we
> > > could exchange ideas on this. I haven't exactly thought this through in
> > > detail, but I think my requirements come down to being able to store
> > > arbitrary information in a Description, a key.
> > >
> > > I'm interesting in testing with Scala, and there are (at least) two
> > generic
> > > problems:
> > >
> > > 1) Tests which are not linked to a specific method, but are run as
> > separate
> > > tests. Scalatest for example allows the following
> > >
> > > @RunWith(classOf[JUnitRunner])
> > > class FileSpec extends FlatSpec with ShouldMatchers {
> > >
> > > "A Stack" should "pop values in last-in-first-out order" in {
> > > }
> > >
> > > it should "throw NoSuchElementException if an empty stack is popped"
> in {
> > > }
> > > }
> > >
> > > This creates a single method with the two tests within. However, the
> > > JUnitRunner runs the single method but records the two tests
> separately.
> > > This can be seen in Eclipse, see attached scalatest1.png. Now if I
> double
> > > click on the test name in the junit view, it tries to find a method of
> > that
> > > name, and fails. Obviously, to correct this would require a fix to
> > Eclipse,
> > > but if I could store the file & line number instead of the method
> name, I
> > > could jump to the appropriate line.
> > >
> > > 2) Scala allows spaces and other characters in method names, but these
> > > names are munged on compilation. For instance, space gets translated to
> > > $0020.
> > >
> > > class FileTest extends AssertionsForJUnit {
> > > @Test def `foo bar`() = {
> > >
> > > }
> > > }
> > >
> > > Here the jump to in Eclipse works correctly, but the displayed
> > description
> > > in Eclipse isn't nice, see scalatest2.png.
> > >
> > > So in both cases, the string I would like to display (in Eclipse,
> > surefire
> > > whatever) doesn't match the method name, either because there is no
> > method
> > > name or because It's not the same thing. I think that I could work
> around
> > > most issues is Eclipse etc if the test runner could add some arbitrary
> > > information to the description, which it could subsequently use. A
> string
> > > would be fine, because I could define the contents of that String, but
> > > arbitrary data would be even better. String would be simpler for
> > everyone I
> > > think.
> > >
> > > Please note that I'm just thinking aloud at the minute, and I haven't
> > > actually done any proof of concepts as yet, but if you have any ideas,
> > > please feel free to take this to email.
> > >
> > > Matthew Farwell.
> > >
> > > [Non-text portions of this message have been removed]
> > >
> > >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>


[Non-text portions of this message have been removed]

I'm using a test suite class to collect up a bunch of individual testcase
classes.   I also use the "junitreport" ant task to create an html report of the
test results.   Currently, my test results are formatted such that all the
individual test methods are organized only the test suite classname in the
report, and I lose visibility as to which testcase class the test methods belong
to.

Instead, I would like the test results to be organized such that the individual
testcase classnames appear in the report with the associated results for each of
their test methods.

Does anyone know if it's possible to configure the "junitreport" ant task to do
this?

Thanks

Phil,

Hopefully someone here knows.  However, the junitreport task was
written by people on the ant project, so you may want to ask on their
lists, as well.

Good luck,

    David

On Mon, Nov 7, 2011 at 10:24 AM, Phil A <padamstx@...> wrote:
> I'm using a test suite class to collect up a bunch of individual testcase
classes.   I also use the "junitreport" ant task to create an html report of the
test results.   Currently, my test results are formatted such that all the
individual test methods are organized only the test suite classname in the
report, and I lose visibility as to which testcase class the test methods belong
to.
>
> Instead, I would like the test results to be organized such that the
individual testcase classnames appear in the report with the associated results
for each of their test methods.
>
> Does anyone know if it's possible to configure the "junitreport" ant task to
do this?
>
> Thanks
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Hi Phil,

ANT's junit task can include multiple <formatter> sub-elements. For your
use-case I would suggest:

1) to use the default XML formatter and post-process the output using XSLT
to emit a HTML with individual test cases,
2) write a custome formatter class (very simple) that emits individual
plain-text or HTML files for each test case or suite, then aggregate.
3) write a formatter that System.println's directly, without creating any
files. Your "report" is then the build file's output.

We are using option (3) along with a custom runner, but you can peek here
at what it looks like:
http://builds.carrot2.org/download/C2HEAD-CORE-JOB1/build_logs/C2HEAD-CORE-JOB1-\
975.log

Dawid

On Wed, Nov 9, 2011 at 1:48 AM, David Saff <david@...> wrote:

> **
>
>
> Phil,
>
> Hopefully someone here knows. However, the junitreport task was
> written by people on the ant project, so you may want to ask on their
> lists, as well.
>
> Good luck,
>
> David
>
>
> On Mon, Nov 7, 2011 at 10:24 AM, Phil A <padamstx@...> wrote:
> > I'm using a test suite class to collect up a bunch of individual
> testcase classes.   I also use the "junitreport" ant task to create an html
> report of the test results.   Currently, my test results are formatted such
> that all the individual test methods are organized only the test suite
> classname in the report, and I lose visibility as to which testcase class
> the test methods belong to.
> >
> > Instead, I would like the test results to be organized such that the
> individual testcase classnames appear in the report with the associated
> results for each of their test methods.
> >
> > Does anyone know if it's possible to configure the "junitreport" ant
> task to do this?
> >
> > Thanks
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>


[Non-text portions of this message have been removed]

So does this mean that adding a Map<String,Object> is being considered
(i.e. having Description implement Map) ?

On 7 November 2011 20:22, Kevin Cooney <kcooney@...> wrote:

>
> It's perfectly fine to add optional properties to the existing Description
> class, as long as Description.equals() and Description.hashCode() are
> updated, and all of the properties that are added have equals() methods
> that obey the general contract of Object.equals().
>


[Non-text portions of this message have been removed]

Hi,

Is it possible to customize the "got ..." message when writing custom Matchers?
I found this message in the Archives:

http://groups.yahoo.com/group/junit/message/21939

which talks about a describeMismatch method but I don't see in the javadocs for
the Matcher or SelfDescribing interfaces.

Am I missing something?

Thanks,

Jason

I'm not sure what David is considering. I'm just pointing out why making
Description final is a good idea.

-- Kevin


On Thu, Nov 10, 2011 at 2:40 AM, Neale Upstone
<neale.upstone@...>wrote:

> **
>
>
> So does this mean that adding a Map<String,Object> is being considered
> (i.e. having Description implement Map) ?
>
>
> On 7 November 2011 20:22, Kevin Cooney <kcooney@...> wrote:
>
> >
> > It's perfectly fine to add optional properties to the existing
> Description
> > class, as long as Description.equals() and Description.hashCode() are
> > updated, and all of the properties that are added have equals() methods
> > that obey the general contract of Object.equals().
> >
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

JUnit uses version 1.1 of the Hamcrest library and it doesn't have that
method. Hamcrest 1.2 and newer have it.


Jason_Bodnar wrote on 10.11.2011 19:21:
> Hi,
>
> Is it possible to customize the "got ..." message when writing custom
> Matchers? I found this message in the Archives:
>
> http://groups.yahoo.com/group/junit/message/21939
>
> which talks about a describeMismatch method but I don't see in the
> javadocs for the Matcher or SelfDescribing interfaces.
>
> Am I missing something?
>
> Thanks,
>
> Jason
>

--
Esko Luontola
www.orfjackal.net

We are all developing new features and bug fixes in to JUnit. You had to already
see that this is not short time to accept a new design.

When I submit something, then only Kent and David are announced. I would expect
that much more developers are announced as well, or even the entire group. This
may speed up.

What are you suggesting to speed up the acceptance?

Hi All,

I have right now created an open group on the LinkedIn for the JUnit which we
can enjoy.

You are All Welcome to join!

It is a good style that the professionals use in order to share their knowledge,
code, asking questions, sending announcements, or even posting jobs, and
discussing and clarifying detailed issues regarding the technologies -JUnit in
our case.

http://www.linkedin.com/groups?home=&gid=4184979&trk=anet_ug_hm

With Best Regards,
Tibor

I was going to start a new thread, but I'll plug in here, to keep the discussion
in one place.

I have had the same problem, different solution and the patch is in github:
https://github.com/KentBeck/junit/pull/145

Florian has same problem, different approach:
https://github.com/KentBeck/junit/pull/358

May the Powers That Be realize that this is a very annoying aspect of JUnit
(forcing us to count lines when a test fails) and consider including any of
these patches in the JUnit tree.

Best regards,
Dimitar

Hi everyone,

I've created a custom Runner meant to help with performance and scalability
testing. It lets you create tests like this:

@RunWith(PerfRunner.class)
public class MultiThreadUsageExampleTest {

   @Test
   public void demonstrationTest(
       @Varying(from=1, to=10) int threadCount,
       @Varying(from=100000, to=1000000, step=100000) int entryCount) {

       // code here that relies on threadCount and entryCount
   }

}

In the above example, my runner calls demonstrationTest() 100 times, with
parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10, 1000000).

What I want to do now is produce charts of how long each call took. There are a
lot of approaches, but I don't know which is "right."

1. Register a TestListener and write my own reports to disk from it (but is
there a generic way to register listeners?)
2. Create my own main program that runs the test suite, and have it create the
report when the suite is done (maybe this is the only way to register a
TestListener anyhow?)
3. Transform the XML output of the ant junit task (but then it's bound to Ant)
4. Transform the XML output of the maven-surefure-plugin (but then it's bound to
Maven)
5. Write performance results to disk directly from my TestRunner (should work in
any environment, but feels like a dirty approach)

Other ideas are welcome too, of course. My test runner is open source and
published on GitHub; I want a solution that can work for everyone. So I think
ideas 3 and 4 are out.

Has anyone else thought about getting data from custom test runners into
human-readable reports?

-Jonathan

Hi!

Just my 2 cents:
  - an ant-script could still be used in maven (it has plugins for that purpose).
It may need a bit of tinkering - I'm not very good in maven and ant.
  - Code injection via AspectJ could work well. Place a cutpoint on the method
looping over the test again and again and measure the time before and after.

Cheers,
  Georg

--- In junit@yahoogroups.com, "jfuerth" <jfuerth@...> wrote:
>
> Hi everyone,
>
> I've created a custom Runner meant to help with performance and scalability
testing. It lets you create tests like this:
>
> @RunWith(PerfRunner.class)
> public class MultiThreadUsageExampleTest {
>
>   @Test
>   public void demonstrationTest(
>       @Varying(from=1, to=10) int threadCount,
>       @Varying(from=100000, to=1000000, step=100000) int entryCount) {
>
>       // code here that relies on threadCount and entryCount
>   }
>
> }
>
> In the above example, my runner calls demonstrationTest() 100 times, with
parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10, 1000000).
>
> What I want to do now is produce charts of how long each call took. There are
a lot of approaches, but I don't know which is "right."
>
> 1. Register a TestListener and write my own reports to disk from it (but is
there a generic way to register listeners?)
> 2. Create my own main program that runs the test suite, and have it create the
report when the suite is done (maybe this is the only way to register a
TestListener anyhow?)
> 3. Transform the XML output of the ant junit task (but then it's bound to Ant)
> 4. Transform the XML output of the maven-surefure-plugin (but then it's bound
to Maven)
> 5. Write performance results to disk directly from my TestRunner (should work
in any environment, but feels like a dirty approach)
>
> Other ideas are welcome too, of course. My test runner is open source and
published on GitHub; I want a solution that can work for everyone. So I think
ideas 3 and 4 are out.
>
> Has anyone else thought about getting data from custom test runners into
human-readable reports?
>
> -Jonathan
>

Salut,

The best way would be to create your own RunListener (btw TestListener is
JUnit3, RunListener is Junit4). Then, you can plug this into standard
tools. For instance, in maven surefire, you can specify:

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <version>2.10</version>
   <configuration>
     <properties>
       <property>
         <name>listener</name>

<value>com.mycompany.MyResultListener,com.mycompany.MyResultListener2</value>
       </property>
   </configuration>
</plugin>

I think you can do similar things with ant. So if you do 1, 3 and 4 aren't
a problem.

Cordialement,

Matthew Farwell.

Le 14 décembre 2011 23:01, jfuerth <jfuerth@...> a écrit :

> **
>
> What I want to do now is produce charts of how long each call took. There
> are a lot of approaches, but I don't know which is "right."
>
> 1. Register a TestListener and write my own reports to disk from it (but
> is there a generic way to register listeners?)
> 2. Create my own main program that runs the test suite, and have it create
> the report when the suite is done (maybe this is the only way to register a
> TestListener anyhow?)
> 3. Transform the XML output of the ant junit task (but then it's bound to
> Ant)
> 4. Transform the XML output of the maven-surefure-plugin (but then it's
> bound to Maven)
> 5. Write performance results to disk directly from my TestRunner (should
> work in any environment, but feels like a dirty approach)
>
> Other ideas are welcome too, of course. My test runner is open source and
> published on GitHub; I want a solution that can work for everyone. So I
> think ideas 3 and 4 are out.
>
> Has anyone else thought about getting data from custom test runners into
> human-readable reports?
>
> -Jonathan
>
>
>


[Non-text portions of this message have been removed]

I should probably also note for completeness that Google Caliper is
specifically designed for micro-benchmarking (with or without
parameterization):

http://code.google.com/p/caliper/

Dawid

On Thu, Dec 15, 2011 at 11:45 AM, Georg <le_garcon_enerve@...> wrote:

> **
>
>
>
> Hi!
>
> Just my 2 cents:
> - an ant-script could still be used in maven (it has plugins for that
> purpose). It may need a bit of tinkering - I'm not very good in maven and
> ant.
> - Code injection via AspectJ could work well. Place a cutpoint on the
> method looping over the test again and again and measure the time before
> and after.
>
> Cheers,
> Georg
>
>
> --- In junit@yahoogroups.com, "jfuerth" <jfuerth@...> wrote:
> >
> > Hi everyone,
> >
> > I've created a custom Runner meant to help with performance and
> scalability testing. It lets you create tests like this:
> >
> > @RunWith(PerfRunner.class)
> > public class MultiThreadUsageExampleTest {
> >
> > @Test
> > public void demonstrationTest(
> > @Varying(from=1, to=10) int threadCount,
> > @Varying(from=100000, to=1000000, step=100000) int entryCount) {
> >
> > // code here that relies on threadCount and entryCount
> > }
> >
> > }
> >
> > In the above example, my runner calls demonstrationTest() 100 times,
> with parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10,
> 1000000).
> >
> > What I want to do now is produce charts of how long each call took.
> There are a lot of approaches, but I don't know which is "right."
> >
> > 1. Register a TestListener and write my own reports to disk from it (but
> is there a generic way to register listeners?)
> > 2. Create my own main program that runs the test suite, and have it
> create the report when the suite is done (maybe this is the only way to
> register a TestListener anyhow?)
> > 3. Transform the XML output of the ant junit task (but then it's bound
> to Ant)
> > 4. Transform the XML output of the maven-surefure-plugin (but then it's
> bound to Maven)
> > 5. Write performance results to disk directly from my TestRunner (should
> work in any environment, but feels like a dirty approach)
> >
> > Other ideas are welcome too, of course. My test runner is open source
> and published on GitHub; I want a solution that can work for everyone. So I
> think ideas 3 and 4 are out.
> >
> > Has anyone else thought about getting data from custom test runners into
> human-readable reports?
> >
> > -Jonathan
> >
>
>
>


[Non-text portions of this message have been removed]

You may want to check out JUnitBenchmarks -- it does what I think you want
to achieve:
http://labs.carrotsearch.com/junit-benchmarks.html

Dawid

On Wed, Dec 14, 2011 at 11:01 PM, jfuerth <jfuerth@...> wrote:

> **
>
>
> Hi everyone,
>
> I've created a custom Runner meant to help with performance and
> scalability testing. It lets you create tests like this:
>
> @RunWith(PerfRunner.class)
> public class MultiThreadUsageExampleTest {
>
> @Test
> public void demonstrationTest(
> @Varying(from=1, to=10) int threadCount,
> @Varying(from=100000, to=1000000, step=100000) int entryCount) {
>
> // code here that relies on threadCount and entryCount
> }
>
> }
>
> In the above example, my runner calls demonstrationTest() 100 times, with
> parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10,
> 1000000).
>
> What I want to do now is produce charts of how long each call took. There
> are a lot of approaches, but I don't know which is "right."
>
> 1. Register a TestListener and write my own reports to disk from it (but
> is there a generic way to register listeners?)
> 2. Create my own main program that runs the test suite, and have it create
> the report when the suite is done (maybe this is the only way to register a
> TestListener anyhow?)
> 3. Transform the XML output of the ant junit task (but then it's bound to
> Ant)
> 4. Transform the XML output of the maven-surefure-plugin (but then it's
> bound to Maven)
> 5. Write performance results to disk directly from my TestRunner (should
> work in any environment, but feels like a dirty approach)
>
> Other ideas are welcome too, of course. My test runner is open source and
> published on GitHub; I want a solution that can work for everyone. So I
> think ideas 3 and 4 are out.
>
> Has anyone else thought about getting data from custom test runners into
> human-readable reports?
>
> -Jonathan
>
>
>


[Non-text portions of this message have been removed]

Thanks for the pointer, Dawid. I have to admit I didn't find
JUnitBenchmarks in my searches. I went looking for something back in
September and again earier this week and came up empty.

As far as what I'm trying to accomplish, it's kind of two things.

First, I want to microbenchmark the performance of some internal data
structures we're experimenting with for a web framework. I need to vary
more than one parameter for these tests: number of threads allowed, number
of messages to send through it, size of those messages, timing of the
messages, and more. A single test method is likely to run thousands of
times and produce multiple 2D charts worth of data. It looks like Google
Caliper (thanks for that pointer too!) may be well suited to this task.

Second, I want to run end-to-end scalability tests in various Java EE 6
containers. These tests will have many of the same variables. For these
tests, I think Caliper is out. We're going to need Arquillian to wrangle
all the server setup and teardown work. Arquillian is a JUnit 4 test runner
too--I haven't thought through how to reconcile my test runner with
Arquillian's, but I work on the same team as the Arquillian guys, and I'm
hoping we can figure something out. It will probably involve pointing
Arquillian at my test runner, or vice-versa.

Thank you once again for the responses. I really appreciate it!

Cheers,

-Jonathan

On Thu, Dec 15, 2011 at 3:47 AM, Dawid Weiss <dawid.weiss@...> wrote:

> **
>
>
> You may want to check out JUnitBenchmarks -- it does what I think you want
> to achieve:
> http://labs.carrotsearch.com/junit-benchmarks.html
>
> Dawid
>
> On Wed, Dec 14, 2011 at 11:01 PM, jfuerth <jfuerth@...> wrote:
>
> > **
>
> >
> >
> > Hi everyone,
> >
> > I've created a custom Runner meant to help with performance and
> > scalability testing. It lets you create tests like this:
> >
> > @RunWith(PerfRunner.class)
> > public class MultiThreadUsageExampleTest {
> >
> > @Test
> > public void demonstrationTest(
> > @Varying(from=1, to=10) int threadCount,
> > @Varying(from=100000, to=1000000, step=100000) int entryCount) {
> >
> > // code here that relies on threadCount and entryCount
> > }
> >
> > }
> >
> > In the above example, my runner calls demonstrationTest() 100 times, with
> > parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10,
> > 1000000).
> >
> > What I want to do now is produce charts of how long each call took. There
> > are a lot of approaches, but I don't know which is "right."
> >
> > 1. Register a TestListener and write my own reports to disk from it (but
> > is there a generic way to register listeners?)
> > 2. Create my own main program that runs the test suite, and have it
> create
> > the report when the suite is done (maybe this is the only way to
> register a
> > TestListener anyhow?)
> > 3. Transform the XML output of the ant junit task (but then it's bound to
> > Ant)
> > 4. Transform the XML output of the maven-surefure-plugin (but then it's
> > bound to Maven)
> > 5. Write performance results to disk directly from my TestRunner (should
> > work in any environment, but feels like a dirty approach)
> >
> > Other ideas are welcome too, of course. My test runner is open source and
> > published on GitHub; I want a solution that can work for everyone. So I
> > think ideas 3 and 4 are out.
> >
> > Has anyone else thought about getting data from custom test runners into
> > human-readable reports?
> >
> > -Jonathan
> >
> >
> >
>
> [Non-text portions of this message have been removed]
>
>
>


[Non-text portions of this message have been removed]

Thanks Matthew.

I will try writing out the reports from a RunListener. If they can be
registered by both the Ant and Maven plugins, that's pretty good.

Eclipse is another story: after a cursory search, I don't see a way to
register custom RunListeners in Eclipse's JUnit launcher. There is a
method in org.eclipse.jdt.junit.JUnitCore for registering
RunListeners, but nothing I can find in the UI for specifying one.
Worst case, I suppose I could just register the RunListener from
within my Runner, perhaps only when enabled by a system property.
That's not as bad as my original picture of option 5.

-Jonathan

On Thu, Dec 15, 2011 at 7:40 AM, Matthew Farwell <matthew@...> wrote:
> Salut,
>
> The best way would be to create your own RunListener (btw TestListener is
> JUnit3, RunListener is Junit4). Then, you can plug this into standard
> tools. For instance, in maven surefire, you can specify:
>
> <plugin>
>  <groupId>org.apache.maven.plugins</groupId>
>  <artifactId>maven-surefire-plugin</artifactId>
>  <version>2.10</version>
>  <configuration>
>    <properties>
>      <property>
>        <name>listener</name>
>
> <value>com.mycompany.MyResultListener,com.mycompany.MyResultListener2</value>
>      </property>
>  </configuration>
> </plugin>
>
> I think you can do similar things with ant. So if you do 1, 3 and 4 aren't
> a problem.
>
> Cordialement,
>
> Matthew Farwell.
>
> Le 14 décembre 2011 23:01, jfuerth <jfuerth@...> a écrit :
>
>> **
>>
>> What I want to do now is produce charts of how long each call took. There
>> are a lot of approaches, but I don't know which is "right."
>>
>> 1. Register a TestListener and write my own reports to disk from it (but
>> is there a generic way to register listeners?)
>> 2. Create my own main program that runs the test suite, and have it create
>> the report when the suite is done (maybe this is the only way to register a
>> TestListener anyhow?)
>> 3. Transform the XML output of the ant junit task (but then it's bound to
>> Ant)
>> 4. Transform the XML output of the maven-surefure-plugin (but then it's
>> bound to Maven)
>> 5. Write performance results to disk directly from my TestRunner (should
>> work in any environment, but feels like a dirty approach)
>>
>> Other ideas are welcome too, of course. My test runner is open source and
>> published on GitHub; I want a solution that can work for everyone. So I
>> think ideas 3 and 4 are out.
>>
>> Has anyone else thought about getting data from custom test runners into
>> human-readable reports?
>>
>> -Jonathan
>>
>>
>>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Am 15.12.2011 um 17:23 schrieb Jonathan Fuerth:
> Eclipse is another story: after a cursory search, I don't see a way to
> register custom RunListeners in Eclipse's JUnit launcher. There is a
> method in org.eclipse.jdt.junit.JUnitCore for registering
> RunListeners, but nothing I can find in the UI for specifying one.
>

I think this method is intended to be used by other Eclipse plugins, I don't
think there's an UI for that. You could wrap your custom listener in an Eclipse
plugin, though. Of course, that would only be worth the effort if you intend to
use it a lot.

Cheers,

Marc



[Non-text portions of this message have been removed]

On Thu, Dec 15, 2011 at 2:38 PM, Marc Philipp <mphilipp82@...> wrote:

> **
>
>
> Am 15.12.2011 um 17:23 schrieb Jonathan Fuerth:
>
> > Eclipse is another story: after a cursory search, I don't see a way to
> > register custom RunListeners in Eclipse's JUnit launcher. There is a
> > method in org.eclipse.jdt.junit.JUnitCore for registering
> > RunListeners, but nothing I can find in the UI for specifying one.
> >
>
> I think this method is intended to be used by other Eclipse plugins, I
> don't think there's an UI for that. You could wrap your custom listener in
> an Eclipse plugin, though. Of course, that would only be worth the effort
> if you intend to use it a lot.
>
>
Yes, I think I'd probably cheat and register the listener directly from
within the runner. If every JUnit launcher needs its own custom
configuration or plugin, that's not a very good experience for users of the
framework.

-Jonathan


[Non-text portions of this message have been removed]

Hi Jonathan,

Out of curiosity: have you considered implementing your framework extension as a
TestRule instead of a Runner?

Cheers,
Stephan

--- In junit@yahoogroups.com, "jfuerth" <jfuerth@...> wrote:
>
> Hi everyone,
>
> I've created a custom Runner meant to help with performance and scalability
testing. It lets you create tests like this:
>
> @RunWith(PerfRunner.class)
> public class MultiThreadUsageExampleTest {
>
>   @Test
>   public void demonstrationTest(
>       @Varying(from=1, to=10) int threadCount,
>       @Varying(from=100000, to=1000000, step=100000) int entryCount) {
>
>       // code here that relies on threadCount and entryCount
>   }
>
> }
>
> In the above example, my runner calls demonstrationTest() 100 times, with
parameter values (1, 100000), (2, 100000), ... (1, 200000), ... (10, 1000000).
>
> What I want to do now is produce charts of how long each call took. There are
a lot of approaches, but I don't know which is "right."
>
> 1. Register a TestListener and write my own reports to disk from it (but is
there a generic way to register listeners?)
> 2. Create my own main program that runs the test suite, and have it create the
report when the suite is done (maybe this is the only way to register a
TestListener anyhow?)
> 3. Transform the XML output of the ant junit task (but then it's bound to Ant)
> 4. Transform the XML output of the maven-surefure-plugin (but then it's bound
to Maven)
> 5. Write performance results to disk directly from my TestRunner (should work
in any environment, but feels like a dirty approach)
>
> Other ideas are welcome too, of course. My test runner is open source and
published on GitHub; I want a solution that can work for everyone. So I think
ideas 3 and 4 are out.
>
> Has anyone else thought about getting data from custom test runners into
human-readable reports?
>
> -Jonathan
>

When there are no tests matching the Include/Exclude filters, Categories
throws an InitializationError.  Can this behavior be changed?  IMO, if
nothing matches the criteria, the test execution should pass.

Currently, we're using a hack around this problem:

public class NullTest {
     @Test
     public void test() {
     }
}

Thanks,
Noel


[Non-text portions of this message have been removed]