Hi,

I maintain the junit provider for maven-surefire, and I'm looking into
transitioning from  a Junitcore+RunListener based approach to an approach based
on something like BlockJUnit4ClassRunner.

Unfortunately I cannot subclass the BlockJUnit4ClassRunner since the client code
itself may need to do that, so I'd somehow need to put "my"
Runner in front of the client-supplied one, maybe something proxy like.

Anyone have any tips on this, maybe some existing code that does this ?

Kristian

Hi Kristian

Do you really want a transition from RunListener?

To an outside observer - a JUnit Runner "describes" its test-executions and
reports their outcome to the RunNotifier, which forward the info to RunListener,
which is the interface that 3rd-party test-running applications (e.g. IDEs,
Maven-Surefire etc) are expected to use.

I have seen IDE-project test-extensions try to go beyond RunListener/RunNotifier
and start to joggle with Runner internals (e.g. test-instance creation/setup,
method-executions etc) and the result is something I don't like much. It works
for simple cases but with customized Runner-classes and new JUnit-features (e.g.
the introduction of JUnit-Rules) the result is usually bad and it discourages
the IDE users (i.e. the developers) from taking full advantage of JUnit!

However, Maven (versions 2.0-2.2) and its Surefire-plugin has handled the
JUnit-integration wonderfully, as far as I know, so your post worries me a
little. What is it that you want to accomplish by moving away from the
successful(!) "Junitcore+RunListener based approach"?

Henrik


--- In junit@yahoogroups.com, kristian@... wrote:
>
> Hi,
>
> I maintain the junit provider for maven-surefire, and I'm looking into
transitioning from  a Junitcore+RunListener based approach to an approach based
on something like BlockJUnit4ClassRunner.
>
> Unfortunately I cannot subclass the BlockJUnit4ClassRunner since the client
code itself may need to do that, so I'd somehow need to put "my"
> Runner in front of the client-supplied one, maybe something proxy like.
>
> Anyone have any tips on this, maybe some existing code that does this ?
>
> Kristian
>

Kristian,

In reading your original mail, I was also confused as to what you wanted to
achieve. Could you expand a little please?

Thanks.

Matthew Farwell.

Le 3 janvier 2012 22:45, hkaipe <hkaipe@...> a écrit :

> **
>
> Hi Kristian
>
> Do you really want a transition from RunListener?
>
> To an outside observer - a JUnit Runner "describes" its test-executions
> and reports their outcome to the RunNotifier, which forward the info to
> RunListener, which is the interface that 3rd-party test-running
> applications (e.g. IDEs, Maven-Surefire etc) are expected to use.
>
> I have seen IDE-project test-extensions try to go beyond
> RunListener/RunNotifier and start to joggle with Runner internals (e.g.
> test-instance creation/setup, method-executions etc) and the result is
> something I don't like much. It works for simple cases but with customized
> Runner-classes and new JUnit-features (e.g. the introduction of
> JUnit-Rules) the result is usually bad and it discourages the IDE users
> (i.e. the developers) from taking full advantage of JUnit!
>
> However, Maven (versions 2.0-2.2) and its Surefire-plugin has handled the
> JUnit-integration wonderfully, as far as I know, so your post worries me a
> little. What is it that you want to accomplish by moving away from the
> successful(!) "Junitcore+RunListener based approach"?
>
> Henrik
>
>
> --- In junit@yahoogroups.com, kristian@... wrote:
> >
> > Hi,
> >
> > I maintain the junit provider for maven-surefire, and I'm looking into
> transitioning from a Junitcore+RunListener based approach to an approach
> based on something like BlockJUnit4ClassRunner.
> >
> > Unfortunately I cannot subclass the BlockJUnit4ClassRunner since the
> client code itself may need to do that, so I'd somehow need to put "my"
> > Runner in front of the client-supplied one, maybe something proxy like.
> >
> > Anyone have any tips on this, maybe some existing code that does this ?
> >
> > Kristian
> >
>
>
>


[Non-text portions of this message have been removed]

Sorry, I was doing just what I sometimes dislike of our own users ;)

The real problem I want to solve is associating console output from a running
thread to a given test-case (for parallel runs). We have this fairly well
covered already and we manage to capture everything that happens within a given
test-case for that test-case.

The real problem is that the RunListener API does not demarcate test-classes
properly, so I am unable to distinguish console output happening in @AfterClass
of one test with @BeforeClass on the test-class on the same thread.

This problem has been in the corner of my eye for quite some time and I've been
trying to find a decent angle to solve it from. I promise not to do anything
that is frowned upon, which is why I asked ;)

Kristian


--- In junit@yahoogroups.com, Matthew Farwell <matthew@...> wrote:
>
> Kristian,
>
> In reading your original mail, I was also confused as to what you wanted to
> achieve. Could you expand a little please?
>
> Thanks.
>
> Matthew Farwell.
>
> Le 3 janvier 2012 22:45, hkaipe <hkaipe@...> a écrit :
>
> > **
> >
> > Hi Kristian
> >
> > Do you really want a transition from RunListener?
> >
> > To an outside observer - a JUnit Runner "describes" its test-executions
> > and reports their outcome to the RunNotifier, which forward the info to
> > RunListener, which is the interface that 3rd-party test-running
> > applications (e.g. IDEs, Maven-Surefire etc) are expected to use.
> >
> > I have seen IDE-project test-extensions try to go beyond
> > RunListener/RunNotifier and start to joggle with Runner internals (e.g.
> > test-instance creation/setup, method-executions etc) and the result is
> > something I don't like much. It works for simple cases but with customized
> > Runner-classes and new JUnit-features (e.g. the introduction of
> > JUnit-Rules) the result is usually bad and it discourages the IDE users
> > (i.e. the developers) from taking full advantage of JUnit!
> >
> > However, Maven (versions 2.0-2.2) and its Surefire-plugin has handled the
> > JUnit-integration wonderfully, as far as I know, so your post worries me a
> > little. What is it that you want to accomplish by moving away from the
> > successful(!) "Junitcore+RunListener based approach"?
> >
> > Henrik
> >
> >
> > --- In junit@yahoogroups.com, kristian@ wrote:
> > >
> > > Hi,
> > >
> > > I maintain the junit provider for maven-surefire, and I'm looking into
> > transitioning from a Junitcore+RunListener based approach to an approach
> > based on something like BlockJUnit4ClassRunner.
> > >
> > > Unfortunately I cannot subclass the BlockJUnit4ClassRunner since the
> > client code itself may need to do that, so I'd somehow need to put "my"
> > > Runner in front of the client-supplied one, maybe something proxy like.
> > >
> > > Anyone have any tips on this, maybe some existing code that does this ?
> > >
> > > Kristian
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>

Dear Kristian,

My first thought - and I hope not to sound arrogant - would be to
  - to remove the the output altogether (who reads it anyhow once the test is run
successfully the first time ;-); if one needs to read the output of the test
runs, what's the point of unit tests?!).
  - or to convert it into assertions (they also work nicely in methods not marked
with @Test).

If you still think you need the output, why not use a logger (log4j), which adds
all the information you need? Refactoring should be very easy.

As I'm starting to learn and play with AspectJ, I'd try that (but would not bet
money on this being a good idea). This would allow to intercept calls to
System.out.print() and alike, and tweak it print the argument(s) of the
print()-statement augmented with the class to which the calling method belongs.




--- In junit@yahoogroups.com, kristian@... wrote:
>
> Sorry, I was doing just what I sometimes dislike of our own users ;)
>
> The real problem I want to solve is associating console output from a running
thread to a given test-case (for parallel runs). We have this fairly well
covered already and we manage to capture everything that happens within a given
test-case for that test-case.
>
> The real problem is that the RunListener API does not demarcate test-classes
properly, so I am unable to distinguish console output happening in @AfterClass
of one test with @BeforeClass on the test-class on the same thread.
>
> This problem has been in the corner of my eye for quite some time and I've
been trying to find a decent angle to solve it from. I promise not to do
anything that is frowned upon, which is why I asked ;)
>
> Kristian
>
>
> --- In junit@yahoogroups.com, Matthew Farwell <matthew@> wrote:
> >
> > Kristian,
> >
> > In reading your original mail, I was also confused as to what you wanted to
> > achieve. Could you expand a little please?
> >
> > Thanks.
> >
> > Matthew Farwell.
> >
> > Le 3 janvier 2012 22:45, hkaipe <hkaipe@> a écrit :
> >
> > > **
> > >
> > > Hi Kristian
> > >
> > > Do you really want a transition from RunListener?
> > >
> > > To an outside observer - a JUnit Runner "describes" its test-executions
> > > and reports their outcome to the RunNotifier, which forward the info to
> > > RunListener, which is the interface that 3rd-party test-running
> > > applications (e.g. IDEs, Maven-Surefire etc) are expected to use.
> > >
> > > I have seen IDE-project test-extensions try to go beyond
> > > RunListener/RunNotifier and start to joggle with Runner internals (e.g.
> > > test-instance creation/setup, method-executions etc) and the result is
> > > something I don't like much. It works for simple cases but with customized
> > > Runner-classes and new JUnit-features (e.g. the introduction of
> > > JUnit-Rules) the result is usually bad and it discourages the IDE users
> > > (i.e. the developers) from taking full advantage of JUnit!
> > >
> > > However, Maven (versions 2.0-2.2) and its Surefire-plugin has handled the
> > > JUnit-integration wonderfully, as far as I know, so your post worries me a
> > > little. What is it that you want to accomplish by moving away from the
> > > successful(!) "Junitcore+RunListener based approach"?
> > >
> > > Henrik
> > >
> > >
> > > --- In junit@yahoogroups.com, kristian@ wrote:
> > > >
> > > > Hi,
> > > >
> > > > I maintain the junit provider for maven-surefire, and I'm looking into
> > > transitioning from a Junitcore+RunListener based approach to an approach
> > > based on something like BlockJUnit4ClassRunner.
> > > >
> > > > Unfortunately I cannot subclass the BlockJUnit4ClassRunner since the
> > > client code itself may need to do that, so I'd somehow need to put "my"
> > > > Runner in front of the client-supplied one, maybe something proxy like.
> > > >
> > > > Anyone have any tips on this, maybe some existing code that does this ?
> > > >
> > > > Kristian
> > > >
> > >
> > >
> > >
> >
> >
> > [Non-text portions of this message have been removed]
> >
>

Hi Kristian,

We had a similar issue. I was debating for a longer while if it makes sense
to run parallel tests within the same VM. In many situations this speeds
things a lot and has a nice memory conserving footprint but you cannot
guarantee tests won't interact with each other (in the simplest case via
mutating system properties). Finally the decision was to do it like Apache
Lucene did it -- split tests into parallel slave VMs with repeatable suite
execution order and sequential test execution within a single VM. This is
implemented in an ANT task called junit4 here.

https://github.com/carrotsearch/randomizedtesting/tree/master/ant-junit4

JUnit tests inside will give you a hint how to use it, the code is still
very fresh so documentation is scarce.

Now... this was a digression. Back to your question (because we did go that
route as well, only backed up), one possible solution to isolating
sysout/syserrs of parallel tests in a single VM is:

1) isolate each suite (for @BeforeClass/ @AfterClass hooks) into a separate
thread which you have under control. This can be either a custom @Runner or
a Computer, many ways to achieve it.

2) write a PrintStream implementation that redirects to a thread-local
buffer. This gist implements a similar idea to demux multiple thread
streams so that each one's output appears on a single line; you'll get the
idea I'm sure.

https://gist.github.com/1561833

3) once you have the above the rest is easy: replace default
System.out/System.err (I assume no security manager is present or it allows
such replacement), clear the buffer before suite, run your suite, flush the
buffer. The buffer's content will be your thread's output even though
system.* are singletons.

Like I said though -- I would consider using multiple jvms for running
parallel tests instead.

Dawid

On Wed, Jan 4, 2012 at 7:51 AM, <kristian@...> wrote:

> **
>
>
> Sorry, I was doing just what I sometimes dislike of our own users ;)
>
> The real problem I want to solve is associating console output from a
> running thread to a given test-case (for parallel runs). We have this
> fairly well covered already and we manage to capture everything that
> happens within a given test-case for that test-case.
>
> The real problem is that the RunListener API does not demarcate
> test-classes properly, so I am unable to distinguish console output
> happening in @AfterClass of one test with @BeforeClass on the test-class on
> the same thread.
>
> This problem has been in the corner of my eye for quite some time and I've
> been trying to find a decent angle to solve it from. I promise not to do
> anything that is frowned upon, which is why I asked ;)
>
> Kristian
>
>
> --- In junit@yahoogroups.com, Matthew Farwell <matthew@...> wrote:
> >
> > Kristian,
> >
> > In reading your original mail, I was also confused as to what you wanted
> to
> > achieve. Could you expand a little please?
> >
> > Thanks.
> >
> > Matthew Farwell.
> >
> > Le 3 janvier 2012 22:45, hkaipe <hkaipe@...> a écrit :
>
> >
> > > **
> > >
> > > Hi Kristian
> > >
> > > Do you really want a transition from RunListener?
> > >
> > > To an outside observer - a JUnit Runner "describes" its test-executions
> > > and reports their outcome to the RunNotifier, which forward the info to
> > > RunListener, which is the interface that 3rd-party test-running
> > > applications (e.g. IDEs, Maven-Surefire etc) are expected to use.
> > >
> > > I have seen IDE-project test-extensions try to go beyond
> > > RunListener/RunNotifier and start to joggle with Runner internals (e.g.
> > > test-instance creation/setup, method-executions etc) and the result is
> > > something I don't like much. It works for simple cases but with
> customized
> > > Runner-classes and new JUnit-features (e.g. the introduction of
> > > JUnit-Rules) the result is usually bad and it discourages the IDE users
> > > (i.e. the developers) from taking full advantage of JUnit!
> > >
> > > However, Maven (versions 2.0-2.2) and its Surefire-plugin has handled
> the
> > > JUnit-integration wonderfully, as far as I know, so your post worries
> me a
> > > little. What is it that you want to accomplish by moving away from the
> > > successful(!) "Junitcore+RunListener based approach"?
> > >
> > > Henrik
> > >
> > >
> > > --- In junit@yahoogroups.com, kristian@ wrote:
> > > >
> > > > Hi,
> > > >
> > > > I maintain the junit provider for maven-surefire, and I'm looking
> into
> > > transitioning from a Junitcore+RunListener based approach to an
> approach
> > > based on something like BlockJUnit4ClassRunner.
> > > >
> > > > Unfortunately I cannot subclass the BlockJUnit4ClassRunner since the
> > > client code itself may need to do that, so I'd somehow need to put "my"
> > > > Runner in front of the client-supplied one, maybe something proxy
> like.
> > > >
> > > > Anyone have any tips on this, maybe some existing code that does
> this ?
> > > >
> > > > Kristian
> > > >
> > >
> > >
> > >
> >
> >
> > [Non-text portions of this message have been removed]
> >
>
>
>


[Non-text portions of this message have been removed]

maven-surefire now also supports concurrent forks, and they don't have any of
the interaction problems you describe - as you know. Since surefire is an
all-purpose runner, we try to support any "sensible" mode of execution, and
there are good use cases for both models of execution.

Your first suggestion seems to work quite well, using the Computer I just wrap
the created runner in another runner that attaches the testclass to the thread
(and delegates on to the target Runner), which we pick up in reporting.

Surefire does all the other stuff already, but I'm thinking we might
be deprecating "both" mode threading support because it becomes
such a pain to do correctly, at least it requires a tight collaboration between
the RunListener and the Computer.


Kristian

--- In junit@yahoogroups.com, Dawid Weiss <dawid.weiss@...> wrote:
>
> Hi Kristian,
>
> We had a similar issue. I was debating for a longer while if it makes sense
> to run parallel tests within the same VM. In many situations this speeds
> things a lot and has a nice memory conserving footprint but you cannot
> guarantee tests won't interact with each other (in the simplest case via
> mutating system properties). Finally the decision was to do it like Apache
> Lucene did it -- split tests into parallel slave VMs with repeatable suite
> execution order and sequential test execution within a single VM. This is
> implemented in an ANT task called junit4 here.
>
> https://github.com/carrotsearch/randomizedtesting/tree/master/ant-junit4
>
> JUnit tests inside will give you a hint how to use it, the code is still
> very fresh so documentation is scarce.
>
> Now... this was a digression. Back to your question (because we did go that
> route as well, only backed up), one possible solution to isolating
> sysout/syserrs of parallel tests in a single VM is:
>
> 1) isolate each suite (for @BeforeClass/ @AfterClass hooks) into a separate
> thread which you have under control. This can be either a custom @Runner or
> a Computer, many ways to achieve it.
>
> 2) write a PrintStream implementation that redirects to a thread-local
> buffer. This gist implements a similar idea to demux multiple thread
> streams so that each one's output appears on a single line; you'll get the
> idea I'm sure.
>
> https://gist.github.com/1561833
>
> 3) once you have the above the rest is easy: replace default
> System.out/System.err (I assume no security manager is present or it allows
> such replacement), clear the buffer before suite, run your suite, flush the
> buffer. The buffer's content will be your thread's output even though
> system.* are singletons.
>
> Like I said though -- I would consider using multiple jvms for running
> parallel tests instead.
>
> Dawid
>
> On Wed, Jan 4, 2012 at 7:51 AM, <kristian@...> wrote:
>
> > **
> >
> >
> > Sorry, I was doing just what I sometimes dislike of our own users ;)
> >
> > The real problem I want to solve is associating console output from a
> > running thread to a given test-case (for parallel runs). We have this
> > fairly well covered already and we manage to capture everything that
> > happens within a given test-case for that test-case.
> >
> > The real problem is that the RunListener API does not demarcate
> > test-classes properly, so I am unable to distinguish console output
> > happening in @AfterClass of one test with @BeforeClass on the test-class on
> > the same thread.
> >
> > This problem has been in the corner of my eye for quite some time and I've
> > been trying to find a decent angle to solve it from. I promise not to do
> > anything that is frowned upon, which is why I asked ;)
> >
> > Kristian
> >
> >
> > --- In junit@yahoogroups.com, Matthew Farwell <matthew@> wrote:
> > >
> > > Kristian,
> > >
> > > In reading your original mail, I was also confused as to what you wanted
> > to
> > > achieve. Could you expand a little please?
> > >
> > > Thanks.
> > >
> > > Matthew Farwell.
> > >
> > > Le 3 janvier 2012 22:45, hkaipe <hkaipe@> a écrit :
> >
> > >
> > > > **
> > > >
> > > > Hi Kristian
> > > >
> > > > Do you really want a transition from RunListener?
> > > >
> > > > To an outside observer - a JUnit Runner "describes" its test-executions
> > > > and reports their outcome to the RunNotifier, which forward the info to
> > > > RunListener, which is the interface that 3rd-party test-running
> > > > applications (e.g. IDEs, Maven-Surefire etc) are expected to use.
> > > >
> > > > I have seen IDE-project test-extensions try to go beyond
> > > > RunListener/RunNotifier and start to joggle with Runner internals (e.g.
> > > > test-instance creation/setup, method-executions etc) and the result is
> > > > something I don't like much. It works for simple cases but with
> > customized
> > > > Runner-classes and new JUnit-features (e.g. the introduction of
> > > > JUnit-Rules) the result is usually bad and it discourages the IDE users
> > > > (i.e. the developers) from taking full advantage of JUnit!
> > > >
> > > > However, Maven (versions 2.0-2.2) and its Surefire-plugin has handled
> > the
> > > > JUnit-integration wonderfully, as far as I know, so your post worries
> > me a
> > > > little. What is it that you want to accomplish by moving away from the
> > > > successful(!) "Junitcore+RunListener based approach"?
> > > >
> > > > Henrik
> > > >
> > > >
> > > > --- In junit@yahoogroups.com, kristian@ wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I maintain the junit provider for maven-surefire, and I'm looking
> > into
> > > > transitioning from a Junitcore+RunListener based approach to an
> > approach
> > > > based on something like BlockJUnit4ClassRunner.
> > > > >
> > > > > Unfortunately I cannot subclass the BlockJUnit4ClassRunner since the
> > > > client code itself may need to do that, so I'd somehow need to put "my"
> > > > > Runner in front of the client-supplied one, maybe something proxy
> > like.
> > > > >
> > > > > Anyone have any tips on this, maybe some existing code that does
> > this ?
> > > > >
> > > > > Kristian
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>

Surefire is maven's default test-runner. It serves a bucketload of use-cases,
some of which are *far* off from what I'd do in my own tests. If the user
decides to use loggers or system out is not really something we want to dictate.

As a humoristic example, let me mention the issue we had on this guy spawning
threads from a test that were producing console output for the entire test run
that was collected by a JVM shutdown hook.

Kristian


--- In junit@yahoogroups.com, "Georg" <le_garcon_enerve@...> wrote:
>
> Dear Kristian,
>
> My first thought - and I hope not to sound arrogant - would be to
>  - to remove the the output altogether (who reads it anyhow once the test is
run successfully the first time ;-); if one needs to read the output of the test
runs, what's the point of unit tests?!).
>  - or to convert it into assertions (they also work nicely in methods not
marked with @Test).
>
> If you still think you need the output, why not use a logger (log4j), which
adds all the information you need? Refactoring should be very easy.
>
> As I'm starting to learn and play with AspectJ, I'd try that (but would not
bet money on this being a good idea). This would allow to intercept calls to
System.out.print() and alike, and tweak it print the argument(s) of the
print()-statement augmented with the class to which the calling method belongs.
>
>
>
>
> --- In junit@yahoogroups.com, kristian@ wrote:
> >
> > Sorry, I was doing just what I sometimes dislike of our own users ;)
> >
> > The real problem I want to solve is associating console output from a
running thread to a given test-case (for parallel runs). We have this fairly
well covered already and we manage to capture everything that happens within a
given test-case for that test-case.
> >
> > The real problem is that the RunListener API does not demarcate test-classes
properly, so I am unable to distinguish console output happening in @AfterClass
of one test with @BeforeClass on the test-class on the same thread.
> >
> > This problem has been in the corner of my eye for quite some time and I've
been trying to find a decent angle to solve it from. I promise not to do
anything that is frowned upon, which is why I asked ;)
> >
> > Kristian
> >
> >
> > --- In junit@yahoogroups.com, Matthew Farwell <matthew@> wrote:
> > >
> > > Kristian,
> > >
> > > In reading your original mail, I was also confused as to what you wanted
to
> > > achieve. Could you expand a little please?
> > >
> > > Thanks.
> > >
> > > Matthew Farwell.
> > >
> > > Le 3 janvier 2012 22:45, hkaipe <hkaipe@> a écrit :
> > >
> > > > **
> > > >
> > > > Hi Kristian
> > > >
> > > > Do you really want a transition from RunListener?
> > > >
> > > > To an outside observer - a JUnit Runner "describes" its test-executions
> > > > and reports their outcome to the RunNotifier, which forward the info to
> > > > RunListener, which is the interface that 3rd-party test-running
> > > > applications (e.g. IDEs, Maven-Surefire etc) are expected to use.
> > > >
> > > > I have seen IDE-project test-extensions try to go beyond
> > > > RunListener/RunNotifier and start to joggle with Runner internals (e.g.
> > > > test-instance creation/setup, method-executions etc) and the result is
> > > > something I don't like much. It works for simple cases but with
customized
> > > > Runner-classes and new JUnit-features (e.g. the introduction of
> > > > JUnit-Rules) the result is usually bad and it discourages the IDE users
> > > > (i.e. the developers) from taking full advantage of JUnit!
> > > >
> > > > However, Maven (versions 2.0-2.2) and its Surefire-plugin has handled
the
> > > > JUnit-integration wonderfully, as far as I know, so your post worries me
a
> > > > little. What is it that you want to accomplish by moving away from the
> > > > successful(!) "Junitcore+RunListener based approach"?
> > > >
> > > > Henrik
> > > >
> > > >
> > > > --- In junit@yahoogroups.com, kristian@ wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I maintain the junit provider for maven-surefire, and I'm looking into
> > > > transitioning from a Junitcore+RunListener based approach to an approach
> > > > based on something like BlockJUnit4ClassRunner.
> > > > >
> > > > > Unfortunately I cannot subclass the BlockJUnit4ClassRunner since the
> > > > client code itself may need to do that, so I'd somehow need to put "my"
> > > > > Runner in front of the client-supplied one, maybe something proxy
like.
> > > > >
> > > > > Anyone have any tips on this, maybe some existing code that does this
?
> > > > >
> > > > > Kristian
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> >
>

> maven-surefire now also supports concurrent forks, and they don't have any of
the interaction problems you describe - as you know. Since surefire is an >
all-purpose runner, we try to support any "sensible" mode of execution, and
there are good use cases for both models of execution.

I was thinking about ANT actually, not maven (surefire). I admit we
also needed some extra control over suite allocation scheduling and
more detailed reporting, so surefire wasn't a perfect fit for us. It
is a great tool though, I agree with that.

Dawid

Oh, I also didn't mention one more thing -- if you have tests that
fork subthreads which in turn write to System.out/err then you will
also need  a smarter way to track which suite a thread belongs to.
This can be done with a parent thread group (in most cases works) but
like you said -- I think it's just a pain and there is no way to
implement it correctly for all possible use cases.

Dawid

On Thu, Jan 5, 2012 at 12:52 PM, Dawid Weiss <dawid.weiss@...> wrote:
>> maven-surefire now also supports concurrent forks, and they don't have any of
the interaction problems you describe - as you know. Since surefire is an >
all-purpose runner, we try to support any "sensible" mode of execution, and
there are good use cases for both models of execution.
>
> I was thinking about ANT actually, not maven (surefire). I admit we
> also needed some extra control over suite allocation scheduling and
> more detailed reporting, so surefire wasn't a perfect fit for us. It
> is a great tool though, I agree with that.
>
> Dawid

I'm already working on something like that. I'm writing a new test
runner called Jumi http://jumi.fi/ which will have native support for
running tests in parallel in the same JVM, and it take care of capturing
the System.out/err of individual tests.

Jumi will do the output capturing by replacing System.out/err with its
own implementation, which saves all print events (the same way it saves
test lifecycle events), synchronizes both out and err (I think all
current IDEs and build tools fail this:
https://github.com/orfjackal/misc-tools/blob/master/src/test/java/net/orfjackal/\
bugs/SystemOutErrBugTest.java),
and also takes care of capturing the output of threads spawned by the
test thread. For long-lived threads, such as AWT event dispatch thread,
I will also try to figure out something (I'll timestamp all print
events, so it should be possible to figure out roughly which test
printed things in the AWT thread).

The output capturing is at the top of my TODO list, so I will have it
implemented after a couple of productive afternoons when I have time to
work on it. Soon after that I'll make the first alpha release.

I've looked a bit at how to integrate Jumi with maven-surefire. Jumi and
Surefire have some overlap, for example both of them take care of
classloading and starting a JVM process. One option is to use Surefire's
classloading and use only the core runner parts of Jumi. Otherwise Jumi
would need information that what are the dependency JARs, what are the
current project's classes, maybe also that which of them are test and
production classes (Jumi will do caching of the dependencies' class
loaders to speed up the tests). That would require changes to Surefire,
or then a completely new Maven plugin.


kristian@... wrote on 4.1.2012 8:51:
> Sorry, I was doing just what I sometimes dislike of our own users ;)
>
> The real problem I want to solve is associating console output from a
> running thread to a given test-case (for parallel runs). We have this
> fairly well covered already and we manage to capture everything that
> happens within a given test-case for that test-case.
>
> The real problem is that the RunListener API does not demarcate
> test-classes properly, so I am unable to distinguish console output
> happening in @AfterClass of one test with @BeforeClass on the test-class
> on the same thread.
>
> This problem has been in the corner of my eye for quite some time and
> I've been trying to find a decent angle to solve it from. I promise not
> to do anything that is frowned upon, which is why I asked ;)
>
> Kristian
>

--
Esko Luontola
www.orfjackal.net

You may want to look into RandomizedRunner/JUnit4 task -- it has most
of the things you mentioned (synchronized i/o events with reporting,
capturing runaway threads and the like).

The code is here:
https://github.com/carrotsearch/randomizedtesting/

I have a talk about randomized testing which includes the above
concepts at Lucene EuroCon Barcelona, the video is here:
http://vimeo.com/32087114

Dawid

On Fri, Jan 6, 2012 at 1:29 PM, Esko Luontola <esko.luontola@...> wrote:
>
>
>
> I'm already working on something like that. I'm writing a new test
> runner called Jumi http://jumi.fi/ which will have native support for
> running tests in parallel in the same JVM, and it take care of capturing
> the System.out/err of individual tests.
>
> Jumi will do the output capturing by replacing System.out/err with its
> own implementation, which saves all print events (the same way it saves
> test lifecycle events), synchronizes both out and err (I think all
> current IDEs and build tools fail this:
>
https://github.com/orfjackal/misc-tools/blob/master/src/test/java/net/orfjackal/\
bugs/SystemOutErrBugTest.java),
> and also takes care of capturing the output of threads spawned by the
> test thread. For long-lived threads, such as AWT event dispatch thread,
> I will also try to figure out something (I'll timestamp all print
> events, so it should be possible to figure out roughly which test
> printed things in the AWT thread).
>
> The output capturing is at the top of my TODO list, so I will have it
> implemented after a couple of productive afternoons when I have time to
> work on it. Soon after that I'll make the first alpha release.
>
> I've looked a bit at how to integrate Jumi with maven-surefire. Jumi and
> Surefire have some overlap, for example both of them take care of
> classloading and starting a JVM process. One option is to use Surefire's
> classloading and use only the core runner parts of Jumi. Otherwise Jumi
> would need information that what are the dependency JARs, what are the
> current project's classes, maybe also that which of them are test and
> production classes (Jumi will do caching of the dependencies' class
> loaders to speed up the tests). That would require changes to Surefire,
> or then a completely new Maven plugin.
>
> kristian@... wrote on 4.1.2012 8:51:
>
>
> > Sorry, I was doing just what I sometimes dislike of our own users ;)
> >
> > The real problem I want to solve is associating console output from a
> > running thread to a given test-case (for parallel runs). We have this
> > fairly well covered already and we manage to capture everything that
> > happens within a given test-case for that test-case.
> >
> > The real problem is that the RunListener API does not demarcate
> > test-classes properly, so I am unable to distinguish console output
> > happening in @AfterClass of one test with @BeforeClass on the test-class
> > on the same thread.
> >
> > This problem has been in the corner of my eye for quite some time and
> > I've been trying to find a decent angle to solve it from. I promise not
> > to do anything that is frowned upon, which is why I asked ;)
> >
> > Kristian
> >
>
> --
> Esko Luontola
> www.orfjackal.net
>
>

Actually this thread wasn't related to the standard JUnit Parameterized runner,
which is a failure. It was related to JUnitParams runner
(http://code.google.com/p/junitparams) which BTW. has been released today in the
new version.
Not only does JUnitParams solve the issue stated in the github comments you
posted, but many others as well... You're welcome to use it ;-)

Pawel

--- In junit@yahoogroups.com, "foo_bar_baz_qux" <dimitar.dimitrov@...> wrote:
>
> I was going to start a new thread, but I'll plug in here, to keep the
discussion in one place.
>
> I have had the same problem, different solution and the patch is in github:
https://github.com/KentBeck/junit/pull/145
>
> Florian has same problem, different approach:
https://github.com/KentBeck/junit/pull/358
>
> May the Powers That Be realize that this is a very annoying aspect of JUnit
(forcing us to count lines when a test fails) and consider including any of
these patches in the JUnit tree.
>
> Best regards,
> Dimitar
>

All,

Sorry for the radio silence that followed this post in last 2011.
Starting today, I've joined the Android team at Google, working on the
Google Books app
(http://saffgreenbar.blogspot.com/2012/01/new-role-at-google.html)

This means that for the first time in three years, I'm now a full-time
user of JUnit again, not just a maintainer.  Starting today, I'm going
to try to get back to one-week maximum latency for new code reviews
and communication around JUnit.  If you've gotten stuck in the backlog
I've built up in the interim, feel free to re-forward and jump the
queue.

Also, if it feels like it's taking a long time to get a code review,
please feel free to use this list to ask other devs to take a first
look: often, the first 80% of improvements, especially around naming
and documentation, can be provided by any suitably experienced
developer.

Thanks for your patience and for supporting each other within the community.

    David Saff

On Mon, Oct 31, 2011 at 2:51 PM, David Saff <david@...> wrote:
> All,
>
> I've been very lucky for several years to had both a family and an
> employer that have left room for me to contribute to JUnit for free.
> However, for at least the next month, two launches are combining to
> absorb the lion's share of my time.  One is named Micah, and is rather
> user-friendly most of the time, but has a somewhat draining
> maintenance schedule, and sporadic uptime.  The other doesn't yet have
> a last name that we're sharing, although I expect the first name will
> be "Google".
>
> Thanks for understanding if your great questions and valuable patches
> are responded to with more delay over the coming weeks.
>
>   David Saff

That is great, besides the fact that I will need to explain my team members and
the procurement department why am I dragging another library in the project, who
will support it and what will happen when the few committers move on to the next
big thing.

We need what is in the package to work - I took the time to implement a
solution, but apparently JUnit has turned into a personal playground for a
chosen few who are not interested in mundane day to day scenarios.

These days I am slowly building support to migrate all project's tests to
TestNG.


--- In junit@yahoogroups.com, "p.lipinski" <p.lipinski@...> wrote:
>
> Actually this thread wasn't related to the standard JUnit Parameterized
runner, which is a failure. It was related to JUnitParams runner
(http://code.google.com/p/junitparams) which BTW. has been released today in the
new version.
> Not only does JUnitParams solve the issue stated in the github comments you
posted, but many others as well... You're welcome to use it ;-)
>
> Pawel
>

Dimitar,

Patch processing has been slow on JUnit, as I shared with the list
that my paid work and family commitments have been taking up more of
my time over the last few months than before.  I hope you do find a
solution that works for you, whether it involves the overhead of
freely reusing others' efforts or of writing your own.

    David Saff

On Mon, Jan 23, 2012 at 10:08 AM, foo_bar_baz_qux
<dimitar.dimitrov@...> wrote:
> That is great, besides the fact that I will need to explain my team members
and the procurement department why am I dragging another library in the project,
who will support it and what will happen when the few committers move on to the
next big thing.
>
> We need what is in the package to work - I took the time to implement a
solution, but apparently JUnit has turned into a personal playground for a
chosen few who are not interested in mundane day to day scenarios.
>
> These days I am slowly building support to migrate all project's tests to
TestNG.
>
>
> --- In junit@yahoogroups.com, "p.lipinski" <p.lipinski@...> wrote:
>>
>> Actually this thread wasn't related to the standard JUnit Parameterized
runner, which is a failure. It was related to JUnitParams runner
(http://code.google.com/p/junitparams) which BTW. has been released today in the
new version.
>> Not only does JUnitParams solve the issue stated in the github comments you
posted, but many others as well... You're welcome to use it ;-)
>>
>> Pawel
>>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

All,

I'm trying to catch up on the Pull Request queue, with some help.
Here's a pull request that's essentially ready to go, it just needs
someone with git skills to merge it with HEAD.

This change would add assertGreaterThan and assertLessThan to the
standard assertions.

https://github.com/KentBeck/junit/pull/228

Is anyone motivated and up to the challenge?  Thanks,

    David Saff

Yeah, I can do it. What exactly needs to be done? A squash and push?

Thanks.

Matthew Farwell.

Le 31 janvier 2012 20:49, David Saff <david@...> a écrit :

> **
>
>
> All,
>
> I'm trying to catch up on the Pull Request queue, with some help.
> Here's a pull request that's essentially ready to go, it just needs
> someone with git skills to merge it with HEAD.
>
> This change would add assertGreaterThan and assertLessThan to the
> standard assertions.
>
> https://github.com/KentBeck/junit/pull/228
>
> Is anyone motivated and up to the challenge? Thanks,
>
> David Saff
>
>


[Non-text portions of this message have been removed]

Hi David,

I rebased @leet3lite's master branch onto @KentBeck's and resolved two
conflicts.

Here's a new pull request:
https://github.com/KentBeck/junit/pull/376

Please let me know if this one works better.

Cheers, Marc


Am 31.01.2012 um 20:49 schrieb David Saff:

> All,
>
> I'm trying to catch up on the Pull Request queue, with some help.
> Here's a pull request that's essentially ready to go, it just needs
> someone with git skills to merge it with HEAD.
>
> This change would add assertGreaterThan and assertLessThan to the
> standard assertions.
>
> https://github.com/KentBeck/junit/pull/228
>
> Is anyone motivated and up to the challenge? Thanks,
>
> David Saff
>



[Non-text portions of this message have been removed]

I am new to JUnit.Entire project has developed using spring and JSPs. Now my job
is review code and notice whether it follows coding standards or not.They didn't
write any code for JUnit.
I want to use JUnit for this purpose. Is it possible with JUnit.If it is
possible can I write any extra code for this
Thanks

Hi!

Please feel fre to take some code for this out of "unofficialjunit" on
sourceforge.

Regards,
 Georg


[Non-text portions of this message have been removed]

Hi All,

Is there a way to get more detail info after running test cases? like spend
how many time for each test case? list all passed test cases other than
just failed?

I am able to get some limit info form TestResult, but not enough to me.

any suggestions would be appreciate!

I launch test by TestRunner BTW.

Thanks
Wensi


[Non-text portions of this message have been removed]

Forgive me if this is a FAQ, but I have tried digging through the documentation
and the answer was not apparent.

I am writing some test cases.  As part of the testing, I want to keep a log of
the responses from the code for debugging purposes.  It would be really helpful
to have some simple process to print out the tests name and other properties
into the log.  Since there are hundreds of tests, I would like to make this a
common step (ie call a method) to do this.  Problem is finding the name of the
test itself (without having to pass it into the logging method or hacking the
stack).

I see there is a Description class in the runner.  Is there any way to access
the runner (i.e. a singleton or static getter) from the test case?

You should write a @Rule that will intercept the Description associated
with the current test case and store it somewhere (or use it directly).
Check out the javadoc for @Rule.

http://kentbeck.github.com/junit/javadoc/latest/org/junit/rules/TestRule.html


Dawid

On Wed, Feb 15, 2012 at 3:54 PM, Wayne <wdtj@...> wrote:

> **
>
>
> Forgive me if this is a FAQ, but I have tried digging through the
> documentation and the answer was not apparent.
>
> I am writing some test cases. As part of the testing, I want to keep a log
> of the responses from the code for debugging purposes. It would be really
> helpful to have some simple process to print out the tests name and other
> properties into the log. Since there are hundreds of tests, I would like to
> make this a common step (ie call a method) to do this. Problem is finding
> the name of the test itself (without having to pass it into the logging
> method or hacking the stack).
>
> I see there is a Description class in the runner. Is there any way to
> access the runner (i.e. a singleton or static getter) from the test case?
>
>
>


[Non-text portions of this message have been removed]

See also
https://github.com/KentBeck/junit/blob/master/src/main/java/org/junit/rules/Test\
Name.java

On Wed, Feb 15, 2012 at 6:05 PM, Dawid Weiss <dawid.weiss@...> wrote:
> You should write a @Rule that will intercept the Description associated
> with the current test case and store it somewhere (or use it directly).
> Check out the javadoc for @Rule.
>
> http://kentbeck.github.com/junit/javadoc/latest/org/junit/rules/TestRule.html
>
>
> Dawid
>
> On Wed, Feb 15, 2012 at 3:54 PM, Wayne <wdtj@...> wrote:
>
>> **
>>
>>
>> Forgive me if this is a FAQ, but I have tried digging through the
>> documentation and the answer was not apparent.
>>
>> I am writing some test cases. As part of the testing, I want to keep a log
>> of the responses from the code for debugging purposes. It would be really
>> helpful to have some simple process to print out the tests name and other
>> properties into the log. Since there are hundreds of tests, I would like to
>> make this a common step (ie call a method) to do this. Problem is finding
>> the name of the test itself (without having to pass it into the logging
>> method or hacking the stack).
>>
>> I see there is a Description class in the runner. Is there any way to
>> access the runner (i.e. a singleton or static getter) from the test case?
>>
>>
>>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Hello,

I wan't to ask for a solution to stop and/or destroy a JUnitCore.
Like pleaseStop() on the RunNotifier.

I already tried ExecutorsService with no success.
My only not yet tested solution is to System.exit my java programm after
starting a new process of it.

Hi all
I would like to know  how to set an order in Junit,
Details: the Junit tests are executing from quickbuild.
Thanks

Hi,

If you're asking about the ordering of tests, it's discouraged in JUnit.
For more information, see this stackoverflow Answer:

http://stackoverflow.com/questions/7845929/has-junit4-begun-supporting-ordering-\
of-test-is-it-intentional/7846712#7846712


Matthew Farwell.

Le 22 février 2012 15:27, nirh20 <nir.heifetz@...> a écrit :

> **
>
>
> Hi all
> I would like to know how to set an order in Junit,
> Details: the Junit tests are executing from quickbuild.
> Thanks
>
>
>


[Non-text portions of this message have been removed]

Thank you very much!
I will look on it


--- In junit@yahoogroups.com, Matthew Farwell <matthew@...> wrote:
>
> Hi,
>
> If you're asking about the ordering of tests, it's discouraged in JUnit.
> For more information, see this stackoverflow Answer:
>
>
http://stackoverflow.com/questions/7845929/has-junit4-begun-supporting-ordering-\
of-test-is-it-intentional/7846712#7846712
>
>
> Matthew Farwell.
>
> Le 22 février 2012 15:27, nirh20 <nir.heifetz@...> a écrit :
>
> > **
> >
> >
> > Hi all
> > I would like to know how to set an order in Junit,
> > Details: the Junit tests are executing from quickbuild.
> > Thanks
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>

Hi,

I've submitted a pull request for this
https://github.com/KentBeck/junit/pull/386.

@SortMethodsWith(MethodSorters.NAME_ASC)
public class MyTest {
}


There are four possibilities for the MethodSorters:

MethodSorters.DEFAULT: the default value, deterministic, but not predictable
MethodSorters.JVM: the order in which the tests are returned by the JVM, i.e.
there is no sorting done
MethodSorters.NAME_ASC: sorted in order of method name, ascending
MethodSorters.NAME_DESC: sorter in order of method name, descending

I would love to have feedback on this proposal.

Thanks.

Matthew Farwell.

--- In junit@yahoogroups.com, David Saff <david@...> wrote:
>
> On Thu, Sep 22, 2011 at 9:08 AM, mjfarwell <matthew@...> wrote:
> > If you change the order then at least change it so that if people have
problems due to ordering they can easily change the order. Yes, I know that
their tests are broken, but they probably do too. If the order is based upon the
hashCode, I have to either run that test with a custom Runner (and make sure
that it's usable in Eclipse, Jenkins, maven, etc. etc.)[*]. If the order is
simply the name, then all I have to do to 'fix' my problem is change the name.
>
> That's a fair request.  Actually, I think the code has evolved to the
> point that adding a feature to ParentRunner to support this would be
> remarkably easy:
>
> @RunWith(Suite.class)
> @SuiteClasses({...})
> public class MySuite {
>   @SortWith public static Sorter alpha = Sorters.ALPHABETICAL;
>   @FilterWith public static Filter category = new
> CategoryFilter(MyCategory.class);
> }
>
> Anyone want to take a crack at it?
>
>    David Saff