The way I have it set up, MyTest1 and MyTest2 are not discovered by
the normal runner, so they never run by themselves.  But otherwise,
yes, MyTest1 does not run as part of MyTestSuite and MyTest2 does run
as part of MyTestSuite.

Adding @RunWith(JUnit4.class) to MyTest2 makes no difference, it still runs.


David Saff wrote on 01/11/10 06:32 PM:
>
>
> Bill,
>
> I would expect what you're trying to do to work if you and I both
> understand correctly how subclassing Suite works.
>
> So you're saying that if MyTest1 has RunWith(Parameterized), and
> MyTest2 does not, then:
>
> MyTest1 runs by itself
> MyTest2 runs by itself
> MyTest1 does not run as part of MyTestSuite
> MyTest2 does run as part of MyTestSuite
>
> Do I have that right? If so, what if MyTest2 is annotated with
> @RunWith(JUnit4.class) [which _should_ be a no-op?]
>
> David Saff
>
> On Mon, Jan 11, 2010 at 4:20 PM, Bill Shannon <Bill.Shannon@...
> <mailto:Bill.Shannon%40sun.com>> wrote:
>  > I created a Suite that allows me to run tests in a separate class loader.
>  > I use it like this:
>  >
>  > @RunWith(ClassLoaderSuite.class)
>  > @SuiteClasses({ MyTest1.class, MyTest2.class })
>  > @TestClass(ClassToTest.class)
>  > public class MyTestSuite {
>  > }
>  >
>  > If I want one of those tests to be parameterized, should I expect to
>  > be able to do:
>  >
>  > @RunWith(Parameterized.class)
>  > public class MyTest1 {
>  > ...
>  > }
>  >
>  > It doesn't seem to work.
>  >
>  > What would I have to do in ClassLoaderSuite to make this work?
>  >
>  > FYI, you can find ClassLoaderSuite.java at:
>  >
>
http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java/\
com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>  >
>  >
>  > ------------------------------------
>  >
>  > Yahoo! Groups Links
>  >
>  >
>  >
>  >
>
>

I am trying to create a per-class aggregation of the output from the junitcore
run-listener, as is done by default in for instance maven surefire-plugin.

I have a solution that works, but unfortunately it's a kind of ketchup-bottle
solution; I have to wait for the entire test-run to finish before I can know
which tests have been run for the individual test classes, mostly because I do
not have any sensible way to know which tests will be run for a given class.

Anyone have any suggestions as to how to solve this ?

Kristian

testRunStarted gives a Description for the top-level suite, which can
be recursively descended to find which tests will be run for each
test.

By the way, in 4.9, we will most likely be deprecating some of
Description's API and introducing a new abstraction in order to
separate the actually fairly different interfaces needed for "what
tests are going to be run" vs. "what test is currently being run".
Just keep an eye out.

    David Saff

On Wed, Jan 13, 2010 at 4:45 AM, krosenvold <kristian@...> wrote:
> I am trying to create a per-class aggregation of the output from the junitcore
run-listener, as is done by default in for instance maven surefire-plugin.
>
> I have a solution that works, but unfortunately it's a kind of ketchup-bottle
solution; I have to wait for the entire test-run to finish before I can know
which tests have been run for the individual test classes, mostly because I do
not have any sensible way to know which tests will be run for a given class.
>
> Anyone have any suggestions as to how to solve this ?
>
> Kristian
>
>
>
>
>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

I've inherited a project using a mix of JUnit 3.8.1 and 4.4. The modules written
for a Java 1.4 deployment is tested with 3.8.1, the modules that needs a 1.5
environment, is tested with 4.4. (Hey, I've inherited it, ok?)

I've been looking for at document describing how to upgrade from 3.8.1 to 4.x,
but haven't found any. So I'll ask here.

Is it possible to run any 4.x JUnit with Java 1.4?
Is it possible to run the 3.8.1 tests with JUnit 4.x? Simply trying this gave me
some strange errors:
java.lang.Exception: Test class should have exactly one public constructor
and
java.lang.Exception: No runnable methods

M.

In my code, I would need to do something like

if (CodeISNotRunningInJUnit()) {
     //do some operations that must not be executed if tests are running
}


reason is that the database helper class used by application and junit tests
does some background operations that must NOT be executed while running junit
tests.

is there a way to implement the method CodeISNotRunningInJUnit() ?

thank you
  « Return to Thread: detect if junit is runnin

2010/1/13 iltrex76 <albmarchi@...>

>
>
> In my code, I would need to do something like
>
> if (CodeISNotRunningInJUnit()) {
> //do some operations that must not be executed if tests are running
> }
>
> reason is that the database helper class used by application and junit
> tests does some background operations that must NOT be executed while
> running junit tests.
>
> is there a way to implement the method CodeISNotRunningInJUnit() ?
>
> thank you
> « Return to Thread: detect if junit is runnin
>
> Extract into an interface, and pass in different implementations based on
the environment.

Cheers,

James


[Non-text portions of this message have been removed]

M,

You should be able to run the 3.8.1 tests with JUnit 4.  What steps
did you take?

   David Saff

On Wed, Jan 13, 2010 at 4:52 PM, blirp2001
<martin.liste.larsson@...> wrote:
> I've inherited a project using a mix of JUnit 3.8.1 and 4.4. The modules
written for a Java 1.4 deployment is tested with 3.8.1, the modules that needs a
1.5 environment, is tested with 4.4. (Hey, I've inherited it, ok?)
>
> I've been looking for at document describing how to upgrade from 3.8.1 to 4.x,
but haven't found any. So I'll ask here.
>
> Is it possible to run any 4.x JUnit with Java 1.4?
> Is it possible to run the 3.8.1 tests with JUnit 4.x? Simply trying this gave
me some strange errors:
> java.lang.Exception: Test class should have exactly one public constructor
> and
> java.lang.Exception: No runnable methods
>
> M.
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Since I couldn't find any documentation, I just changed the version number
from 3.8.1 to 4.7 in my pom.xml-file (since that's the latest in the
maven repo).
Then 'mvn clean install'.

Quite a few tests ran (~1000) but 5 failed with the given strange errors.
It seems the reason is that the classnames start with Test. Changing
TestOperationDAO to MockOperationDAO solves the problem.

But still... is there any documentation on what has changed since 3.x?

On Thu, Jan 14, 2010 at 2:51 PM, David Saff <david@...> wrote:
>
>
>
> M,
>
> You should be able to run the 3.8.1 tests with JUnit 4. What steps
> did you take?
>
> David Saff
>
> On Wed, Jan 13, 2010 at 4:52 PM, blirp2001
> <martin.liste.larsson@...> wrote:
> > I've inherited a project using a mix of JUnit 3.8.1 and 4.4. The modules
written for a Java 1.4 deployment is tested with 3.8.1, the modules that needs a
1.5 environment, is tested with 4.4. (Hey, I've inherited it, ok?)
> >
> > I've been looking for at document describing how to upgrade from 3.8.1 to
4.x, but haven't found any. So I'll ask here.
> >
> > Is it possible to run any 4.x JUnit with Java 1.4?
> > Is it possible to run the 3.8.1 tests with JUnit 4.x? Simply trying this
gave me some strange errors:
> > java.lang.Exception: Test class should have exactly one public constructor
> > and
> > java.lang.Exception: No runnable methods
> >
> > M.
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>

There are many ways to handle this. The most "standard" is to use a different
helper under test than in production (that share a common interface or base
class) that behave differently as required.

Another is to use a Dependency Injection framework (such as Guice or Spring) to
insert the right one in production vs. testing.

A more offbeat way, which is generally more around configuration than the way
you describe (starting background threads) is a system property. For example we
use it to make sure no actual customers get e-mails during QA tests against
copies of production data.

If you want to be (in my view) really ugly, you could just analyze the call
stack and look for any junit classes via Thread.currentThread().getStackTrace().

Those last two should really be a last resort because basically, from a TDD
perspective, this requirement to run sometimes with the background thread and
sometimes without is driving the design to abstract your database helper. Don't
fight that, listen to what the test is telling you (unless of course you have a
specific reason to know better).

Yishai

--- In junit@yahoogroups.com, "iltrex76" <albmarchi@...> wrote:
>
> In my code, I would need to do something like
>
> if (CodeISNotRunningInJUnit()) {
>     //do some operations that must not be executed if tests are running
> }
>
>
> reason is that the database helper class used by application and junit tests
does some background operations that must NOT be executed while running junit
tests.
>
> is there a way to implement the method CodeISNotRunningInJUnit() ?
>
> thank you
>  « Return to Thread: detect if junit is runnin
>

That's a problem with your Maven configuration. Not sure why it only surfaces
with JUnit 4, but you have to configure the include patterns of
maven-surefire-plugin (<includes/>) to only match test classes. One of the
default patterns is "**/Test*.java" which explains your problem with
TestOperationDAO. See the documentation at:
http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html

Cheers,
Peter


martin liste larsson wrote:
>
> It seems the reason is that the classnames start with Test. Changing
>
--
View this message in context:
http://old.nabble.com/Upgrading-from-3.8.1-to-4.x--tp27153980p27169769.html
Sent from the JUnit - User mailing list archive at Nabble.com.

Anyone have any ideas?

Bill Shannon wrote on 01/12/2010 11:04 AM:
> The way I have it set up, MyTest1 and MyTest2 are not discovered by
> the normal runner, so they never run by themselves.  But otherwise,
> yes, MyTest1 does not run as part of MyTestSuite and MyTest2 does run
> as part of MyTestSuite.
>
> Adding @RunWith(JUnit4.class) to MyTest2 makes no difference, it still
> runs.
>
>
> David Saff wrote on 01/11/10 06:32 PM:
>>
>>
>> Bill,
>>
>> I would expect what you're trying to do to work if you and I both
>> understand correctly how subclassing Suite works.
>>
>> So you're saying that if MyTest1 has RunWith(Parameterized), and
>> MyTest2 does not, then:
>>
>> MyTest1 runs by itself
>> MyTest2 runs by itself
>> MyTest1 does not run as part of MyTestSuite
>> MyTest2 does run as part of MyTestSuite
>>
>> Do I have that right? If so, what if MyTest2 is annotated with
>> @RunWith(JUnit4.class) [which _should_ be a no-op?]
>>
>> David Saff
>>
>> On Mon, Jan 11, 2010 at 4:20 PM, Bill Shannon <Bill.Shannon@...
>> <mailto:Bill.Shannon%40sun.com>> wrote:
>>  > I created a Suite that allows me to run tests in a separate class
>> loader.
>>  > I use it like this:
>>  >
>>  > @RunWith(ClassLoaderSuite.class)
>>  > @SuiteClasses({ MyTest1.class, MyTest2.class })
>>  > @TestClass(ClassToTest.class)
>>  > public class MyTestSuite {
>>  > }
>>  >
>>  > If I want one of those tests to be parameterized, should I expect to
>>  > be able to do:
>>  >
>>  > @RunWith(Parameterized.class)
>>  > public class MyTest1 {
>>  > ...
>>  > }
>>  >
>>  > It doesn't seem to work.
>>  >
>>  > What would I have to do in ClassLoaderSuite to make this work?
>>  >
>>  > FYI, you can find ClassLoaderSuite.java at:
>>  >
>>
http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java/\
com/sun/mail/test/ClassLoaderSuite.java?rev=231
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>>
>>  >
>>  >
>>  > ------------------------------------
>>  >
>>  > Yahoo! Groups Links
>>  >
>>  >
>>  >
>>  >
>>
>>
>

 
Hi,
I am trying to use the new experimental Categories Feature. Unfortunately it
doesn't seem to be compatible with Parameterized Tests. I run the following in
eclipse, but since these are
 
When defining  Tests and TestSuite as below, the execution of the suite results
in errors/failures. The exact error depends on the order in which the tests are
specified in the Suite (see below).
 
So I guess my points/ questions are:
- The behavior of a TestSuite should not depend on the order of contained Tests
- If a Runner isn't supported by a Suite Runner, this should result in an
instructive error message
- Is there a Category compatible version of the Parameterized runner?
 
Thanx
Jens
 
 
 
With the normal Test first I get this call stack on the console:
 
JUnit version 4.8.1
Exception in thread "main" java.lang.NullPointerException
  at org.junit.runner.Description.createSuiteDescription(Description.java:
72)
  at org.junit.experimental.categories.Categories$CategoryFilter.parentDes
cription(Categories.java:127)
  at org.junit.experimental.categories.Categories$CategoryFilter.categorie
s(Categories.java:121)
  at org.junit.experimental.categories.Categories$CategoryFilter.hasCorrec
tCategoryAnnotation(Categories.java:106)
  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
(Categories.java:97)
  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
(Categories.java:100)
  at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:299)
  at org.junit.runners.ParentRunner.getFilteredChildren(ParentRunner.java:
277)
  at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:225)
  at org.junit.runners.Suite.describeChild(Suite.java:123)
  at org.junit.runners.Suite.describeChild(Suite.java:24)
  at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:226)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:156)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
  at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
  at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
  at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
 
--------------------------------------------------------------------------------\
----
With the Parameterized Test first, I get a Failure Trace (which also isn't very
instructive):
 
JUnit version 4.8.1
.E
Time: 0,006
There was 1 failure:
1) initializationError(de.schauderhaft.junit.categories.ParameterTokenSuite)
java.lang.NullPointerException
  at org.junit.runner.Description.createSuiteDescription(Description.java:
72)
  at org.junit.experimental.categories.Categories$CategoryFilter.parentDes
cription(Categories.java:127)
  at org.junit.experimental.categories.Categories$CategoryFilter.categorie
s(Categories.java:121)
  at org.junit.experimental.categories.Categories$CategoryFilter.hasCorrec
tCategoryAnnotation(Categories.java:106)
  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
(Categories.java:97)
  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
(Categories.java:100)
  at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:299)
  at org.junit.runners.ParentRunner.filter(ParentRunner.java:254)
  at org.junit.experimental.categories.Categories.(Categories.java:1
42)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou
rce)
  at java.lang.reflect.Constructor.newInstance(Unknown Source)
  at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBui
lder.java:35)
  at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(Annotated
Builder.java:24)
  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilde
r.java:57)
  at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForC
lass(AllDefaultPossibilitiesBuilder.java:29)
  at org.junit.runner.Computer.getRunner(Computer.java:38)
  at org.junit.runner.Computer$1.runnerForClass(Computer.java:29)
  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilde
r.java:57)
  at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:93)
  at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:84)
  at org.junit.runners.Suite.(Suite.java:79)
  at org.junit.runner.Computer.getSuite(Computer.java:26)
  at org.junit.runner.Request.classes(Request.java:69)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
  at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
  at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
  at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
FAILURES!!!
Tests run: 1,  Failures: 1
 
 
Class Definitions
--------------------------------------------------------------------------------\
----
 
package de.schauderhaft.junit.categories;
import java.util.Collection;
import java.util.Collections;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
@RunWith(Parameterized.class)
public class ParameterizedTestA {
public ParameterizedTestA(String a) {
  }
@Parameters
  public static Collection getParameters() {
  return Collections.singletonList(new String[] { "a" });
  }
@Test
  public void testSomething() {
  Assert.assertTrue(true);
  }
--------------------------------------------------------------------------------\
----
package de.schauderhaft.junit.categories;
import org.junit.Assert;
import org.junit.Test;
import org.junit.experimental.categories.Category;
@Category(Token.class)
public class SomeTestB {
@Test
  public void testSomething() {
  Assert.assertTrue(true);
  }
}
 
--------------------------------------------------------------------------------\
----

package de.schauderhaft.junit.categories;
import org.junit.experimental.categories.Categories;
import org.junit.experimental.categories.Categories.IncludeCategory;
import org.junit.runner.RunWith;
import org.junit.runners.Suite.SuiteClasses;
@RunWith(Categories.class)
@IncludeCategory(Token.class)
@SuiteClasses( { SomeTestB.class, ParameterizedTestA.class }) // switch order of
classes for slightly different behaviour
public class ParameterTokenSuite {
}
 
}
--------------------------------------------------------------------------------\
----
 
 

--


[Non-text portions of this message have been removed]

Hi All -

I have a theory enabled test case, with few data points. When my theory
method is executed with by the Theories runner, for few data points my
theory fails.

Currently, the failer message says the instance variable name (or) the array
name with the index. I expect the data point value to be displayed, rather
than its variable.

Is there a way to fix this problem?

Regards
Sai


[Non-text portions of this message have been removed]

There's no default way.  You could patch the Theory runner yourself,
or submit a feature request at junit.org.  Thanks!

    David Saff

On Sun, Jan 17, 2010 at 8:29 AM, B. Sai Prasad, M.Phil, SCJP, MCP,
MCTS, SSGB, PMP, PMI-SP <sai.prasad.b@...> wrote:
> Hi All -
>
> I have a theory enabled test case, with few data points. When my theory
> method is executed with by the Theories runner, for few data points my
> theory fails.
>
> Currently, the failer message says the instance variable name (or) the array
> name with the index. I expect the data point value to be displayed, rather
> than its variable.
>
> Is there a way to fix this problem?
>
> Regards
> Sai
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Jens,

Definitely a bug.  Have you already logged it at junit.org as well?
If not, I will, and we'll get on it soon.

    David Saff

On Fri, Jan 15, 2010 at 4:00 AM,  <jens.schauder@...> wrote:
>
> Hi,
> I am trying to use the new experimental Categories Feature. Unfortunately it
doesn't seem to be compatible with Parameterized Tests. I run the following in
eclipse, but since these are
>
> When defining  Tests and TestSuite as below, the execution of the suite
results in errors/failures. The exact error depends on the order in which the
tests are specified in the Suite (see below).
>
> So I guess my points/ questions are:
> - The behavior of a TestSuite should not depend on the order of contained
Tests
> - If a Runner isn't supported by a Suite Runner, this should result in an
instructive error message
> - Is there a Category compatible version of the Parameterized runner?
>
> Thanx
> Jens
>
>
>
> With the normal Test first I get this call stack on the console:
>
> JUnit version 4.8.1
> Exception in thread "main" java.lang.NullPointerException
>  at org.junit.runner.Description.createSuiteDescription(Description.java:
> 72)
>  at org.junit.experimental.categories.Categories$CategoryFilter.parentDes
> cription(Categories.java:127)
>  at org.junit.experimental.categories.Categories$CategoryFilter.categorie
> s(Categories.java:121)
>  at org.junit.experimental.categories.Categories$CategoryFilter.hasCorrec
> tCategoryAnnotation(Categories.java:106)
>  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
> (Categories.java:97)
>  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
> (Categories.java:100)
>  at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:299)
>  at org.junit.runners.ParentRunner.getFilteredChildren(ParentRunner.java:
> 277)
>  at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:225)
>  at org.junit.runners.Suite.describeChild(Suite.java:123)
>  at org.junit.runners.Suite.describeChild(Suite.java:24)
>  at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:226)
>  at org.junit.runner.JUnitCore.run(JUnitCore.java:156)
>  at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>  at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>  at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
>  at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
>  at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
>
>
--------------------------------------------------------------------------------\
----
> With the Parameterized Test first, I get a Failure Trace (which also isn't
very instructive):
>
> JUnit version 4.8.1
> .E
> Time: 0,006
> There was 1 failure:
> 1) initializationError(de.schauderhaft.junit.categories.ParameterTokenSuite)
> java.lang.NullPointerException
>  at org.junit.runner.Description.createSuiteDescription(Description.java:
> 72)
>  at org.junit.experimental.categories.Categories$CategoryFilter.parentDes
> cription(Categories.java:127)
>  at org.junit.experimental.categories.Categories$CategoryFilter.categorie
> s(Categories.java:121)
>  at org.junit.experimental.categories.Categories$CategoryFilter.hasCorrec
> tCategoryAnnotation(Categories.java:106)
>  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
> (Categories.java:97)
>  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
> (Categories.java:100)
>  at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:299)
>  at org.junit.runners.ParentRunner.filter(ParentRunner.java:254)
>  at org.junit.experimental.categories.Categories.(Categories.java:1
> 42)
>  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou
> rce)
>  at java.lang.reflect.Constructor.newInstance(Unknown Source)
>  at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBui
> lder.java:35)
>  at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(Annotated
> Builder.java:24)
>  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilde
> r.java:57)
>  at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForC
> lass(AllDefaultPossibilitiesBuilder.java:29)
>  at org.junit.runner.Computer.getRunner(Computer.java:38)
>  at org.junit.runner.Computer$1.runnerForClass(Computer.java:29)
>  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilde
> r.java:57)
>  at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:93)
>  at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:84)
>  at org.junit.runners.Suite.(Suite.java:79)
>  at org.junit.runner.Computer.getSuite(Computer.java:26)
>  at org.junit.runner.Request.classes(Request.java:69)
>  at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>  at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
>  at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
>  at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
> FAILURES!!!
> Tests run: 1,  Failures: 1
>
>
> Class Definitions
--------------------------------------------------------------------------------\
----
>
> package de.schauderhaft.junit.categories;
> import java.util.Collection;
> import java.util.Collections;
> import org.junit.Assert;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.Parameterized;
> import org.junit.runners.Parameterized.Parameters;
> @RunWith(Parameterized.class)
> public class ParameterizedTestA {
> public ParameterizedTestA(String a) {
>  }
> @Parameters
>  public static Collection getParameters() {
>  return Collections.singletonList(new String[] { "a" });
>  }
> @Test
>  public void testSomething() {
>  Assert.assertTrue(true);
>  }
>
--------------------------------------------------------------------------------\
----
> package de.schauderhaft.junit.categories;
> import org.junit.Assert;
> import org.junit.Test;
> import org.junit.experimental.categories.Category;
> @Category(Token.class)
> public class SomeTestB {
> @Test
>  public void testSomething() {
>  Assert.assertTrue(true);
>  }
> }
>
>
--------------------------------------------------------------------------------\
----
>
> package de.schauderhaft.junit.categories;
> import org.junit.experimental.categories.Categories;
> import org.junit.experimental.categories.Categories.IncludeCategory;
> import org.junit.runner.RunWith;
> import org.junit.runners.Suite.SuiteClasses;
> @RunWith(Categories.class)
> @IncludeCategory(Token.class)
> @SuiteClasses( { SomeTestB.class, ParameterizedTestA.class }) // switch order
of classes for slightly different behaviour
> public class ParameterTokenSuite {
> }
>
> }
>
--------------------------------------------------------------------------------\
----
>
>
>
> --
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Bill,

Jens recently posted a helpful bug with Suite and Parameterized.  It's
possible the behavior you're seeing is related.  Can you reduce your
problem down to a small amount of code you can share?  Thanks,

    David Saff

On Thu, Jan 14, 2010 at 10:17 PM, Bill Shannon <Bill.Shannon@...> wrote:
> Anyone have any ideas?
>
> Bill Shannon wrote on 01/12/2010 11:04 AM:
>> The way I have it set up, MyTest1 and MyTest2 are not discovered by
>> the normal runner, so they never run by themselves.  But otherwise,
>> yes, MyTest1 does not run as part of MyTestSuite and MyTest2 does run
>> as part of MyTestSuite.
>>
>> Adding @RunWith(JUnit4.class) to MyTest2 makes no difference, it still
>> runs.
>>
>>
>> David Saff wrote on 01/11/10 06:32 PM:
>>>
>>>
>>> Bill,
>>>
>>> I would expect what you're trying to do to work if you and I both
>>> understand correctly how subclassing Suite works.
>>>
>>> So you're saying that if MyTest1 has RunWith(Parameterized), and
>>> MyTest2 does not, then:
>>>
>>> MyTest1 runs by itself
>>> MyTest2 runs by itself
>>> MyTest1 does not run as part of MyTestSuite
>>> MyTest2 does run as part of MyTestSuite
>>>
>>> Do I have that right? If so, what if MyTest2 is annotated with
>>> @RunWith(JUnit4.class) [which _should_ be a no-op?]
>>>
>>> David Saff
>>>
>>> On Mon, Jan 11, 2010 at 4:20 PM, Bill Shannon <Bill.Shannon@...
>>> <mailto:Bill.Shannon%40sun.com>> wrote:
>>>  > I created a Suite that allows me to run tests in a separate class
>>> loader.
>>>  > I use it like this:
>>>  >
>>>  > @RunWith(ClassLoaderSuite.class)
>>>  > @SuiteClasses({ MyTest1.class, MyTest2.class })
>>>  > @TestClass(ClassToTest.class)
>>>  > public class MyTestSuite {
>>>  > }
>>>  >
>>>  > If I want one of those tests to be parameterized, should I expect to
>>>  > be able to do:
>>>  >
>>>  > @RunWith(Parameterized.class)
>>>  > public class MyTest1 {
>>>  > ...
>>>  > }
>>>  >
>>>  > It doesn't seem to work.
>>>  >
>>>  > What would I have to do in ClassLoaderSuite to make this work?
>>>  >
>>>  > FYI, you can find ClassLoaderSuite.java at:
>>>  >
>>>
http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java/\
com/sun/mail/test/ClassLoaderSuite.java?rev=231
>>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>>>
>>>  >
>>>  >
>>>  > ------------------------------------
>>>  >
>>>  > Yahoo! Groups Links
>>>  >
>>>  >
>>>  >
>>>  >
>>>
>>>
>>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

All,

There are many things I love about github, but in the past, I have
found it someone bewildering to keep up with the various ways that
maintenance work can be brought to my attention, including issues,
direct messages, and pull requests, which has allowed a lot of
communication to back up, and now it's not clear to me how the various
proposals inter-relate, so I've dropped some valuable conversations.
I'm sorry.

So, I'm going to try again, but I'm wiping the slate clean.  If you've
sent an issue, and are just waiting for it to get fixed, you have
nothing more to do.  If you've sent a pull request in the past, please
send it again, and also link to the commit from the github issue.  If
you've tried to send me a message direct through github, please try
again, or, even better, ping me directly at david@....

Again, my apologies.

   David Saff

Hi,

I have submitted this issue in the past
(http://github.com/KentBeck/junit/issues#issue/41) but now it has become more
relevant (and perhaps a more definitive solution presents itself):

When I went to implement categories in my project (instead of some rather ugly
global system variables and a bunch of assumeTrue peppered hither and yon), I
found I couldn't use it because it doesn't expose the Suite constructor that
allows you to pass an array of classes (which is what I was using to create
dynamic suites until now).

I think this points to a @SuiteMethod annotation (or just make the current
@SuiteClasses annotation able to annotate a method, and if it does require the
method to be public, static, take no parameters and return a Class array), so
that it combines well with Categories and any other future subclass of Suite
that wants to collect the classes from the annotation.

By the way, this and one other issue (better naming of parameterized tests
Issues 24, 44, 46 and I though one more where I put in my two cents) makes me
want to get involved in GitHub to suggest some implementations. What is the best
getting started guide for a Windows user out there?

I heard from someone else off-list that, due to the design of JUnit
runners, this almost certainly isn't going to work.

All of the relevant code is part of an existing open source project,
and I can tell you how to reproduce the problem with that code.  Is
that sufficient?

Or are you looking for a completely standalone project that demonstrates
the problem?

I guess I'm not sure whether you're still confused about what it is I'm
trying to do, or whether you're sure it *should* work and you want a
reproducible test case so that you can debug it.  In the latter case,
how would you like me to deliver that?  (Note that my experience with
JUnit is limited.  For instance, I've only ever run it as part of a
maven project.)


David Saff wrote on 01/19/10 06:37 AM:
>
>
> Bill,
>
> Jens recently posted a helpful bug with Suite and Parameterized. It's
> possible the behavior you're seeing is related. Can you reduce your
> problem down to a small amount of code you can share? Thanks,
>
> David Saff
>
> On Thu, Jan 14, 2010 at 10:17 PM, Bill Shannon <Bill.Shannon@...
> <mailto:Bill.Shannon%40sun.com>> wrote:
>  > Anyone have any ideas?
>  >
>  > Bill Shannon wrote on 01/12/2010 11:04 AM:
>  >> The way I have it set up, MyTest1 and MyTest2 are not discovered by
>  >> the normal runner, so they never run by themselves.  But otherwise,
>  >> yes, MyTest1 does not run as part of MyTestSuite and MyTest2 does run
>  >> as part of MyTestSuite.
>  >>
>  >> Adding @RunWith(JUnit4.class) to MyTest2 makes no difference, it still
>  >> runs.
>  >>
>  >>
>  >> David Saff wrote on 01/11/10 06:32 PM:
>  >>>
>  >>>
>  >>> Bill,
>  >>>
>  >>> I would expect what you're trying to do to work if you and I both
>  >>> understand correctly how subclassing Suite works.
>  >>>
>  >>> So you're saying that if MyTest1 has RunWith(Parameterized), and
>  >>> MyTest2 does not, then:
>  >>>
>  >>> MyTest1 runs by itself
>  >>> MyTest2 runs by itself
>  >>> MyTest1 does not run as part of MyTestSuite
>  >>> MyTest2 does run as part of MyTestSuite
>  >>>
>  >>> Do I have that right? If so, what if MyTest2 is annotated with
>  >>> @RunWith(JUnit4.class) [which _should_ be a no-op?]
>  >>>
>  >>> David Saff
>  >>>
>  >>> On Mon, Jan 11, 2010 at 4:20 PM, Bill Shannon <Bill.Shannon@...
> <mailto:Bill.Shannon%40sun.com>
>  >>> <mailto:Bill.Shannon%40sun.com>> wrote:
>  >>>  > I created a Suite that allows me to run tests in a separate class
>  >>> loader.
>  >>>  > I use it like this:
>  >>>  >
>  >>>  > @RunWith(ClassLoaderSuite.class)
>  >>>  > @SuiteClasses({ MyTest1.class, MyTest2.class })
>  >>>  > @TestClass(ClassToTest.class)
>  >>>  > public class MyTestSuite {
>  >>>  > }
>  >>>  >
>  >>>  > If I want one of those tests to be parameterized, should I expect to
>  >>>  > be able to do:
>  >>>  >
>  >>>  > @RunWith(Parameterized.class)
>  >>>  > public class MyTest1 {
>  >>>  > ...
>  >>>  > }
>  >>>  >
>  >>>  > It doesn't seem to work.
>  >>>  >
>  >>>  > What would I have to do in ClassLoaderSuite to make this work?
>  >>>  >
>  >>>  > FYI, you can find ClassLoaderSuite.java at:
>  >>>  >
>  >>>
>
http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java/\
com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>  >>>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>
>  >>>
>  >>>  >
>  >>>  >
>  >>>  > ------------------------------------
>  >>>  >
>  >>>  > Yahoo! Groups Links
>  >>>  >
>  >>>  >
>  >>>  >
>  >>>  >
>  >>>
>  >>>
>  >>
>  >
>  >
>  >
>  > ------------------------------------
>  >
>  > Yahoo! Groups Links
>  >
>  >
>  >
>  >
>
>

Bill,

Nothing you've said so far seems impossible, so I am indeed just
trying to replicate the bug you're facing, for debugging.  Steps to
reproduce on open-source code are a great start.  Thanks,

    David Saff

On Tue, Jan 19, 2010 at 12:50 PM, Bill Shannon <Bill.Shannon@...> wrote:
> I heard from someone else off-list that, due to the design of JUnit
> runners, this almost certainly isn't going to work.
>
> All of the relevant code is part of an existing open source project,
> and I can tell you how to reproduce the problem with that code.  Is
> that sufficient?
>
> Or are you looking for a completely standalone project that demonstrates
> the problem?
>
> I guess I'm not sure whether you're still confused about what it is I'm
> trying to do, or whether you're sure it *should* work and you want a
> reproducible test case so that you can debug it.  In the latter case,
> how would you like me to deliver that?  (Note that my experience with
> JUnit is limited.  For instance, I've only ever run it as part of a
> maven project.)
>
>
> David Saff wrote on 01/19/10 06:37 AM:
>>
>>
>> Bill,
>>
>> Jens recently posted a helpful bug with Suite and Parameterized. It's
>> possible the behavior you're seeing is related. Can you reduce your
>> problem down to a small amount of code you can share? Thanks,
>>
>> David Saff
>>
>> On Thu, Jan 14, 2010 at 10:17 PM, Bill Shannon <Bill.Shannon@...
>> <mailto:Bill.Shannon%40sun.com>> wrote:
>>  > Anyone have any ideas?
>>  >
>>  > Bill Shannon wrote on 01/12/2010 11:04 AM:
>>  >> The way I have it set up, MyTest1 and MyTest2 are not discovered by
>>  >> the normal runner, so they never run by themselves.  But otherwise,
>>  >> yes, MyTest1 does not run as part of MyTestSuite and MyTest2 does run
>>  >> as part of MyTestSuite.
>>  >>
>>  >> Adding @RunWith(JUnit4.class) to MyTest2 makes no difference, it still
>>  >> runs.
>>  >>
>>  >>
>>  >> David Saff wrote on 01/11/10 06:32 PM:
>>  >>>
>>  >>>
>>  >>> Bill,
>>  >>>
>>  >>> I would expect what you're trying to do to work if you and I both
>>  >>> understand correctly how subclassing Suite works.
>>  >>>
>>  >>> So you're saying that if MyTest1 has RunWith(Parameterized), and
>>  >>> MyTest2 does not, then:
>>  >>>
>>  >>> MyTest1 runs by itself
>>  >>> MyTest2 runs by itself
>>  >>> MyTest1 does not run as part of MyTestSuite
>>  >>> MyTest2 does run as part of MyTestSuite
>>  >>>
>>  >>> Do I have that right? If so, what if MyTest2 is annotated with
>>  >>> @RunWith(JUnit4.class) [which _should_ be a no-op?]
>>  >>>
>>  >>> David Saff
>>  >>>
>>  >>> On Mon, Jan 11, 2010 at 4:20 PM, Bill Shannon <Bill.Shannon@...
>> <mailto:Bill.Shannon%40sun.com>
>>  >>> <mailto:Bill.Shannon%40sun.com>> wrote:
>>  >>>  > I created a Suite that allows me to run tests in a separate class
>>  >>> loader.
>>  >>>  > I use it like this:
>>  >>>  >
>>  >>>  > @RunWith(ClassLoaderSuite.class)
>>  >>>  > @SuiteClasses({ MyTest1.class, MyTest2.class })
>>  >>>  > @TestClass(ClassToTest.class)
>>  >>>  > public class MyTestSuite {
>>  >>>  > }
>>  >>>  >
>>  >>>  > If I want one of those tests to be parameterized, should I expect to
>>  >>>  > be able to do:
>>  >>>  >
>>  >>>  > @RunWith(Parameterized.class)
>>  >>>  > public class MyTest1 {
>>  >>>  > ...
>>  >>>  > }
>>  >>>  >
>>  >>>  > It doesn't seem to work.
>>  >>>  >
>>  >>>  > What would I have to do in ClassLoaderSuite to make this work?
>>  >>>  >
>>  >>>  > FYI, you can find ClassLoaderSuite.java at:
>>  >>>  >
>>  >>>
>>
http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java/\
com/sun/mail/test/ClassLoaderSuite.java?rev=231
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>>  >>>
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>
>>  >>>
>>  >>>  >
>>  >>>  >
>>  >>>  > ------------------------------------
>>  >>>  >
>>  >>>  > Yahoo! Groups Links
>>  >>>  >
>>  >>>  >
>>  >>>  >
>>  >>>  >
>>  >>>
>>  >>>
>>  >>
>>  >
>>  >
>>  >
>>  > ------------------------------------
>>  >
>>  > Yahoo! Groups Links
>>  >
>>  >
>>  >
>>  >
>>
>>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Hi
thanx for the confirmation.

No I didn't file it at junit.org.
I couldn't find the correct place to do that ... although I admit I didn't try
very hard.

Jens

--- In junit@yahoogroups.com, David Saff <david@...> wrote:
>
> Jens,
>
> Definitely a bug.  Have you already logged it at junit.org as well?
> If not, I will, and we'll get on it soon.
>
>    David Saff
>
> On Fri, Jan 15, 2010 at 4:00 AM,  <jens.schauder@...> wrote:
> >
> > Hi,
> > I am trying to use the new experimental Categories Feature. Unfortunately it
doesn't seem to be compatible with Parameterized Tests. I run the following in
eclipse, but since these are
> >
> > When defining  Tests and TestSuite as below, the execution of the suite
results in errors/failures. The exact error depends on the order in which the
tests are specified in the Suite (see below).
> >
> > So I guess my points/ questions are:
> > - The behavior of a TestSuite should not depend on the order of contained
Tests
> > - If a Runner isn't supported by a Suite Runner, this should result in an
instructive error message
> > - Is there a Category compatible version of the Parameterized runner?
> >
> > Thanx
> > Jens
> >
> >
> >
> > With the normal Test first I get this call stack on the console:
> >
> > JUnit version 4.8.1
> > Exception in thread "main" java.lang.NullPointerException
> >  at org.junit.runner.Description.createSuiteDescription(Description.java:
> > 72)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.parentDes
> > cription(Categories.java:127)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.categorie
> > s(Categories.java:121)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.hasCorrec
> > tCategoryAnnotation(Categories.java:106)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
> > (Categories.java:97)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
> > (Categories.java:100)
> >  at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:299)
> >  at org.junit.runners.ParentRunner.getFilteredChildren(ParentRunner.java:
> > 277)
> >  at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:225)
> >  at org.junit.runners.Suite.describeChild(Suite.java:123)
> >  at org.junit.runners.Suite.describeChild(Suite.java:24)
> >  at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:226)
> >  at org.junit.runner.JUnitCore.run(JUnitCore.java:156)
> >  at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
> >  at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> >  at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
> >  at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
> >  at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
> >
> >
--------------------------------------------------------------------------------\
----
> > With the Parameterized Test first, I get a Failure Trace (which also isn't
very instructive):
> >
> > JUnit version 4.8.1
> > .E
> > Time: 0,006
> > There was 1 failure:
> > 1) initializationError(de.schauderhaft.junit.categories.ParameterTokenSuite)
> > java.lang.NullPointerException
> >  at org.junit.runner.Description.createSuiteDescription(Description.java:
> > 72)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.parentDes
> > cription(Categories.java:127)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.categorie
> > s(Categories.java:121)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.hasCorrec
> > tCategoryAnnotation(Categories.java:106)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
> > (Categories.java:97)
> >  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
> > (Categories.java:100)
> >  at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:299)
> >  at org.junit.runners.ParentRunner.filter(ParentRunner.java:254)
> >  at org.junit.experimental.categories.Categories.(Categories.java:1
> > 42)
> >  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou
> > rce)
> >  at java.lang.reflect.Constructor.newInstance(Unknown Source)
> >  at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBui
> > lder.java:35)
> >  at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(Annotated
> > Builder.java:24)
> >  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilde
> > r.java:57)
> >  at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForC
> > lass(AllDefaultPossibilitiesBuilder.java:29)
> >  at org.junit.runner.Computer.getRunner(Computer.java:38)
> >  at org.junit.runner.Computer$1.runnerForClass(Computer.java:29)
> >  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilde
> > r.java:57)
> >  at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:93)
> >  at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:84)
> >  at org.junit.runners.Suite.(Suite.java:79)
> >  at org.junit.runner.Computer.getSuite(Computer.java:26)
> >  at org.junit.runner.Request.classes(Request.java:69)
> >  at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> >  at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
> >  at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
> >  at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
> > FAILURES!!!
> > Tests run: 1,  Failures: 1
> >
> >
> > Class Definitions
--------------------------------------------------------------------------------\
----
> >
> > package de.schauderhaft.junit.categories;
> > import java.util.Collection;
> > import java.util.Collections;
> > import org.junit.Assert;
> > import org.junit.Test;
> > import org.junit.runner.RunWith;
> > import org.junit.runners.Parameterized;
> > import org.junit.runners.Parameterized.Parameters;
> > @RunWith(Parameterized.class)
> > public class ParameterizedTestA {
> > public ParameterizedTestA(String a) {
> >  }
> > @Parameters
> >  public static Collection getParameters() {
> >  return Collections.singletonList(new String[] { "a" });
> >  }
> > @Test
> >  public void testSomething() {
> >  Assert.assertTrue(true);
> >  }
> >
--------------------------------------------------------------------------------\
----
> > package de.schauderhaft.junit.categories;
> > import org.junit.Assert;
> > import org.junit.Test;
> > import org.junit.experimental.categories.Category;
> > @Category(Token.class)
> > public class SomeTestB {
> > @Test
> >  public void testSomething() {
> >  Assert.assertTrue(true);
> >  }
> > }
> >
> >
--------------------------------------------------------------------------------\
----
> >
> > package de.schauderhaft.junit.categories;
> > import org.junit.experimental.categories.Categories;
> > import org.junit.experimental.categories.Categories.IncludeCategory;
> > import org.junit.runner.RunWith;
> > import org.junit.runners.Suite.SuiteClasses;
> > @RunWith(Categories.class)
> > @IncludeCategory(Token.class)
> > @SuiteClasses( { SomeTestB.class, ParameterizedTestA.class }) // switch
order of classes for slightly different behaviour
> > public class ParameterTokenSuite {
> > }
> >
> > }
> >
--------------------------------------------------------------------------------\
----
> >
> >
> >
> > --
> >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>

Ok, here's how I reproduced the problem using JavaMail.

You can find JavaMail at http://kenai.com/projects/javamail.
You'll need to check out the source code using Mercurial.
You'll find instruction here:
http://kenai.com/projects/javamail/pages/BuildInstructions

Most of the tests are in the workspace at
mail/src/test/java/javax/mail/internet.

Here's what I did to test this...

InternetAddressTest.java is a parameterized test.  Copy it to IA.java,
then change IA.java so that all instances of "InternetAddressTest"
are changed to "IA".  Edit the constructor or the data() method to
include a println so you can see when the test is actually being run.

ParameterListTestSuite.java uses my ClassLoaderSuite, and contains a
list of tests to run in a separate class loader.  Edit it and add
IA.class to the list.

Now run "mvn" in the "mail" directory.  The IA.java test won't
be run.

Let me know if that's not clear enough, or if you have trouble reproducing
it.

Thanks for your help!


David Saff wrote on 01/19/2010 07:02 PM:
>
>
> Bill,
>
> Nothing you've said so far seems impossible, so I am indeed just
> trying to replicate the bug you're facing, for debugging. Steps to
> reproduce on open-source code are a great start. Thanks,
>
> David Saff
>
> On Tue, Jan 19, 2010 at 12:50 PM, Bill Shannon <Bill.Shannon@...
> <mailto:Bill.Shannon%40sun.com>> wrote:
>  > I heard from someone else off-list that, due to the design of JUnit
>  > runners, this almost certainly isn't going to work.
>  >
>  > All of the relevant code is part of an existing open source project,
>  > and I can tell you how to reproduce the problem with that code.  Is
>  > that sufficient?
>  >
>  > Or are you looking for a completely standalone project that demonstrates
>  > the problem?
>  >
>  > I guess I'm not sure whether you're still confused about what it is I'm
>  > trying to do, or whether you're sure it *should* work and you want a
>  > reproducible test case so that you can debug it.  In the latter case,
>  > how would you like me to deliver that?  (Note that my experience with
>  > JUnit is limited.  For instance, I've only ever run it as part of a
>  > maven project.)
>  >
>  >
>  > David Saff wrote on 01/19/10 06:37 AM:
>  >>
>  >>
>  >> Bill,
>  >>
>  >> Jens recently posted a helpful bug with Suite and Parameterized. It's
>  >> possible the behavior you're seeing is related. Can you reduce your
>  >> problem down to a small amount of code you can share? Thanks,
>  >>
>  >> David Saff
>  >>
>  >> On Thu, Jan 14, 2010 at 10:17 PM, Bill Shannon <Bill.Shannon@...
> <mailto:Bill.Shannon%40sun.com>
>  >> <mailto:Bill.Shannon%40sun.com>> wrote:
>  >>  > Anyone have any ideas?
>  >>  >
>  >>  > Bill Shannon wrote on 01/12/2010 11:04 AM:
>  >>  >> The way I have it set up, MyTest1 and MyTest2 are not discovered by
>  >>  >> the normal runner, so they never run by themselves.  But otherwise,
>  >>  >> yes, MyTest1 does not run as part of MyTestSuite and MyTest2
> does run
>  >>  >> as part of MyTestSuite.
>  >>  >>
>  >>  >> Adding @RunWith(JUnit4.class) to MyTest2 makes no difference, it
> still
>  >>  >> runs.
>  >>  >>
>  >>  >>
>  >>  >> David Saff wrote on 01/11/10 06:32 PM:
>  >>  >>>
>  >>  >>>
>  >>  >>> Bill,
>  >>  >>>
>  >>  >>> I would expect what you're trying to do to work if you and I both
>  >>  >>> understand correctly how subclassing Suite works.
>  >>  >>>
>  >>  >>> So you're saying that if MyTest1 has RunWith(Parameterized), and
>  >>  >>> MyTest2 does not, then:
>  >>  >>>
>  >>  >>> MyTest1 runs by itself
>  >>  >>> MyTest2 runs by itself
>  >>  >>> MyTest1 does not run as part of MyTestSuite
>  >>  >>> MyTest2 does run as part of MyTestSuite
>  >>  >>>
>  >>  >>> Do I have that right? If so, what if MyTest2 is annotated with
>  >>  >>> @RunWith(JUnit4.class) [which _should_ be a no-op?]
>  >>  >>>
>  >>  >>> David Saff
>  >>  >>>
>  >>  >>> On Mon, Jan 11, 2010 at 4:20 PM, Bill Shannon
> <Bill.Shannon@... <mailto:Bill.Shannon%40sun.com>
>  >> <mailto:Bill.Shannon%40sun.com>
>  >>  >>> <mailto:Bill.Shannon%40sun.com>> wrote:
>  >>  >>>  > I created a Suite that allows me to run tests in a separate
> class
>  >>  >>> loader.
>  >>  >>>  > I use it like this:
>  >>  >>>  >
>  >>  >>>  > @RunWith(ClassLoaderSuite.class)
>  >>  >>>  > @SuiteClasses({ MyTest1.class, MyTest2.class })
>  >>  >>>  > @TestClass(ClassToTest.class)
>  >>  >>>  > public class MyTestSuite {
>  >>  >>>  > }
>  >>  >>>  >
>  >>  >>>  > If I want one of those tests to be parameterized, should I
> expect to
>  >>  >>>  > be able to do:
>  >>  >>>  >
>  >>  >>>  > @RunWith(Parameterized.class)
>  >>  >>>  > public class MyTest1 {
>  >>  >>>  > ...
>  >>  >>>  > }
>  >>  >>>  >
>  >>  >>>  > It doesn't seem to work.
>  >>  >>>  >
>  >>  >>>  > What would I have to do in ClassLoaderSuite to make this work?
>  >>  >>>  >
>  >>  >>>  > FYI, you can find ClassLoaderSuite.java at:
>  >>  >>>  >
>  >>  >>>
>  >>
>
http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java/\
com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>
>  >>  >>>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>>
>  >>  >>>
>  >>  >>>  >
>  >>  >>>  >
>  >>  >>>  > ------------------------------------
>  >>  >>>  >
>  >>  >>>  > Yahoo! Groups Links
>  >>  >>>  >
>  >>  >>>  >
>  >>  >>>  >
>  >>  >>>  >
>  >>  >>>
>  >>  >>>
>  >>  >>
>  >>  >
>  >>  >
>  >>  >
>  >>  > ------------------------------------
>  >>  >
>  >>  > Yahoo! Groups Links
>  >>  >
>  >>  >
>  >>  >
>  >>  >
>  >>
>  >>
>  >
>  >
>  >
>  > ------------------------------------
>  >
>  > Yahoo! Groups Links
>  >
>  >
>  >
>  >
>
>

I'm excited about the new Categories feature.  I have one question though that I
haven't been able to work out:

Must all of the specific tests be enumerated when creating a suite (e.g.
DatabaseTestSuite)?

Including every categorized test explicitly via @SuiteClasses in the suite (or
suites) seems error prone, especially on an active project where new tests are
added frequently.  It also works against DRY.

If not supported out of the box, perhaps there's an extension that does it or
it'll be included in a future version?

Regards,
Christian

The ability to automatically gather test classes from the classpath is
provided by Johannes Link's cpsuite extension.  You can then wrap a
Categories runner around that to get what you want.

I could see cpsuite's functionality as a core part of JUnit.  Submit a
feature request, and vote!

    David Saff

On Wed, Jan 20, 2010 at 12:03 AM, xianpants <cnelson@...> wrote:
> I'm excited about the new Categories feature.  I have one question though that
I haven't been able to work out:
>
> Must all of the specific tests be enumerated when creating a suite (e.g.
DatabaseTestSuite)?
>
> Including every categorized test explicitly via @SuiteClasses in the suite (or
suites) seems error prone, especially on an active project where new tests are
added frequently.  It also works against DRY.
>
> If not supported out of the box, perhaps there's an extension that does it or
it'll be included in a future version?
>
> Regards,
> Christian
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Bill,

Thanks for that.  Can you also create a bug to track it at github?
I'll try to take a deeper look soon.

    David Saff

On Tue, Jan 19, 2010 at 11:00 PM, Bill Shannon <Bill.Shannon@...> wrote:
> Ok, here's how I reproduced the problem using JavaMail.
>
> You can find JavaMail at http://kenai.com/projects/javamail.
> You'll need to check out the source code using Mercurial.
> You'll find instruction here:
> http://kenai.com/projects/javamail/pages/BuildInstructions
>
> Most of the tests are in the workspace at
> mail/src/test/java/javax/mail/internet.
>
> Here's what I did to test this...
>
> InternetAddressTest.java is a parameterized test.  Copy it to IA.java,
> then change IA.java so that all instances of "InternetAddressTest"
> are changed to "IA".  Edit the constructor or the data() method to
> include a println so you can see when the test is actually being run.
>
> ParameterListTestSuite.java uses my ClassLoaderSuite, and contains a
> list of tests to run in a separate class loader.  Edit it and add
> IA.class to the list.
>
> Now run "mvn" in the "mail" directory.  The IA.java test won't
> be run.
>
> Let me know if that's not clear enough, or if you have trouble reproducing
> it.
>
> Thanks for your help!
>
>
> David Saff wrote on 01/19/2010 07:02 PM:
>>
>>
>> Bill,
>>
>> Nothing you've said so far seems impossible, so I am indeed just
>> trying to replicate the bug you're facing, for debugging. Steps to
>> reproduce on open-source code are a great start. Thanks,
>>
>> David Saff
>>
>> On Tue, Jan 19, 2010 at 12:50 PM, Bill Shannon <Bill.Shannon@...
>> <mailto:Bill.Shannon%40sun.com>> wrote:
>>  > I heard from someone else off-list that, due to the design of JUnit
>>  > runners, this almost certainly isn't going to work.
>>  >
>>  > All of the relevant code is part of an existing open source project,
>>  > and I can tell you how to reproduce the problem with that code.  Is
>>  > that sufficient?
>>  >
>>  > Or are you looking for a completely standalone project that demonstrates
>>  > the problem?
>>  >
>>  > I guess I'm not sure whether you're still confused about what it is I'm
>>  > trying to do, or whether you're sure it *should* work and you want a
>>  > reproducible test case so that you can debug it.  In the latter case,
>>  > how would you like me to deliver that?  (Note that my experience with
>>  > JUnit is limited.  For instance, I've only ever run it as part of a
>>  > maven project.)
>>  >
>>  >
>>  > David Saff wrote on 01/19/10 06:37 AM:
>>  >>
>>  >>
>>  >> Bill,
>>  >>
>>  >> Jens recently posted a helpful bug with Suite and Parameterized. It's
>>  >> possible the behavior you're seeing is related. Can you reduce your
>>  >> problem down to a small amount of code you can share? Thanks,
>>  >>
>>  >> David Saff
>>  >>
>>  >> On Thu, Jan 14, 2010 at 10:17 PM, Bill Shannon <Bill.Shannon@...
>> <mailto:Bill.Shannon%40sun.com>
>>  >> <mailto:Bill.Shannon%40sun.com>> wrote:
>>  >>  > Anyone have any ideas?
>>  >>  >
>>  >>  > Bill Shannon wrote on 01/12/2010 11:04 AM:
>>  >>  >> The way I have it set up, MyTest1 and MyTest2 are not discovered by
>>  >>  >> the normal runner, so they never run by themselves.  But otherwise,
>>  >>  >> yes, MyTest1 does not run as part of MyTestSuite and MyTest2
>> does run
>>  >>  >> as part of MyTestSuite.
>>  >>  >>
>>  >>  >> Adding @RunWith(JUnit4.class) to MyTest2 makes no difference, it
>> still
>>  >>  >> runs.
>>  >>  >>
>>  >>  >>
>>  >>  >> David Saff wrote on 01/11/10 06:32 PM:
>>  >>  >>>
>>  >>  >>>
>>  >>  >>> Bill,
>>  >>  >>>
>>  >>  >>> I would expect what you're trying to do to work if you and I both
>>  >>  >>> understand correctly how subclassing Suite works.
>>  >>  >>>
>>  >>  >>> So you're saying that if MyTest1 has RunWith(Parameterized), and
>>  >>  >>> MyTest2 does not, then:
>>  >>  >>>
>>  >>  >>> MyTest1 runs by itself
>>  >>  >>> MyTest2 runs by itself
>>  >>  >>> MyTest1 does not run as part of MyTestSuite
>>  >>  >>> MyTest2 does run as part of MyTestSuite
>>  >>  >>>
>>  >>  >>> Do I have that right? If so, what if MyTest2 is annotated with
>>  >>  >>> @RunWith(JUnit4.class) [which _should_ be a no-op?]
>>  >>  >>>
>>  >>  >>> David Saff
>>  >>  >>>
>>  >>  >>> On Mon, Jan 11, 2010 at 4:20 PM, Bill Shannon
>> <Bill.Shannon@... <mailto:Bill.Shannon%40sun.com>
>>  >> <mailto:Bill.Shannon%40sun.com>
>>  >>  >>> <mailto:Bill.Shannon%40sun.com>> wrote:
>>  >>  >>>  > I created a Suite that allows me to run tests in a separate
>> class
>>  >>  >>> loader.
>>  >>  >>>  > I use it like this:
>>  >>  >>>  >
>>  >>  >>>  > @RunWith(ClassLoaderSuite.class)
>>  >>  >>>  > @SuiteClasses({ MyTest1.class, MyTest2.class })
>>  >>  >>>  > @TestClass(ClassToTest.class)
>>  >>  >>>  > public class MyTestSuite {
>>  >>  >>>  > }
>>  >>  >>>  >
>>  >>  >>>  > If I want one of those tests to be parameterized, should I
>> expect to
>>  >>  >>>  > be able to do:
>>  >>  >>>  >
>>  >>  >>>  > @RunWith(Parameterized.class)
>>  >>  >>>  > public class MyTest1 {
>>  >>  >>>  > ...
>>  >>  >>>  > }
>>  >>  >>>  >
>>  >>  >>>  > It doesn't seem to work.
>>  >>  >>>  >
>>  >>  >>>  > What would I have to do in ClassLoaderSuite to make this work?
>>  >>  >>>  >
>>  >>  >>>  > FYI, you can find ClassLoaderSuite.java at:
>>  >>  >>>  >
>>  >>  >>>
>>  >>
>>
http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java/\
com/sun/mail/test/ClassLoaderSuite.java?rev=231
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>>  >>
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>
>>  >>  >>>
>>  >>
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>>  >>
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>>
>>  >>  >>>
>>  >>  >>>  >
>>  >>  >>>  >
>>  >>  >>>  > ------------------------------------
>>  >>  >>>  >
>>  >>  >>>  > Yahoo! Groups Links
>>  >>  >>>  >
>>  >>  >>>  >
>>  >>  >>>  >
>>  >>  >>>  >
>>  >>  >>>
>>  >>  >>>
>>  >>  >>
>>  >>  >
>>  >>  >
>>  >>  >
>>  >>  > ------------------------------------
>>  >>  >
>>  >>  > Yahoo! Groups Links
>>  >>  >
>>  >>  >
>>  >>  >
>>  >>  >
>>  >>
>>  >>
>>  >
>>  >
>>  >
>>  > ------------------------------------
>>  >
>>  > Yahoo! Groups Links
>>  >
>>  >
>>  >
>>  >
>>
>>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Sorry--Issue Tracker, on the left side.  Or go straight here:

http://github.com/KentBeck/junit/issues#sort=votes

    David Saff

On Wed, Jan 20, 2010 at 3:06 AM, jens_schauder <jens.schauder@...> wrote:
> Hi
> thanx for the confirmation.
>
> No I didn't file it at junit.org.
> I couldn't find the correct place to do that ... although I admit I didn't try
very hard.
>
> Jens
>
> --- In junit@yahoogroups.com, David Saff <david@...> wrote:
>>
>> Jens,
>>
>> Definitely a bug.  Have you already logged it at junit.org as well?
>> If not, I will, and we'll get on it soon.
>>
>>    David Saff
>>
>> On Fri, Jan 15, 2010 at 4:00 AM,  <jens.schauder@...> wrote:
>> >
>> > Hi,
>> > I am trying to use the new experimental Categories Feature. Unfortunately
it doesn't seem to be compatible with Parameterized Tests. I run the following
in eclipse, but since these are
>> >
>> > When defining  Tests and TestSuite as below, the execution of the suite
results in errors/failures. The exact error depends on the order in which the
tests are specified in the Suite (see below).
>> >
>> > So I guess my points/ questions are:
>> > - The behavior of a TestSuite should not depend on the order of contained
Tests
>> > - If a Runner isn't supported by a Suite Runner, this should result in an
instructive error message
>> > - Is there a Category compatible version of the Parameterized runner?
>> >
>> > Thanx
>> > Jens
>> >
>> >
>> >
>> > With the normal Test first I get this call stack on the console:
>> >
>> > JUnit version 4.8.1
>> > Exception in thread "main" java.lang.NullPointerException
>> >  at org.junit.runner.Description.createSuiteDescription(Description.java:
>> > 72)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.parentDes
>> > cription(Categories.java:127)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.categorie
>> > s(Categories.java:121)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.hasCorrec
>> > tCategoryAnnotation(Categories.java:106)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
>> > (Categories.java:97)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
>> > (Categories.java:100)
>> >  at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:299)
>> >  at org.junit.runners.ParentRunner.getFilteredChildren(ParentRunner.java:
>> > 277)
>> >  at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:225)
>> >  at org.junit.runners.Suite.describeChild(Suite.java:123)
>> >  at org.junit.runners.Suite.describeChild(Suite.java:24)
>> >  at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:226)
>> >  at org.junit.runner.JUnitCore.run(JUnitCore.java:156)
>> >  at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>> >  at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> >  at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
>> >  at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
>> >  at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
>> >
>> >
--------------------------------------------------------------------------------\
----
>> > With the Parameterized Test first, I get a Failure Trace (which also isn't
very instructive):
>> >
>> > JUnit version 4.8.1
>> > .E
>> > Time: 0,006
>> > There was 1 failure:
>> > 1)
initializationError(de.schauderhaft.junit.categories.ParameterTokenSuite)
>> > java.lang.NullPointerException
>> >  at org.junit.runner.Description.createSuiteDescription(Description.java:
>> > 72)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.parentDes
>> > cription(Categories.java:127)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.categorie
>> > s(Categories.java:121)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.hasCorrec
>> > tCategoryAnnotation(Categories.java:106)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
>> > (Categories.java:97)
>> >  at org.junit.experimental.categories.Categories$CategoryFilter.shouldRun
>> > (Categories.java:100)
>> >  at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:299)
>> >  at org.junit.runners.ParentRunner.filter(ParentRunner.java:254)
>> >  at org.junit.experimental.categories.Categories.(Categories.java:1
>> > 42)
>> >  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
>> > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou
>> > rce)
>> >  at java.lang.reflect.Constructor.newInstance(Unknown Source)
>> >  at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBui
>> > lder.java:35)
>> >  at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(Annotated
>> > Builder.java:24)
>> >  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilde
>> > r.java:57)
>> >  at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForC
>> > lass(AllDefaultPossibilitiesBuilder.java:29)
>> >  at org.junit.runner.Computer.getRunner(Computer.java:38)
>> >  at org.junit.runner.Computer$1.runnerForClass(Computer.java:29)
>> >  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilde
>> > r.java:57)
>> >  at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:93)
>> >  at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:84)
>> >  at org.junit.runners.Suite.(Suite.java:79)
>> >  at org.junit.runner.Computer.getSuite(Computer.java:26)
>> >  at org.junit.runner.Request.classes(Request.java:69)
>> >  at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> >  at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
>> >  at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
>> >  at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
>> > FAILURES!!!
>> > Tests run: 1,  Failures: 1
>> >
>> >
>> > Class Definitions
--------------------------------------------------------------------------------\
----
>> >
>> > package de.schauderhaft.junit.categories;
>> > import java.util.Collection;
>> > import java.util.Collections;
>> > import org.junit.Assert;
>> > import org.junit.Test;
>> > import org.junit.runner.RunWith;
>> > import org.junit.runners.Parameterized;
>> > import org.junit.runners.Parameterized.Parameters;
>> > @RunWith(Parameterized.class)
>> > public class ParameterizedTestA {
>> > public ParameterizedTestA(String a) {
>> >  }
>> > @Parameters
>> >  public static Collection getParameters() {
>> >  return Collections.singletonList(new String[] { "a" });
>> >  }
>> > @Test
>> >  public void testSomething() {
>> >  Assert.assertTrue(true);
>> >  }
>> >
--------------------------------------------------------------------------------\
----
>> > package de.schauderhaft.junit.categories;
>> > import org.junit.Assert;
>> > import org.junit.Test;
>> > import org.junit.experimental.categories.Category;
>> > @Category(Token.class)
>> > public class SomeTestB {
>> > @Test
>> >  public void testSomething() {
>> >  Assert.assertTrue(true);
>> >  }
>> > }
>> >
>> >
--------------------------------------------------------------------------------\
----
>> >
>> > package de.schauderhaft.junit.categories;
>> > import org.junit.experimental.categories.Categories;
>> > import org.junit.experimental.categories.Categories.IncludeCategory;
>> > import org.junit.runner.RunWith;
>> > import org.junit.runners.Suite.SuiteClasses;
>> > @RunWith(Categories.class)
>> > @IncludeCategory(Token.class)
>> > @SuiteClasses( { SomeTestB.class, ParameterizedTestA.class }) // switch
order of classes for slightly different behaviour
>> > public class ParameterTokenSuite {
>> > }
>> >
>> > }
>> >
--------------------------------------------------------------------------------\
----
>> >
>> >
>> >
>> > --
>> >
>> >
>> > [Non-text portions of this message have been removed]
>> >
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

On Tue, Jan 19, 2010 at 6:07 PM, yme0987654321 <yme0987654321@...> wrote:
> Hi,
>
> I have submitted this issue in the past
(http://github.com/KentBeck/junit/issues#issue/41) but now it has become more
relevant (and perhaps a more definitive solution presents itself):
>
> When I went to implement categories in my project (instead of some rather ugly
global system variables and a bunch of assumeTrue peppered hither and yon), I
found I couldn't use it because it doesn't expose the Suite constructor that
allows you to pass an array of classes (which is what I was using to create
dynamic suites until now).
>
> I think this points to a @SuiteMethod annotation (or just make the current
@SuiteClasses annotation able to annotate a method, and if it does require the
method to be public, static, take no parameters and return a Class array), so
that it combines well with Categories and any other future subclass of Suite
that wants to collect the classes from the annotation.
>
> By the way, this and one other issue (better naming of parameterized tests
Issues 24, 44, 46 and I though one more where I put in my two cents) makes me
want to get involved in GitHub to suggest some implementations. What is the best
getting started guide for a Windows user out there?

I assume you mean a getting-started guide for github in particular?  I
found the on-site help sufficient, but I think Kent found that things
didn't Just Work on Windows.  Kent?

    David Saff

>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Done.  http://github.com/KentBeck/junit/issues/issue/73

David Saff wrote on 01/20/2010 06:02 AM:
>
>
> Bill,
>
> Thanks for that. Can you also create a bug to track it at github?
> I'll try to take a deeper look soon.
>
> David Saff
>
> On Tue, Jan 19, 2010 at 11:00 PM, Bill Shannon <Bill.Shannon@...
> <mailto:Bill.Shannon%40sun.com>> wrote:
>  > Ok, here's how I reproduced the problem using JavaMail.
>  >
>  > You can find JavaMail at http://kenai.com/projects/javamail.
> <http://kenai.com/projects/javamail.>
>  > You'll need to check out the source code using Mercurial.
>  > You'll find instruction here:
>  > http://kenai.com/projects/javamail/pages/BuildInstructions
> <http://kenai.com/projects/javamail/pages/BuildInstructions>
>  >
>  > Most of the tests are in the workspace at
>  > mail/src/test/java/javax/mail/internet.
>  >
>  > Here's what I did to test this...
>  >
>  > InternetAddressTest.java is a parameterized test.  Copy it to IA.java,
>  > then change IA.java so that all instances of "InternetAddressTest"
>  > are changed to "IA".  Edit the constructor or the data() method to
>  > include a println so you can see when the test is actually being run.
>  >
>  > ParameterListTestSuite.java uses my ClassLoaderSuite, and contains a
>  > list of tests to run in a separate class loader.  Edit it and add
>  > IA.class to the list.
>  >
>  > Now run "mvn" in the "mail" directory.  The IA.java test won't
>  > be run.
>  >
>  > Let me know if that's not clear enough, or if you have trouble
> reproducing
>  > it.
>  >
>  > Thanks for your help!
>  >
>  >
>  > David Saff wrote on 01/19/2010 07:02 PM:
>  >>
>  >>
>  >> Bill,
>  >>
>  >> Nothing you've said so far seems impossible, so I am indeed just
>  >> trying to replicate the bug you're facing, for debugging. Steps to
>  >> reproduce on open-source code are a great start. Thanks,
>  >>
>  >> David Saff
>  >>
>  >> On Tue, Jan 19, 2010 at 12:50 PM, Bill Shannon <Bill.Shannon@...
> <mailto:Bill.Shannon%40sun.com>
>  >> <mailto:Bill.Shannon%40sun.com>> wrote:
>  >>  > I heard from someone else off-list that, due to the design of JUnit
>  >>  > runners, this almost certainly isn't going to work.
>  >>  >
>  >>  > All of the relevant code is part of an existing open source project,
>  >>  > and I can tell you how to reproduce the problem with that code.  Is
>  >>  > that sufficient?
>  >>  >
>  >>  > Or are you looking for a completely standalone project that
> demonstrates
>  >>  > the problem?
>  >>  >
>  >>  > I guess I'm not sure whether you're still confused about what it
> is I'm
>  >>  > trying to do, or whether you're sure it *should* work and you want a
>  >>  > reproducible test case so that you can debug it.  In the latter case,
>  >>  > how would you like me to deliver that?  (Note that my experience with
>  >>  > JUnit is limited.  For instance, I've only ever run it as part of a
>  >>  > maven project.)
>  >>  >
>  >>  >
>  >>  > David Saff wrote on 01/19/10 06:37 AM:
>  >>  >>
>  >>  >>
>  >>  >> Bill,
>  >>  >>
>  >>  >> Jens recently posted a helpful bug with Suite and Parameterized.
> It's
>  >>  >> possible the behavior you're seeing is related. Can you reduce your
>  >>  >> problem down to a small amount of code you can share? Thanks,
>  >>  >>
>  >>  >> David Saff
>  >>  >>
>  >>  >> On Thu, Jan 14, 2010 at 10:17 PM, Bill Shannon
> <Bill.Shannon@... <mailto:Bill.Shannon%40sun.com>
>  >> <mailto:Bill.Shannon%40sun.com>
>  >>  >> <mailto:Bill.Shannon%40sun.com>> wrote:
>  >>  >>  > Anyone have any ideas?
>  >>  >>  >
>  >>  >>  > Bill Shannon wrote on 01/12/2010 11:04 AM:
>  >>  >>  >> The way I have it set up, MyTest1 and MyTest2 are not
> discovered by
>  >>  >>  >> the normal runner, so they never run by themselves.  But
> otherwise,
>  >>  >>  >> yes, MyTest1 does not run as part of MyTestSuite and MyTest2
>  >> does run
>  >>  >>  >> as part of MyTestSuite.
>  >>  >>  >>
>  >>  >>  >> Adding @RunWith(JUnit4.class) to MyTest2 makes no difference, it
>  >> still
>  >>  >>  >> runs.
>  >>  >>  >>
>  >>  >>  >>
>  >>  >>  >> David Saff wrote on 01/11/10 06:32 PM:
>  >>  >>  >>>
>  >>  >>  >>>
>  >>  >>  >>> Bill,
>  >>  >>  >>>
>  >>  >>  >>> I would expect what you're trying to do to work if you and
> I both
>  >>  >>  >>> understand correctly how subclassing Suite works.
>  >>  >>  >>>
>  >>  >>  >>> So you're saying that if MyTest1 has
> RunWith(Parameterized), and
>  >>  >>  >>> MyTest2 does not, then:
>  >>  >>  >>>
>  >>  >>  >>> MyTest1 runs by itself
>  >>  >>  >>> MyTest2 runs by itself
>  >>  >>  >>> MyTest1 does not run as part of MyTestSuite
>  >>  >>  >>> MyTest2 does run as part of MyTestSuite
>  >>  >>  >>>
>  >>  >>  >>> Do I have that right? If so, what if MyTest2 is annotated with
>  >>  >>  >>> @RunWith(JUnit4.class) [which _should_ be a no-op?]
>  >>  >>  >>>
>  >>  >>  >>> David Saff
>  >>  >>  >>>
>  >>  >>  >>> On Mon, Jan 11, 2010 at 4:20 PM, Bill Shannon
>  >> <Bill.Shannon@... <mailto:Bill.Shannon%40sun.com>
> <mailto:Bill.Shannon%40sun.com>
>  >>  >> <mailto:Bill.Shannon%40sun.com>
>  >>  >>  >>> <mailto:Bill.Shannon%40sun.com>> wrote:
>  >>  >>  >>>  > I created a Suite that allows me to run tests in a separate
>  >> class
>  >>  >>  >>> loader.
>  >>  >>  >>>  > I use it like this:
>  >>  >>  >>>  >
>  >>  >>  >>>  > @RunWith(ClassLoaderSuite.class)
>  >>  >>  >>>  > @SuiteClasses({ MyTest1.class, MyTest2.class })
>  >>  >>  >>>  > @TestClass(ClassToTest.class)
>  >>  >>  >>>  > public class MyTestSuite {
>  >>  >>  >>>  > }
>  >>  >>  >>>  >
>  >>  >>  >>>  > If I want one of those tests to be parameterized, should I
>  >> expect to
>  >>  >>  >>>  > be able to do:
>  >>  >>  >>>  >
>  >>  >>  >>>  > @RunWith(Parameterized.class)
>  >>  >>  >>>  > public class MyTest1 {
>  >>  >>  >>>  > ...
>  >>  >>  >>>  > }
>  >>  >>  >>>  >
>  >>  >>  >>>  > It doesn't seem to work.
>  >>  >>  >>>  >
>  >>  >>  >>>  > What would I have to do in ClassLoaderSuite to make this
> work?
>  >>  >>  >>>  >
>  >>  >>  >>>  > FYI, you can find ClassLoaderSuite.java at:
>  >>  >>  >>>  >
>  >>  >>  >>>
>  >>  >>
>  >>
>
http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java/\
com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>
>  >>  >>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>>
>  >>  >>  >>>
>  >>  >>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>
>  >>  >>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>
>  >>
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231
>
<http://kenai.com/projects/javamail/sources/mercurial/content/mail/src/test/java\
/com/sun/mail/test/ClassLoaderSuite.java?rev=231>>>>
>  >>  >>  >>>
>  >>  >>  >>>  >
>  >>  >>  >>>  >
>  >>  >>  >>>  > ------------------------------------
>  >>  >>  >>>  >
>  >>  >>  >>>  > Yahoo! Groups Links
>  >>  >>  >>>  >
>  >>  >>  >>>  >
>  >>  >>  >>>  >
>  >>  >>  >>>  >
>  >>  >>  >>>
>  >>  >>  >>>
>  >>  >>  >>
>  >>  >>  >
>  >>  >>  >
>  >>  >>  >
>  >>  >>  > ------------------------------------
>  >>  >>  >
>  >>  >>  > Yahoo! Groups Links
>  >>  >>  >
>  >>  >>  >
>  >>  >>  >
>  >>  >>  >
>  >>  >>
>  >>  >>
>  >>  >
>  >>  >
>  >>  >
>  >>  > ------------------------------------
>  >>  >
>  >>  > Yahoo! Groups Links
>  >>  >
>  >>  >
>  >>  >
>  >>  >
>  >>
>  >>
>  >
>  >
>  >
>  > ------------------------------------
>  >
>  > Yahoo! Groups Links
>  >
>  >
>  >
>  >
>
>

Besides the usual Git teething pains, what I had the most trouble with was
PuTTY.  I know what to do now, but it wasn't obvious. Unfortunately, I don't
understand it well enough to explain it.

Cheers,

Kent

On Jan 20, 2010, at 6:48 AM, David Saff wrote:

> On Tue, Jan 19, 2010 at 6:07 PM, yme0987654321 <yme0987654321@...>
wrote:
>> Hi,
>>
>> I have submitted this issue in the past
(http://github.com/KentBeck/junit/issues#issue/41) but now it has become more
relevant (and perhaps a more definitive solution presents itself):
>>
>> When I went to implement categories in my project (instead of some rather
ugly global system variables and a bunch of assumeTrue peppered hither and yon),
I found I couldn't use it because it doesn't expose the Suite constructor that
allows you to pass an array of classes (which is what I was using to create
dynamic suites until now).
>>
>> I think this points to a @SuiteMethod annotation (or just make the current
@SuiteClasses annotation able to annotate a method, and if it does require the
method to be public, static, take no parameters and return a Class array), so
that it combines well with Categories and any other future subclass of Suite
that wants to collect the classes from the annotation.
>>
>> By the way, this and one other issue (better naming of parameterized tests
Issues 24, 44, 46 and I though one more where I put in my two cents) makes me
want to get involved in GitHub to suggest some implementations. What is the best
getting started guide for a Windows user out there?
>
> I assume you mean a getting-started guide for github in particular?  I
> found the on-site help sufficient, but I think Kent found that things
> didn't Just Work on Windows.  Kent?
>
>   David Saff

I've discovered that my ClassLoaderSuite that loads each test class in its
own ClassLoader also needs to set the thread's context class loader before
running the test, and restore it after running the test.  If I override the
runChild method, how do I find the test class (so I can find its ClassLoader)?
The javadocs say that runChild runs "the test corresponding to child", but
"child" isn't a parameter.  I assume it means the test corresponding to
"runner", but I don't see how to get from "runner" to the test class.

Do I need to implement my own Runner, and thus my own RunnerBuilder?
I'm not sure how to connect all these pieces together.

I suppose I could create a delegating RunnerBuilder to wrap the one
I'm given, and have it replace the Runners with delegating Runners
that can also remember the class that the Runner was created for,
but is there a better way?

If I assume the runner is always going to be a ParentRunner, I can use
parent.getTestClass().getJavaClass() to get what I want.  But how safe
is it to assume that the runner will always be a ParentRunner?