As per the other replies I would suggest that you want to "mirror" your
source directories with test directories. That is, don't mix production
code and test code in the same directories. Just let your production
code package structure induce a test code directory structure. Now you
can use your IDE to launch tests at the approporate level by choosing
the appropriate directory.
If you find you need greater control than that, I've found a utility
called ClasspathSuite
<http://www.johanneslink.net/projects/cpsuite.html> handy for running
different sets of tests in Eclipse (especially if you want to run tests
across more than one Eclipse project). I can choose which tests to run
based on test class naming conventions.
Below is the body of the class that I use to run my project's unit tests
but not my project's integration tests:
@RunWith(ClasspathSuite.class)
@SuiteTypes({ JUNIT38_TEST_CLASSES, TEST_CLASSES })
@ClassnameFilters({".*(Behaviou?r|Spec|Test)", "!.*IntegrationTest"})
public class AllUnitTests {}
Next I make a launch target in Eclipse that runs AllUnitTests as a JUnit
Test.
As you might guess from the ClassnameFilters above, a project history
that includes a mixture of folks from Canada and from the States
experimenting with BDD, requires me to use the regular expression
`Behaviou?r` to match a few of the project's unit tests. Sigh.
Good luck,
Alistair
--- In junit@yahoogroups.com, "David Parks" <davidparks21@...> wrote:
>
> Hi there, I'm new at JUnit, and just want to start using it with a new
project.
>
> I'm trying to figure out the best way to easily expand on a set of
test classes.
>
> It seems like I should configure a TestSuite of some sort. But I'm not
sure if I should use the DirectoryBuilder, or if I should manually
maintain a class that launches my tests, or use annotations.
>
> It looks like there is a way to simply annotate the classes I want to
run and then to be able to kick off various test suites. But the docs
don't have many examples of this. And it seems like there are many ways
to do the same thing.
>
> The cookbook just mentions using
JUnitCore.runClasses(TestClass1.class, ...), but that doesn't seem very
extendable.
>
> I'm just looking for a simple best practice. I'd like to not have to
update a main() method each time I want to add a new test class. I'd
like to simply kick off one or another suite of test classes from my IDE
in most cases and get a nicely printed result printed to the console.
>
> Thanks!
> David
>
[Non-text portions of this message have been removed]
I don't really know enough about your exact situation to be sure, but
you might find this tool to be useful: ClasspathSuite
<http://www.johanneslink.net/projects/cpsuite.html> . I'm finding the
tool to be quite handy on a current project of mine.
Alistair
--- In junit@yahoogroups.com, Jean-Philippe Daigle <jpdaigle@...> wrote:
>
> Hello all,
>
> After years of joy with JUnit 3, we're starting to look at JUnit 4 to
> take advantage of Filters. We're looking to incrementally start
> tagging our hundreds of test methods with attributes to control
> whether they're run or not, and provide a Filter implementation to do
> the selection logic.
>
> I haven't found much documentation on using the
> org.junit.runner.manipulation.Filter class; it seems to be very much
> something applied to a Runner or Request instance, not a property of
> the test suite itself.
>
> Since we have a large investment in organizing tests using a number of
> classes exposing a suite() method (JUnit3), it seems I want to use the
> JUnit4TestAdapter class, but once instantiated from within a suite()
> method, calling setFilter() on it has no effect. It seems that I need
> to convince the runner itself (which could be Eclipse, Ant, etc...) to
> call filter(), so unless there's something I'm misunderstanding, I
> can't go down this path.
>
> A second approach that seems like it would work would be to use an
> @RunsWith annotation to set the org.junit.runners.Suite runner to run
> a suite crafted to replicate the organization we're doing today with
> suite() methods. I've been able to apply a Filter from within my own
> Runner implementation (subclassing BlockJUnit4ClassRunner), but I
> can't see a way to apply a Filter if I'm using the Suite runner.
>
> Any ideas?
>
> Jean-Philippe
> Ottawa, Canada
>
[Non-text portions of this message have been removed]
Hi – I wanted to automatically sweep tests into suites as part of my
continuous builds, so I derived a runner from Suite which finds all test
classes in a package. The runner works just fine, but the results
display is less than expected.
I have one class in my testing support package with a @RunWith
annotation for my runner. Set a property to the package under test and
tell JUnit to run the annotated class, and all tests in that package are
executed. The name of the suite is reported as the name of the class
which has the @RunWith annotation, in both Ant and IntelliJ. My runner
has an override for ParentRunner.getName() which returns the name of the
package under test. I verified that the string gets into the runner's
Description object. What am I missing?
Environment:
* JUnit: 4.5
* Ant: 1.7.0
* IntelliJ IDEA: 8.1
Thanks for whatever direction you can provide.
[Non-text portions of this message have been removed]
It may be that ant is ignoring getDescription for the parent class.
Have you tried Johannes Link's cpsuite?
David Saff
On Wed, Apr 1, 2009 at 2:47 PM, stujpa <stujpa@...> wrote:
> Hi – I wanted to automatically sweep tests into suites as part of my
> continuous builds, so I derived a runner from Suite which finds all test
> classes in a package. The runner works just fine, but the results
> display is less than expected.
> I have one class in my testing support package with a @RunWith
> annotation for my runner. Set a property to the package under test and
> tell JUnit to run the annotated class, and all tests in that package are
> executed. The name of the suite is reported as the name of the class
> which has the @RunWith annotation, in both Ant and IntelliJ. My runner
> has an override for ParentRunner.getName() which returns the name of the
> package under test. I verified that the string gets into the runner's
> Description object. What am I missing?
> Environment:
> * JUnit: 4.5
> * Ant: 1.7.0
> * IntelliJ IDEA: 8.1
>
> Thanks for whatever direction you can provide.
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
On Tue, Mar 31, 2009 at 12:36 PM, jduprez4junit <jethrie@...> wrote:
> I just happened to stumble on a fairly big heap of tests which are organized
in a class hierarchy (OrderDaoTestCase DaoTestCase extends BaseTestCase extends
TestCase). jUnit4 annotations are disregarded in OrderDaoTestCase, as it
indirectly extends TestCase.
>
> If I want to benefit from jUnit4 annotations in OrderDaoTestCase, I can
either:
> - break this hierarchy before extending BaseTestCase: this would be heavy work
(lots of utilities in BaseTestcase, could have been written as static utilities
or helper classes, but the legacy situation is that most of these useful
utilities are based on instance methods or protected fields).
> - simply making BaseTestCase not extend TestCase: that would break those test
subclasses that do rely on being indirect TestCase subclasses (ex: non-static
assert methods).
If you annotate OrderDaoTestCase with @RunWith(JUnit4.class), does
everything work?
David Saff
>> 2) Where did you look in the documentation? Where would be a natural place
to see this behavior described?
>
> OK this is a trick question because I mentioned "the documentation" :o)
> Your question made me search for the docs I had read, and I realized that most
of them are not from the jUnit team (or on the jUnit website).
> As far as the standard doc is concerned, well, this issue is probably too
far-fetched to deserve an entry in the short cookbook (which no more describes
3.8-style anyway).
> I expected I would find it in the FAQ (must not be as frequent as I thought).
>
> This is probably merely a subject for a tutorial on migrating from 3.8, which
may seem obsolete as of today, but I'm sure there still exists lots of 3.8 test
code in the field.
As you've gathered, we've been relying on Google and the far corners
of the Internet to do some of our documentation for us, as we
concentrate our limited time on functionality. This is missing.
Perhaps you could log a feature request at SourceForge to update the
FAQ? Thanks,
David Saff
Hi Junit,
I set up a Facebook profile where I can post my pictures, videos and events and
I want to add you as a friend so you can see it. First, you need to join
Facebook! Once you join, you can also create your own profile.
Thanks,
John
To sign up for Facebook, follow the link below:
http://www.facebook.com/p.php?i=588889670&k=Z6AUQ5WRP25CUGBAS144PQ&r
junit@yahoogroups.com was invited to join Facebook by John Van V. If you do not
wish to receive this type of email from Facebook in the future, please click on
the link below to unsubscribe.
http://www.facebook.com/o.php?k=7ebb4d&u=561222115&mid=41bf20G217391e3G0G8
Facebook's offices are located at 156 University Ave., Palo Alto, CA 94301.
[Non-text portions of this message have been removed]
I tried the steps required for Installation
Below are the installation steps for installing JUnit:
1.unzip the junit4.6.zip file
2.add junit-4.6.jar to the CLASSPATH.
For example: set classpath=%classpath%;INSTALL_DIR\junit-4.6.jar;INSTALL_DIR
test the installation by running java org.junit.runner.JUnitCore
org.junit.tests.AllTests
I did everythg that was asked but i cudnt find org.junit.runner file? and if I
using java org.junit.tests.AllTests ,it throws an exception that main class is
not defined!!
Please help!!
Sorry to all--that was supposed to be to author only.
David Saff
On Mon, Apr 6, 2009 at 10:02 AM, David Saff <david@...> wrote:
> John,
>
> Was this a mistake?
>
> David Saff
>
> On Sat, Apr 4, 2009 at 4:24 PM, John Van V. <john.van.v@...> wrote:
>> Hi Junit,
>>
>> I set up a Facebook profile where I can post my pictures, videos and events
and I want to add you as a friend so you can see it. First, you need to join
Facebook! Once you join, you can also create your own profile.
>>
>> Thanks,
>> John
>>
>> To sign up for Facebook, follow the link below:
>> http://www.facebook.com/p.php?i=588889670&k=Z6AUQ5WRP25CUGBAS144PQ&r
>>
>>
>> junit@yahoogroups.com was invited to join Facebook by John Van V. If you do
not wish to receive this type of email from Facebook in the future, please click
on the link below to unsubscribe.
>> http://www.facebook.com/o.php?k=7ebb4d&u=561222115&mid=41bf20G217391e3G0G8
>> Facebook's offices are located at 156 University Ave., Palo Alto, CA 94301.
>>
>>
>>
>> [Non-text portions of this message have been removed]
>>
>>
>>
>> ------------------------------------
>>
>> Yahoo! Groups Links
>>
>>
>>
>>
>
John,
Was this a mistake?
David Saff
On Sat, Apr 4, 2009 at 4:24 PM, John Van V. <john.van.v@...> wrote:
> Hi Junit,
>
> I set up a Facebook profile where I can post my pictures, videos and events
and I want to add you as a friend so you can see it. First, you need to join
Facebook! Once you join, you can also create your own profile.
>
> Thanks,
> John
>
> To sign up for Facebook, follow the link below:
> http://www.facebook.com/p.php?i=588889670&k=Z6AUQ5WRP25CUGBAS144PQ&r
>
>
> junit@yahoogroups.com was invited to join Facebook by John Van V. If you do
not wish to receive this type of email from Facebook in the future, please click
on the link below to unsubscribe.
> http://www.facebook.com/o.php?k=7ebb4d&u=561222115&mid=41bf20G217391e3G0G8
> Facebook's offices are located at 156 University Ave., Palo Alto, CA 94301.
>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
Sadia,
Where did you get a 4.6 zip? 4.5 is the most recent official release. Thanks,
David Saff
On Thu, Apr 2, 2009 at 12:49 AM, sadia_in2006 <sadia_in2006@...> wrote:
> I tried the steps required for Installation
> Below are the installation steps for installing JUnit:
> 1.unzip the junit4.6.zip file
> 2.add junit-4.6.jar to the CLASSPATH.
>
> For example: set classpath=%classpath%;INSTALL_DIR\junit-4.6.jar;INSTALL_DIR
> test the installation by running java org.junit.runner.JUnitCore
org.junit.tests.AllTests
>
> I did everythg that was asked but i cudnt find org.junit.runner file? and if I
using java org.junit.tests.AllTests ,it throws an exception that main class is
not defined!!
>
>
> Please help!!
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
Ya It is Junit4.5 I could get it installed and tested on eclipse but not on
command prompt.It always says classdefnotfound error..even though I set the
classpath to directory containing the junit.
--- On Mon, 6/4/09, David Saff <david@...> wrote:
From: David Saff <david@...>
Subject: Re: [junit] difficulty in Installation of Junit4.5
To: junit@yahoogroups.com
Date: Monday, 6 April, 2009, 7:36 PM
Sadia,
Where did you get a 4.6 zip? 4.5 is the most recent official release. Thanks,
David Saff
On Thu, Apr 2, 2009 at 12:49 AM, sadia_in2006 <sadia_in2006@ yahoo.co. in>
wrote:
> I tried the steps required for Installation
> Below are the installation steps for installing JUnit:
> 1.unzip the junit4.6.zip file
> 2.add junit-4.6.jar to the CLASSPATH.
>
> For example: set classpath=%classpat h%;INSTALL_ DIR\junit- 4.6.jar;INSTALL_
DIR
> test the installation by running java org.junit.runner. JUnitCore
org.junit.tests. AllTests
>
> I did everythg that was asked but i cudnt find org.junit.runner file? and if I
using java org.junit.tests. AllTests ,it throws an exception that main class is
not defined!!
>
>
> Please help!!
>
>
>
> ------------ --------- --------- ------
>
> Yahoo! Groups Links
>
>
>
>
Add more friends to your messenger and enjoy! Go to
http://messenger.yahoo.com/invite/
[Non-text portions of this message have been removed]
Hi,
I'm using the latest version of JUnit with Java 1.5, WebLogic 9.2.2. I have a
couple of methods in my TestCase class ...
public void testSuccessfulDefaultRedirect() {
...
}
public void testSuccessfulDefaultRedirectNoLogger() {
...
}
within each, I'm invoking a class that has a static block, i.e.
static {
final RoutingEnginePropertiesMgr rePropMgr =
RoutingEnginePropertiesMgr.getInstance();
if (rePropMgr != null) {
final String loggerName = rePropMgr.getLogger();
if (loggerName != null) {
logger = Logger.getLogger(loggerName);
logger.info("AccountsAction logger initialized.");
} // if
} // if
}
Is there anything I can do in the second call to force the class to re-execute
the static block again?
Thanks, - Dave
The usage of static blocks leads to software that is very untestable…
I would argue that a static block is much like calling a static method in a
constructor. This anti-pattern is talked about here…
http://misko.hevery.com/code-reviewers-guide/flaw-constructor-does-real-work/
I used to love placing things into static blocks. But after thinking a while,
and reviewing the above article, I’ve changed my mind.
Big Mike
From: junit@yahoogroups.com [mailto:junit@yahoogroups.com] On Behalf Of
dave.alvarado@...
Sent: Monday, April 06, 2009 10:26 AM
To: junit@yahoogroups.com;
Subject: [junit] Possible to re-execute static blocks again from one method to
the next?
Hi,
I'm using the latest version of JUnit with Java 1.5, WebLogic 9.2.2. I have a
couple of methods in my TestCase class ...
public void testSuccessfulDefaultRedirect() {
...
}
public void testSuccessfulDefaultRedirectNoLogger() {
...
}
within each, I'm invoking a class that has a static block, i.e.
static {
final RoutingEnginePropertiesMgr rePropMgr =
RoutingEnginePropertiesMgr.getInstance();
if (rePropMgr != null) {
final String loggerName = rePropMgr.getLogger();
if (loggerName != null) {
logger = Logger.getLogger(loggerName);
logger.info("AccountsAction logger initialized.");
} // if
} // if
}
Is there anything I can do in the second call to force the class to re-execute
the static block again?
Thanks, - Dave
[Non-text portions of this message have been removed]
Can you recreate your exact steps? Do you have the string INSTALL_DIR
in your path?
David Saff
On Mon, Apr 6, 2009 at 11:18 AM, sadia tahseen <sadia_in2006@...> wrote:
> Ya It is Junit4.5 I could get it installed and tested on eclipse but not on
command prompt.It always says classdefnotfound error..even though I set the
classpath to directory containing the junit.
>
> --- On Mon, 6/4/09, David Saff <david@...> wrote:
>
>
> From: David Saff <david@...>
> Subject: Re: [junit] difficulty in Installation of Junit4.5
> To: junit@yahoogroups.com
> Date: Monday, 6 April, 2009, 7:36 PM
>
>
>
>
>
>
> Sadia,
>
> Where did you get a 4.6 zip? 4.5 is the most recent official release. Thanks,
>
> David Saff
>
> On Thu, Apr 2, 2009 at 12:49 AM, sadia_in2006 <sadia_in2006@ yahoo.co. in>
wrote:
>> I tried the steps required for Installation
>> Below are the installation steps for installing JUnit:
>> 1.unzip the junit4.6.zip file
>> 2.add junit-4.6.jar to the CLASSPATH.
>>
>> For example: set classpath=%classpat h%;INSTALL_ DIR\junit- 4.6.jar;INSTALL_
DIR
>> test the installation by running java org.junit.runner. JUnitCore
org.junit.tests. AllTests
>>
>> I did everythg that was asked but i cudnt find org.junit.runner file? and if
I using java org.junit.tests. AllTests ,it throws an exception that main class
is not defined!!
>>
>>
>> Please help!!
>>
>>
>>
>> ------------ --------- --------- ------
>>
>> Yahoo! Groups Links
>>
>>
>>
>>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Add more friends to your messenger and enjoy! Go to
http://messenger.yahoo.com/invite/
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
Sadia:
For a jar file, you need to put the whole path, including the jar file itself,
into your classpath, not just the directory that the jar file is in.
Something like: set
CLASSPATH=d:\java\lib\junit-4.5.jar;d:\java\lib\log4j-1.2.15.jar
David
sadia tahseen wrote:
>
>
> Ya It is Junit4.5 I could get it installed and tested on eclipse but not
> on command prompt.It always says classdefnotfound error..even though I
> set the classpath to directory containing the junit.
>
> --- On Mon, 6/4/09, David Saff <david@...
> <mailto:david%40saff.net>> wrote:
>
> From: David Saff <david@... <mailto:david%40saff.net>>
> Subject: Re: [junit] difficulty in Installation of Junit4.5
> To: junit@yahoogroups.com <mailto:junit%40yahoogroups.com>
> Date: Monday, 6 April, 2009, 7:36 PM
>
> Sadia,
>
> Where did you get a 4.6 zip? 4.5 is the most recent official release.
> Thanks,
>
> David Saff
>
> On Thu, Apr 2, 2009 at 12:49 AM, sadia_in2006 <sadia_in2006@ yahoo.co.
> in> wrote:
>> I tried the steps required for Installation
>> Below are the installation steps for installing JUnit:
>> 1.unzip the junit4.6.zip file
>> 2.add junit-4.6.jar to the CLASSPATH.
>>
>> For example: set classpath=%classpat h%;INSTALL_ DIR\junit-
> 4.6.jar;INSTALL_ DIR
>> test the installation by running java org.junit.runner. JUnitCore
> org.junit.tests. AllTests
>>
>> I did everythg that was asked but i cudnt find org.junit.runner file?
> and if I using java org.junit.tests. AllTests ,it throws an exception
> that main class is not defined!!
>>
>>
>> Please help!!
>>
>>
>>
>> ------------ --------- --------- ------
>>
>> Yahoo! Groups Links
>>
>>
>>
>>
>
> Add more friends to your messenger and enjoy! Go to
> http://messenger.yahoo.com/invite/ <http://messenger.yahoo.com/invite/>
>
> [Non-text portions of this message have been removed]
>
>
Would anyone know why ant (I'm using 1.7) is not recognizing JUnit4 (I'm using
4.5) tests? I'm getting "No tests found in ...". It seems to me that ant is
running the 3.x test runner since, if I change my test methods to begin with
'test' it runs. I'm using the @Test annotation otherwise. I've added the
junit-4.5.jar to my class path and also added it to the ant home lib directory.
What else can I try?
Example trivial tests I'm using and ant test target:
public class MathTest {
int i = 7;
int j = -9;
double x = 72.3;
double y = 0.34;
@Test
public void absoluteValue() {
System.out.println("|" + i + "| is " +
Math.abs(i));
System.out.println("|" + j + "| is " +
Math.abs(j));
assertEquals(9, Math.abs(j));
}
@Test
public void roundingValue() {
System.out.println(x + " is approximately " +
Math.round(x));
System.out.println(y + " is approximately " +
Math.round(y));
assertEquals(72, Math.round(x));
}
}
<target name="test" depends="test-compile">
<junit fork="yes" printsummary="yes">
<test name="${test.class}" if="test.class"/>
<classpath refid="base.path"/>
<formatter type="plain" usefile="false" unless="testfile"/>
<formatter type="xml" usefile="true" if="testfile"/>
</junit>
</target>
Thanks,
Scott
[Non-text portions of this message have been removed]
Sorry for the late response to all this. Can you log a feature
request with what you'd like to see? Giving Parameterized more love
is one of our goals for 4.7. Thanks,
David
On Mon, Mar 9, 2009 at 10:02 PM, anfernee_xu <anfernee.xu@...> wrote:
> --- In junit@yahoogroups.com, David Saff <david@...> wrote:
>>
>> Anfernee,
>>
>> Can you check the feature requests at
>>
>> http://sourceforge.net/projects/junit/
>>
>> and make sure that there's one there for what you want? Thanks,
>>
>> David Saff
>>
>> On Mon, Mar 9, 2009 at 9:48 AM, David Saff <david@...> wrote:
>> > Anfernee,
>> >
>> > There is currently no such API, but it's a very popular feature
>> > request. I'll take a stab at it soon.
>> >
>> > David Saff
>> >
>> > On Mon, Mar 9, 2009 at 3:28 AM, anfernee_xu <anfernee.xu@...> wrote:
>> >> With JUnit4, we can have parameterized tests with @RunWith(value =
Parameterized.class) and @Parameters,
>> >>
>> >> But in any JUnit reporting interface(UI, xml and its API), it seems
there's no API to know the parameters each test runs with, this is useful
feature, as it's easy to identify what parameters result in particular test
fails.
>> >>
>> >> Can we have an enhancement in Description class, so we can have a new API
>> >> suppose
>> >> Map getParameters()
>> >>
>> >> thus reporting layer can have access to the parameters and be able to show
them to users.
>> >>
>> >> --Anfernee
>> >>
>> >>
>> >>
>> >> ------------------------------------
>> >>
>> >> Yahoo! Groups Links
>> >>
>> >>
>> >>
>> >>
>> >
>>
>
> Hi David,
>
> I checked the request tracker page,
http://sourceforge.net/tracker/index.php?func=detail&aid=1742040&group_id=15278&\
atid=365278 is similar to what I ask for, but it's more better to have an
explicit API to get those parameters rather than implicitly include them in test
name.
>
> Thanks
>
> Anfernee
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
Sorry for the long delay in response.
This is a good idea, I think. I wonder if it needs to be in the same
runner as Suite, or could it be a different Runner?
David Saff
On Wed, Feb 25, 2009 at 10:34 AM, yme0987654321 <yme0987654321@...> wrote:
> I have the following challenge, with a proposed solution. If there is
> interest, I would be happy to contribute a patch to the JUnit project
> to address it. On the other hand, if there is a way to solve this
> problem differently in JUnit already, please let me know.
>
> Ant and IDEA (the IDE from Intellij) don't agree on how to gather
> classes for test. IDEA inspects the actual class for things that
> extend TestCase or have a JUnit annotation on the class (or have a
> static suite method) and executes the test. Ant gathers tests via
> class names. To use the Ant method with JUnit outside of Ant, you can
> use JUnitCore, and just pass it an array of classes gathered in the
> Ant form. However IDEA doesn't report those results in its typical
> fashion. It uses its own custom runner to gather and run these tests.
>
> The only option I see in the JUnit current code is to make a custom
> runner that knows how to extract a Class[] array and pass it on to the
> standard runner, and then use the RunWith tag to annotate an arbitrary
> class to get IDEA to think that it is running a test suite. I haven't
> actually done this to see if IDEA has any issues, but it seems like it
> should work.
>
> My proposal is to add an annotation to the Suite class called
> SuiteMethod that would annotate a method that returns a Class[] array.
> The Suite would then be built from the two annotations,
> Suite.SuiteClasses plus any methods on the class (we could enforce
> that they be public, static, take no parameters and return a Class[]
> array) that are annotated with Suite.SuiteMethod and build the Suite
> as a combination of all of the Arrays returned.
>
> What do you think?
>
>
>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
All,
This is a test of the "with enough eyeballs, all bugs are shallow"
hypothesis.
I am experiencing a race condition with timeouts. Say I have a test like
this:
public class WaitTooLong {
@Test(timeout=10) public void ripVanWinkle() throws
InterruptedException {
Thread.sleep(100);
}
}
When I run this I get one of two errors, non-deterministically:
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at UnpredictableTest.takeTen(UnpredictableTest.java:5)
...
or
java.lang.Exception: test timed out after 10 milliseconds
at
org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.j
ava:48)
...
Here's the relevant code in JUnit (FailOnTimeout):
public void evaluate() throws Throwable {
ExecutorService service=
Executors.newSingleThreadExecutor();
Callable<Object> callable= new Callable<Object>() {
public Object call() throws Exception {
try {
fNext.evaluate();
} catch (Throwable e) {
throw new ExecutionException(e);
}
return null;
}
};
Future<Object> result= service.submit(callable);
service.shutdown();
try {
boolean terminated=
service.awaitTermination(fTimeout, TimeUnit.MILLISECONDS);
if (!terminated)
service.shutdownNow();
result.get(0, TimeUnit.MILLISECONDS); // throws the
exception if one occurred during the invocation
} catch (TimeoutException e) {
throw new Exception(String.format("test timed out
after %d milliseconds", fTimeout));
} catch (ExecutionException e) {
throw unwrap(e);
}
}
It seems that sometimes the TimeoutException gets thrown and caught and the
Exception thrown. Sometimes, though, the InterruptedException that is
terminating the test-running thread gets thrown/caught first. I've stared at
this enough minutes to be ready for help.
One strangeness is that if I run the test programmatically
("JUnitCore.runClass(WaitTooLong.class)") the Exception is always thrown. If
I run the test with the Eclipse JUnit runner or with JUnit Max, the results
are non-deterministic.
Any ideas for making this deterministic? I would prefer the second result
above, if I can get it.
Regards,
Kent Beck
Three Rivers Institute
Oops, sorry about the formatting. I'm afraid you'll have to go look at the
JUnit source to read it easily.
Regards,
Kent
_____
From: junit@yahoogroups.com [mailto:junit@yahoogroups.com] On Behalf Of
kentb
Sent: Thursday, April 09, 2009 1:53 PM
To: junit@yahoogroups.com
Subject: [junit] Race condition with timeouts
All,
This is a test of the "with enough eyeballs, all bugs are shallow"
hypothesis.
I am experiencing a race condition with timeouts. Say I have a test like
this:
public class WaitTooLong {
@Test(timeout=10) public void ripVanWinkle() throws
InterruptedException {
Thread.sleep(100);
}
}
When I run this I get one of two errors, non-deterministically:
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at UnpredictableTest.takeTen(UnpredictableTest.java:5)
...
or
java.lang.Exception: test timed out after 10 milliseconds
at
org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.j
ava:48)
...
Here's the relevant code in JUnit (FailOnTimeout):
public void evaluate() throws Throwable {
ExecutorService service=
Executors.newSingleThreadExecutor();
Callable<Object> callable= new Callable<Object>() {
public Object call() throws Exception {
try {
fNext.evaluate();
} catch (Throwable e) {
throw new ExecutionException(e);
}
return null;
}
};
Future<Object> result= service.submit(callable);
service.shutdown();
try {
boolean terminated=
service.awaitTermination(fTimeout, TimeUnit.MILLISECONDS);
if (!terminated)
service.shutdownNow();
result.get(0, TimeUnit.MILLISECONDS); // throws the
exception if one occurred during the invocation
} catch (TimeoutException e) {
throw new Exception(String.format("test timed out
after %d milliseconds", fTimeout));
} catch (ExecutionException e) {
throw unwrap(e);
}
}
It seems that sometimes the TimeoutException gets thrown and caught and the
Exception thrown. Sometimes, though, the InterruptedException that is
terminating the test-running thread gets thrown/caught first. I've stared at
this enough minutes to be ready for help.
One strangeness is that if I run the test programmatically
("JUnitCore.runClass(WaitTooLong.class)") the Exception is always thrown. If
I run the test with the Eclipse JUnit runner or with JUnit Max, the results
are non-deterministic.
Any ideas for making this deterministic? I would prefer the second result
above, if I can get it.
Regards,
Kent Beck
Three Rivers Institute
[Non-text portions of this message have been removed]
Hi Kent,
On Apr 9, 2009, at 3:52 PM, kentb wrote:
> All,
>
> This is a test of the "with enough eyeballs, all bugs are shallow"
> hypothesis.
>
> I am experiencing a race condition with timeouts. Say I have a test
> like
> this:
> <snip>
>
Just curious ... how many cores in your machine?
cas
[Non-text portions of this message have been removed]
Two.
Kent
_____
From: junit@yahoogroups.com [mailto:junit@yahoogroups.com] On Behalf Of
Charles Sharp
Sent: Thursday, April 09, 2009 1:59 PM
To: junit@yahoogroups.com
Subject: Re: [junit] Race condition with timeouts
Hi Kent,
On Apr 9, 2009, at 3:52 PM, kentb wrote:
> All,
>
> This is a test of the "with enough eyeballs, all bugs are shallow"
> hypothesis.
>
> I am experiencing a race condition with timeouts. Say I have a test
> like
> this:
> <snip>
>
Just curious ... how many cores in your machine?
cas
[Non-text portions of this message have been removed]
[Non-text portions of this message have been removed]
A timeout of 10ms is very small, too close to the granularity of the
OS clock for me to feel comfortable.
How does the behaviour change as you change the timeout durations?
--Nat
On 09/04/2009, kentb <kentb@...> wrote:
> All,
>
> This is a test of the "with enough eyeballs, all bugs are shallow"
> hypothesis.
>
> I am experiencing a race condition with timeouts. Say I have a test like
> this:
>
> public class WaitTooLong {
> @Test(timeout=10) public void ripVanWinkle() throws
> InterruptedException {
> Thread.sleep(100);
> }
> }
>
> When I run this I get one of two errors, non-deterministically:
>
> java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at UnpredictableTest.takeTen(UnpredictableTest.java:5)
> ...
>
> or
>
> java.lang.Exception: test timed out after 10 milliseconds
> at
> org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.j
> ava:48)
> ...
>
> Here's the relevant code in JUnit (FailOnTimeout):
>
> public void evaluate() throws Throwable {
> ExecutorService service=
> Executors.newSingleThreadExecutor();
> Callable<Object> callable= new Callable<Object>() {
> public Object call() throws Exception {
> try {
> fNext.evaluate();
> } catch (Throwable e) {
> throw new ExecutionException(e);
> }
> return null;
> }
> };
> Future<Object> result= service.submit(callable);
> service.shutdown();
> try {
> boolean terminated=
> service.awaitTermination(fTimeout, TimeUnit.MILLISECONDS);
> if (!terminated)
> service.shutdownNow();
> result.get(0, TimeUnit.MILLISECONDS); // throws the
> exception if one occurred during the invocation
> } catch (TimeoutException e) {
> throw new Exception(String.format("test timed out
> after %d milliseconds", fTimeout));
> } catch (ExecutionException e) {
> throw unwrap(e);
> }
> }
>
> It seems that sometimes the TimeoutException gets thrown and caught and the
> Exception thrown. Sometimes, though, the InterruptedException that is
> terminating the test-running thread gets thrown/caught first. I've stared at
> this enough minutes to be ready for help.
>
> One strangeness is that if I run the test programmatically
> ("JUnitCore.runClass(WaitTooLong.class)") the Exception is always thrown. If
> I run the test with the Eclipse JUnit runner or with JUnit Max, the results
> are non-deterministic.
>
> Any ideas for making this deterministic? I would prefer the second result
> above, if I can get it.
>
> Regards,
>
> Kent Beck
> Three Rivers Institute
>
>
--
http://www.natpryce.com
Nat Pryce wrote:
>
>
> A timeout of 10ms is very small, too close to the granularity of the
> OS clock for me to feel comfortable.
>
> How does the behaviour change as you change the timeout durations?
>
> --Nat
>
> .
>
>
Note that on some JVM's the behaviour of sleep itself changes when using
small timeouts. In particular a sleep of 2ms will actually do what it
says on Windows even though the normal OS clock is 10ms. As far as I can
recall sleeps which are multiples of 10ms behave in the regular fashion.
Mark Thornton
I'm assuming that the InterruptedException is caused by
service.shutdownNow(), which, according to javadoc, "typical
implementations will cancel via Thread.interrupt()".
What is causing the TimeoutException? Unfortunately, your code throws
away the original stacktrace, and I don't know the executor API well
enough to guess...
Cheers, Ilja
2009/4/9 kentb <kentb@...>:
>
>
> All,
>
> This is a test of the "with enough eyeballs, all bugs are shallow"
> hypothesis.
>
> I am experiencing a race condition with timeouts. Say I have a test like
> this:
>
> public class WaitTooLong {
> @Test(timeout=10) public void ripVanWinkle() throws
> InterruptedException {
> Thread.sleep(100);
> }
> }
>
> When I run this I get one of two errors, non-deterministically:
>
> java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at UnpredictableTest.takeTen(UnpredictableTest.java:5)
> ...
>
> or
>
> java.lang.Exception: test timed out after 10 milliseconds
> at
> org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.j
> ava:48)
> ...
>
> Here's the relevant code in JUnit (FailOnTimeout):
>
> public void evaluate() throws Throwable {
> ExecutorService service=
> Executors.newSingleThreadExecutor();
> Callable<Object> callable= new Callable<Object>() {
> public Object call() throws Exception {
> try {
> fNext.evaluate();
> } catch (Throwable e) {
> throw new ExecutionException(e);
> }
> return null;
> }
> };
> Future<Object> result= service.submit(callable);
> service.shutdown();
> try {
> boolean terminated=
> service.awaitTermination(fTimeout, TimeUnit.MILLISECONDS);
> if (!terminated)
> service.shutdownNow();
> result.get(0, TimeUnit.MILLISECONDS); // throws the
> exception if one occurred during the invocation
> } catch (TimeoutException e) {
> throw new Exception(String.format("test timed out
> after %d milliseconds", fTimeout));
> } catch (ExecutionException e) {
> throw unwrap(e);
> }
> }
>
> It seems that sometimes the TimeoutException gets thrown and caught and the
> Exception thrown. Sometimes, though, the InterruptedException that is
> terminating the test-running thread gets thrown/caught first. I've stared at
> this enough minutes to be ready for help.
>
> One strangeness is that if I run the test programmatically
> ("JUnitCore.runClass(WaitTooLong.class)") the Exception is always thrown. If
> I run the test with the Eclipse JUnit runner or with JUnit Max, the results
> are non-deterministic.
>
> Any ideas for making this deterministic? I would prefer the second result
> above, if I can get it.
>
> Regards,
>
> Kent Beck
> Three Rivers Institute
>
>
Why do you need this first :
service.awaitTermination(fTimeout, TimeUnit.MILLISECONDS);
Maybe just :
result.get(0, TimeUnit.MILLISECONDS);
with a shutdownNow after in a finally block if required.
I would expect you always get InterrpuptedException because you are sending
it to the service thread via shutdownNow. Once you get the result from your
Future, you should get it.
However, I guess that because your timeout is very small, the get method
might times out before the service thread is reactivated. So in those case,
you get only your TimeOut and throw your exception.
Gilles Scokart
2009/4/10 Ilja Preuß <iljapreuss@...>
>
>
> I'm assuming that the InterruptedException is caused by
> service.shutdownNow(), which, according to javadoc, "typical
> implementations will cancel via Thread.interrupt()".
>
> What is causing the TimeoutException? Unfortunately, your code throws
> away the original stacktrace, and I don't know the executor API well
> enough to guess...
>
> Cheers, Ilja
>
> 2009/4/9 kentb <kentb@... <kentb%40earthlink.net>>:
>
> >
> >
> > All,
> >
> > This is a test of the "with enough eyeballs, all bugs are shallow"
> > hypothesis.
> >
> > I am experiencing a race condition with timeouts. Say I have a test like
> > this:
> >
> > public class WaitTooLong {
> > @Test(timeout=10) public void ripVanWinkle() throws
> > InterruptedException {
> > Thread.sleep(100);
> > }
> > }
> >
> > When I run this I get one of two errors, non-deterministically:
> >
> > java.lang.InterruptedException: sleep interrupted
> > at java.lang.Thread.sleep(Native Method)
> > at UnpredictableTest.takeTen(UnpredictableTest.java:5)
> > ...
> >
> > or
> >
> > java.lang.Exception: test timed out after 10 milliseconds
> > at
> >
> org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.j
> > ava:48)
> > ...
> >
> > Here's the relevant code in JUnit (FailOnTimeout):
> >
> > public void evaluate() throws Throwable {
> > ExecutorService service=
> > Executors.newSingleThreadExecutor();
> > Callable<Object> callable= new Callable<Object>() {
> > public Object call() throws Exception {
> > try {
> > fNext.evaluate();
> > } catch (Throwable e) {
> > throw new ExecutionException(e);
> > }
> > return null;
> > }
> > };
> > Future<Object> result= service.submit(callable);
> > service.shutdown();
> > try {
> > boolean terminated=
> > service.awaitTermination(fTimeout, TimeUnit.MILLISECONDS);
> > if (!terminated)
> > service.shutdownNow();
> > result.get(0, TimeUnit.MILLISECONDS); // throws the
> > exception if one occurred during the invocation
> > } catch (TimeoutException e) {
> > throw new Exception(String.format("test timed out
> > after %d milliseconds", fTimeout));
> > } catch (ExecutionException e) {
> > throw unwrap(e);
> > }
> > }
> >
> > It seems that sometimes the TimeoutException gets thrown and caught and
> the
> > Exception thrown. Sometimes, though, the InterruptedException that is
> > terminating the test-running thread gets thrown/caught first. I've stared
> at
> > this enough minutes to be ready for help.
> >
> > One strangeness is that if I run the test programmatically
> > ("JUnitCore.runClass(WaitTooLong.class)") the Exception is always thrown.
> If
> > I run the test with the Eclipse JUnit runner or with JUnit Max, the
> results
> > are non-deterministic.
> >
> > Any ideas for making this deterministic? I would prefer the second result
> > above, if I can get it.
> >
> > Regards,
> >
> > Kent Beck
> > Three Rivers Institute
> >
> >
>
>
[Non-text portions of this message have been removed]
Just wondering if the junitext project is still active. I'd like to
contribute to it, but the mailing list and web site looks stale.
I'd also like to move the project to github. To stay in line with JUnit.
Plus git is alot better then SVN.
Big Mike
[Non-text portions of this message have been removed]
Send Email
