I have found that the reported total number of tests run is wrong when
some of the tests fail. I see figures like "Runs: 1007/1000".

I'm running the tests using the graphical TestRunner in Eclipse 3.1.

Further investigation suggests that that total number of tests is only
wrong when tests fail (84 failures above). A dummy test which never
fails almost report the correct number of tests run.

I have created a simple case which replicates the problem (below),
based on using a test name which does not exist, so all tests fail.
I have also replicated the problem with a test case which fails about
10% of the time based on the value of Math.random().

With concurrentUsers=1, the count is correct. With concurrentUsers=2,
the count is only wrong occassionally. With concurrentUsers>=3 the
count is almost always wrong.

This isn't going to put me off using JUnitPerf. I downloaded for the
first time today, and used it to confirm that my code wasn't
threadsafe in no time flat. Thank you for creating a valuable tool.


Ken.




import junit.framework.Test;
import junit.framework.TestCase;

import com.clarkware.junitperf.LoadTest;

public class WrongCountTest extends TestCase {

public WrongCountTest(String name) {
super(name);
}

public static Test suite() {
int concurrentUsers = 10;
int iterations = 100;
Test test = new WrongCountTest("testDoesNotExist");
Test loadTest = new LoadTest(test, concurrentUsers, iterations);
return loadTest;
}
}