Chad,

Do the superclass and subclass @BeforeClass methods have the same name?

    David Saff

On Sat, Dec 19, 2009 at 6:13 AM, cretz@... <chad.retz@...> wrote:
> I have an abstract base class that defines a static @BeforeClass setUp method.
When this class is extended, the static setUp method is not called. The javadoc
says it will call a super class's @BeforeClass before the current class's. Am I
missing something?
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

I don't even have a @BeforeClass method in the subclass. Only in the superclass.
I figured it out, I was extending from TestCase also (which I know isn't JUnit 4
style). If I don't extend TestCase, it works. Example base class:

public abstract class MyTestBase extends TestCase {

     protected static boolean setUpExecuted = false;

     @BeforeClass
     public static void oneTimeSetUp() {
         setUpExecuted = true;
     }
}

and example extension:

public class MyTest extends MyTestBase {

     @Test
     public void testWhetherOneTimeSetUpExecuted() {
         Assert.assertTrue("One time setUp not executed",
                 setUpExecuted);
     }
}

I didn't check the JUnit source to see if it uses an instanceof to determine
whether to use JUnit 3 style or JUnit 4 style. My problem is I need to extend
from a third party lib (SeleniumTestCase) and use protected stuff in there. What
I'm gonna do is just have another delegating class extending it and expose the
protected object I need.

So, I have solved it in a workaround fashion. Other people have too (ref:
http://rockhoppertech.com/blogs/archives/45). I was just wondering if this is by
intention?

--- In junit@yahoogroups.com, David Saff <david@...> wrote:
>
> Chad,
>
> Do the superclass and subclass @BeforeClass methods have the same name?
>
>    David Saff
>
> On Sat, Dec 19, 2009 at 6:13 AM, cretz@... <chad.retz@...> wrote:
> > I have an abstract base class that defines a static @BeforeClass setUp
method. When this class is extended, the static setUp method is not called. The
javadoc says it will call a super class's @BeforeClass before the current
class's. Am I missing something?
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>

I think the java documentation for @BeforeClass is wrong; static methods
will not be inherited and so, can't be called your runner. - Sai

On Sat, Dec 19, 2009 at 4:43 PM, cretz@... <chad.retz@...>wrote:

>
>
> I have an abstract base class that defines a static @BeforeClass setUp
> method. When this class is extended, the static setUp method is not called.
> The javadoc says it will call a super class's @BeforeClass before the
> current class's. Am I missing something?
>
>
>


[Non-text portions of this message have been removed]

Not entirely. My point isn't (just) that "wouldn't it be nice if JUnit had all
the options for automatic test collection that ANT, Maven and IDE's have." But
rather "wouldn't it be nice if the different components of putting together a
test run were pluggable, so that if ANT gathers tests in one way, it wouldn't
stop something else from, in the same test run, gather information about the
test results because those two concepts are decoupled."

What I am saying doesn't really change the merit of Global Rules, I'm just
seeing Global Rules in this context as really an attempt to gather information
about a test run, which strikes me as more in the area of the Runner.

I meant what I said as food for thought to influence the design, not as a
specific feature request. Thanks for the attention ;).

--- In junit@yahoogroups.com, David Saff <david@...> wrote:
>
> Yishai,
>
> I see what you're asking for (automatic test collection) as a powerful
> feature request on its own, but I feel that it could be implemented
> without Global Rules, and Global Rules could be implemented without
> automatic test detection.  Do I understand your point correctly?
>
>    David
>
> On Thu, Dec 17, 2009 at 11:49 AM, yme0987654321 <yme0987654321@...> wrote:
> > I would suggest another way of thinking about this. It would be harder to
get to full implementation, perhaps, but I think much more powerful in the long
run.
> >
> > Right now, a lot of effort is put into collecting the right classes to run
for a test. JUnit at its core has its way (basically spelling them all out in a
Suite or on the command line), ANT has its own runner, as does Maven. IDEs do
something else, again introducing their own runner.
> >
> > It would seem to be more powerful/better if the JUnit framework itself
provided the uber-Runner that others could hook into to provide providers for
collecting the tests and receive information to capture results, so that all
systems are running the same overall framework and individual needs can be
reliably plugged into it.
> >
> > This way if a system (say ANT or an IDE) needs to be able to handle
capturing results, they don't also have to provide the gathering capabilities.
Those gathering capabilities can be something standardized (I think we need
something more than just listing them that JUnit provides now, either slurping
up class files from specified directories as ANT does, or something that
examines all available classpaths like IDEs do).
> >
> > If something like that existed, then the fact that Maven is running the
tests wouldn't matter because JUnit would still be able to take the additional
result capturing mechanism required by your TestUI.
> >
> > I don't think the above (long winded) statement is that different from what
you are saying, I'm just suggesting the relationship between the components be
better mapped, more formal, than a feature stuck in here or there to hook into a
different runner.
> >
> > Yishai
> >
> > --- In junit@yahoogroups.com, Daniel Brolund <daniel.brolund@> wrote:
> >>
> >> Ok, I tried to sum things up. Please add comments if I omitted anything.
> >>
> >> http://github.com/KentBeck/junit/issues/#issue/69
> >>
> >> I hope it is concise enough.
> >>
> >> Don't forget to vote!
> >> :-)
> >>
> >> Cheers
> >> Daniel
> >>
> >> On Wed, Dec 16, 2009 at 8:54 PM, David Saff <david@> wrote:
> >>
> >> >
> >> >
> >> > All,
> >> >
> >> > If someone comes up with a concise feature request at github, it
> >> > definitely sounds like there's enough votes on this thread to move it
> >> > pretty high up the queue.
> >> >
> >> > David Saff
> >> >
> >> >
> >> > On Mon, Dec 14, 2009 at 3:36 PM, Bogdan
> >> > <bogdan.mocanu.notifications@<bogdan.mocanu.notifications%40gmail.com>>
> >> > wrote:
> >> > > I agree Joakim, setting a JVM parameter for each executed session of
> >> > JUnit tests can be a very nasty solution. BUT in some cases this solution
> >> > can be a desirable one. Therefore I think this solution, along with
others
> >> > (see below) should be provided.
> >> > >
> >> > > My current need for global rules is:
> >> > > - I am developing a Swing console, called TestUI, that will allow you
to
> >> > visually see the results of your tests, the tests that failed and those
that
> >> > succeeded. You will also be able to specify a path to watch for Surefire
> >> > reports, therefore combining analytical functions with report functions
(the
> >> > report functions are those for which I need global rules).
> >> > >
> >> > > @David: I took a look at RunListener, and indeed it solves my problem
> >> > (just like a TestWatchman-derived rule would), except that it is very
> >> > inconvenient to use. For example, at work I have 110 modules, and a total
> >> > number of 800 unit tests. I would like to globally hook up my
> >> > TestUIAgentRule, to report to my TestUI console each test that is
executed.
> >> > >
> >> > > I execute the tests from Eclipse or from Maven, during build sessions.
> >> > Therefore I cannot use RunListener or a Rule for this. I need some global
> >> > mechanism to enable a Rule for all the executed tests.
> >> > >
> >> > > Except for JVM params, here is another idea: having a junit
> >> > properties/XML configuration file. It has been suggested on this message
> >> > group that junit should have a configuration file, and I am PRO this
> >> > solution.
> >> > >
> >> > > In such a config file (available soomewhere in the classpath, like the
> >> > log4j.properties), we could specifiy:
> >> > > - which tests to execute/exclude/ignore
> >> > > - global rules
> >> > > - test suites to run/exclude
> >> > > - categories to run/exclude
> >> > >
> >> > > This could also solve this problem:
> >> > http://tech.groups.yahoo.com/group/junit/message/22184. The request here
> >> > (read the first post in the thread) asks for a GUI where unit tests could
be
> >> > disabled, like light bulbs (PS: my TestUI console will be able to do
this,
> >> > sometime in the
> >> > > future releases).
> >> > >
> >> > > What do you think of this?
> >> > >
> >> > > Cheers,
> >> > > Bogdan
> >> > >
> >> > > --- In junit@yahoogroups.com <junit%40yahoogroups.com>, Joakim Ohlrogge
> >> > <joakim.ohlrogge@> wrote:
> >> > >>
> >> > >> Hi Bogdan,
> >> > >>
> >> > >> I can see where you're coming from with system properties to solve
this.
> >> > >> I would much rather have some way of solving this where you would not
> >> > have
> >> > >> to rely on a property being set at the JVM-level. The reason is that
> >> > it's
> >> > >> very inconvenient to have to set run parameters if you just want to
> >> > rerun a
> >> > >> single test (in eclipse that's a run configuration rather than just
> >> > >> rightclicking on a test in the runner). If JUnit instead would look
for
> >> > a
> >> > >> standard property-file or similar (like junit-rules.properties in the
> >> > root
> >> > >> of your class-path and merge all found resources found like
> >> > >> getClass().getClassLoader().getResources("/junit-rules.properties")
then
> >> > it
> >> > >> would just work with no external configuration like launch parameters
or
> >> > >> script parameters.
> >> > >>
> >> > >> With VM-parameters I don't see that the feature would be useful for
> >> > anything
> >> > >> that influences the testrun, just for collecting information about the
> >> > run.
> >> > >>
> >> > >>
> >> > >> -----------------------------------------------------
> >> > >> Joakim Ohlrogge
> >> > >> Agical AB
> >> > >> Västerlånggatan 79, 2 tr
> >> > >> 111 29 Stockholm, SWEDEN
> >> > >>
> >> > >> Mobile: +46-708-754004
> >> > >> Blog: johlrogge.wordpress.com
> >> > >> E-mail: joakim.ohlrogge@
> >> > >>
> >> > >>
> >> > >> On Sat, Dec 5, 2009 at 8:46 PM, Bogdan <
> >> > >> bogdan.mocanu.notifications@> wrote:
> >> > >>
> >> > >> >
> >> > >> >
> >> > >> > Hi guys,
> >> > >> >
> >> > >> > thank you very much for your responses.
> >> > >> >
> >> > >> > I didn't know about the RunListener, so I will definitely take a
look
> >> > at
> >> > >> > it, and come back with an answer regarding it.
> >> > >> >
> >> > >> > What I was thinking about the global rules is to be able to
configure
> >> > >> > something like global rules, that are specified as a startup
parameter
> >> > (read
> >> > >> > Java option) when starting the test (in the project at my office, we
> >> > have
> >> > >> > like 750 tests, and therefore would work just nice to configure this
> >> > option
> >> > >> > just for my env; probably the RunListener will work just like that,
so
> >> > as I
> >> > >> > said, I will read about this).
> >> > >> >
> >> > >> > I was thinking about something like this:
> >> > >> >
> >> > >> > java
> >> > >> >
> >> >
-Djunit.globalRules=com.test.myrules.SomeNiceRule,com.test.myrules.SomeNiceRule2
> >> > >> > ..........
> >> > >> >
> >> > >> > Cheers,
> >> > >> > Bogdan
> >> > >> >
> >> > >> >
> >> > >> > --- In junit@yahoogroups.com <junit%40yahoogroups.com> <junit%
> >> > 40yahoogroups.com>, "Bogdan"
> >> > >> > <bogdan.mocanu.notifications@> wrote:
> >> > >> > >
> >> > >> > > Hi everyone,
> >> > >> > >
> >> > >> > > as we have it currently, we can place a Rule in a test, and that
> >> > rule
> >> > >> > will be activated for that test. To have the rule working in several
> >> > tests,
> >> > >> > you have to place the Rule in each test.
> >> > >> > >
> >> > >> > > However, I am searching for a way to activate a Rule for all the
> >> > tests
> >> > >> > that are executed, without needing to add the rule to each test.
> >> > Perhaps by
> >> > >> > specifying a Java property parameter when running the tests (I could
> >> > also
> >> > >> > set this as a global Java option, therefore being used upon each
> >> > execution
> >> > >> > of the JVM).
> >> > >> > >
> >> > >> > > Do you know if this is possible?
> >> > >> > >
> >> > >> > > The reason why I need this, is: I am making the plans and
gathering
> >> > the
> >> > >> > ideas for creating a simple Swing application that will display the
> >> > results
> >> > >> > of the executed tests (just like the JUnit view in
> >> > Eclipse/Idea/Netbeans,
> >> > >> > but perhaps with more details, and also available for anyone, not
just
> >> > IDE
> >> > >> > users). Currently I developed a small agent rule, that you can place
> >> > in your
> >> > >> > test, and it will report to the console the status of your tests
(the
> >> > Swing
> >> > >> > console runs remotely).
> >> > >> > >
> >> > >> > > This is ok, but it means that you will have to add this rule to
each
> >> > and
> >> > >> > every test that you have. It also means that will not be an easy
task
> >> > to
> >> > >> > disable this rule, if you don't want it anymore.
> >> > >> > >
> >> > >> > > Therefore a possibility to enable Rules globally for all tests I
> >> > think is
> >> > >> > the best solution for my case, and also a good tool to have for
> >> > various
> >> > >> > other tasks.
> >> > >> > >
> >> > >> > > What do you think? Also, what do you think about the Swing console
> >> > idea?
> >> > >> > >
> >> > >> > > Cheers,
> >> > >> > > Bogdan
> >> > >> > >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >>
> >> > >>
> >> > >> [Non-text portions of this message have been removed]
> >> > >>
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > ------------------------------------
> >> > >
> >> > > Yahoo! Groups Links
> >> > >
> >> > >
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> ---------------------------------------------------------
> >> Daniel Brolund
> >> Agical AB - www.agical.com
> >>
> >> work: daniel.brolund@
> >> phone: +46708754002
> >> blog:http://danielbrolund.wordpress.com
> >> twitter: @danielbrolund
> >> private: daniel.brolund@
> >>
> >>
> >> [Non-text portions of this message have been removed]
> >>
> >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>

It sounds to me like the perfect excuse for a Code Camp. Invite the people who
write the JUnit runners for Ant, Maven, Eclipse, IntelliJ, and NetBeans to get
together for three days and see if we can expand the runner in org.junit.runners
so that they can all shrink or eliminate their runners. Any suggestions for a
sponsor?

Kent

On Dec 20, 2009, at 4:37 AM, yme0987654321 wrote:

> Not entirely. My point isn't (just) that "wouldn't it be nice if JUnit had all
the options for automatic test collection that ANT, Maven and IDE's have." But
rather "wouldn't it be nice if the different components of putting together a
test run were pluggable, so that if ANT gathers tests in one way, it wouldn't
stop something else from, in the same test run, gather information about the
test results because those two concepts are decoupled."
>
> What I am saying doesn't really change the merit of Global Rules, I'm just
seeing Global Rules in this context as really an attempt to gather information
about a test run, which strikes me as more in the area of the Runner.
>
> I meant what I said as food for thought to influence the design, not as a
specific feature request. Thanks for the attention ;).
>
> --- In junit@yahoogroups.com, David Saff <david@...> wrote:
> >
> > Yishai,
> >
> > I see what you're asking for (automatic test collection) as a powerful
> > feature request on its own, but I feel that it could be implemented
> > without Global Rules, and Global Rules could be implemented without
> > automatic test detection. Do I understand your point correctly?
> >
> > David
> >
> > On Thu, Dec 17, 2009 at 11:49 AM, yme0987654321 <yme0987654321@...> wrote:
> > > I would suggest another way of thinking about this. It would be harder to
get to full implementation, perhaps, but I think much more powerful in the long
run.
> > >
> > > Right now, a lot of effort is put into collecting the right classes to run
for a test. JUnit at its core has its way (basically spelling them all out in a
Suite or on the command line), ANT has its own runner, as does Maven. IDEs do
something else, again introducing their own runner.
> > >
> > > It would seem to be more powerful/better if the JUnit framework itself
provided the uber-Runner that others could hook into to provide providers for
collecting the tests and receive information to capture results, so that all
systems are running the same overall framework and individual needs can be
reliably plugged into it.
> > >
> > > This way if a system (say ANT or an IDE) needs to be able to handle
capturing results, they don't also have to provide the gathering capabilities.
Those gathering capabilities can be something standardized (I think we need
something more than just listing them that JUnit provides now, either slurping
up class files from specified directories as ANT does, or something that
examines all available classpaths like IDEs do).
> > >
> > > If something like that existed, then the fact that Maven is running the
tests wouldn't matter because JUnit would still be able to take the additional
result capturing mechanism required by your TestUI.
> > >
> > > I don't think the above (long winded) statement is that different from
what you are saying, I'm just suggesting the relationship between the components
be better mapped, more formal, than a feature stuck in here or there to hook
into a different runner.
> > >
> > > Yishai
> > >
> > > --- In junit@yahoogroups.com, Daniel Brolund <daniel.brolund@> wrote:
> > >>
> > >> Ok, I tried to sum things up. Please add comments if I omitted anything.
> > >>
> > >> http://github.com/KentBeck/junit/issues/#issue/69
> > >>
> > >> I hope it is concise enough.
> > >>
> > >> Don't forget to vote!
> > >> :-)
> > >>
> > >> Cheers
> > >> Daniel
> > >>
> > >> On Wed, Dec 16, 2009 at 8:54 PM, David Saff <david@> wrote:
> > >>
> > >> >
> > >> >
> > >> > All,
> > >> >
> > >> > If someone comes up with a concise feature request at github, it
> > >> > definitely sounds like there's enough votes on this thread to move it
> > >> > pretty high up the queue.
> > >> >
> > >> > David Saff
> > >> >
> > >> >
> > >> > On Mon, Dec 14, 2009 at 3:36 PM, Bogdan
> > >> > <bogdan.mocanu.notifications@<bogdan.mocanu.notifications%40gmail.com>>
> > >> > wrote:
> > >> > > I agree Joakim, setting a JVM parameter for each executed session of
> > >> > JUnit tests can be a very nasty solution. BUT in some cases this
solution
> > >> > can be a desirable one. Therefore I think this solution, along with
others
> > >> > (see below) should be provided.
> > >> > >
> > >> > > My current need for global rules is:
> > >> > > - I am developing a Swing console, called TestUI, that will allow you
to
> > >> > visually see the results of your tests, the tests that failed and those
that
> > >> > succeeded. You will also be able to specify a path to watch for
Surefire
> > >> > reports, therefore combining analytical functions with report functions
(the
> > >> > report functions are those for which I need global rules).
> > >> > >
> > >> > > @David: I took a look at RunListener, and indeed it solves my problem
> > >> > (just like a TestWatchman-derived rule would), except that it is very
> > >> > inconvenient to use. For example, at work I have 110 modules, and a
total
> > >> > number of 800 unit tests. I would like to globally hook up my
> > >> > TestUIAgentRule, to report to my TestUI console each test that is
executed.
> > >> > >
> > >> > > I execute the tests from Eclipse or from Maven, during build
sessions.
> > >> > Therefore I cannot use RunListener or a Rule for this. I need some
global
> > >> > mechanism to enable a Rule for all the executed tests.
> > >> > >
> > >> > > Except for JVM params, here is another idea: having a junit
> > >> > properties/XML configuration file. It has been suggested on this
message
> > >> > group that junit should have a configuration file, and I am PRO this
> > >> > solution.
> > >> > >
> > >> > > In such a config file (available soomewhere in the classpath, like
the
> > >> > log4j.properties), we could specifiy:
> > >> > > - which tests to execute/exclude/ignore
> > >> > > - global rules
> > >> > > - test suites to run/exclude
> > >> > > - categories to run/exclude
> > >> > >
> > >> > > This could also solve this problem:
> > >> > http://tech.groups.yahoo.com/group/junit/message/22184. The request
here
> > >> > (read the first post in the thread) asks for a GUI where unit tests
could be
> > >> > disabled, like light bulbs (PS: my TestUI console will be able to do
this,
> > >> > sometime in the
> > >> > > future releases).
> > >> > >
> > >> > > What do you think of this?
> > >> > >
> > >> > > Cheers,
> > >> > > Bogdan
> > >> > >
> > >> > > --- In junit@yahoogroups.com <junit%40yahoogroups.com>, Joakim
Ohlrogge
> > >> > <joakim.ohlrogge@> wrote:
> > >> > >>
> > >> > >> Hi Bogdan,
> > >> > >>
> > >> > >> I can see where you're coming from with system properties to solve
this.
> > >> > >> I would much rather have some way of solving this where you would
not
> > >> > have
> > >> > >> to rely on a property being set at the JVM-level. The reason is that
> > >> > it's
> > >> > >> very inconvenient to have to set run parameters if you just want to
> > >> > rerun a
> > >> > >> single test (in eclipse that's a run configuration rather than just
> > >> > >> rightclicking on a test in the runner). If JUnit instead would look
for
> > >> > a
> > >> > >> standard property-file or similar (like junit-rules.properties in
the
> > >> > root
> > >> > >> of your class-path and merge all found resources found like
> > >> > >> getClass().getClassLoader().getResources("/junit-rules.properties")
then
> > >> > it
> > >> > >> would just work with no external configuration like launch
parameters or
> > >> > >> script parameters.
> > >> > >>
> > >> > >> With VM-parameters I don't see that the feature would be useful for
> > >> > anything
> > >> > >> that influences the testrun, just for collecting information about
the
> > >> > run.
> > >> > >>
> > >> > >>
> > >> > >> -----------------------------------------------------
> > >> > >> Joakim Ohlrogge
> > >> > >> Agical AB
> > >> > >> Västerlånggatan 79, 2 tr
> > >> > >> 111 29 Stockholm, SWEDEN
> > >> > >>
> > >> > >> Mobile: +46-708-754004
> > >> > >> Blog: johlrogge.wordpress.com
> > >> > >> E-mail: joakim.ohlrogge@
> > >> > >>
> > >> > >>
> > >> > >> On Sat, Dec 5, 2009 at 8:46 PM, Bogdan <
> > >> > >> bogdan.mocanu.notifications@> wrote:
> > >> > >>
> > >> > >> >
> > >> > >> >
> > >> > >> > Hi guys,
> > >> > >> >
> > >> > >> > thank you very much for your responses.
> > >> > >> >
> > >> > >> > I didn't know about the RunListener, so I will definitely take a
look
> > >> > at
> > >> > >> > it, and come back with an answer regarding it.
> > >> > >> >
> > >> > >> > What I was thinking about the global rules is to be able to
configure
> > >> > >> > something like global rules, that are specified as a startup
parameter
> > >> > (read
> > >> > >> > Java option) when starting the test (in the project at my office,
we
> > >> > have
> > >> > >> > like 750 tests, and therefore would work just nice to configure
this
> > >> > option
> > >> > >> > just for my env; probably the RunListener will work just like
that, so
> > >> > as I
> > >> > >> > said, I will read about this).
> > >> > >> >
> > >> > >> > I was thinking about something like this:
> > >> > >> >
> > >> > >> > java
> > >> > >> >
> > >> >
-Djunit.globalRules=com.test.myrules.SomeNiceRule,com.test.myrules.SomeNiceRule2
> > >> > >> > ..........
> > >> > >> >
> > >> > >> > Cheers,
> > >> > >> > Bogdan
> > >> > >> >
> > >> > >> >
> > >> > >> > --- In junit@yahoogroups.com <junit%40yahoogroups.com> <junit%
> > >> > 40yahoogroups.com>, "Bogdan"
> > >> > >> > <bogdan.mocanu.notifications@> wrote:
> > >> > >> > >
> > >> > >> > > Hi everyone,
> > >> > >> > >
> > >> > >> > > as we have it currently, we can place a Rule in a test, and that
> > >> > rule
> > >> > >> > will be activated for that test. To have the rule working in
several
> > >> > tests,
> > >> > >> > you have to place the Rule in each test.
> > >> > >> > >
> > >> > >> > > However, I am searching for a way to activate a Rule for all the
> > >> > tests
> > >> > >> > that are executed, without needing to add the rule to each test.
> > >> > Perhaps by
> > >> > >> > specifying a Java property parameter when running the tests (I
could
> > >> > also
> > >> > >> > set this as a global Java option, therefore being used upon each
> > >> > execution
> > >> > >> > of the JVM).
> > >> > >> > >
> > >> > >> > > Do you know if this is possible?
> > >> > >> > >
> > >> > >> > > The reason why I need this, is: I am making the plans and
gathering
> > >> > the
> > >> > >> > ideas for creating a simple Swing application that will display
the
> > >> > results
> > >> > >> > of the executed tests (just like the JUnit view in
> > >> > Eclipse/Idea/Netbeans,
> > >> > >> > but perhaps with more details, and also available for anyone, not
just
> > >> > IDE
> > >> > >> > users). Currently I developed a small agent rule, that you can
place
> > >> > in your
> > >> > >> > test, and it will report to the console the status of your tests
(the
> > >> > Swing
> > >> > >> > console runs remotely).
> > >> > >> > >
> > >> > >> > > This is ok, but it means that you will have to add this rule to
each
> > >> > and
> > >> > >> > every test that you have. It also means that will not be an easy
task
> > >> > to
> > >> > >> > disable this rule, if you don't want it anymore.
> > >> > >> > >
> > >> > >> > > Therefore a possibility to enable Rules globally for all tests I
> > >> > think is
> > >> > >> > the best solution for my case, and also a good tool to have for
> > >> > various
> > >> > >> > other tasks.
> > >> > >> > >
> > >> > >> > > What do you think? Also, what do you think about the Swing
console
> > >> > idea?
> > >> > >> > >
> > >> > >> > > Cheers,
> > >> > >> > > Bogdan
> > >> > >> > >
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >>
> > >> > >>
> > >> > >> [Non-text portions of this message have been removed]
> > >> > >>
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > ------------------------------------
> > >> > >
> > >> > > Yahoo! Groups Links
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> ---------------------------------------------------------
> > >> Daniel Brolund
> > >> Agical AB - www.agical.com
> > >>
> > >> work: daniel.brolund@
> > >> phone: +46708754002
> > >> blog:http://danielbrolund.wordpress.com
> > >> twitter: @danielbrolund
> > >> private: daniel.brolund@
> > >>
> > >>
> > >> [Non-text portions of this message have been removed]
> > >>
> > >
> > >
> > >
> > >
> > > ------------------------------------
> > >
> > > Yahoo! Groups Links
> > >
> > >
> > >
> > >
> >
>
>



[Non-text portions of this message have been removed]

I have a scenario to test where in I need to make sure that tests are
executed in particular order...
How can I achieve this?

Are the test cases added with annotation @Test gets executed in the order
they are written?

Do test cases get executed randomly or do they follow any specific order?

-----
--
Regards,
Yogesh Patil.
--
View this message in context:
http://old.nabble.com/Order-of-execution-for-tests-tp26877097p26877097.html
Sent from the JUnit - User mailing list archive at Nabble.com.

I am using JExample to make sure that all required test cases has been
executed before executing the test case which is dependent on others.

But it seems like @AfterClass annotation stops working while running test
with JExample....

At the same time @BeforeClass annotation works perfectly fine...

Is it a bug in JExample or am I missing something?

-----
--
Regards,
Yogesh Patil.
--
View this message in context:
http://old.nabble.com/Use-of-JExample-for-maintaining-order-of-execution-for-tes\
ts-tp26877151p26877151.html
Sent from the JUnit - User mailing list archive at Nabble.com.

Tests are executed in an unspecified order. There is no way to define an order,
because in my experience independent tests are much more valuable than
order-dependent tests.

Every time I've had order dependencies I have had a problem with the design of
the underlying system. Investing my available time in improving the design has
had a much higher payoff than investing that same time into shuffling tests
around.

Independent tests enable interesting transformations like parallel execution or
the kind of dynamic reordering that let JUnit Max deliver failures more quickly.

Regards,

Kent

On Dec 21, 2009, at 10:28 PM, yogesh.patil wrote:

>
> I have a scenario to test where in I need to make sure that tests are
> executed in particular order...
> How can I achieve this?
>
> Are the test cases added with annotation @Test gets executed in the order
> they are written?
>
> Do test cases get executed randomly or do they follow any specific order?
>
> -----
> --
> Regards,
> Yogesh Patil.
> --
> View this message in context:
http://old.nabble.com/Order-of-execution-for-tests-tp26877097p26877097.html
> Sent from the JUnit - User mailing list archive at Nabble.com.
>
>



[Non-text portions of this message have been removed]

Sounds like a JExample bug to me, but I can't be sure.

    David Saff

On Tue, Dec 22, 2009 at 1:29 AM, yogesh.patil <patilyogeshp@...> wrote:
>
> I am using JExample to make sure that all required test cases has been
> executed before executing the test case which is dependent on others.
>
> But it seems like @AfterClass annotation stops working while running test
> with JExample....
>
> At the same time @BeforeClass annotation works perfectly fine...
>
> Is it a bug in JExample or am I missing something?
>
> -----
> --
> Regards,
> Yogesh Patil.
> --
> View this message in context:
http://old.nabble.com/Use-of-JExample-for-maintaining-order-of-execution-for-tes\
ts-tp26877151p26877151.html
> Sent from the JUnit - User mailing list archive at Nabble.com.
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

On Sat, Dec 19, 2009 at 3:51 PM, cretz@... <chad.retz@...> wrote:
>
>
>
> I don't even have a @BeforeClass method in the subclass. Only in the
superclass. I figured it out, I was extending from TestCase also (which I know
isn't JUnit 4 style). If I don't extend TestCase, it works. Example base class:
>
> public abstract class MyTestBase extends TestCase {
>
>    protected static boolean setUpExecuted = false;
>
>    @BeforeClass
>    public static void oneTimeSetUp() {
>        setUpExecuted = true;
>    }
> }
>
> and example extension:
>
> public class MyTest extends MyTestBase {
>
>    @Test
>    public void testWhetherOneTimeSetUpExecuted() {
>        Assert.assertTrue("One time setUp not executed",
>                setUpExecuted);
>    }
> }
>
> I didn't check the JUnit source to see if it uses an instanceof to determine
whether to use JUnit 3 style or JUnit 4 style. My problem is I need to extend
from a third party lib (SeleniumTestCase) and use protected stuff in there. What
I'm gonna do is just have another delegating class extending it and expose the
protected object I need.
>
> So, I have solved it in a workaround fashion. Other people have too (ref:
http://rockhoppertech.com/blogs/archives/45). I was just wondering if this is by
intention?

Yes, if you extend TestCase, you get the same behavior that JUnit 3
would have given you.  To override that, you can annotate the subclass
or baseclass with @RunWith(JUnit4.class)

    David Saff

>
> --- In junit@yahoogroups.com, David Saff <david@...> wrote:
>>
>> Chad,
>>
>> Do the superclass and subclass @BeforeClass methods have the same name?
>>
>>    David Saff
>>
>> On Sat, Dec 19, 2009 at 6:13 AM, cretz@... <chad.retz@...> wrote:
>> > I have an abstract base class that defines a static @BeforeClass setUp
method. When this class is extended, the static setUp method is not called. The
javadoc says it will call a super class's @BeforeClass before the current
class's. Am I missing something?
>> >
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

I think the javadoc is correct.  If you have a failing test, please let me know.

    David Saff

On Sat, Dec 19, 2009 at 8:43 AM, B. Sai Prasad, M.Phil, SCJP, MCP,
MCTS, SSGB, PMP, PMI-SP <sai.prasad.b@...> wrote:
> I think the java documentation for @BeforeClass is wrong; static methods
> will not be inherited and so, can't be called your runner. - Sai
>
> On Sat, Dec 19, 2009 at 4:43 PM, cretz@... <chad.retz@...>wrote:
>
>>
>>
>> I have an abstract base class that defines a static @BeforeClass setUp
>> method. When this class is extended, the static setUp method is not called.
>> The javadoc says it will call a super class's @BeforeClass before the
>> current class's. Am I missing something?
>>
>>
>>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Hi Kent,

A few precisions:

On Wed, Dec 23, 2009 at 7:50 AM, Kent Beck <kentb@...> wrote:

> Tests are executed in an unspecified order. There is no way to define an
> order, because in my experience independent tests are much more valuable
> than order-dependent tests.
>
> Every time I've had order dependencies I have had a problem with the design
> of the underlying system. Investing my available time in improving the
> design has had a much higher payoff than investing that same time into
> shuffling tests around.
>

I found that if the testing framework takes care of removing certain
practical obstacles, dependent testing is actually very easy to implement
and to use.  For example, if method c() depends on b() and a() and you only
want to run c(), you should be able to say "run c()" and the framework will
figure out which dependencies need to be run.

My experience with TestNG showed me that the fears often associated with
test dependencies are very overstated, and dependent testing is actually
extremely useful for a lot of scenarios such as web testing (where
navigating from page to page adds state at each step, and having to recreate
that state from scratch would be very expensive).  Users of Selenium +
TestNG with dependencies can probably make a good case at giving you more
details on this.



>
> Independent tests enable interesting transformations like parallel
> execution or the kind of dynamic reordering that let JUnit Max deliver
> failures more quickly.
>

Dependencies and parallelism are actually not mutually exclusive at all.
  Here are a couple of articles I wrote that describe the multithreaded
topological sort algorithm that I implemented in TestNG:

http://beust.com/weblog/archives/000535.html
http://beust.com/weblog/archives/000536.html

This should be especially of interest to developers interested in hard
concurrency issues, but all this complexity is completely hidden away from
users.

With TestNG, you specify your dependencies (e.g. "@Test(dependsOnMethods =
"f")), then your thread pool size (thread-pool-size=15) and TestNG will
figure out how to run all your tests (both dependent and non dependent ones)
in the most efficient parallel manner by using the threads you supplied as
well as possible.

--
***Cédric
*


[Non-text portions of this message have been removed]

On 12/23/2009 03:08 PM, Cédric Beust ? wrote:
>
> Users of Selenium +
> TestNG with dependencies can probably make a good case at giving you more
> details on this.
>

I agree with Cédric. We use selenium, and the order and dependencies for
us is really important.

However, this is not unit testing. So I think it's understandable why
Junit doesn't have this features implemented.

Even though, I still prefer to use Junit framework. And we use ant to
solve dependency matters.
We still have some issues with ant tasks most probably because of my
lack of knowledge with the tool; but junit+selenium+ant works really
good for our integration tests.

pura vida.


[Non-text portions of this message have been removed]

Can we agree that when someone says test A "depends on" test B, they
may mean one of these, but not necessarily both?

1) If test A fails, test B can provide no additional information, so
it's a waste of time to run B, and a waste of the developers' time to
report B as a failure to investigate (efficiency)
2) Test A's execution changes global state in a way that test B needs
in order to run correctly.  (correctness)

If I had to guess, the original poster appears to be asking for
ordering for correctness, but Cedric's blog post seems to be handling
ordering for efficiency.

I'm happy to discuss my opinions on either or both, but first, do we
agree on the distinction?

    David Saff

2009/12/23 Cédric Beust ♔ <cbeust@...>:
> Hi Kent,
>
> A few precisions:
>
> On Wed, Dec 23, 2009 at 7:50 AM, Kent Beck <kentb@...> wrote:
>
>> Tests are executed in an unspecified order. There is no way to define an
>> order, because in my experience independent tests are much more valuable
>> than order-dependent tests.
>>
>> Every time I've had order dependencies I have had a problem with the design
>> of the underlying system. Investing my available time in improving the
>> design has had a much higher payoff than investing that same time into
>> shuffling tests around.
>>
>
> I found that if the testing framework takes care of removing certain
> practical obstacles, dependent testing is actually very easy to implement
> and to use.  For example, if method c() depends on b() and a() and you only
> want to run c(), you should be able to say "run c()" and the framework will
> figure out which dependencies need to be run.



>
> My experience with TestNG showed me that the fears often associated with
> test dependencies are very overstated, and dependent testing is actually
> extremely useful for a lot of scenarios such as web testing (where
> navigating from page to page adds state at each step, and having to recreate
> that state from scratch would be very expensive).  Users of Selenium +
> TestNG with dependencies can probably make a good case at giving you more
> details on this.
>
>
>
>>
>> Independent tests enable interesting transformations like parallel
>> execution or the kind of dynamic reordering that let JUnit Max deliver
>> failures more quickly.
>>
>
> Dependencies and parallelism are actually not mutually exclusive at all.
>  Here are a couple of articles I wrote that describe the multithreaded
> topological sort algorithm that I implemented in TestNG:
>
> http://beust.com/weblog/archives/000535.html
> http://beust.com/weblog/archives/000536.html
>
> This should be especially of interest to developers interested in hard
> concurrency issues, but all this complexity is completely hidden away from
> users.
>
> With TestNG, you specify your dependencies (e.g. "@Test(dependsOnMethods =
> "f")), then your thread pool size (thread-pool-size=15) and TestNG will
> figure out how to run all your tests (both dependent and non dependent ones)
> in the most efficient parallel manner by using the threads you supplied as
> well as possible.



>
> --
> ***Cédric
> *
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Hi David,

In my experience, these two cases are usually equivalent so the distinction
doesn't provide much value.  What's the point of declaring that b() depends
on a() if b() doesn't need any state created by a()?  By doing this, you are
causing your test run to needlessly take longer if you are running in
parallel (since b() can't be run until after a() has completed, while it
could be run in parallel if you didn't declare that dependency).

I'm going to think more about this and see if I can find a counter example
to prove my impression wrong.

--
***Cédric
*


On Wed, Dec 23, 2009 at 1:46 PM, David Saff <david@...> wrote:

>
>
> Can we agree that when someone says test A "depends on" test B, they
> may mean one of these, but not necessarily both?
>
> 1) If test A fails, test B can provide no additional information, so
> it's a waste of time to run B, and a waste of the developers' time to
> report B as a failure to investigate (efficiency)
> 2) Test A's execution changes global state in a way that test B needs
> in order to run correctly. (correctness)
>
> If I had to guess, the original poster appears to be asking for
> ordering for correctness, but Cedric's blog post seems to be handling
> ordering for efficiency.
>
> I'm happy to discuss my opinions on either or both, but first, do we
> agree on the distinction?
>
> David Saff
>
> 2009/12/23 Cédric Beust ♔ <cbeust@... <cbeust%40google.com>>:
>
> > Hi Kent,
> >
> > A few precisions:
> >
> > On Wed, Dec 23, 2009 at 7:50 AM, Kent Beck
<kentb@...<kentb%40earthlink.net>>
> wrote:
> >
> >> Tests are executed in an unspecified order. There is no way to define an
> >> order, because in my experience independent tests are much more valuable
> >> than order-dependent tests.
> >>
> >> Every time I've had order dependencies I have had a problem with the
> design
> >> of the underlying system. Investing my available time in improving the
> >> design has had a much higher payoff than investing that same time into
> >> shuffling tests around.
> >>
> >
> > I found that if the testing framework takes care of removing certain
> > practical obstacles, dependent testing is actually very easy to implement
> > and to use.  For example, if method c() depends on b() and a() and you
> only
> > want to run c(), you should be able to say "run c()" and the framework
> will
> > figure out which dependencies need to be run.
>
> >
> > My experience with TestNG showed me that the fears often associated with
> > test dependencies are very overstated, and dependent testing is actually
> > extremely useful for a lot of scenarios such as web testing (where
> > navigating from page to page adds state at each step, and having to
> recreate
> > that state from scratch would be very expensive).  Users of Selenium +
> > TestNG with dependencies can probably make a good case at giving you more
> > details on this.
> >
> >
> >
> >>
> >> Independent tests enable interesting transformations like parallel
> >> execution or the kind of dynamic reordering that let JUnit Max deliver
> >> failures more quickly.
> >>
> >
> > Dependencies and parallelism are actually not mutually exclusive at all.
> >  Here are a couple of articles I wrote that describe the multithreaded
> > topological sort algorithm that I implemented in TestNG:
> >
> > http://beust.com/weblog/archives/000535.html
> > http://beust.com/weblog/archives/000536.html
> >
> > This should be especially of interest to developers interested in hard
> > concurrency issues, but all this complexity is completely hidden away
> from
> > users.
> >
> > With TestNG, you specify your dependencies (e.g. "@Test(dependsOnMethods
> =
> > "f")), then your thread pool size (thread-pool-size=15) and TestNG will
> > figure out how to run all your tests (both dependent and non dependent
> ones)
> > in the most efficient parallel manner by using the threads you supplied
> as
> > well as possible.
>
> >
> > --
> > ***Cédric
> > *
> >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>


[Non-text portions of this message have been removed]

Hi Cédric,

The cases are quite different. For case 1, consider...

Test A verifies that addition works correctly.

Test B verifies that multiplication works correctly by
comparing the result of a multiplication operation to
the result of a number of additions.

Test B "depends" on Test A in the sense that it requires
functionality that is tested by Test A. However, it can
be run independently, since Test A does not create any
sort of state that Test B relies upon.

In the wild, this situation comes up quite frequently when
a class under test implements equality. If equality isn't
working, lots of tests involving objects of that class will
fail, but the real error is the one given by the test of
equality.

Charlie

> -----Original Message-----
> From: junit@yahoogroups.com [mailto:junit@yahoogroups.com] On
> Behalf Of Cédric Beust ?
> Sent: Wednesday, December 23, 2009 2:08 PM
> To: junit@yahoogroups.com
> Subject: Re: [junit] Order of execution for tests
>
> Hi David,
>
> In my experience, these two cases are usually equivalent so
> the distinction doesn't provide much value.  What's the point
> of declaring that b() depends on a() if b() doesn't need any
> state created by a()?  By doing this, you are causing your
> test run to needlessly take longer if you are running in
> parallel (since b() can't be run until after a() has
> completed, while it could be run in parallel if you didn't
> declare that dependency).
>
> I'm going to think more about this and see if I can find a
> counter example to prove my impression wrong.
>
> --
> ***Cédric
> *
>
>
> On Wed, Dec 23, 2009 at 1:46 PM, David Saff <david@...> wrote:
>
> >
> >
> > Can we agree that when someone says test A "depends on" test B, they
> > may mean one of these, but not necessarily both?
> >
> > 1) If test A fails, test B can provide no additional information, so
> > it's a waste of time to run B, and a waste of the
> developers' time to
> > report B as a failure to investigate (efficiency)
> > 2) Test A's execution changes global state in a way that
> test B needs
> > in order to run correctly. (correctness)
> >
> > If I had to guess, the original poster appears to be asking for
> > ordering for correctness, but Cedric's blog post seems to
> be handling
> > ordering for efficiency.
> >
> > I'm happy to discuss my opinions on either or both, but first, do we
> > agree on the distinction?
> >
> > David Saff
> >
> > 2009/12/23 Cédric Beust ? <cbeust@... <cbeust%40google.com>>:
> >
> > > Hi Kent,
> > >
> > > A few precisions:
> > >
> > > On Wed, Dec 23, 2009 at 7:50 AM, Kent Beck
> <kentb@...<kentb%40earthlink.net>>
> > wrote:
> > >
> > >> Tests are executed in an unspecified order. There is no
> way to define an
> > >> order, because in my experience independent tests are
> much more valuable
> > >> than order-dependent tests.
> > >>
> > >> Every time I've had order dependencies I have had a
> problem with the
> > design
> > >> of the underlying system. Investing my available time in
> improving the
> > >> design has had a much higher payoff than investing that
> same time into
> > >> shuffling tests around.
> > >>
> > >
> > > I found that if the testing framework takes care of
> removing certain
> > > practical obstacles, dependent testing is actually very
> easy to implement
> > > and to use.  For example, if method c() depends on b()
> and a() and you
> > only
> > > want to run c(), you should be able to say "run c()" and
> the framework
> > will
> > > figure out which dependencies need to be run.
> >
> > >
> > > My experience with TestNG showed me that the fears often
> associated with
> > > test dependencies are very overstated, and dependent
> testing is actually
> > > extremely useful for a lot of scenarios such as web testing (where
> > > navigating from page to page adds state at each step, and
> having to
> > recreate
> > > that state from scratch would be very expensive).  Users
> of Selenium +
> > > TestNG with dependencies can probably make a good case at
> giving you more
> > > details on this.
> > >
> > >
> > >
> > >>
> > >> Independent tests enable interesting transformations
> like parallel
> > >> execution or the kind of dynamic reordering that let
> JUnit Max deliver
> > >> failures more quickly.
> > >>
> > >
> > > Dependencies and parallelism are actually not mutually
> exclusive at all.
> > >  Here are a couple of articles I wrote that describe the
> multithreaded
> > > topological sort algorithm that I implemented in TestNG:
> > >
> > > http://beust.com/weblog/archives/000535.html
> > > http://beust.com/weblog/archives/000536.html
> > >
> > > This should be especially of interest to developers
> interested in hard
> > > concurrency issues, but all this complexity is completely
> hidden away
> > from
> > > users.
> > >
> > > With TestNG, you specify your dependencies (e.g.
> "@Test(dependsOnMethods
> > =
> > > "f")), then your thread pool size (thread-pool-size=15)
> and TestNG will
> > > figure out how to run all your tests (both dependent and
> non dependent
> > ones)
> > > in the most efficient parallel manner by using the
> threads you supplied
> > as
> > > well as possible.
> >
> > >
> > > --
> > > ***Cédric
> > > *
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> > >
> > >
> > > ------------------------------------
> > >
> > > Yahoo! Groups Links
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Hi Charlie,

On Wed, Dec 23, 2009 at 5:52 PM, Charlie Poole
<charlie@...>wrote:

>
>
> Hi Cédric,
>
> The cases are quite different. For case 1, consider...
>
> Test A verifies that addition works correctly.
>
> Test B verifies that multiplication works correctly by
> comparing the result of a multiplication operation to
> the result of a number of additions.
>
> Test B "depends" on Test A in the sense that it requires
> functionality that is tested by Test A. However, it can
> be run independently, since Test A does not create any
> sort of state that Test B relies upon.
>
> In the wild, this situation comes up quite frequently when
> a class under test implements equality. If equality isn't
> working, lots of tests involving objects of that class will
> fail, but the real error is the one given by the test of
> equality.
>
Yes, that's a good example.  I guess we could call it a layered dependency,
which is similar to the connection between unit and functional tests:  in
theory, you really only need to have functional tests to assert that the
system is working as expected, but in practice, unit tests verify that the
system works at a more primitive level and they can therefore help find
problems faster.

Using this analogy, it looks like all functional tests should implicitly
depend on unit tests, since it makes little sense to run functional tests if
the primitive functionalities of your code base (captured by unit tests) are
not working correctly.

--
***Cédric
*


[Non-text portions of this message have been removed]

Hi David,

> Can we agree that when someone says test A "depends on" test
> B, they may mean one of these, but not necessarily both?
>
> 1) If test A fails, test B can provide no additional
> information, so it's a waste of time to run B, and a waste of
> the developers' time to report B as a failure to investigate
> (efficiency)
> 2) Test A's execution changes global state in a way that test
> B needs in order to run correctly.  (correctness)

On another list, I identified the same variations of dependency but
with the variation that for case 1. We might run B anyway but could
highlight the error in A in some way. That approach doesn't require
ordering the tests in any particular way, which has some advantages.
It still promotes developer efficiency while not worrying about
execution efficiency.

Charlie

On Wed, Dec 23, 2009 at 6:09 PM, Charlie Poole
<charlie@...>wrote:

>
>
> On another list, I identified the same variations of dependency but
> with the variation that for case 1. We might run B anyway but could
> highlight the error in A in some way. That approach doesn't require
> ordering the tests in any particular way, which has some advantages.
> It still promotes developer efficiency while not worrying about
> execution efficiency.
>

Yes, we call these "soft dependencies".

If "b" has a hard dependency on "a" and "a" fails, "a" will be marked
"FAILED" and b "SKIPPED".

If "b" has a soft dependency on "a", "b" will still run even if "a" fails
but a warning could be displayed in the log.

Every time we have discussed this potential feature for TestNG, the
discussion ended up being non conclusive on the real interest and the
existing ability to mark certain methods as "alwaysRun" already fills a big
part of this scenario, so I always ended up punting on it, but I'd be happy
to hear if someone can come up with a good use case for soft dependencies...

--
***Cédric
*


[Non-text portions of this message have been removed]

Hi Cédric,

> > On another list, I identified the same variations of dependency but
> > with the variation that for case 1. We might run B anyway but could
> > highlight the error in A in some way. That approach doesn't require
> > ordering the tests in any particular way, which has some advantages.
> > It still promotes developer efficiency while not worrying about
> > execution efficiency.
> >
>
> Yes, we call these "soft dependencies".
>
> If "b" has a hard dependency on "a" and "a" fails, "a" will
> be marked "FAILED" and b "SKIPPED".
>
> If "b" has a soft dependency on "a", "b" will still run even
> if "a" fails but a warning could be displayed in the log.

This is an implementation-based view of the tests: i.e. they
will either be run or not run.

The distinction I was trying to make - and I think David as
well - is based on the cause and nature of the dependency.
For the sake of argument, I'll try to redefine what you're
calling Hard and Soft along those lines.

Hard: Test A does something (sets state) upon which B depends.
For example, A may write to a data store and B might try to
read the information back. Or A might create a complex object
and B might try to call a method on that object. If we allow
this sort of thing, then there is no choice but to run A first
and it makes sense to not run B at all if A fails. However,
I would not want a unit test framework to allow it.

Soft: Test A tests some functionality upon which some other
functionality, tested by B, depends. Therefore, if test A
fails, Test B will either fail as well or be inconclusive,
so nothing further can be learned from test B. In this case,
a testing framework could respond in a number of ways to
the dependency:
1) Run the tests in order, skipping B if A fails.
2) Run the tests in an arbitrary order, but skip B if A
already ran and failed.
3) Run the tests in an arbitrary order, not skipping any
of them but organize reporting in such a way that a failure
of A takes precedence over a failure of B.

As a framework author, I can see that allowing the user to
specify ordering is a simpler implementation. But I think
it's more interesting to imagine the user specifying the
nature of the dependencies and allowing the framework to
figure out a strategy.

Charlie

> Every time we have discussed this potential feature for
> TestNG, the discussion ended up being non conclusive on the
> real interest and the existing ability to mark certain
> methods as "alwaysRun" already fills a big part of this
> scenario, so I always ended up punting on it, but I'd be
> happy to hear if someone can come up with a good use case for
> soft dependencies...
>
> --
> ***Cédric
> *
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Hello:

     I'm trying to download junit code using SVN. I only get a empty folder
whithout any data and a message saying that revision 0 is downloaded.

     I'm using https://junit.svn.sourceforge.net/svnroot/junit

     ¿Can I donwload it from another url?

     Thanks.

now the junit move to the github.
You can use the git to clone the source from the github.

On 12/27/2009 08:12 AM, tferrerp wrote:
>
> Hello:
>
> I'm trying to download junit code using SVN. I only get a empty folder
> whithout any data and a message saying that revision 0 is downloaded.
>
> I'm using https://junit.svn.sourceforge.net/svnroot/junit
> <https://junit.svn.sourceforge.net/svnroot/junit>
>
> ¿Can I donwload it from another url?
>
> Thanks.
>
>



[Non-text portions of this message have been removed]

hi,

http://github.com/KentBeck/junit/downloads

good day,
nico





On Sun, Dec 27, 2009 at 1:12 AM, tferrerp <tferrerp@...> wrote:

>
>
> Hello:
>
> I'm trying to download junit code using SVN. I only get a empty folder
> whithout any data and a message saying that revision 0 is downloaded.
>
> I'm using https://junit.svn.sourceforge.net/svnroot/junit
>
> ¿Can I donwload it from another url?
>
> Thanks.
>
>
>


[Non-text portions of this message have been removed]

Hi Nico:

     I'm nice to meet you. I had used your url and I had download source code

     Thanks.




________________________________
From: Nicolas Dermine <nicolas.dermine@...>
To: junit@yahoogroups.com
Sent: Sun, December 27, 2009 7:54:53 AM
Subject: Re: [junit] Download source code junit

hi,

http://github.com/KentBeck/junit/downloads

good day,
nico





On Sun, Dec 27, 2009 at 1:12 AM, tferrerp <tferrerp@...> wrote:

>
>
> Hello:
>
> I'm trying to download junit code using SVN. I only get a empty folder
> whithout any data and a message saying that revision 0 is downloaded.
>
> I'm using https://junit.svn.sourceforge.net/svnroot/junit
>
> ¿Can I donwload it from another url?
>
> Thanks.
>
> 
>


[Non-text portions of this message have been removed]



------------------------------------

Yahoo! Groups Links






[Non-text portions of this message have been removed]

Thank you very much.




________________________________
From: ginkgo_w <ginkgo_w@...>
To: junit@yahoogroups.com
Sent: Sun, December 27, 2009 8:44:41 PM
Subject: Re: [junit] Download source code junit

 
now the junit move to the github.
You can use the git to clone the source from the github.

On 12/27/2009 08:12 AM, tferrerp wrote:
>
> Hello:
>
> I'm trying to download junit code using SVN. I only get a empty folder
> whithout any data and a message saying that revision 0 is downloaded.
>
> I'm using https://junit. svn.sourceforge. net/svnroot/ junit
> <https://junit. svn.sourceforge. net/svnroot/ junit>
>
> ¿Can I donwload it from another url?
>
> Thanks.
>
>

[Non-text portions of this message have been removed]







[Non-text portions of this message have been removed]

Charlie,

>As a framework author, I can see that allowing the user to
>specify ordering is a simpler implementation. But I think
>it's more interesting to imagine the user specifying the
>nature of the dependencies and allowing the framework to
>figure out a strategy.

I am very much with you on this one.  In my experience, there have been a lot of
times when functions under test within the same class depend upon each other,
and it would have been great to have the test harness know that information. 
When I was refactoring code for a new framework package, I would have loved to
have the UI to display non-dependent tests first as it would have shown me very
clearly the source of the problem.

Specifying only the relationships in code and leaving the display strategy to
the test harness would allow more flexible display and execution of these tests
based upon the user's goals rather than upon the dependencies themselves.  I
could imagine myself asking the UI only to display the least dependent
functions' errors and to hide all other errors for quick debugging on sweeping
optimizations.  I could also want to see the dependent tests in a tree in case I
am changing a single class and want to understand the effects of what I am
touching.  I could also want to see all tests in a single list with the least
dependent at the top and the most dependent on the bottom when running
regression tests.  As you can see, the chosen view depends more upon what I am
doing than upon the test dependencies themselves.

Finally, being able to list dependencies between test classes in addition to
test cases would also JUnit to report which components of the system depend upon
each other.  This simple extension would greatly increase the power of the
dependency models for larger systems (particularly regression tests).

Thank you,

Rocco

From: junit@yahoogroups.com [mailto:junit@yahoogroups.com] On Behalf Of Charlie
Poole
Sent: Thursday, December 24, 2009 12:09 AM
To: junit@yahoogroups.com
Subject: RE: [junit] Order of execution for tests



Hi Cédric,

> > On another list, I identified the same variations of dependency but
> > with the variation that for case 1. We might run B anyway but could
> > highlight the error in A in some way. That approach doesn't require
> > ordering the tests in any particular way, which has some advantages.
> > It still promotes developer efficiency while not worrying about
> > execution efficiency.
> >
>
> Yes, we call these "soft dependencies".
>
> If "b" has a hard dependency on "a" and "a" fails, "a" will
> be marked "FAILED" and b "SKIPPED".
>
> If "b" has a soft dependency on "a", "b" will still run even
> if "a" fails but a warning could be displayed in the log.

This is an implementation-based view of the tests: i.e. they
will either be run or not run.

The distinction I was trying to make - and I think David as
well - is based on the cause and nature of the dependency.
For the sake of argument, I'll try to redefine what you're
calling Hard and Soft along those lines.

Hard: Test A does something (sets state) upon which B depends.
For example, A may write to a data store and B might try to
read the information back. Or A might create a complex object
and B might try to call a method on that object. If we allow
this sort of thing, then there is no choice but to run A first
and it makes sense to not run B at all if A fails. However,
I would not want a unit test framework to allow it.

Soft: Test A tests some functionality upon which some other
functionality, tested by B, depends. Therefore, if test A
fails, Test B will either fail as well or be inconclusive,
so nothing further can be learned from test B. In this case,
a testing framework could respond in a number of ways to
the dependency:
1) Run the tests in order, skipping B if A fails.
2) Run the tests in an arbitrary order, but skip B if A
already ran and failed.
3) Run the tests in an arbitrary order, not skipping any
of them but organize reporting in such a way that a failure
of A takes precedence over a failure of B.

As a framework author, I can see that allowing the user to
specify ordering is a simpler implementation. But I think
it's more interesting to imagine the user specifying the
nature of the dependencies and allowing the framework to
figure out a strategy.

Charlie

> Every time we have discussed this potential feature for
> TestNG, the discussion ended up being non conclusive on the
> real interest and the existing ability to mark certain
> methods as "alwaysRun" already fills a big part of this
> scenario, so I always ended up punting on it, but I'd be
> happy to hear if someone can come up with a good use case for
> soft dependencies...
>
> --
> ***Cédric
> *
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>



[Non-text portions of this message have been removed]

It seems to me that a coverage tool like Clover could discover this information
dynamically with the user having to specify more meta-information about tests
that could be wrong and would need maintenance. Isn't removing tedium why we
have computers in the first place?

Kent

On Dec 29, 2009, at 2:28 PM, Pigneri, Rocco wrote:

> Charlie,
>
> >As a framework author, I can see that allowing the user to
> >specify ordering is a simpler implementation. But I think
> >it's more interesting to imagine the user specifying the
> >nature of the dependencies and allowing the framework to
> >figure out a strategy.
>
> I am very much with you on this one. In my experience, there have been a lot
of times when functions under test within the same class depend upon each other,
and it would have been great to have the test harness know that information.
When I was refactoring code for a new framework package, I would have loved to
have the UI to display non-dependent tests first as it would have shown me very
clearly the source of the problem.
>
> Specifying only the relationships in code and leaving the display strategy to
the test harness would allow more flexible display and execution of these tests
based upon the user's goals rather than upon the dependencies themselves. I
could imagine myself asking the UI only to display the least dependent
functions' errors and to hide all other errors for quick debugging on sweeping
optimizations. I could also want to see the dependent tests in a tree in case I
am changing a single class and want to understand the effects of what I am
touching. I could also want to see all tests in a single list with the least
dependent at the top and the most dependent on the bottom when running
regression tests. As you can see, the chosen view depends more upon what I am
doing than upon the test dependencies themselves.
>
> Finally, being able to list dependencies between test classes in addition to
test cases would also JUnit to report which components of the system depend upon
each other. This simple extension would greatly increase the power of the
dependency models for larger systems (particularly regression tests).
>
> Thank you,
>
> Rocco
>
> From: junit@yahoogroups.com [mailto:junit@yahoogroups.com] On Behalf Of
Charlie Poole
> Sent: Thursday, December 24, 2009 12:09 AM
> To: junit@yahoogroups.com
> Subject: RE: [junit] Order of execution for tests
>
> Hi Cédric,
>
> > > On another list, I identified the same variations of dependency but
> > > with the variation that for case 1. We might run B anyway but could
> > > highlight the error in A in some way. That approach doesn't require
> > > ordering the tests in any particular way, which has some advantages.
> > > It still promotes developer efficiency while not worrying about
> > > execution efficiency.
> > >
> >
> > Yes, we call these "soft dependencies".
> >
> > If "b" has a hard dependency on "a" and "a" fails, "a" will
> > be marked "FAILED" and b "SKIPPED".
> >
> > If "b" has a soft dependency on "a", "b" will still run even
> > if "a" fails but a warning could be displayed in the log.
>
> This is an implementation-based view of the tests: i.e. they
> will either be run or not run.
>
> The distinction I was trying to make - and I think David as
> well - is based on the cause and nature of the dependency.
> For the sake of argument, I'll try to redefine what you're
> calling Hard and Soft along those lines.
>
> Hard: Test A does something (sets state) upon which B depends.
> For example, A may write to a data store and B might try to
> read the information back. Or A might create a complex object
> and B might try to call a method on that object. If we allow
> this sort of thing, then there is no choice but to run A first
> and it makes sense to not run B at all if A fails. However,
> I would not want a unit test framework to allow it.
>
> Soft: Test A tests some functionality upon which some other
> functionality, tested by B, depends. Therefore, if test A
> fails, Test B will either fail as well or be inconclusive,
> so nothing further can be learned from test B. In this case,
> a testing framework could respond in a number of ways to
> the dependency:
> 1) Run the tests in order, skipping B if A fails.
> 2) Run the tests in an arbitrary order, but skip B if A
> already ran and failed.
> 3) Run the tests in an arbitrary order, not skipping any
> of them but organize reporting in such a way that a failure
> of A takes precedence over a failure of B.
>
> As a framework author, I can see that allowing the user to
> specify ordering is a simpler implementation. But I think
> it's more interesting to imagine the user specifying the
> nature of the dependencies and allowing the framework to
> figure out a strategy.
>
> Charlie
>
> > Every time we have discussed this potential feature for
> > TestNG, the discussion ended up being non conclusive on the
> > real interest and the existing ability to mark certain
> > methods as "alwaysRun" already fills a big part of this
> > scenario, so I always ended up punting on it, but I'd be
> > happy to hear if someone can come up with a good use case for
> > soft dependencies...
> >
> > --
> > ***Cédric
> > *
> >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
> [Non-text portions of this message have been removed]
>
>



[Non-text portions of this message have been removed]

Kent,

That's a good point, especially about the maintenance.  I hadn't thought of
doing something like that.  I'll have to give it a try next time I am in
development mode.

Rocco

-----Original Message-----
From: junit@yahoogroups.com [mailto:junit@yahoogroups.com] On Behalf Of Kent
Beck
Sent: Wednesday, December 30, 2009 12:09 PM
To: junit@yahoogroups.com
Subject: Re: [junit] Order of execution for tests

It seems to me that a coverage tool like Clover could discover this information
dynamically with the user having to specify more meta-information about tests
that could be wrong and would need maintenance. Isn't removing tedium why we
have computers in the first place?

Kent

On Dec 29, 2009, at 2:28 PM, Pigneri, Rocco wrote:

> Charlie,
>
> >As a framework author, I can see that allowing the user to
> >specify ordering is a simpler implementation. But I think
> >it's more interesting to imagine the user specifying the
> >nature of the dependencies and allowing the framework to
> >figure out a strategy.
>
> I am very much with you on this one. In my experience, there have been a lot
of times when functions under test within the same class depend upon each other,
and it would have been great to have the test harness know that information.
When I was refactoring code for a new framework package, I would have loved to
have the UI to display non-dependent tests first as it would have shown me very
clearly the source of the problem.
>
> Specifying only the relationships in code and leaving the display strategy to
the test harness would allow more flexible display and execution of these tests
based upon the user's goals rather than upon the dependencies themselves. I
could imagine myself asking the UI only to display the least dependent
functions' errors and to hide all other errors for quick debugging on sweeping
optimizations. I could also want to see the dependent tests in a tree in case I
am changing a single class and want to understand the effects of what I am
touching. I could also want to see all tests in a single list with the least
dependent at the top and the most dependent on the bottom when running
regression tests. As you can see, the chosen view depends more upon what I am
doing than upon the test dependencies themselves.
>
> Finally, being able to list dependencies between test classes in addition to
test cases would also JUnit to report which components of the system depend upon
each other. This simple extension would greatly increase the power of the
dependency models for larger systems (particularly regression tests).
>
> Thank you,
>
> Rocco
>
> From: junit@yahoogroups.com [mailto:junit@yahoogroups.com] On Behalf Of
Charlie Poole
> Sent: Thursday, December 24, 2009 12:09 AM
> To: junit@yahoogroups.com
> Subject: RE: [junit] Order of execution for tests
>
> Hi Cédric,
>
> > > On another list, I identified the same variations of dependency but
> > > with the variation that for case 1. We might run B anyway but could
> > > highlight the error in A in some way. That approach doesn't require
> > > ordering the tests in any particular way, which has some advantages.
> > > It still promotes developer efficiency while not worrying about
> > > execution efficiency.
> > >
> >
> > Yes, we call these "soft dependencies".
> >
> > If "b" has a hard dependency on "a" and "a" fails, "a" will
> > be marked "FAILED" and b "SKIPPED".
> >
> > If "b" has a soft dependency on "a", "b" will still run even
> > if "a" fails but a warning could be displayed in the log.
>
> This is an implementation-based view of the tests: i.e. they
> will either be run or not run.
>
> The distinction I was trying to make - and I think David as
> well - is based on the cause and nature of the dependency.
> For the sake of argument, I'll try to redefine what you're
> calling Hard and Soft along those lines.
>
> Hard: Test A does something (sets state) upon which B depends.
> For example, A may write to a data store and B might try to
> read the information back. Or A might create a complex object
> and B might try to call a method on that object. If we allow
> this sort of thing, then there is no choice but to run A first
> and it makes sense to not run B at all if A fails. However,
> I would not want a unit test framework to allow it.
>
> Soft: Test A tests some functionality upon which some other
> functionality, tested by B, depends. Therefore, if test A
> fails, Test B will either fail as well or be inconclusive,
> so nothing further can be learned from test B. In this case,
> a testing framework could respond in a number of ways to
> the dependency:
> 1) Run the tests in order, skipping B if A fails.
> 2) Run the tests in an arbitrary order, but skip B if A
> already ran and failed.
> 3) Run the tests in an arbitrary order, not skipping any
> of them but organize reporting in such a way that a failure
> of A takes precedence over a failure of B.
>
> As a framework author, I can see that allowing the user to
> specify ordering is a simpler implementation. But I think
> it's more interesting to imagine the user specifying the
> nature of the dependencies and allowing the framework to
> figure out a strategy.
>
> Charlie
>
> > Every time we have discussed this potential feature for
> > TestNG, the discussion ended up being non conclusive on the
> > real interest and the existing ability to mark certain
> > methods as "alwaysRun" already fills a big part of this
> > scenario, so I always ended up punting on it, but I'd be
> > happy to hear if someone can come up with a good use case for
> > soft dependencies...
> >
> > --
> > ***Cédric
> > *
> >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
> [Non-text portions of this message have been removed]
>
>



[Non-text portions of this message have been removed]



------------------------------------

Yahoo! Groups Links

Hi Kent,

> It seems to me that a coverage tool like Clover could
> discover this information dynamically with[out] the user having to
> specify more meta-information about tests that could be wrong
> and would need maintenance. Isn't removing tedium why we have
> computers in the first place?

As far as I know, coverage tools deal well with direct dependencies
but not with logical (interpretive) dependency.

For example, if we have

   Test A --> SUT A --> SUT B
   Test B --> SUT B

Even in this simple case, coverage analysis won't tell us anything
about the relationship between A and B, although we could make some
deductions by examining the results.

If there were some way to tell the framework "Test A depends
on functionality, which is tested directly by B" then I can
imagine several good outcomes:

1) The test result output could be made much clearer - by
emphasizing the B result when both tests had failed. Some
approaches I can think include subordinating the A result
to that of B, highlighting B, not showing A at all and
marking A as inconclusive.

2) Users would be discouraged from seeking annotations that
provide for direct ordering of tests or the actual "hard"
dependencies.

3) Less important, but as an optimization, the framework
could suppress running of A if B had already failed.

Charlie

> Kent
>
> On Dec 29, 2009, at 2:28 PM, Pigneri, Rocco wrote:
>
> > Charlie,
> >
> > >As a framework author, I can see that allowing the user to specify
> > >ordering is a simpler implementation. But I think it's more
> > >interesting to imagine the user specifying the nature of the
> > >dependencies and allowing the framework to figure out a strategy.
> >
> > I am very much with you on this one. In my experience,
> there have been a lot of times when functions under test
> within the same class depend upon each other, and it would
> have been great to have the test harness know that
> information. When I was refactoring code for a new framework
> package, I would have loved to have the UI to display
> non-dependent tests first as it would have shown me very
> clearly the source of the problem.
> >
> > Specifying only the relationships in code and leaving the
> display strategy to the test harness would allow more
> flexible display and execution of these tests based upon the
> user's goals rather than upon the dependencies themselves. I
> could imagine myself asking the UI only to display the least
> dependent functions' errors and to hide all other errors for
> quick debugging on sweeping optimizations. I could also want
> to see the dependent tests in a tree in case I am changing a
> single class and want to understand the effects of what I am
> touching. I could also want to see all tests in a single list
> with the least dependent at the top and the most dependent on
> the bottom when running regression tests. As you can see, the
> chosen view depends more upon what I am doing than upon the
> test dependencies themselves.
> >
> > Finally, being able to list dependencies between test
> classes in addition to test cases would also JUnit to report
> which components of the system depend upon each other. This
> simple extension would greatly increase the power of the
> dependency models for larger systems (particularly regression tests).
> >
> > Thank you,
> >
> > Rocco
> >
> > From: junit@yahoogroups.com [mailto:junit@yahoogroups.com]
> On Behalf
> > Of Charlie Poole
> > Sent: Thursday, December 24, 2009 12:09 AM
> > To: junit@yahoogroups.com
> > Subject: RE: [junit] Order of execution for tests
> >
> > Hi Cédric,
> >
> > > > On another list, I identified the same variations of dependency
> > > > but with the variation that for case 1. We might run B
> anyway but
> > > > could highlight the error in A in some way. That
> approach doesn't
> > > > require ordering the tests in any particular way, which
> has some advantages.
> > > > It still promotes developer efficiency while not worrying about
> > > > execution efficiency.
> > > >
> > >
> > > Yes, we call these "soft dependencies".
> > >
> > > If "b" has a hard dependency on "a" and "a" fails, "a" will be
> > > marked "FAILED" and b "SKIPPED".
> > >
> > > If "b" has a soft dependency on "a", "b" will still run
> even if "a"
> > > fails but a warning could be displayed in the log.
> >
> > This is an implementation-based view of the tests: i.e. they will
> > either be run or not run.
> >
> > The distinction I was trying to make - and I think David as
> well - is
> > based on the cause and nature of the dependency.
> > For the sake of argument, I'll try to redefine what you're calling
> > Hard and Soft along those lines.
> >
> > Hard: Test A does something (sets state) upon which B depends.
> > For example, A may write to a data store and B might try to
> read the
> > information back. Or A might create a complex object and B
> might try
> > to call a method on that object. If we allow this sort of
> thing, then
> > there is no choice but to run A first and it makes sense to
> not run B
> > at all if A fails. However, I would not want a unit test
> framework to
> > allow it.
> >
> > Soft: Test A tests some functionality upon which some other
> > functionality, tested by B, depends. Therefore, if test A
> fails, Test
> > B will either fail as well or be inconclusive, so nothing
> further can
> > be learned from test B. In this case, a testing framework could
> > respond in a number of ways to the dependency:
> > 1) Run the tests in order, skipping B if A fails.
> > 2) Run the tests in an arbitrary order, but skip B if A already ran
> > and failed.
> > 3) Run the tests in an arbitrary order, not skipping any of
> them but
> > organize reporting in such a way that a failure of A takes
> precedence
> > over a failure of B.
> >
> > As a framework author, I can see that allowing the user to specify
> > ordering is a simpler implementation. But I think it's more
> > interesting to imagine the user specifying the nature of the
> > dependencies and allowing the framework to figure out a strategy.
> >
> > Charlie
> >
> > > Every time we have discussed this potential feature for
> TestNG, the
> > > discussion ended up being non conclusive on the real interest and
> > > the existing ability to mark certain methods as
> "alwaysRun" already
> > > fills a big part of this scenario, so I always ended up
> punting on
> > > it, but I'd be happy to hear if someone can come up with
> a good use
> > > case for soft dependencies...
> > >
> > > --
> > > ***Cédric
> > > *
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> > >
> > >
> > > ------------------------------------
> > >
> > > Yahoo! Groups Links
> > >
> > >
> > >
> > >
> >
> > [Non-text portions of this message have been removed]
> >
> >
>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Agree with Charlie:  I think it's impossible for a framework to
automatically determine this kind of dependency.  Even assuming that the
framework would go inside the body of test methods and actually understand
what's going on in there, it's just not possible to infer from that
information which methods depend on which ones.

Explicit (@Test(dependsOnGroups = "init")) is good, and it opens the door
for additional tools to use this information in different ways, such as
showing a graph of dependencies or optimizing the thread pool allocation,
like I did in the article I posted earlier.

--
***Cédric
*

On Wed, Dec 30, 2009 at 12:11 PM, Charlie Poole <charlie@...
> wrote:

>
>
> Hi Kent,
>
> > It seems to me that a coverage tool like Clover could
> > discover this information dynamically with[out] the user having to
> > specify more meta-information about tests that could be wrong
> > and would need maintenance. Isn't removing tedium why we have
> > computers in the first place?
>
> As far as I know, coverage tools deal well with direct dependencies
> but not with logical (interpretive) dependency.
>
> For example, if we have
>
> Test A --> SUT A --> SUT B
> Test B --> SUT B
>
> Even in this simple case, coverage analysis won't tell us anything
> about the relationship between A and B, although we could make some
> deductions by examining the results.
>
> If there were some way to tell the framework "Test A depends
> on functionality, which is tested directly by B" then I can
> imagine several good outcomes:
>
> 1) The test result output could be made much clearer - by
> emphasizing the B result when both tests had failed. Some
> approaches I can think include subordinating the A result
> to that of B, highlighting B, not showing A at all and
> marking A as inconclusive.
>
> 2) Users would be discouraged from seeking annotations that
> provide for direct ordering of tests or the actual "hard"
> dependencies.
>
> 3) Less important, but as an optimization, the framework
> could suppress running of A if B had already failed.
>
> Charlie
>
>
> > Kent
> >
> > On Dec 29, 2009, at 2:28 PM, Pigneri, Rocco wrote:
> >
> > > Charlie,
> > >
> > > >As a framework author, I can see that allowing the user to specify
> > > >ordering is a simpler implementation. But I think it's more
> > > >interesting to imagine the user specifying the nature of the
> > > >dependencies and allowing the framework to figure out a strategy.
> > >
> > > I am very much with you on this one. In my experience,
> > there have been a lot of times when functions under test
> > within the same class depend upon each other, and it would
> > have been great to have the test harness know that
> > information. When I was refactoring code for a new framework
> > package, I would have loved to have the UI to display
> > non-dependent tests first as it would have shown me very
> > clearly the source of the problem.
> > >
> > > Specifying only the relationships in code and leaving the
> > display strategy to the test harness would allow more
> > flexible display and execution of these tests based upon the
> > user's goals rather than upon the dependencies themselves. I
> > could imagine myself asking the UI only to display the least
> > dependent functions' errors and to hide all other errors for
> > quick debugging on sweeping optimizations. I could also want
> > to see the dependent tests in a tree in case I am changing a
> > single class and want to understand the effects of what I am
> > touching. I could also want to see all tests in a single list
> > with the least dependent at the top and the most dependent on
> > the bottom when running regression tests. As you can see, the
> > chosen view depends more upon what I am doing than upon the
> > test dependencies themselves.
> > >
> > > Finally, being able to list dependencies between test
> > classes in addition to test cases would also JUnit to report
> > which components of the system depend upon each other. This
> > simple extension would greatly increase the power of the
> > dependency models for larger systems (particularly regression tests).
> > >
> > > Thank you,
> > >
> > > Rocco
> > >
> > > From: junit@yahoogroups.com <junit%40yahoogroups.com> [mailto:
> junit@yahoogroups.com <junit%40yahoogroups.com>]
> > On Behalf
> > > Of Charlie Poole
> > > Sent: Thursday, December 24, 2009 12:09 AM
> > > To: junit@yahoogroups.com <junit%40yahoogroups.com>
> > > Subject: RE: [junit] Order of execution for tests
> > >
> > > Hi Cédric,
> > >
> > > > > On another list, I identified the same variations of dependency
> > > > > but with the variation that for case 1. We might run B
> > anyway but
> > > > > could highlight the error in A in some way. That
> > approach doesn't
> > > > > require ordering the tests in any particular way, which
> > has some advantages.
> > > > > It still promotes developer efficiency while not worrying about
> > > > > execution efficiency.
> > > > >
> > > >
> > > > Yes, we call these "soft dependencies".
> > > >
> > > > If "b" has a hard dependency on "a" and "a" fails, "a" will be
> > > > marked "FAILED" and b "SKIPPED".
> > > >
> > > > If "b" has a soft dependency on "a", "b" will still run
> > even if "a"
> > > > fails but a warning could be displayed in the log.
> > >
> > > This is an implementation-based view of the tests: i.e. they will
> > > either be run or not run.
> > >
> > > The distinction I was trying to make - and I think David as
> > well - is
> > > based on the cause and nature of the dependency.
> > > For the sake of argument, I'll try to redefine what you're calling
> > > Hard and Soft along those lines.
> > >
> > > Hard: Test A does something (sets state) upon which B depends.
> > > For example, A may write to a data store and B might try to
> > read the
> > > information back. Or A might create a complex object and B
> > might try
> > > to call a method on that object. If we allow this sort of
> > thing, then
> > > there is no choice but to run A first and it makes sense to
> > not run B
> > > at all if A fails. However, I would not want a unit test
> > framework to
> > > allow it.
> > >
> > > Soft: Test A tests some functionality upon which some other
> > > functionality, tested by B, depends. Therefore, if test A
> > fails, Test
> > > B will either fail as well or be inconclusive, so nothing
> > further can
> > > be learned from test B. In this case, a testing framework could
> > > respond in a number of ways to the dependency:
> > > 1) Run the tests in order, skipping B if A fails.
> > > 2) Run the tests in an arbitrary order, but skip B if A already ran
> > > and failed.
> > > 3) Run the tests in an arbitrary order, not skipping any of
> > them but
> > > organize reporting in such a way that a failure of A takes
> > precedence
> > > over a failure of B.
> > >
> > > As a framework author, I can see that allowing the user to specify
> > > ordering is a simpler implementation. But I think it's more
> > > interesting to imagine the user specifying the nature of the
> > > dependencies and allowing the framework to figure out a strategy.
> > >
> > > Charlie
> > >
> > > > Every time we have discussed this potential feature for
> > TestNG, the
> > > > discussion ended up being non conclusive on the real interest and
> > > > the existing ability to mark certain methods as
> > "alwaysRun" already
> > > > fills a big part of this scenario, so I always ended up
> > punting on
> > > > it, but I'd be happy to hear if someone can come up with
> > a good use
> > > > case for soft dependencies...
> > > >
> > > > --
> > > > ***Cédric
> > > > *
> > > >
> > > >
> > > > [Non-text portions of this message have been removed]
> > > >
> > > >
> > > >
> > > > ------------------------------------
> > > >
> > > > Yahoo! Groups Links
> > > >
> > > >
> > > >
> > > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> > >
> >
> >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>


[Non-text portions of this message have been removed]