For those of you who didn't read the original thread:

As often happens a discussion about using or not using mocks sparked.
I decided to break out a different thread on a subject I think is
interesting. Using mock setup as a metric.

As a reply to an interesting post by James Carr I wrote this:

> I think when using mocks, absurd abmounts of setup code is an
> indication that your design could be improved. I have no concrete
> metrics for you but I would start looking if I can simplify my design
> if I require more than 10 or so lines of setup. Sometimes it is hard.
> Usually you are doing too much at the time and then srp can be your
> guide.

Now back to the discussion with comments in the reply:

> Joakim,
>
> You're correct. Yesterday when my partner and I were looking over the test
> and class, we realized that the class had taken on too much responsibility
> and broke it up into three seperate objects.

Hi James, from an interaction based testing perspective that is
usually the right thing to do when you see lots of setup code.

>
> The interesting thing was that one of the new tests (with methods refactored
> from the old) for the new objects didn't need any mocks, and the amount of
> setup for mocks across tests dropped dramatically.

Maybe you created a class that I tend to call a "leaf"-class. Classes
that have no colaborators. Those are the classes I generally use a
statebased approach to test. So in contrast to a common opinion: "use
mocks to mock out external resources, amen". I actually tend to use
statebasaed tests for "THE" class that is responsible for managing
resources on a filesystem for instance. While everything else that
needs resources "interact" with my resource-manager mock.

> So I guess this is really
> more of a code smell ... too many mocks or too much mock setup is an
> indicator that objects may be doing too much?

I firmly beleive that is the case. A collegue of mine (and a fellow
developer of rMock), Daniel Brolund, said something like "When you
need a lot of setup-code you have most certainly violated the SRP
principle". I can't recall a situation when that was wrong, I think
you could even remove the "most certainly" and it would still stand,
but since I'm not completely sure I tend to leave that "exit" in.

One thing that is for sure is that when doing interactionbased testing
is that you will end up in a design that has a lot of
"interaction-based" character. You will get something that is very
different than if you used a statebased approach. Some people don't
like what those designs look like. I like how they work and I agree
with what Nat Pryce wrote:

"I think that the most important benefit of interaction based testing
is that it helps reduce the amount of mutable state in a program"

That quote and a lot of good IB meat can be read here
http://nat.truemesh.com/archives/000342.html

I totally agree with this and I think that statebased tests tend to
drive too much mutable state into the code. Or rather, that is one
mistake you are likely to make with statebased tests. In contrast with
interactionbased tests you tend to make the mistake of having too many
and too specific expectations.
A lot of people think that Interactionbased tests tend to be brittle
and expose too much of the internals of a class. While that is a
correct assumption based on their experience it is not how it /has to/
be. Learning mocks is easy, /understanding/ how to use them properly
is harder.

We are toying with the thought of creating a "nose" in rMock. The nose
would be able to sniff upp smells in your tests like when you have too
many mocks or too many expectations. Here are some examples that we
could sniff for:
- number of mocks
- total number of expectations
- number of expectaions per mock
- stack depth before the mock (we need to eperimet a bit here but
possibly we could detect if you are testing too big chunks or too many
classes at the time)

Do you think learning how to ue mocks would be easier if such a
feature was in place?. rMock could fail the test if some metric had a
"bad" value and direct you to somewhere on the rMock site where the
smell is explained.

Any input here would be welcome.
Best regards!
Joakim Ohlrogge