The short Fast and Filthy answer is...

Use valgrind see http://valgrind.org

On Wed, 3 Dec 2008, Alan Baljeu wrote:


> Problem: there is a bug, namely memory corruption. Demonstrating
> the bug requires elaborate setup and catching the access violation
> that shows up. Except for this situation, the function works quite
> well, although it has no unit tests.

The realist in me says... There is a bug... that he knows about. And
maybe a couple he doesn't!

Part of the problem of MegaFunctions is don't really know exactly what
they should be doing. If you could write down in few understandable
lines.. odds on the function would not many more lines than
that. ie. The very notion "this function does not behave according to
spec" is fuzzy because the spec is fuzzy.

> Writing good tests may be
> complicated because the datastructure it takes and produces is
> large, and there are many cases to cover.

Break up your notion of "test" into the following varieties...

Precondition asserts - If you know, by some line in the megafunction,
that certain assumptions must hold, "executably document"
them. ie. Write an assert that expresses and tests that
assumption. Explicitly label that assert as a precondition
assert. (Yes, yes, I know that precondition asserts are usually on the
parameters of a function, give me time and you will see where I'm
heading.)

Invariant checks - You are clearly maintaining some fairly complex
data structure(s?). Create a function that that checks that data
structure. Inevitable such a data structure is not just a "Bag of
Bits". The bits in a data structure (if it is in a Valid State) can never
have just any old value. There are range constraints, and parts that
are constrained to vary in some set relation with other
parts. "Executably document" those constraints in function that checks
all those constraints and throws a wobbly if any are violated.

Invoke that invariant check _at every point_ in the where you believe
the data structure to be in "a Valid State".

> Fixing the bug requires dramatic changes to the function. Careless
> changes will break the function. Careful changes might not break
> anything but it's hard to tell because there are no tests.
> Interactively I could try stuff and see what happens. I'll have a
> good indication if things are okay or not, but this is the time
> consuming way.

I keep telling people around me there is nothing new about unit tests,
nothing strange. Programmers have _always_ turned on logging and, as
you say, "tried stuff and see what happens", and have always had a
"Good indication if things are not".

All I'm asking is that "Oracle", that knowledge you have when you look
at the log trace and say "Ah yes, that looks right", to be captured
executably as an assert.

So at those logging points you have now, add a "postcondition assert"
that checks that the effect of the above lines were as you expected.

Even if it is really dumb and one off like, "If we're running test
dataset 5, the result should be 3".

Yes, yes, I know "Postcondition asserts" are usually done at the end
of a routine. But humour me and label them "postcondition" and scatter
them liberally inside this mega routine.

Yes, yes, I know "If we are running test dataset 5, the result should
be 3" is really lame coding, you wouldn't want to see in production
code, but bear with me.

An excellent place to look is at comments, wherever possible try and
replace comments by "executable comments".

So there you are, without changing a single line of existing code,
only adding side effect free code, you have strongly braced it with
unit tests.

And if you have added a couple of post checks like "we should have
freed this memory by now", you may have found and fixed the bug by
this stage.

In which case you're finished, check in and close the issue.

> How would you proceed?

By now you probably have a fairly good idea of the data structures
involved. See if you can copy and paste largish chunks of the code
into an entirely new class(es?) that encapsulates the data
structure(s?).

Hopefully you already have chunks of the invariant for
that class written, you should be able to invoke that invariant check
at the start and end of _every_ public method for that class.

Some of the precondition asserts you have written probably can be
slipped into the front of the various methods on that class.

Some of the postcondition asserts you have written probably can be
slipped into the Unit Test of the various methods on that class.

(See, that lame "If we running test dataset 5" code doesn't look so
lame now! Now, we only put "result must equal 3" in the unit test case
where we're using dataset 5!)

Once the unit tests on that class is running and you're reasonably
happy, replacing chunks of the megafunction by references to that
class and methods on that class.

Replace the least bit that can be expected to compile and run
correctly, even if you have to do some silly copy stuff into the
class, and copy stuff out again.

Compile and run and get working again. It should be utterly unchanged.

Gradually eat away more and more of the code until everything that
changes or accesses the data structure encapsulated by the class you
extracted is done via the methods on the class.

Repeat as often as needed.

By now the megafunction is starting to look quite a bit smaller. The
invariants are now mostly in terms of invariants of the worker
classes.

Probably now you can crumble the whole mega function into a couple of
smaller functions.

Warning! There is probably an irreducible core to the function, don't
endlessly slave to reduce it. Accept that some stuff is just complex
and brace it well with pre and posts and invariants.


John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : john.carter@...
New Zealand