-------------------------------------------------------------
Attend the Newest Web Seminar -
Brought to you by Better Software Magazine & StickyMinds.com
-------------------------------------------------------------
Techniques to Optimize Your Performance-Test Planning
Register Now!       http://www.sqe.com/WB1006EM1
Date:               Tuesday, October 10, 2006, 2:00 PM EST
Featured Speakers:  Dale Perry,  Consultant,
                     Software Quality Engineering
                     Joe Fernandes, Director of Product Management,
                   Empirix
Sponsored by:       Empirix
Learn More:         http://www.sqe.com/WB1006EM1
-------------------------------------------------------------
During functional testing, we focus on the behavior of application
features and execution scenarios on an individual basis. During
performance-testing, we focus on the speed and capacity of the
application rather than functionality. Planning an effective
performance-test requires an understanding of the basic difference
between these two types of tests. For effective performance-tests,
we must create an operational profile defining the types of users
and the load each will put on the system. From a performance-perspective,
it is the effect of this mix of workloads and interactions that we seek
to understand.
During this Web seminar, Dale Perry, consultant for Software Quality
Engineering, will discuss how creating a representative load profile
is one of the key requirements for successful performance-test
planning and critical to effective tool usage. We will look at what
makes up a load profile and how to develop one for your project.
Whether your application is Web, client/server, mainframe, or even
embedded, the same concepts of operational profiles apply.
Later, Joe Fernandes of Empirix will discuss the importance of
adding a solid performance-test plan to your current test process.
Whether testing is done manually or leveraging automated test tools,
functional testing prior to application launch is, for the most part,
an accepted part of the development process. Good performance-testing
is far less common.  Whether it's due to lack of time, resources,
knowledge, or commitment, performance-test practices at many organizations
range from limited to non-existent.  During this session, Joe will discuss
performance-test execution and provide practical tips and techniques for
adding performance-testing to your current test process.
---------------------------------------------------------------
                   Software Quality Engineering
  Conferences * Training * eTraining * Consulting * eNewsletters
     StickyMinds.com * Better Software Magazine * Web Seminars
http://www.sqe.com
http://www.StickyMinds.com
http://www.BetterSoftware.com

My article "Test Driven Database Development" appears in the current issue
of TASSQuarterly magazine (http://www.tassq.org/quarterly/).  The issue
itself is available at
http://www.tassq.org/quarterly/docs/tassq_magazine-0609.pdf .  Just like you
can take a TDD-based approach to developing application code, you can do the
same thing with your database schema.  I personally think that it's time
that we start adopting quality practices for database development.  If you
work in an organization which considers data to be a corporate asset, and/or
you're implementing functionality in your database via stored procedures,
stored functions, or even OO code, then shouldn't you also have a regression
test suite in place?  Later this week at www.ddj.com they'll be posting my
November column which shares the results of a survey sharing which explored
the current state of data management within the IT community.  It wasn't a
pretty picture, and there is a clear need for improvement -- TDDD will
likely prove to be part of your improvement strategy.

As an aside, TASSQuarterly is a great magazine if you're interested in
testing and quality issues.  I highly recommend it.

- Scott
Scott W. Ambler
Practice Leader Agile Development, IBM
http://www-306.ibm.com/software/rational/bios/ambler.html

Every organization gets the process that it deserves.

Better Software magazine?s Feature Article
    By Randall Rice
    I am a big fan of risk-based testing. I have developed, used, and taught
    risk-based testing for the past eighteen years. Because we rarely have
    enough time to perform the level of testing we?d like, we should
    prioritize our tests. This is best done by understanding various product
    risks and letting those risks guide our work. However, we must remember
    that risk-based testing contains risks. We must not be lulled into a
    false sense of security when using this technique.

    Read More: http://www.stickyminds.com/BetterSoftware/magazine.asp

Looking for new development techniques to build better software?
Attend these new Development courses to improve your current skill set
and take back new techniques.

                      Practical Test Driven Development
                       http://www.sqe.com/trainingtdd

                         Agile Development Practices
                       http://www.sqe.com/trainingadp

Pair two courses in San Diego and save $250! Contact our Client Support
Group
at Group at sqeinfo@..., 904.278.0524, or 888.268.8770 for more
information.
***************************************************************************
***
                           TRAINING INFORMATION

2006 Schedule           http://www.sqe.com/locations.asp
Public Training         http://www.sqe.com/training.asp
Register Now            http://www.sqe.com/register

***************************************************************************
***

            Practical Test Driven Development (3 days)
     A Revolutionary Approach to Software Design and Programming
                  http://www.sqe.com/trainingtdd
           San Diego, California * November 6-8, 2006

-Practice using test-first design development methods
-Experience writing unit tests before writing production code
-Automate all unit testing with open-source xUnit
-Use the open source Fitnesse tool to automate acceptance testing
  during - not after - development
-Gain experience developing programs in small verifiable steps
  for better designs
-Use test driven development to add new functionality to
  applications without adding bugs
-Learn how to refactor (re-design) existing applications to
  make them more maintainable

Register Now!
---> http://www.sqe.com/register

***************************************************************************
***

               Agile Development Practices (2 days)
        How to Run a Successful Software Development Project
                  http://www.sqe.com/trainingadp
             San Diego, California * November 9-10, 2006

-Developing and confirming requirements with user stories
-Planning for small development iterations and internal releases
-Sizing and justifying agile development projects
-Employing automated unit and acceptance testing to drive
  project success
-Using continuous integration to insure working software every day
-Building cohesive development teams with agile project management
-Determining when and when not to use agile development practices

Register Now!
---> http://www.sqe.com/register

***************************************************************************
***

In early July I ran a survey for Dr. Dobb's Journal investigating the
current state of data management practices. Links to the article, a summary
presentation file, a summary of the questions and results, and a CSV file of
the source data (without email addresses) can be found at
http://www.ambysoft.com/surveys/dataManagementAugust2006.html . I've decided
to share the source data in the hopes that others will choose to analyze it
and publish their own findings.

Some highlights include:
1. 96% of organizations consider data to be a corporate asset, yet only 40%
have tests in place to validate that data and only 63% of those with tests
allow developers to run them. Of the orgs with a test suite in place, less
than 35% had even discussed the concept.
2. 64% of organizations implement mission critical functionality in the
database, yet only 46% had tests in place (66% allowed developers to run
them), and less than 40% had discussed the concept.
3. 62% had data quality problems in production, yet only one third seemed to
have a viable strategy in place to deal with the issue (e.g. refactor it
safely over time). I'm not sure that people really understood what database
refactoring is all about, so I'm exploring this issue in a follow-up survey.
I'm also exploring database testing in greater detail too because I suspect
those numbers are a bit optimistic.
4. In many organizations data professionals and developers were not working
well together: 66% said that developers sometimes needed to go around their
data groups. When developers do this about 25% of the time it's because they
didn't know better, so we have an education issue on our hands. 75% of the
time it appears that the data folks are perceived to be too hard to work
with, too slow to respond, or simply don't bring sufficient value to the
overall effort. Clearly we need to find ways to work together more
effectively (see www.agiledata.org for a few thoughts).

If we write high-quality code, but our data still suffers from quality
challenges, have we really done our jobs as IT professionals? I'm not so
sure. I'm a firm believer that data, and data management activities, are
important to the success of a software development project, regardless of
how agile or not agile that project is. If your software manipulates data
then clearly you need to consider data management issues. I believe that we
need to develop techniques and approaches to data management which are much
more agile than we currently see today.

- Scott
Scott W. Ambler
Practice Leader Agile Development, IBM
http://www-306.ibm.com/software/rational/bios/ambler.html

Every organization gets the process that it deserves.

Last night 60 Minutes did a piece on the "No Fly List" and the ongoing
data quality problems surrounding it.  If you missed the show there's a
write up at
http://www.cbsnews.com/stories/2006/10/05/60minutes/main2066624.shtml .

- Scott

I forwarded the original CFP a few weeks ago and just wanted to share this
reminder with everyone.  I've spoken at this conference before and can
highly recommend it.  I'd personally love to see talk proposals around agile
database techniques and agile modeling submitted.  We need to get the word
out, and quite frankly it can be great learning experience attending a
conference.

- Scott

> All,
> If you have already submitted an abstract for Collaborate '07 - thank you.
>
> I know that the deadline is not until November 17th - however, this year,
> we are trying to ensure that all of the topics of interest to Developer
> Track attendees are covered. To that end, we want to get as many abstracts
> in as early as possible so we can balance the number of topics covered and
> reach out to fill any gaps that we see in the program.
>
> I don't want to see more than 2-3 submissions per person, but I really
> would like to have some choices. Some of you are very good speakers but
> you might submit an abstract on a topic that is well covered elsewhere. It
> might save us some round trips if I have additional abstracts from you to
> select from.
>
> Please try to submit something in the next week or so using the following
> link:
> <http://w3.ioug.org/callforspeakers07>
>
> Paul
>
> Dr. Paul Dorsey
> Collaborate '07 Developer Track Co-FAM

Thanks, Scott, I'm looking forward to digging into this.

"If your software manipulates data", you say? I guess maybe I'm
naive, but is there software of any significance that doesn't? Or is
that your point, perhaps? Back in the day, they called our
profession DATA Processing...

Bill L



--- In agileDatabases@yahoogroups.com, "Scott W. Ambler" <swa@...>
wrote:
>
> In early July I ran a survey for Dr. Dobb's Journal investigating
the
> current state of data management practices. Links to the article,
a summary
> presentation file, a summary of the questions and results, and a
CSV file of
> the source data (without email addresses) can be found at
> http://www.ambysoft.com/surveys/dataManagementAugust2006.html .
I've decided
> to share the source data in the hopes that others will choose to
analyze it
> and publish their own findings.
>
> Some highlights include:
> 1. 96% of organizations consider data to be a corporate asset, yet
only 40%
> have tests in place to validate that data and only 63% of those
with tests
> allow developers to run them. Of the orgs with a test suite in
place, less
> than 35% had even discussed the concept.
> 2. 64% of organizations implement mission critical functionality
in the
> database, yet only 46% had tests in place (66% allowed developers
to run
> them), and less than 40% had discussed the concept.
> 3. 62% had data quality problems in production, yet only one third
seemed to
> have a viable strategy in place to deal with the issue (e.g.
refactor it
> safely over time). I'm not sure that people really understood what
database
> refactoring is all about, so I'm exploring this issue in a follow-
up survey.
> I'm also exploring database testing in greater detail too because
I suspect
> those numbers are a bit optimistic.
> 4. In many organizations data professionals and developers were
not working
> well together: 66% said that developers sometimes needed to go
around their
> data groups. When developers do this about 25% of the time it's
because they
> didn't know better, so we have an education issue on our hands.
75% of the
> time it appears that the data folks are perceived to be too hard
to work
> with, too slow to respond, or simply don't bring sufficient value
to the
> overall effort. Clearly we need to find ways to work together more
> effectively (see www.agiledata.org for a few thoughts).
>
> If we write high-quality code, but our data still suffers from
quality
> challenges, have we really done our jobs as IT professionals? I'm
not so
> sure. I'm a firm believer that data, and data management
activities, are
> important to the success of a software development project,
regardless of
> how agile or not agile that project is. If your software
manipulates data
> then clearly you need to consider data management issues. I
believe that we
> need to develop techniques and approaches to data management which
are much
> more agile than we currently see today.
>
> - Scott
> Scott W. Ambler
> Practice Leader Agile Development, IBM
> http://www-306.ibm.com/software/rational/bios/ambler.html
>
> Every organization gets the process that it deserves.
>

I certainly can't think of a good example of a system that doesn't
manipulate data although am open minded to the concept.  ;-)

- Scott

On Mon, October 9, 2006 8:47 pm, Bill Lewis said:
> Thanks, Scott, I'm looking forward to digging into this.
>
> "If your software manipulates data", you say? I guess maybe I'm
> naive, but is there software of any significance that doesn't? Or is
> that your point, perhaps? Back in the day, they called our
> profession DATA Processing...
>
> Bill L
>
Practice Leader Agile Development, IBM Rational
http://www-306.ibm.com/software/rational/bios/ambler.html

Refactoring Databases (
http://www.ambysoft.com/books/refactoringDatabases.html ) is now
available.

Yes, I much preferred it when the profession was called "data
processing' instead of "information technology" (for example) where we
go to school to study "computer science".  I wrote a lament on this,
combining it with some personal musings and statements about women in
computing (I suspect there are is a bigger percentage of us over-40
women in "data processing" since we didn't first have to decide to
major in a machine, for example).

If interested, see
http://www.tincat-group.com/mewsings/2006/01/data-movement.html

--dawn

Dawn M. Wolthuis
Tincat Group, Inc.

On 10/9/06, Bill Lewis <datamodel@...> wrote:
>
>
> Thanks, Scott, I'm looking forward to digging into this.
>
>  "If your software manipulates data", you say? I guess maybe I'm
>  naive, but is there software of any significance that doesn't? Or is
>  that your point, perhaps? Back in the day, they called our
>  profession DATA Processing...
>
>  Bill L

On Tue, October 10, 2006 5:56 pm, Adrian Walker said:
>
> Yes indeed.  So, how to go about that?

In the DDJ article, at
http://www.ddj.com/dept/architect/193104893?cid=Ambysoft , there is a
sidebar entitled "A New Vision for Data Management".  It basically repeats
some of the things that I've been saying for years at www.agiledata.org.
As you can see it's really radical ideas such as taking a collaborative
approach, improving your approach to education of both developers and data
folks, do continuous db testing, that data folks need to adopt
evolutionary approaches which reflect the approaches of developers, and
you need to adopt realistic strategies to actually fix production data
problems.  The survey explored these issues, and the current picture isn't
pretty.  There seems to be a good argument that the majority of
organizations need to do a bit of a rethink when it comes to data
management.

>
> In order to test the data quality, one needs to understand the data.
> Often,
> much of the knowledge needed is locked up in the applications.

That's the current situation, probably the result of the current approach.
  Gut feel tells me that it's not working so well, is it?  ;-)

But what if we developed the tests for our database in parallel to
developing the tests for our application code?  What if we developed each
test before the production code/schema which fulfills that test?  Wouldn't
the knowledge about the data, and the design for that matter, now be
captured in the tests?

Wouldn't people now be much more motivated to keep that information up to
date because the "design docs" (the tests) actually add real concrete
value to them?  Within the agile community we definitely have a track
record for keeping our developer tests in sync with our production code,
surely we could do the same with database tests?

Wouldn't we finally have a real chance at getting out of the "the design
is out of sync with the code anti-pattern" that we've been trapped in for
decades because we've now moved to a more valuable form of capturing the
design?


>
> For example, in   www.reengineeringllc.com/demo_agents/MedMine2.agent   ,
> much of the data is meaningless by itself.  It takes on meaning from the
> English rules that are run over it.
>
> So, I'd argue that agile rules in open vocabulary, executable English
> constitute a promising approach to agile data quality.  Particularly since
> they can be used to automatically generate and run complex SQL "under the
> covers", with results explained in English, at the business level.
>
> What do you think?

I think that it's an interesting approach but that we'd still need to
test.  The rules need to be testable as does the data, otherwise we
wouldn't be able to trust the results, would we?

- Scott
Practice Leader Agile Development, IBM Rational
http://www-306.ibm.com/software/rational/bios/ambler.html

Refactoring Databases (
http://www.ambysoft.com/books/refactoringDatabases.html ) is now
available.

???????????????????????????? ????????????? ??????
???????????

Looks like my email address is being spoofed.  The previous message
that went out from "me" had a virus attached.

Sorry about that.

- Scott

On Wed, October 11, 2006 10:43 am, Garris, Nicole said:
> Scott Ambler wrote:
>
> (snip)
> But what if we developed the tests for our database in parallel to
> developing the tests for our application code? What if we developed each
> test before the production code/schema which fulfills that test?
> Wouldn't
> the knowledge about the data, and the design for that matter, now be
> captured in the tests? (snip)
>
> My response:
>
> As a DBA, I've been considering your recommendation and wondering how I
> would test my databases. The bottom line is whether the database returns
> data correctly to the application.

That would be a black box testing point of view.  That's a part of the
overall testing picture, but it's not the entire picture.  I suspect that
we need to consider "white/clear box database tests" too.

> The physical database design is
> irrelevant in this respect, as long as it returns data correctly within
> an acceptable amount of time. So I would test:

Shouldn't the database owner be responsible for ensuring that the
internals of the database work, just like the owner of a component would
be responsible for testing the internals of that component?  Why are
databases special?

>
> *         Queries: Does the query return the correct data?
>
> *         Updates: These are tested by an update followed by a
> query(ies). After the update is performed, do the queries return the
> correct data?

Yes.  This would be black box, interface-level database testing.  A very
important thing to do, IMHO.


>
> *         Both of these tests require populating the database with test
> data, and checking the query results against expected results.
>
> I would argue that anything you can do in a database can and should be
> tested as described above. Furthermore, the above tests are just a
> subset of the usual application testing. Typically these tests are
> performed as a part of application testing-aren't they?

What if you have 100 apps hitting the same database?  Does it make sense
that 100 app teams have to write tests like that?  Perhaps they should
write some, but I suspect that there's a fair bit of overlap, particularly
if they're invoking stored procedures or other shared database access
code.

Furthermore, do you expect that those 100 app teams will fully test?  Or
test consistently?  For example, I may expect a query to respond with the
answer "Blue", you may expect it to respond "Red", who is right?  One of
us?  Both?  Neither?

>
> So I must be missing something.

Assume that you've chosen to implement RI rules via triggers in your
database.  Not exactly a radical strategy.  Someone comes along and drops
a delete trigger, either on purpose because they thought it was a good
idea or accidently.  Here's you choice:
1. You can have internal, clear box tests in place which validate the
implementation of the appropriate RI rules (e.g. that trigger you just
dropped).  Clearly you can write a test that does this.
2. You can have a black box test which only validates things at the
interface.  Hopefully you'd catch it after the data has been corrupted.
Clearly you can write a test like this.
3. Not do any database testing at all (which seems to be what a lot of
people do in practice).  Clearly you can not write a test this way. ;-)

Which is a safer approach?

Also, think of all the data quality problems that you've run into over
time.  Are you absolutely sure that you can catch those problems only with
black-box tests?  I bet you could, but I bet it would be a lot harder than
writing black box tests when they make sense and clear box tests when they
make sense.  My philosophy is use the best technique for the situation,
and I just don't think that black box tests are always the best option.

- Scott
Practice Leader Agile Development, IBM Rational
http://www-306.ibm.com/software/rational/bios/ambler.html

Refactoring Databases (
http://www.ambysoft.com/books/refactoringDatabases.html ) is now
available.

On Thu, October 12, 2006 12:31 pm, Garris, Nicole said:
> You've raised some interesting points. Thanks for responding.

My pleasure.

>
>
>
> I would argue that most DBAs do at least the equivalent of "unit
> testing" for stored procedures, triggers, and other code that they
> write. Rigorous test cases can be built for these objects (and test data
> is needed too). I (and I think most DBAs) also test the patches,
> updates, and configuration changes we apply to our databases.
>
>
>
> What puzzles me is how to test the structure of a table or index, or a
> database design in general.

Easy things would be existence tests -- has anyone dropped that index,
perhaps to do a table load, but forgotten to put it back?

> Of course I sight-verify these objects. I

That will only work for smaller dbs, but even for small ones you still
have a regression testing problem.


> also review the requirements against the objects after I've built them,
> and I check the DDL against the actual table structure, both of which
> are forms of testing. But how to rigorously test them in a manner that
> is analogous to building and running test cases and verifying actual
> against expected results-I don't know how to do that. Unless checking
> DDL against the table structure is the equivalent of verifying actual to
> expected results, but that just doesn't seem right to me.

Why not test column invariants?  If you have a day of the week column (OK,
goofy example), shouldn't you validate that you're only storing Monday,
Tuesday, ... in it?

Similarly, what about table invariants?  e.g. My start date must be before
my end date.

>
>
>
> And if I make a structural change to an existing table, then a
> completely thorough test of that change would test all 100 apps hitting
> that table. Of course these tests could be run intelligently, some apps
> probably access the same table columns in the same way and the redundant
> tests need not be run.

Gut feel tells me that if you make a structural change, and don't support
a transition period
(http://www.agiledata.org/essays/databaseRefactoring.html), then if those
apps have tests then more than a few tests will fail.  If they don't have
tests, then they've got it coming I suppose.

>
>
>
> I'm not sure whether the issue is white-box versus black-box testing (my
> opinion is that both have their place),

Mine too, which I why I think we need both.

> but whether the testing can be
> automated. I don't have any experience with this, but it seems very

If we can develop something like JUnit which tests Java, can't we develop
something to test our databases.

Or, perhaps something a little more pertinent, if Microsoft can add
database testing tools for SQL Server in the beta version of VSTS for Data
Professionals, couldn't we do that for other database products?

> useful to me to be able to start a regression test suite when I go home
> at night and come in the next day and pick up the results. I'm trying to
> think of how to do that with database DDL.

I'm more concerned with the actual schema, and the data within the
database, than with the DDL.  Having validated DDL is also a good thing of
course.

Better yet, if Java developers can take a TDD approach to application
code, and if they can do regression testing many times a day, why can't we
do that in the database too?  Why are Java programmers, or C# programmers,
or ... special?  Other than of course the seem to have better tools and
many of them have a testing mindset.

I think that we can raise the bar on the level of quality that we see in
the data world.

- Scott

On Thu, October 12, 2006 4:25 pm, Garris, Nicole said:
> My response:
>
> Verifying the validity of data (commonly stored in a database) is the
> job of a different specialty area-Data Quality.

I'm not as concerned about who does it as I am about it being done.  If
your organization, for whatever reason, decides to split tasks between
different roles then that's your business.  If a DQ person is assigned
this work, they still need to do it.

Instead of working separately, perhaps they should pair with a DBA?


> Most DBAs aren't tasked
> with this responsibility.

Perhaps they should be.   Agile programmers have found it to be very
effective to have fully tested source code.  Better yet, they usually take
a test-first approach to development (see
http://www.agiledata.org/essays/tdd.html ).  If programmers are doing that
sort of stuff when they write application code, shouldn't the person(s)
creating database schema be responsible for testing their work too?


> So I won't be checking the data in my
> databases. My job is to apply database-specific constraints (such as
> referential integrity) and to work with the developer when a constraint
> fails to apply. Beyond that, data quality is not my responsibility, and
> I have no mandate to perform the function.

That's one way to work.  I think that there are much better ways to work
where everyone on the project should be concerned with the quality of what
they're producing.  Taking a collaborative, test-first approach seems to
work well.


>
> My feeling is that Data Quality is an extremely important job unto
> itself requiring deep knowledge of the data and the business rules
> regarding the data.

Agreed.

> There are different kinds of DBAs, but in my job I
> don't have access to business data knowledge.

Perhaps you should have that access.  Would it enable you to do a better job?

> I serve developers, not
> end users. Data Quality folks probably need a close working relationship
> with end users who provide and use the information, to help providers
> improve their quality, and understand from the information users which
> quality problems have the biggest impact on the organization. They would
> also work with developers and DBAs to build quality checks into the
> applications and databases.

In agile development we work closely with our stakeholders (end users +
others).  XP includes the concept of "On Site Customer" and Agile Modeling
takes it one step further with Active Stakeholder Participation (
http://www.agilemodeling.com/essays/activeStakeholderParticipation.htm ).
So we have access to those folks on a regular basis (one of the 12 agile
principles is to interact with them daily).  It's a really smart way to
work because you can do work which actually reflects their true needs.

- Scott
Practice Leader Agile Development, IBM Rational
http://www-306.ibm.com/software/rational/bios/ambler.html

Refactoring Databases (
http://www.ambysoft.com/books/refactoringDatabases.html ) is now
available.

On Thu, October 12, 2006 6:13 pm, Garris, Nicole said:
> I'm not sure why you're suggesting that the DBAs perform Data Quality
> work, rather than the developers, or the end users, or a job function
> called Data Quality.

What I'm saying is that the work needs to be done, and that it's probably
a good idea for the people who create the schema to try to do some of the
work to validate that they've done a good job of it.  Experience with
agile programmers seems to show that this approach works for them, and I
suspect that it could also work for data professionals too.  Maybe you
should try it and see what happens.

> As a DBA, my stakeholders are the developers. I try
> to work collaboratively with them and reflect their true database needs
> in my work. I don't typically have contact with the application users or

That's a pretty serious issue that's likely hampering your ability to
produce work that meets their true needs.

Working closely with developers is a very good thing.  Working closely
with developers and stakeholders is an even better thing.

> data users. Data Quality is not just a slight extension of my current
> job description, I would need management to give me these additional
> responsibilities, requisite authority, and additional staffing. If I

Sounds to me that your organization may have some organizational
challenges.  ;-)

You may not be able to accomplish some of the things that I'm talking
about in your current environment due to work restrictions such as you
describe.  You can only hope to do the best that you can do.


> took on Data Quality with current staffing levels I would have to put
> aside DBA in favor of the new job. Don't think my boss would like that.

Or perhaps they'd like the idea that a better overall job was being done
as a resort.  I don't know your boss, so it's hard to say.  Why don't you
consider pointing them at
http://www.ddj.com/dept/architect/193104893?cid=Ambysoft and ask them how
well your organization would have faired on that survey, and if they're
willing to do anything about it?

- Scott
Practice Leader Agile Development, IBM Rational
http://www-306.ibm.com/software/rational/bios/ambler.html

Refactoring Databases (
http://www.ambysoft.com/books/refactoringDatabases.html ) is now
available.

On Thu, October 12, 2006 7:32 pm, Garris, Nicole said:
>
> I don't think testing data values necessarily tests the DBA's work. It
> usually tests application edits and whether the users enter correct
> data.

If your database schema holds that data, I'd be interested in writing
tests that validated that the right thing was happening (and that the
wrong things weren't).

>
> But my stakeholders are the developers. They understand the true needs
> of the end users, don't they? Shouldn't they?

I should hope so.  But wouldn't it be better to get that information from
the actual end users themselves?

I suspect that you're struggling with some of these ideas because your
organization has chosen to organize their staff into well-defined roles.
You do this and someone else does that.  On agile projects you do this and
that and whatever else needs to get done.

- Scott
Practice Leader Agile Development, IBM Rational
http://www-306.ibm.com/software/rational/bios/ambler.html

Refactoring Databases (
http://www.ambysoft.com/books/refactoringDatabases.html ) is now
available.

--- In agileDatabases@yahoogroups.com, "Garris, Nicole"
<Nicole.Garris@...> wrote:
>
> Scott Ambler wrote:
>
> (snip)
> But what if we developed the tests for our database in parallel to
> developing the tests for our application code? What if we developed
each
> test before the production code/schema which fulfills that test?
> Wouldn't
> the knowledge about the data, and the design for that matter, now be
> captured in the tests? (snip)
>
> My response:
>
> As a DBA, I've been considering your recommendation and wondering
how I
> would test my databases. The bottom line is whether the database
returns
> data correctly to the application. The physical database design is
> irrelevant in this respect, as long as it returns data correctly
within
> an acceptable amount of time. So I would test:
>
> *         Queries: Does the query return the correct data?
>
> *         Updates: These are tested by an update followed by a
> query(ies). After the update is performed, do the queries return the
> correct data?
>
> *         Both of these tests require populating the database with
test
> data, and checking the query results against expected results.
>
> I would argue that anything you can do in a database can and should
be
> tested as described above. Furthermore, the above tests are just a
> subset of the usual application testing. Typically these tests are
> performed as a part of application testing-aren't they?
>
> So I must be missing something.
>

I think that DBUnit can be a good candidate for some kinds of db
automated tests.
For those who want to know more about DBUnit I recommend a start
point: http://dbunit.sourceforge.net/resources.html

Garris, Nicole wrote:
> I'm not sure why you're suggesting that the DBAs perform Data Quality
> work, rather than the developers, or the end users, or a job function
> called Data Quality. As a DBA, my stakeholders are the developers. I try
> to work collaboratively with them and reflect their true database needs
> in my work.

Following this whole thread, a thought occurred to me: the relation
between a DBA and a developer is more a provider-client relationship
than a "direct colleague" relationship.
In XP, clients can also write tests: acceptance tests. I think
acceptance tests more resemble the situation than unit tests. The
technical implementation may still be the same, but:
- You'd accept a test written in a different system (FIT, or JUnit
instead of SqlUnit)
- You would write the test together. Like a developer would help a
client with technical details, a DBA would help a developer with
database design or usage issues.
- You would decide the strictness or flexibility together. You may
decide that a certain structure would be good, but too expensive. Or
that developers should restrict themselves in the way calls are made
(not to refer to indexes, or to use only stored procedures, for
instance). Or that structure should be improved in the sense of
security, performance, etc.
- As the acceptance tests of the database can be unit tests in
applications, these tests can be shared using a source code control
system. This way, all applications that use a particular database, use
the same code and have the same documentation (=unit tests). This could
prevent applications from "interpreting" a data structure differently.

Best regards,
Willem Bogaerts

i attached the details.

Thank you

Note: forwarded message attached.