Hello Franck,

Hmm. The design gets ever more ambitious. I am
concerned that if we take on too much we will not
finish. But it would be good to get this right.

So let us leave the transaction cache in
SConnection, or at least in the one default
SDataSet in SConnection. Finders are on
SConnection are just a short cut to the
underlying data set. Let's rename SConnection to
be SCon as it will be very commonly used in the new API.

SCon.begin();

Department sales = SCon.findOrCreate(Departement.meta, "100");

List<Employee> inDeptX = SCon.query(new
SQuery(Employee.meta).eq(DEPARTMENT, sales));

List<Employee> inDeptX = SCon.query(new
SQuery(Employee.meta).eq(DEPARTMENT, sales));

SDataSet myDS = SCon.detachAll();

...

List<Employee> inDeptX2 = myDS.query(new
SQuery(Employee.meta).eq(DEPARTMENT, sales));

myDS.create(new Employee().set(Employee.ID,
"123").set.....); // not part of any transaction

...
SCon.attach(myDS);


Not that much worse than before, and allows the
data set to be queried when detached. Having
only One SDataSet per SConnection makes the
behavior of flush() and commit() well defined.

You are right that it is probably OK just to
always return a list in one hit, and hide
SResultSet and SPreparedStatement completely.
If people need to go one record at a time for
some special reason then they can just get the
raw JDBC driver from SConnection and do it manually.
Best to return a List rather than an Iterator --
avoids hasNext issues and is more flexible and really no more expensive.

(Ideally the SDriver routines should be split
into a third .jdbc jar that is dependent on the
.record jar but not on the .connection jar. But
let us leave that alone for now.)

Wow a big clean up.

We need to walk this through carefully. Probably as follows:-

1. svn copy frank frankPreSplit -- just in case.

2. Rename SConnection to SCon, everywhere.

3. Create methods SCon.query, SCon.find etc. that
just call back to SRecordMeta etc. Update all
code and examples to use them (bigest job.)

4. Move code out of SRecordMeta into SCon (or
better as static methods in SRecordFinder to avoid clutter).

5. Hide (make package local) or remove SResultSet
and SPreparedStatement. Return Lists instead.
(No need for raw SQL option -- raw SQL
fragments can be poked into SQuery, or just call
underlying JDBC driver. Keep SimpleOrm Simple.)
Update all code and examples to use this.
I would also like to automatically flush
before any query is executed. Cheap (needs doing
anyway) and avoids nasty issues about cache consistency.

6. Create new package SimpleOrm.record and move
SRecord*, SField* into it. Update all applications and examples.

7. Cut dependencies and change build.xml to build
two distinct jars, SimpleormRecord.jar and
SimpleormConnection. The latter dependent on the former.

8. Rework SQuery to build a query tree structure.
Implement SDriver methods to convert this to an SQL Query.
Things like Joins will need thought.

9. Write SDataSet query engine (ie. that can query a detached data set).

10. Implement SCon.detachDataSet. and
SCon.attacheDataSet. These need careful
thought. Get rid of other single record detachment -- all or none.

11. Fix up the dozens of loose end, such as generators.
And automatically finding references when detached etc.

Phew. That is a lot. But it would be much
better when done. If we only get as far as step
7 we have still done most of the visible changes
to ordinary code, and got the basic split.

I am busy tomorrow.

This will take at least a full week of each of our time. Do you have it?

Anthony

At 12:24 AM 1/07/2008, Franck Routier wrote:

>Anthony,
>
>maybe we should consider introducing SDataSet (or SDataSetManager ?)
>right now.
>First it would be equivalent to SConnection's transaction cache and
>could NOT be detached "per se".
>But maybe later, it could be.
>
>This could allow to have the following API :
>
>SQuery<Employee> qry = new SQuery(Employee.meta).eq(EMPEE_ID, "123");
>
>SConnection.attach(datasource);
>SConnection.begin();
>SDataSet ds = SConnection.getDataSet();
>Set<Employee> rs = ds.execute(qry);
>
>or a shortcut
>
>Set<Employee> rs = SConnection.execute(qry);
>and SConnection would delegate to its internal SDataSet implementation.
>
>This would be consistant with the idea of applying simple queries to
>SDataSet outside of a database (detached). So in the future, we could do
>this, without breaking the API :
>
>SDataSet ds = SConnection.detachDataSet();
>...in another tier...
>Set<Employee> rs = ds.execute(qry);
>
>Also SFinder could be a subclass of SQuery. And we could do :
>
>SFinder<Employee> finder = new SFinder(Employee.meta);
>finder.setPirmaryKey("123");
>Employee emp = ds.mustFind(finder); // throws exception if not found
>or
>Employee emp = ds.find(finder); // return null if not found
>or
>Set<Employee> emp = ds.execute(finder); // generic execute, return an
>empty set or a single value set
>
>and also :
>
>ds.findOrCreate(finder);
>
>I don't know if we need a SResultSet. Maybe it could remain internal to
>Simpleorm, or at least to the .database part.
>
>Does it make sense ?
>
>Regarding my time, as everybody, I have a ton of important things to
>do :), but I will definitely find some time to get Sorm 3 right !
>
>Regards,
>Franck
>
>Le lundi 30 juin 2008 à 20:44 +1000, Anthony & Melissa Berglas a écrit :
> > Hello Frank,
> >
> > Moving SQuery out is possible. One would have to
> > store the built query as a structure rather than
> > a query string, and then translate to string in .database. eg.
> > eq(this, that) {
> > operationsList.add(new Eq(this, that));
> > }
> > A little bit of extra overhead, but doable. Much
> > better to have query SQL built in SDriver.
> >
> > What would the API look like? Maybe
> >
> > SQuery<Employee> qry = new SQuery(Employee.meta).eq(EMPEE_ID, "123");
> > SResultSet<Employee> rs = new SResultSet(qry).execute();
> > and maybe
> > Employee[] emps = new SFinder(qry).execute();
> >
> > (Cannot have qry.execute() because that crosses the boundary.)
> >
> > Having SQuery in the record part would also mean
> > that one could later apply simple queries to a
> > DataSet (ie.list) of records, outside a database which would be cool.
> >
> > We could also choose to keep the existing API if
> > we have a database accessor interface written in
> > .record but implemented in .database. But we
> > then need to keep a pointer to the
> > implementation, either in a static or another
> > thread local. This produces a slightly neater
> > API but is much less transparent.
> >
> > What do you think? Please propose an API style that would suit you.
> >
> > Going forward, do you have time this week? I
> > would like to start by cutting out the finder
> > code in SRecordMeta, which is an internal change.
> >
> > Sorry about the non-public method. If we split
> > this then all the meta data will need to be
> > public so that the .database part can access it!
> >
> > Regards,
> >
> > Anthony
> >
> > At 07:24 PM 30/06/2008, Franck Routier wrote:
> > >Hello Anthony,
> > >
> > >I totally second this separation. This makes things much cleaner, and
> > >opens the possibility for future evolutions.
> > >
> > >However, I am not sure I want SQuery to be in the database part...
> > >Right now, I build a lot of queries in my business layer, depending
> > on
> > >business rules, and then execute them in Dao's.
> > >Would it be possible to keep SQuery quite conceptual, and let
> > >SConnection/SDriver implement the DB specific part. This would also
> > make
> > >possible to have some different SConnection implementation, say
> > towards
> > >a ldap repository, etc. (well, I know SQuery join and so on are
> > pretty
> > >RDBMS oriented...)
> > >
> > >This would make SQuery kind of a custom query language, which you
> > would
> > >rather avoid, as said in the whitepaper. But we could keep it as it
> > is
> > >now, that is a 'tiny' layer over jdbc AND potentialy other stuffs :)
> > >Would you consider this option ?
> > >
> > >By the way, introducing references to SConnection.getDriver() in
> > SQuery
> > >(for quoting) broke my app. As said before, I tend to build my
> > queries
> > >outside connections. So I temporay commented it out. The option I
> > would
> > >go for is to translate SQuery into sql latter on, in
> > SPreparedStatement
> > >for example. What is your opinion on this ?
> > >
> > >Also be careful to preserve API visibility, at least for read-only
> > >getters. Restricting it to package visibility also breaks a lot of my
> > >code for dynamically building queries.
> > >
> > >
> > >That said, I really like your idea.
> > >
> > >Franck
> > >
> > >
> > >Le lundi 30 juin 2008 à 15:56 +1000, Anthony Berglas a écrit :
> > > > Hello Franck,
> > > >
> > > > I think that as part of 3.0 we should consider splitting SimpleOrm
> > > > into two pieces, each in their own Jars.
> > > >
> > > > simpleorm.records that contains SRecordMeta, SRecordInstance,
> > SField*
> > > >
> > > > simpleorm.database that contains SQuery, SDriver*, SConnection.
> > > >
> > > > The record part should have zero dependency on the database part,
> > and
> > > > should be compiled and built into its own jar, with some test
> > cases of
> > > > its own.
> > > >
> > > > The database part should depend on the records part, obviously.
> > > >
> > > > When working detached, one should only need the .records jar.
> > > >
> > > > This is actually not that hard to do. I have already decoupled
> > > > SQuery from SRecordMeta. A similar trick can be used to decouple
> > > > findOrCreate and friends. And the only other substantial
> > connection
> > > > is where SReferenceField.getRawValue can do a lazy read.
> > > >
> > > > For findOrCreate, the API becomes slightly messier:-
> > > >
> > > > SFinder empf<Employee> = new SFinder(Employee.meta);
> > > > Employee e = empf.findOrCreate("123");
> > > >
> > > > or simply but slightly less efficiently
> > > >
> > > > Employee e = new SFinder(Employee.meta).findOrCreate("123");
> > > >
> > > > (The finder would cache the sets of different types of fields
> > etc.)
> > > >
> > > > vs old
> > > >
> > > > Employee e = Employee.meta.findOrCreate("123");
> > > >
> > > > For the SFieldReference.get the call would become
> > > >
> > > > SFinder deptf<Department> = new SFinder(Department.meta);
> > > > Department dept = deptf.getReference(myEmp, DEPT);(
> > > >
> > > > (For the detached case, no query:-)
> > > > Department dept = myEmp.getUnqueriedReference(DEPT);
> > > >
> > > > One could, of course, give SRecordMeta a default finder, defined
> > > > through an interface in .SRecord, and then keep the existing
> > > > API. But being explicit makes the structure clear.
> > > >
> > > > I have also attached some other prototyping that I was doing in
> > > > SimpleData (it is checked in). But I now think that splitting
> > > > SimpleOrm is a much better idea, and SimpeData should be
> > > > abandoned. I think that this change is actually very good, and
> > would
> > > > lead naturally to DataSets if we wanted to go there later.
> > > >
> > > > My current task is to work on the white paper etc. I would not
> > want
> > > > to start any split until you have caught up with the current
> > changes
> > > > and your application works 100%. (Although I might be tempted to
> > do
> > > > some of the internal rearrangement, moving code from SRecordMeta
> > to
> > > > SFinder.)
> > > >
> > > > One way or the other I am hoping to make your branch the trunk
> > this
> > > > week.
> > > >
> > > > What do you think?
> > > >
> > > > Anthony
> > > >
> > > > Dr Anthony Berglas,
> <mailto:anthony%40berglas.org>anthony@... Mobile: +61 4 4838 8874
> > > > Just because it is possible to push twigs along the ground with
> > ones
> > > > nose
> > > > does not necessarily mean that is the best way to collect
> > firewood.
> > > >
> > > >
> > > >
> > > >
> >
> > Spreadsheet Detective,
> > Southern Cross Software Queensland Pty Limited
> > 54 Gerler Street
> > Bardon, Queensland 4065, Australia.
> >
> > Email:
> <mailto:berglas%40spreadsheetdetective.com>berglas@...
> > www.SpreadsheetDetective.com
> > Ph: +61 427 830248 (Australian Eastern Standard Time)
> >
> > "If the model seems correct only because the numbers look right,
> > then why build the model in the first place?"
> >
> >
> >
> >
> >
>
>

Dr Anthony Berglas, anthony@... Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that is the best way to collect firewood.