Anthony,

I've done some work, namely steps 1 to 6, execpt for the 'flush before
query part'.
Lot of thinks are now public, including some fields. Bad.

Also SRecordMeta select methods should be moved to SDriver (or
SRecordFinder ?).

etc.

Franck


Le mardi 01 juillet 2008 à 16:46 +0200, Franck Routier a écrit :
> Hi Anthony,
>
> that seems fine to me, except for one point that is not that clear :)
>
> shouldn't "myDS.create(new Employee().set(Employee.ID,
> "123").set.....);" rather be called :
>
> myDS.attach(new Employee().set(Employee.ID, "123").set.....);
>
> Also, one should probably be able to attach a single record to SCon,
> without first having a SDataSet (or one must have a convinient way to
> create an empty SDataSet).
>
> So either :
>
> SCon.attach(new Employee().set(...));
>
> or
>
> SDataSet ds = new SDataSet(); // detached
> ds.attach(new Employee().set(...));
> SCon.begin();
> SCon.attach(ds);
>
> or both.
>
> We should probably also find another name for SCon.attach(DataSource).
> This is confusing. I propose SCon.bind(DataSource).
>
> That said, your action plan is alright. I am busy but will definitely
> find some time. The hard part is SDataSet query engine !! This is
> where
> we have to be clever (maintaining some kind of graph of relationships
> between records...)
>
> So let go on... I think we should agree on who does what in what
> order.
> I could tackle with the easy part (1. 2. and 3.) to begin. Do you
> agree
> on that ?
>
> Franck
>
> Le mardi 01 juillet 2008 à 21:18 +1000, Anthony Berglas a écrit :
> > Hello Franck,
> >
> > Hmm. The design gets ever more ambitious. I am
> > concerned that if we take on too much we will not
> > finish. But it would be good to get this right.
> >
> > So let us leave the transaction cache in
> > SConnection, or at least in the one default
> > SDataSet in SConnection. Finders are on
> > SConnection are just a short cut to the
> > underlying data set. Let's rename SConnection to
> > be SCon as it will be very commonly used in the new API.
> >
> > SCon.begin();
> >
> > Department sales = SCon.findOrCreate(Departement.meta, "100");
> >
> > List<Employee> inDeptX = SCon.query(new
> > SQuery(Employee.meta).eq(DEPARTMENT, sales));
> >
> > List<Employee> inDeptX = SCon.query(new
> > SQuery(Employee.meta).eq(DEPARTMENT, sales));
> >
> > SDataSet myDS = SCon.detachAll();
> >
> > ...
> >
> > List<Employee> inDeptX2 = myDS.query(new
> > SQuery(Employee.meta).eq(DEPARTMENT, sales));
> >
> > myDS.create(new Employee().set(Employee.ID,
> > "123").set.....); // not part of any transaction
> >
> > ...
> > SCon.attach(myDS);
> >
> > Not that much worse than before, and allows the
> > data set to be queried when detached. Having
> > only One SDataSet per SConnection makes the
> > behavior of flush() and commit() well defined.
> >
> > You are right that it is probably OK just to
> > always return a list in one hit, and hide
> > SResultSet and SPreparedStatement completely.
> > If people need to go one record at a time for
> > some special reason then they can just get the
> > raw JDBC driver from SConnection and do it manually.
> > Best to return a List rather than an Iterator --
> > avoids hasNext issues and is more flexible and really no more
> > expensive.
> >
> > (Ideally the SDriver routines should be split
> > into a third .jdbc jar that is dependent on the
> > .record jar but not on the .connection jar. But
> > let us leave that alone for now.)
> >
> > Wow a big clean up.
> >
> > We need to walk this through carefully. Probably as follows:-
> >
> > 1. svn copy frank frankPreSplit -- just in case.
> >
> > 2. Rename SConnection to SCon, everywhere.
> >
> > 3. Create methods SCon.query, SCon.find etc. that
> > just call back to SRecordMeta etc. Update all
> > code and examples to use them (bigest job.)
> >
> > 4. Move code out of SRecordMeta into SCon (or
> > better as static methods in SRecordFinder to avoid clutter).
> >
> > 5. Hide (make package local) or remove SResultSet
> > and SPreparedStatement. Return Lists instead.
> > (No need for raw SQL option -- raw SQL
> > fragments can be poked into SQuery, or just call
> > underlying JDBC driver. Keep SimpleOrm Simple.)
> > Update all code and examples to use this.
> > I would also like to automatically flush
> > before any query is executed. Cheap (needs doing
> > anyway) and avoids nasty issues about cache consistency.
> >
> > 6. Create new package SimpleOrm.record and move
> > SRecord*, SField* into it. Update all applications and examples.
> >
> > 7. Cut dependencies and change build.xml to build
> > two distinct jars, SimpleormRecord.jar and
> > SimpleormConnection. The latter dependent on the former.
> >
> > 8. Rework SQuery to build a query tree structure.
> > Implement SDriver methods to convert this to an SQL Query.
> > Things like Joins will need thought.
> >
> > 9. Write SDataSet query engine (ie. that can query a detached data
> > set).
> >
> > 10. Implement SCon.detachDataSet. and
> > SCon.attacheDataSet. These need careful
> > thought. Get rid of other single record detachment -- all or none.
> >
> > 11. Fix up the dozens of loose end, such as generators.
> > And automatically finding references when detached etc.
> >
> > Phew. That is a lot. But it would be much
> > better when done. If we only get as far as step
> > 7 we have still done most of the visible changes
> > to ordinary code, and got the basic split.
> >
> > I am busy tomorrow.
> >
> > This will take at least a full week of each of our time. Do you have
> > it?
> >
> > Anthony
> >
> > At 12:24 AM 1/07/2008, Franck Routier wrote:
> >
> > >Anthony,
> > >
> > >maybe we should consider introducing SDataSet (or
> SDataSetManager ?)
> > >right now.
> > >First it would be equivalent to SConnection's transaction cache and
> > >could NOT be detached "per se".
> > >But maybe later, it could be.
> > >
> > >This could allow to have the following API :
> > >
> > >SQuery<Employee> qry = new SQuery(Employee.meta).eq(EMPEE_ID,
> "123");
> > >
> > >SConnection.attach(datasource);
> > >SConnection.begin();
> > >SDataSet ds = SConnection.getDataSet();
> > >Set<Employee> rs = ds.execute(qry);
> > >
> > >or a shortcut
> > >
> > >Set<Employee> rs = SConnection.execute(qry);
> > >and SConnection would delegate to its internal SDataSet
> > implementation.
> > >
> > >This would be consistant with the idea of applying simple queries
> to
> > >SDataSet outside of a database (detached). So in the future, we
> could
> > do
> > >this, without breaking the API :
> > >
> > >SDataSet ds = SConnection.detachDataSet();
> > >...in another tier...
> > >Set<Employee> rs = ds.execute(qry);
> > >
> > >Also SFinder could be a subclass of SQuery. And we could do :
> > >
> > >SFinder<Employee> finder = new SFinder(Employee.meta);
> > >finder.setPirmaryKey("123");
> > >Employee emp = ds.mustFind(finder); // throws exception if not
> found
> > >or
> > >Employee emp = ds.find(finder); // return null if not found
> > >or
> > >Set<Employee> emp = ds.execute(finder); // generic execute, return
> an
> > >empty set or a single value set
> > >
> > >and also :
> > >
> > >ds.findOrCreate(finder);
> > >
> > >I don't know if we need a SResultSet. Maybe it could remain
> internal
> > to
> > >Simpleorm, or at least to the .database part.
> > >
> > >Does it make sense ?
> > >
> > >Regarding my time, as everybody, I have a ton of important things
> to
> > >do :), but I will definitely find some time to get Sorm 3 right !
> > >
> > >Regards,
> > >Franck
> > >
> > >Le lundi 30 juin 2008 Ã 20:44 +1000, Anthony & Melissa Berglas a
> > écrit :
> > > > Hello Frank,
> > > >
> > > > Moving SQuery out is possible. One would have to
> > > > store the built query as a structure rather than
> > > > a query string, and then translate to string in .database. eg.
> > > > eq(this, that) {
> > > > operationsList.add(new Eq(this, that));
> > > > }
> > > > A little bit of extra overhead, but doable. Much
> > > > better to have query SQL built in SDriver.
> > > >
> > > > What would the API look like? Maybe
> > > >
> > > > SQuery<Employee> qry = new SQuery(Employee.meta).eq(EMPEE_ID,
> > "123");
> > > > SResultSet<Employee> rs = new SResultSet(qry).execute();
> > > > and maybe
> > > > Employee[] emps = new SFinder(qry).execute();
> > > >
> > > > (Cannot have qry.execute() because that crosses the boundary.)
> > > >
> > > > Having SQuery in the record part would also mean
> > > > that one could later apply simple queries to a
> > > > DataSet (ie.list) of records, outside a database which would be
> > cool.
> > > >
> > > > We could also choose to keep the existing API if
> > > > we have a database accessor interface written in
> > > > .record but implemented in .database. But we
> > > > then need to keep a pointer to the
> > > > implementation, either in a static or another
> > > > thread local. This produces a slightly neater
> > > > API but is much less transparent.
> > > >
> > > > What do you think? Please propose an API style that would suit
> > you.
> > > >
> > > > Going forward, do you have time this week? I
> > > > would like to start by cutting out the finder
> > > > code in SRecordMeta, which is an internal change.
> > > >
> > > > Sorry about the non-public method. If we split
> > > > this then all the meta data will need to be
> > > > public so that the .database part can access it!
> > > >
> > > > Regards,
> > > >
> > > > Anthony
> > > >
> > > > At 07:24 PM 30/06/2008, Franck Routier wrote:
> > > > >Hello Anthony,
> > > > >
> > > > >I totally second this separation. This makes things much
> cleaner,
> > and
> > > > >opens the possibility for future evolutions.
> > > > >
> > > > >However, I am not sure I want SQuery to be in the database
> > part...
> > > > >Right now, I build a lot of queries in my business layer,
> > depending
> > > > on
> > > > >business rules, and then execute them in Dao's.
> > > > >Would it be possible to keep SQuery quite conceptual, and let
> > > > >SConnection/SDriver implement the DB specific part. This would
> > also
> > > > make
> > > > >possible to have some different SConnection implementation, say
> > > > towards
> > > > >a ldap repository, etc. (well, I know SQuery join and so on are
> > > > pretty
> > > > >RDBMS oriented...)
> > > > >
> > > > >This would make SQuery kind of a custom query language, which
> you
> > > > would
> > > > >rather avoid, as said in the whitepaper. But we could keep it
> as
> > it
> > > > is
> > > > >now, that is a 'tiny' layer over jdbc AND potentialy other
> > stuffs :)
> > > > >Would you consider this option ?
> > > > >
> > > > >By the way, introducing references to SConnection.getDriver()
> in
> > > > SQuery
> > > > >(for quoting) broke my app. As said before, I tend to build my
> > > > queries
> > > > >outside connections. So I temporay commented it out. The option
> I
> > > > would
> > > > >go for is to translate SQuery into sql latter on, in
> > > > SPreparedStatement
> > > > >for example. What is your opinion on this ?
> > > > >
> > > > >Also be careful to preserve API visibility, at least for
> > read-only
> > > > >getters. Restricting it to package visibility also breaks a lot
> > of my
> > > > >code for dynamically building queries.
> > > > >
> > > > >
> > > > >That said, I really like your idea.
> > > > >
> > > > >Franck
> > > > >
> > > > >
> > > > >Le lundi 30 juin 2008 Ã 15:56 +1000, Anthony Berglas a
> > écrit :
> > > > > > Hello Franck,
> > > > > >
> > > > > > I think that as part of 3.0 we should consider splitting
> > SimpleOrm
> > > > > > into two pieces, each in their own Jars.
> > > > > >
> > > > > > simpleorm.records that contains SRecordMeta,
> SRecordInstance,
> > > > SField*
> > > > > >
> > > > > > simpleorm.database that contains SQuery, SDriver*,
> > SConnection.
> > > > > >
> > > > > > The record part should have zero dependency on the database
> > part,
> > > > and
> > > > > > should be compiled and built into its own jar, with some
> test
> > > > cases of
> > > > > > its own.
> > > > > >
> > > > > > The database part should depend on the records part,
> > obviously.
> > > > > >
> > > > > > When working detached, one should only need the .records
> jar.
> > > > > >
> > > > > > This is actually not that hard to do. I have already
> decoupled
> > > > > > SQuery from SRecordMeta. A similar trick can be used to
> > decouple
> > > > > > findOrCreate and friends. And the only other substantial
> > > > connection
> > > > > > is where SReferenceField.getRawValue can do a lazy read.
> > > > > >
> > > > > > For findOrCreate, the API becomes slightly messier:-
> > > > > >
> > > > > > SFinder empf<Employee> = new SFinder(Employee.meta);
> > > > > > Employee e = empf.findOrCreate("123");
> > > > > >
> > > > > > or simply but slightly less efficiently
> > > > > >
> > > > > > Employee e = new SFinder(Employee.meta).findOrCreate("123");
> > > > > >
> > > > > > (The finder would cache the sets of different types of
> fields
> > > > etc.)
> > > > > >
> > > > > > vs old
> > > > > >
> > > > > > Employee e = Employee.meta.findOrCreate("123");
> > > > > >
> > > > > > For the SFieldReference.get the call would become
> > > > > >
> > > > > > SFinder deptf<Department> = new SFinder(Department.meta);
> > > > > > Department dept = deptf.getReference(myEmp, DEPT);(
> > > > > >
> > > > > > (For the detached case, no query:-)
> > > > > > Department dept = myEmp.getUnqueriedReference(DEPT);
> > > > > >
> > > > > > One could, of course, give SRecordMeta a default finder,
> > defined
> > > > > > through an interface in .SRecord, and then keep the existing
> > > > > > API. But being explicit makes the structure clear.
> > > > > >
> > > > > > I have also attached some other prototyping that I was doing
> > in
> > > > > > SimpleData (it is checked in). But I now think that
> splitting
> > > > > > SimpleOrm is a much better idea, and SimpeData should be
> > > > > > abandoned. I think that this change is actually very good,
> and
> > > > would
> > > > > > lead naturally to DataSets if we wanted to go there later.
> > > > > >
> > > > > > My current task is to work on the white paper etc. I would
> not
> > > > want
> > > > > > to start any split until you have caught up with the current
> > > > changes
> > > > > > and your application works 100%. (Although I might be
> tempted
> > to
> > > > do
> > > > > > some of the internal rearrangement, moving code from
> > SRecordMeta
> > > > to
> > > > > > SFinder.)
> > > > > >
> > > > > > One way or the other I am hoping to make your branch the
> trunk
> > > > this
> > > > > > week.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Anthony
> > > > > >
> > > > > > Dr Anthony Berglas,
> > > <mailto:anthony%40berglas.org>anthony@... Mobile: +61 4
> 4838
> > 8874
> > > > > > Just because it is possible to push twigs along the ground
> > with
> > > > ones
> > > > > > nose
> > > > > > does not necessarily mean that is the best way to collect
> > > > firewood.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > > Spreadsheet Detective,
> > > > Southern Cross Software Queensland Pty Limited
> > > > 54 Gerler Street
> > > > Bardon, Queensland 4065, Australia.
> > > >
> > > > Email:
> > > <mailto:berglas%
> > 40spreadsheetdetective.com>berglas@...
> > > > www.SpreadsheetDetective.com
> > > > Ph: +61 427 830248 (Australian Eastern Standard Time)
> > > >
> > > > "If the model seems correct only because the numbers look right,
> > > > then why build the model in the first place?"
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> > Dr Anthony Berglas, anthony@... Mobile: +61 4 4838 8874
> > Just because it is possible to push twigs along the ground with ones
> > nose
> > does not necessarily mean that is the best way to collect firewood.
> >
> >
> >
> >
> >
>
>
>
>
>