Hello,

I am new to the XML-DBMS. When I attempted to open action.dtd using XML Spy 5.0,
I was told that the file was not well-formed.  The error message "This file is
not well-formed: % expected" appears at the end of every empty element
declaration, such as: <!ELEMENT %p; Namespace EMPTY>.

What is wrong?

Many thanks,

haiying







[Non-text portions of this message have been removed]

1) When you open action.dtd, are you opening it as an XML document or a
DTD? Trying to open it as an XML document will definitely lead to
well-formedness errors. This is because it is a DTD and doesn't have any
elements in it. I'm not sure what the error message means...

2) I just noticed a bug in all the version 2.0 DTDs. Because parameter
entity substitution rules require replacement values to have a single
space added to each end of the entity value, the resulting element type
and attribute declarations are invalid, as a space will separate the
prefix from the element type name. However, you will only hit this bug
if you change the values of the p and s entities, so most people will
never encounter it.

-- Ron

Haiying wrote:

> I am new to the XML-DBMS. When I attempted to open action.dtd using
> XML Spy 5.0, I was told that the file was not well-formed.  The
> error message "This file is not well-formed: % expected" appears
> at the end of every empty element declaration, such as:
> <!ELEMENT %p; Namespace EMPTY>.
>
> What is wrong?

Hai,
    How can i retreive the xml element mapped in  a map file.
For ex:
<ClassMap>
     <ElementType Name="PURCHASEORDER"/>
     <ToClassTable Name="PURCHASEORDER"/>
   <PropertyMap>
     <ElementType Name="No"/>
     <ToColumn Name="RECNO"/>
   </PropertyMap>
</ClassMap>

here i mapped xml element "No" to RECNO.i want to query back what
element i mapped to RECNO.

Thanks
sharada.

One way to do this is to compile the map document into an XMLDBMSMap
object, then use the methods on this and related objects. For example:

    // Get an XMLReader for the Xerces parser, then get a map compiler.

    ParserUtils utils = new ParserUtilsXerces();
    XMLReader xmlReader = utils.getXMLReader(true);
    MapCompiler compiler = new MapCompiler(xmlReader);

    // Get a SAX InputSource over the map document and compile it.

    InputSource src = new InputSource(new FileReader("my_map_file.map"));
    XMLDBMSMap map = compiler.compile(src);

    // Get the ClassTableMap for PURCHASEORDER, then get the ColumnMap
    // for RECNO.

    ClassTableMap ctm = map.getClassTableMap(null, null, null,
"PURCHASEORDER");
    ColumnMap columnMap = ctm.getColumnMap("RECNO");

    // Get the name of the element type to which the RECNO column is
mapped.

    XMLName name = columnMap.getXMLName();
    int type = columnMap.getType();

    // Print out the element type name.

    if (type == PropertyMapBase.ELEMENTTYPE)
    {
       System.out.println("Column RECNO is mapped to the following
element type");
       System.out.println("Namespace URI: " + name.getURI());
       System.out.println("Local name: " + name.getLocalName();
    }
    else
       System.out.println("Column not mapped to an element type");

Does this answer your question? If so, you'll want to look at the
javadocs for the org.xmlmiddleware.xmldbms.maps package. Start with
XMLDBMSMap, which gives an overview.

-- Ron

sharadaj2000 wrote:
>
> Hai,
>    How can i retreive the xml element mapped in  a map file.
> For ex:
> <ClassMap>
>     <ElementType Name="PURCHASEORDER"/>
>     <ToClassTable Name="PURCHASEORDER"/>
>   <PropertyMap>
>     <ElementType Name="No"/>
>     <ToColumn Name="RECNO"/>
>   </PropertyMap>
> </ClassMap>
>
> here i mapped xml element "No" to RECNO.i want to query back what
> element i mapped to RECNO.

Hello Ron et al,

Sorry for being out of the loop for so long but I've been incredibly
buzy.....& now the project is reaching a "Waiting for CapGem to catch up
stage" so I can chill & code more for fun & interest than for tight
deadlines set against changing criteria <only semi G> .

BTW a couple of monhs ago I managed to get cvs working on sourceforge @
http://sourceforge.net/projects/xmlmiddleware/ . The code hasn't changed
mch since then as no time to do anything but consider "customer requests
& enhancements", testing, etc. etc. etc.

Anyway ........I have the execution framework thingy working & it works
really quite well. I have done the jms & xslt bits (& some test plugins
etc) I have not yet done any meaningfull docs but I was
wondering....about what to do now.

There is a GUI builder in there for building map files (version 1) but
really I need to look at this again if anything because I would like to
run it within the framwork itself (which should be possible) & I'm not
happy with a number of the initial UI decisions. The UI decisions are
the main ones but anyway....

I could spend my time :
A) Building a plugin for  XD1
B) Building a plugin for XD 2
C) Reviewing & refining the framework itself
D) Reviewing the GUI Map builder for a user perspective & making it more
"intuitive".
E) Build a GUI map builder for V2 maps etc.

The framework runs off an xml file which is the only thing referenced
(rather than by a host of properties files). The XML file can reference
& then load properties files or the properties can be contained as
<Property Key= Value= /> elements within the "Action elements". As an
example see below.

A) This file loads some "default" properties in the first section
(including a properties file).
B)  It then loads a textfile (an xml file in this case)
C) It then saves the loaded file with a filename of Smeg21.txt
D) It then runs that XML through XSLT
E) It then saves the result via a pattern (you can use a variety of
patterns for multiple produced files (e.g. if you were watching a JMS
topic & wanting to do things & then save the file with a unique
filename). In this case it will save the file as "Mytestfile%.xml" where
the % is replaced with the pattern of your choice (in this case a
timestamp string)
F) It then sends the same file on via JMS to a topic called "testtopic"


<?xml version='1.0' encoding="UTF-8"?>
<Map>
<Default>
<Property Key="Deftest1" Value="DT1"/>
<Property Key="File"
Value="C:\Data\Laptop\Move\aat\XMLDBMS\V2Alpha3\samples\map1.props"/>
<Property Key="test1" Value="Smeg1" />
<Property Key="test2" Value="Smeg2" />
</Default>
<Actions>
<Action Class="net.xmlmiddleware.aaf.plugin.TextFileLoad">
<Property Key="TextFileURL"
Value="C:\Data\Laptop\Move\aat\XSLTTest\PRD.xml"/>
<Action Class="net.xmlmiddleware.aaf.plugin.FileSave">
<Property Key="FileURL" Value="C:\Data\Laptop\Move\aat\XSLTTest\"/>
<Property Key="Filename" Value="Smeg21.txt"/>
</Action>
<Action Class="net.xmlmiddleware.aaf.plugin.ProcessXSLT">
<Property Key="XSLTClass"
Value="net.xmlmiddleware.transform.xslt.ProcessXalan2"/>
<Property Key="XSLTScript"
Value="C:\Data\Laptop\Move\aat\old\allwms\msg_prd.xsl"/>
<Action Class="net.xmlmiddleware.aaf.plugin.FileSave">
<Property Key="Action" Value="TimeStamp"/>
<Property Key="FileURL" Value="C:\Data\Laptop\Move\aat\XSLTTest\"/>
<Property Key="Pattern" Value="Mytestfile%.xml"/>
</Action>
<Action Class="net.xmlmiddleware.aaf.plugin.JMSSend">
<Property Key="JMSContext"
Value="com.swiftmq.jndi.InitialContextFactoryImpl"/>
<Property Key="JMSProviderURL" Value="smqp://sauron:4001"/>
<Property Key="JMSTopic" Value="testtopic"/>
<Property Key="JMSTCF" Value="TopicConnectionFactory"/>
</Action>
</Action>
</Action>
</Actions>

It can be very simple ie. to set up a JMS receiver which simply watches
the same topic & prints to system out all you would need is:

<?xml version='1.0' encoding="UTF-8"?>
<Map>
<Actions>
<Action Class="net.xmlmiddleware.aaf.plugin.JMSReceive">
<Property Key="JMSContext"
Value="com.swiftmq.jndi.InitialContextFactoryImpl"/>
<Property Key="JMSProviderURL" Value="smqp://sauron:4001"/>
<Property Key="JMSTopic" Value="testtopic"/>
<Property Key="JMSTCF" Value="TopicConnectionFactory"/>
</Action>
</Actions>
</Map>

OK so call me rusty but I can do the V1 plugin quickly. What I'd like to
know (other than what people would prefer re the A,B,C etc choices
above) is:

a) What level of plugion would people require? E.g. a "do everything
rely on text values to work out what" approach or the "retreiveBySQL"
(as a plugin vs say the "RetrievebyKey" plugin) or the halfway house way
i.e. do a "retrieve" plugin where you set the "BySQL" or "ByKey" within
the properties etc?

b) Where has Version 2 got to? Are there any incredibly helpfull docs
which I could use to (a) build the plugin & (b) build the GUI MapBuilder?

Adam

Hi all!
does I.E. 5/5 supports in XML?
where can I read about it?
thanks
Taly

Cen-Tex Programmin would like to announce their Newly released
E-GLOBAL Pro ver 1.0 Email Marketing Program
along woth a bonus of 5,000+ (FREE) Email addresses to get you
started.

With this program you can run your own fully automated Email
marketing website.

This program has been successfully tested with over 100,000 Email
addresses with no slow down in sight.

Program specs and demo links are below.
(price is $25.00)
To purchase this program by PayPal Email them at: eglobal@m...

E-GLOBAL Pro Features:

Demo:
Demo Username: asking
Demo Password: jw101871
Demo URL: http://ivsingles.hypermart.net/opt-in/eglobal.pl

1 Integrated banner advertising, fully automated.
2.Can be setup to use PayPals one time purchase or membership options.
3.Compatible with PayPal.
4. New user verification email to sent to user
5. Site Stats
6. Admin program includes, ban user by e-mail, send mail to all
members, manage or delete members, generate list of members email
addresses.
7. Site completely username and password protected.
8. Username and password lookups.
9. Update registration.
10. Change password.

Server requirements are:

1. A Unix server (or compatible)
2. Running at least perl 5.0
3. Sendmail access/or installed on server
4. cgi-bin and perl access
5. ftp access to server
If you are interested in purchasing this very fine and extremely
value added program which also comes with free lifetime e-mail
support, access to free add-ons, bulletin board tech support and
5,000+ Email addresses.

The price is $25.00
To purchase this program Email them at: eglobal@m...

If you do not have a current web server we can install the program on
They can let you know where a web server account can be obtained for
FREE.

They are currently offering this program at the discounted rate to
all members of this group along with the bonus of 5,000+ Email
addresses to help get you started and all the assistance you may need
regarding Email advertising to make your site successful.

Thanks for your time.

Hello , Everyone !
When I type the command : java Transfer -toxml sales.map sales_out.xml Sales 123
There occurs exception as following :

java.sql.SQLException: ResultSet is closed
  at sun.jdbc.odbc.JdbcOdbcResultSet.checkOpen(JdbcOdbcResultSet.java:6356)
  at sun.jdbc.odbc.JdbcOdbcResultSet.clearWarnings(JdbcOdbcResultSet.java:1705)
  at sun.jdbc.odbc.JdbcOdbcResultSet.close(JdbcOdbcResultSet.java:1427)
  at
de.tudarmstadt.ito.xmldbms.DBMSToDOM.processRelatedTables(DBMSToDOM.java:512)
  at
de.tudarmstadt.ito.xmldbms.DBMSToDOM.processClassResultSet(DBMSToDOM.java:419)
  at
de.tudarmstadt.ito.xmldbms.DBMSToDOM.processRootResultSet(DBMSToDOM.java:381)
  at de.tudarmstadt.ito.xmldbms.DBMSToDOM.retrieveDocument(DBMSToDOM.java:198)
  at domtodb.TransferResultSet.toXML(TransferResultSet.java:145)
  at domtodb.TransferResultSet.main(TransferResultSet.java:105)

Why ? Thanks !
Dou Qinghua




[Non-text portions of this message have been removed]

Google sez:

    http://www.xml.com/pub/a/2000/05/03/msie/

Note that this article is two years old. I assume later versions of IE
have even more support.

-- Ron

Taly Reinish wrote:
>
> Hi all!
> does I.E. 5/5 supports in XML?
> where can I read about it?

This is a known bug. It usually occurs only when you are using JDK 1.3.
To solve it, use JDK 1.1.x or 1.2, OR do the code fixes in:

   http://groups.yahoo.com/group/xml-dbms/message/466 (question 3)
   http://groups.yahoo.com/group/xml-dbms/message/473

This has been fixed in version 2.0.

-- Ron

dou wrote:
>
> Hello , Everyone !
> When I type the command : java Transfer -toxml sales.map sales_out.xml Sales
123
> There occurs exception as following :
>
> java.sql.SQLException: ResultSet is closed
>  at sun.jdbc.odbc.JdbcOdbcResultSet.checkOpen(JdbcOdbcResultSet.java:6356)
>  at sun.jdbc.odbc.JdbcOdbcResultSet.clearWarnings(JdbcOdbcResultSet.java:1705)
>  at sun.jdbc.odbc.JdbcOdbcResultSet.close(JdbcOdbcResultSet.java:1427)
>  at
de.tudarmstadt.ito.xmldbms.DBMSToDOM.processRelatedTables(DBMSToDOM.java:512)
>  at
de.tudarmstadt.ito.xmldbms.DBMSToDOM.processClassResultSet(DBMSToDOM.java:419)
>  at
de.tudarmstadt.ito.xmldbms.DBMSToDOM.processRootResultSet(DBMSToDOM.java:381)
>  at de.tudarmstadt.ito.xmldbms.DBMSToDOM.retrieveDocument(DBMSToDOM.java:198)
>  at domtodb.TransferResultSet.toXML(TransferResultSet.java:145)
>  at domtodb.TransferResultSet.main(TransferResultSet.java:105)
>
> Why ? Thanks !

Hello all,
My name is Azzam; I'm a new member in this grate group.
I'm talking to you from Riyadh, Saudi Arabia,. I'm studding computer science
at King Saud University (KSU), college of compute and information science
(CCIS).

I'm at my last year here. I'm studding graduation project I with my
Instructor professor Abad. My subject was chosen by my instructor and its
title is:
“Extraction of Relational Database from XML Document”.

I sow the Yahoo! Groups Link on Your website, so I joined seeking knowledge
and help.

I hope you accept me as a student of yours.
I wish you all the best and I hope you wish me back all the luck, cuz I need
it!

My best regards,,,
Azzam


_________________________________________________________________
The new MSN 8: smart spam protection and 2 months FREE*
http://join.msn.com/?page=features/junkmail

Adam Flinton wrote:

> Sorry for being out of the loop for so long but I've been incredibly
> buzy

Hello! I was beginning to wonder if you'd dropped off the face of the
earth.


> I could spend my time :
> A) Building a plugin for  XD1
> B) Building a plugin for XD 2
> C) Reviewing & refining the framework itself
> D) Reviewing the GUI Map builder for a user perspective & making it more
> "intuitive".
> E) Build a GUI map builder for V2 maps etc.

My choice is:

1) D+E
2) B
3) C
4) A

Version 2 is still in alpha, but people are using it and I'm sure a GUI
map builder would be much appreciated.

> The framework runs off an xml file which is the only thing referenced
> (rather than by a host of properties files). The XML file can reference
> & then load properties files or the properties can be contained as
> <Property Key= Value= /> elements within the "Action elements". As an
> example see below.

[example snipped]

Looks nice -- it appears you can "pipeline" documents, which is one of
the hot new things in XML.

> a) What level of plugion would people require? E.g. a "do everything
> rely on text values to work out what" approach or the "retreiveBySQL"
> (as a plugin vs say the "RetrievebyKey" plugin) or the halfway house way
> i.e. do a "retrieve" plugin where you set the "BySQL" or "ByKey" within
> the properties etc?

I like the "rely on text values" approach -- seems to be the most
extensible.

> b) Where has Version 2 got to? Are there any incredibly helpfull docs
> which I could use to (a) build the plugin & (b) build the GUI MapBuilder?

Version 2.0 is still in alpha. (Currently in alpha 3, although I'm
hoping to get an alpha 4 out shortly.)

For incredibly helpful docs, see the Java Docs. I think the most useful
things for you are:

1) org.xmlmiddleware.xmldbms.tools.Transfer -- for transferring data

2) org.xmlmiddleware.xmldbms.tools.MapManager -- for generating maps
(part of the map editor)

3) org.xmlmiddleware.xmldbms.maps.* -- for building the GUI map editor
(start with XMLDBMSMap for an overview)

Also worth reading are the DTDs: filter.dtd, actions.dtd, and
xmldbms2.dtd

-- Ron

Hey, folks. This is my first message to this mailing list that I've
monitored (very) loosely for a couple of months now. I work in a
non-profit biological research lab (Cold Spring Harbor Laboratory
http://www.cshl.org). Our lab recently became a part of a rather large,
international collaboration, the Haplotype Map project, to product
enormous amounts of biological data (genotypes). Our role will be to
synchronize data handling and build the database to hold the stuff (see
http://www.genome.gov/page.cfm?pageID=10001688 if you're interested in
this thing), plus related tasks.
    We'd originally planned to use a fairly XML-centric approach from the
ground up, for a multitude of reasons. One very strong reason was that we
could use the allegedly powerful XML-capabilities of Oracle 9i XMLDB to
produce a XML-relational schema from our own data handling/exchange XML
Schema definitions.
    To cut this story short, we got our funding cut down quite a bit and
will now not be able to afford the big-buch Oracle licenses we'd need
(3x$15.000). Open source is now pretty much the only option for us
database-wise, which is in fact a blessing in disguise because it will
make the endevour entirely open source (Oracle would have been the only
proprietary component otherwise).

    Now that I'm through with the introduction, I'll get to the point. I've
been looking at the XML-enabled database flora and see that commercial
solutions rule. There's a bunch of non-commercial stuff in the middleware
department, so we're tempted to go with MySQL (already a favorite in my
camp!) or possibly PostGreSQL and some sort of open source middleware
solution in between. The XML-DBMS stuff lookes interesting, but when I had
read about it bit I saw a quite a few references to DTDs, including the
mapping language DTD itself. I may a big mistaken here, since I have not
RTFM'd enough yet in the docs, but is there not yet support for direct XML
Schema definition (XSD) to build the relational database schema etc.?
    My utopian dream here is do something similar to what Oracle XMLDB can
do; register, or otherwise associate, the XSD with the database in such a
way that the strong datatype support in XML Schema can be utilized to
directly build the database schema, without having to instruct the
middleware on what kind of datatype the column should be  for every
element or attribute in a (potentially large and complex) schema...I mean,
   the schema already says in the schema whether, say, an element should be
integer or string.

    Any comments on this thing to help me out here? Thanks.


           Regards,

                 Mummi, CSHL
--
----------------------------------------------------
Gudmundur Arni Thorisson, bioinformatics researcher, B.Sc.
Steinlab, Cold Spring Harbor Laboratory
w-phone#: 516-367-6904
w-fax#:   516-367-8389
1 Bungtown Road
Cold Spring Harbor
11724 New York
USA


[Non-text portions of this message have been removed]

Ronald Bourret wrote:

>Adam Flinton wrote:
>
>
>
>>Sorry for being out of the loop for so long but I've been incredibly
>>buzy
>>
>>
>
>Hello! I was beginning to wonder if you'd dropped off the face of the
>earth.
>
>
>
Nope...just Sweden <GD&R>

>
>
>>I could spend my time :
>>A) Building a plugin for  XD1
>>B) Building a plugin for XD 2
>>C) Reviewing & refining the framework itself
>>D) Reviewing the GUI Map builder for a user perspective & making it more
>>"intuitive".
>>E) Build a GUI map builder for V2 maps etc.
>>
>>
>
>My choice is:
>
>1) D+E
>2) B
>3) C
>4) A
>
>

OK.  I'll have to look at V2 stuff anyway simply so I can both do a
plugin & so I can do a GUI mapbuilder. I'll drag the latest code down
from sourceforge.

>Version 2 is still in alpha, but people are using it and I'm sure a GUI
>map builder would be much appreciated.
>
>

If it makes their life easier <G>.

>
>
>>The framework runs off an xml file which is the only thing referenced
>>(rather than by a host of properties files). The XML file can reference
>>& then load properties files or the properties can be contained as
>><Property Key= Value= /> elements within the "Action elements". As an
>>example see below.
>>
>>
>
>[example snipped]
>
>Looks nice -- it appears you can "pipeline" documents, which is one of
>the hot new things in XML.
>
>

Yup. It's more than that though...It's down to thinking how to makes
things both easier & faster specifically wrt to Java. e.g..the main
objects are built during the first pass & are then stored in a hashtable
such that I manage to get closer to minimising object creation(i.e.
create once & then reuse reuse reuse reuse etc.).


>
>
>>a) What level of plugion would people require? E.g. a "do everything
>>rely on text values to work out what" approach or the "retreiveBySQL"
>>(as a plugin vs say the "RetrievebyKey" plugin) or the halfway house way
>>i.e. do a "retrieve" plugin where you set the "BySQL" or "ByKey" within
>>the properties etc?
>>
>>
>
>I like the "rely on text values" approach -- seems to be the most
>extensible.
>
>
OK that's the simplest because I can literally take the exisiting stuff
straight over.

>
>
>>b) Where has Version 2 got to? Are there any incredibly helpfull docs
>>which I could use to (a) build the plugin & (b) build the GUI MapBuilder?
>>
>>
>
>Version 2.0 is still in alpha. (Currently in alpha 3, although I'm
>hoping to get an alpha 4 out shortly.)
>
>
OK I'll get stuff from CVS

>For incredibly helpful docs, see the Java Docs. I think the most useful
>things for you are:
>
>1) org.xmlmiddleware.xmldbms.tools.Transfer -- for transferring data
>
>2) org.xmlmiddleware.xmldbms.tools.MapManager -- for generating maps
>(part of the map editor)
>
>3) org.xmlmiddleware.xmldbms.maps.* -- for building the GUI map editor
>(start with XMLDBMSMap for an overview)
>
>Also worth reading are the DTDs: filter.dtd, actions.dtd, and
>xmldbms2.dtd
>
>
Thanks will do.

Adam


>-- Ron
>
>
>To post a message, send it to: xml-dbms@yahoogroups.com
>To unsubscribe, send a blank message to: xml-dbms-unsubscribe@yahoogroups.com
>Or join the newsgroup at: news://news.barkto.com/homeless.xml.xml-dbms
>
>Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>

[Dan, I'm CCing this to the list, I hope you don't mind]

    Thanks for the good wishes. I had thought of going native XML, just
looked at the Apache XML-project Xindice thing yesterday. That is
certainly a possibility, though we're talking about a very data-centric,
fairly rigid system here, one that would probably not exploit more than in
part the native-XML capabilities of a native XML-database. Also
XML-relational setup, like the Oracle thing I mentioned, allows one to use
existing SQL/RDMS know-how instead of going full-bore down the new XPath
etc. route.
    Additionally, I have concerns about the scalability and performance part
of the equation. We're talking about genotyping up to 1.5 million SNPs in
  >180 individuals, resulting in up to 400 million genotype records (i.e
smallish XML complex types). That's a mighty big dataset where I come from
anyway, something that I don't know if the emerging native XML-db's can
handle at this time.

    But thanks for the reminder, though. I'll be sure to investigate this
matter a bit more and at least try out a  couple of open-source native XML
solutions :)

	 Regards,


				 Mummi, CSHL

On Tuesday, December 17, 2002, at 03:01 PM, Dan Wolfe wrote:

> Mummi,
>
> Sounds like an interesting project!  Best of luck.
>
> Out of curiosity, have you considered going fully
> native XML, with an open source [native] XML database
> perhaps?  I would be interested to know why or why
> not.
>
> Regards,
>
> Dan Wolfe
>
> --- Gudmundur Arni Thorisson <mummi@...> wrote:
>>    Hey, folks. This is my first message to this
>> mailing list that I've
>> monitored (very) loosely for a couple of months now.
>> I work in a
>> non-profit biological research lab (Cold Spring
>> Harbor Laboratory
>> http://www.cshl.org). Our lab recently became a part
>> of a rather large,
>> international collaboration, the Haplotype Map
>> project, to product
>> enormous amounts of biological data (genotypes). Our
>> role will be to
>> synchronize data handling and build the database to
>> hold the stuff (see
>> http://www.genome.gov/page.cfm?pageID=10001688 if
>> you're interested in
>> this thing), plus related tasks.
>>    We'd originally planned to use a fairly
>> XML-centric approach from the
>> ground up, for a multitude of reasons. One very
>> strong reason was that we
>> could use the allegedly powerful XML-capabilities of
>> Oracle 9i XMLDB to
>> produce a XML-relational schema from our own data
>> handling/exchange XML
>> Schema definitions.
>>    To cut this story short, we got our funding cut
>> down quite a bit and
>> will now not be able to afford the big-buch Oracle
>> licenses we'd need
>> (3x$15.000). Open source is now pretty much the only
>> option for us
>> database-wise, which is in fact a blessing in
>> disguise because it will
>> make the endevour entirely open source (Oracle would
>> have been the only
>> proprietary component otherwise).
>>
>>    Now that I'm through with the introduction, I'll
>> get to the point. I've
>> been looking at the XML-enabled database flora and
>> see that commercial
>> solutions rule. There's a bunch of non-commercial
>> stuff in the middleware
>> department, so we're tempted to go with MySQL
>> (already a favorite in my
>> camp!) or possibly PostGreSQL and some sort of open
>> source middleware
>> solution in between. The XML-DBMS stuff lookes
>> interesting, but when I had
>> read about it bit I saw a quite a few references to
>> DTDs, including the
>> mapping language DTD itself. I may a big mistaken
>> here, since I have not
>> RTFM'd enough yet in the docs, but is there not yet
>> support for direct XML
>> Schema definition (XSD) to build the relational
>> database schema etc.?
>>    My utopian dream here is do something similar to
>> what Oracle XMLDB can
>> do; register, or otherwise associate, the XSD with
>> the database in such a
>> way that the strong datatype support in XML Schema
>> can be utilized to
>> directly build the database schema, without having
>> to instruct the
>> middleware on what kind of datatype the column
>> should be  for every
>> element or attribute in a (potentially large and
>> complex) schema...I mean,
>>   the schema already says in the schema whether,
>> say, an element should be
>> integer or string.
>>
>>    Any comments on this thing to help me out here?
>> Thanks.
>>
>>
>>           Regards,
>>
>>                 Mummi, CSHL
>> --
>> ----------------------------------------------------
>> Gudmundur Arni Thorisson, bioinformatics researcher,
>> B.Sc.
>> Steinlab, Cold Spring Harbor Laboratory
>> w-phone#: 516-367-6904
>> w-fax#:   516-367-8389
>> 1 Bungtown Road
>> Cold Spring Harbor
>> 11724 New York
>> USA
>>
>>
>> [Non-text portions of this message have been
>> removed]
>>
>>
>> ------------------------ Yahoo! Groups Sponsor
>>
>> To post a message, send it to:
>> xml-dbms@yahoogroups.com
>> To unsubscribe, send a blank message to:
>> xml-dbms-unsubscribe@yahoogroups.com
>> Or join the newsgroup at:
>> news://news.barkto.com/homeless.xml.xml-dbms
>>
>> Your use of Yahoo! Groups is subject to
>> http://docs.yahoo.com/info/terms/
>>
>>
>
>
> __________________________________________________
> Do you Yahoo!?
> Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
> http://mailplus.yahoo.com
>

Hi,

    Can you please give some example code to use MS Access
with the xml-dbms1.0.1 software in
http://www.rpbourret.com/xmldbms/xmldbms101.zip


Thanks & Regards,
Venu Madhav

Dear All,

I've been going over the code I wrote for the map builder & it suddenly
struck me that I was rewriting eclipse (www.eclipse.org) right down to
plugins etc. Given how: (a) This seems silly as I use eclipse & does
anyone really need another standalone tool...?
(b) The number of plugins for eclipse is growing fast (esp the free
ones) inc XML editors etc & frankly I see the pace of this picking up
not slowing down.
(c) eclipse seems to be steadily conquering all the other ide'es
especially in the java world & as such an eclipse plugin might be more
usefull to people anyway.
(d) eclipse is both opensource & ported/being ported to just about every OS

I am contemplating (re)doing the mapbuilder as an Eclipse plugin

Obviously this will mean doing the GUI in SWT rather than swing however
the underlying models etc don't need to change as they work on the
JDBC/JDOM Level so....

Does anyone have any massive objections? Does anyone prefer that it be
done as an eclipse plugin?

I have to say that I spent yesterday playing with eclipse/swt & maybe
it's because it was written some time after Swing & maybe it's because
of it's native peers however it seems to be to be a much better GUI toolkit.

Adam

Gudmundur Arni Thorisson wrote:
> The XML-DBMS stuff lookes interesting, but when I had
> read about it bit I saw a quite a few references to DTDs, including the
> mapping language DTD itself. I may a big mistaken here, since I have not
> RTFM'd enough yet in the docs, but is there not yet support for direct XML
> Schema definition (XSD) to build the relational database schema etc.?

This is (unfortunately) correct. There has been some work done in this
regard, but it is now out of date.

A couple of comments here:

1) XML-DBMS only uses DTDs/schemas at design time, and then their use is
optional as a starting point for a map. At run time, all that is needed
is a map and an XML instance. The software reads an element/attribute
from the instance, looks it up in the map, and transfers the data
accordingly. Thus, the problem is not as large as it might seem.

2) The mapping language uses a DTD simply because a DTD is sufficient to
constrain the language. There is no need to introduce the complexity of
XML Schemas. If you want to use XML Schemas, convert the mapping
language DTD to an XML Schema and validate against that.

>    My utopian dream here is do something similar to what Oracle XMLDB can
> do; register, or otherwise associate, the XSD with the database in such a
> way that the strong datatype support in XML Schema can be utilized to
> directly build the database schema, without having to instruct the
> middleware on what kind of datatype the column should be  for every
> element or attribute in a (potentially large and complex) schema...I mean,
>   the schema already says in the schema whether, say, an element should be
> integer or string.

Your dream is a bit utopian. Although it is theoretically possible to
generate a database schema from an XML schema or DTD, the result is
rarely what you want, especially if the schema/DTD is large and complex.
The reason for this is that the XML structure rarely matches the optimal
database structure. For example, consider the following XML:

    <Customer>
       <Name>ABC Industries</Name>
       <Address>
          <Street>123 Main St.</Street>
          <City>Fooville</City>
          <State>CA</State>
          <Country>USA</Country>
          <PostCode>95041</PostCode>
       </Address>
    </Customer>

If you generate a database schema from the DTD/schema for this XML, you
will end up with two tables -- one for customers and one for addresses.
It is unlikely that this is what you want -- most commonly the address
information will be in the customer table.

Other problems include naming problems (including collisions with
existing names in the database), identifying key fields (including when
these should be generated), and determining which way primary key /
foreign key relationships go (that is, does the parent or the child have
the primary key?).

This is not to say that generating a database schema from an XML schema
is not a good thing: it can automate a lot of drudge work. It's just
that the reality is that the resulting schema/map will probably need to
be tweaked by hand.

Also, it's important to remember that this is a design-time operation
that is usually done once, with later minor changes as the XML schema
changes. If your schema has hundreds of element types, you can probably
sit down and add type information by hand in a day or so. Unpleasant,
but possible.

On the other hand, if your schema has thousands of element types, you
really will want to automate this process as much as possible and, if
you do decide to use XML-DBMS, you should give me a yell for some tips
on how you might do this.

-- Ron

Gudmundur Arni Thorisson wrote:

>    Thanks for the good wishes. I had thought of going native XML, just
> looked at the Apache XML-project Xindice thing yesterday. That is
> certainly a possibility, though we're talking about a very data-centric,
> fairly rigid system here, one that would probably not exploit more than in
> part the native-XML capabilities of a native XML-database.

I was curious about this, as the Web page in your previous email did not
provide an XML Schema or DTD. As a general rule, if your schema has a
lot of mixed content or is semi-structured, you should strongly consider
a native XML database. However, if your data is (as you say) fairly
rigid and maps well to the relational model, an XML-enabled database (or
related middleware) should be sufficient as long as it performs well
enough.

What I'm curious about here is that many of the biology DTDs I've seen
are quite semi-structured. Is there something about the SNP data that
makes it more rigid?

> Also
> XML-relational setup, like the Oracle thing I mentioned, allows one to use
> existing SQL/RDMS know-how instead of going full-bore down the new XPath
> etc. route.

A very good point.

>    Additionally, I have concerns about the scalability and performance part
> of the equation. We're talking about genotyping up to 1.5 million SNPs in
>  >180 individuals, resulting in up to 400 million genotype records (i.e
> smallish XML complex types). That's a mighty big dataset where I come from
> anyway, something that I don't know if the emerging native XML-db's can
> handle at this time.

This is largely an unknown. Xindice is apparently written to handle
large numbers of small XML documents, but I don't know what "large
numbers" is. Another open source native XML database to look at (if you
do look at them) is eXist, which is built on top of a relational
database.

-- Ron

Look in the samples directory. Transfer, TransferResultSet, and
GenerateMap are all designed to use Sun's JDBC-ODBC Bridge. Thus, to use
Access, all you need to do is set the URL to point to an ODBC data
source that uses Access. The format of the URL is:

    jdbc:odbc:<ODBC-data-source-name>

You may need to make other changes to the samples as well, such as to
use a different XML parser. For details, see section 7.0 of the
readme.htm file and the comments in the code.

-- Ron

"madhu_yanam " wrote:
>
> Hi,
>
>    Can you please give some example code to use MS Access
> with the xml-dbms1.0.1 software in
> http://www.rpbourret.com/xmldbms/xmldbms101.zip

Thanks a lot, Ron. I agree with much of of what you say in your previous
reply. The reason why I looked at Xindice (and still am) in the first
place is that the records in question are A) fairly rigidly structured B)
and relatively small, but C) large numbers of them. The 'large numbers' is
in the hundreds-millions range. There is a chance that commercial
offerings (i.e. cheaper than Oracle!) could be used, such as Tamino, given
that they have the necessary scalability. But I'll take a look at eXist as
well.
    As for the schema-2-tables issues that you commented on in the other
mail; the reason why our schema structure would be a lot more rigid and
small scale than say BSML or AGAVE is that those are supposed to be
general genome-annotation centric formats, something that can be loaded
into e.g. compliant GUI browsers and viewed and manipulated. This stuff is
supposed to be much more project-specific but extensible and/or easily
transformable (using XML-technologies) into other formats. After all, that
is one of the cool things about XML, right?
    For instance, one of the key entities that should result from this
Haplotype Map project are haplotype blocks, inferred via a certain
statistical procedure from many SNP markers genotyped in multiple
individuals. These markers span a region along a chromosome, thus this
haplotype becomes a sort of a feature on the genome that one wants use as
an annotation along with things like genes, repeats, other markers and so
on. One can imagine a scenario where a full chromosome's worth of these
things, with a span on the chromosome, is transformed from the
haplotype-with-coordinates-on-chromosome structure into the
chromosome-with-haplotype-features structure of, say BSML or similar
annotation XML formats.

    But I dissert. I'm way outside the realm of XML-DBMS now! I hope this
gives an idea of what we want to do, though. I've attached a few key XML
schema definitions that are the first part of a larger system. I  must
emphasize what I said above: the scalability factor in my mind is not the
schema itself, but rather the size of the dataset (+400M records in the
largest record class). But I also think it is important to keep RDMS/SQL
table maintainance to an absolute minimum, possibly even escape it
altogether using a native XML-db

    Thanks again for a helpful reply. Who knows, maybe I'll end up using
XML-DBMS after all...


		        Mummi, CSHL

On Friday, December 20, 2002, at 08:11 AM, Ronald Bourret wrote:

> Gudmundur Arni Thorisson wrote:
>
>>    Thanks for the good wishes. I had thought of going native XML, just
>> looked at the Apache XML-project Xindice thing yesterday. That is
>> certainly a possibility, though we're talking about a very data-centric,
>> fairly rigid system here, one that would probably not exploit more than
>> in
>> part the native-XML capabilities of a native XML-database.
>
> I was curious about this, as the Web page in your previous email did not
> provide an XML Schema or DTD. As a general rule, if your schema has a
> lot of mixed content or is semi-structured, you should strongly consider
> a native XML database. However, if your data is (as you say) fairly
> rigid and maps well to the relational model, an XML-enabled database (or
> related middleware) should be sufficient as long as it performs well
> enough.
>
> What I'm curious about here is that many of the biology DTDs I've seen
> are quite semi-structured. Is there something about the SNP data that
> makes it more rigid?
>
>> Also
>> XML-relational setup, like the Oracle thing I mentioned, allows one to
>> use
>> existing SQL/RDMS know-how instead of going full-bore down the new XPath
>> etc. route.
>
> A very good point.
>
>>    Additionally, I have concerns about the scalability and performance
>> part
>> of the equation. We're talking about genotyping up to 1.5 million SNPs in
>>> 180 individuals, resulting in up to 400 million genotype records (i.e
>> smallish XML complex types). That's a mighty big dataset where I come
>> from
>> anyway, something that I don't know if the emerging native XML-db's can
>> handle at this time.
>
> This is largely an unknown. Xindice is apparently written to handle
> large numbers of small XML documents, but I don't know what "large
> numbers" is. Another open source native XML database to look at (if you
> do look at them) is eXist, which is built on top of a relational
> database.
>
> -- Ron
>
>
> To post a message, send it to: xml-dbms@yahoogroups.com
> To unsubscribe, send a blank message to: xml-dbms-unsubscribe@yahoogroups.
> com
> Or join the newsgroup at: news://news.barkto.com/homeless.xml.xml-dbms
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>

Damn. Forgot to attach the documents!:

        Mummi

   ----------



Begin forwarded message:

> From: Gudmundur Arni Thorisson <mummi@...>
> Date: Fri Dec 20, 2002  02:54:56 PM Etc/GMT
> To: xml-dbms@yahoogroups.com
> Subject: Re: [xml-dbms] XML-DBMS: using with XML Schema, data-centric
> deployment,  large scale biomedical project
> Reply-To: xml-dbms@yahoogroups.com
>
>    Thanks a lot, Ron. I agree with much of of what you say in your
> previous
> reply. The reason why I looked at Xindice (and still am) in the first
> place is that the records in question are A) fairly rigidly structured B)
> and relatively small, but C) large numbers of them. The 'large numbers' is
> in the hundreds-millions range. There is a chance that commercial
> offerings (i.e. cheaper than Oracle!) could be used, such as Tamino, given
> that they have the necessary scalability. But I'll take a look at eXist as
> well.
>    As for the schema-2-tables issues that you commented on in the other
> mail; the reason why our schema structure would be a lot more rigid and
> small scale than say BSML or AGAVE is that those are supposed to be
> general genome-annotation centric formats, something that can be loaded
> into e.g. compliant GUI browsers and viewed and manipulated. This stuff is
> supposed to be much more project-specific but extensible and/or easily
> transformable (using XML-technologies) into other formats. After all, that
> is one of the cool things about XML, right?
>    For instance, one of the key entities that should result from this
> Haplotype Map project are haplotype blocks, inferred via a certain
> statistical procedure from many SNP markers genotyped in multiple
> individuals. These markers span a region along a chromosome, thus this
> haplotype becomes a sort of a feature on the genome that one wants use as
> an annotation along with things like genes, repeats, other markers and so
> on. One can imagine a scenario where a full chromosome's worth of these
> things, with a span on the chromosome, is transformed from the
> haplotype-with-coordinates-on-chromosome structure into the
> chromosome-with-haplotype-features structure of, say BSML or similar
> annotation XML formats.
>
>    But I dissert. I'm way outside the realm of XML-DBMS now! I hope this
> gives an idea of what we want to do, though. I've attached a few key XML
> schema definitions that are the first part of a larger system. I  must
> emphasize what I said above: the scalability factor in my mind is not the
> schema itself, but rather the size of the dataset (+400M records in the
> largest record class). But I also think it is important to keep RDMS/SQL
> table maintainance to an absolute minimum, possibly even escape it
> altogether using a native XML-db
>
>    Thanks again for a helpful reply. Who knows, maybe I'll end up using
> XML-DBMS after all...
>
>
> 		       Mummi, CSHL
>
> On Friday, December 20, 2002, at 08:11 AM, Ronald Bourret wrote:
>
>> Gudmundur Arni Thorisson wrote:
>>
>>>    Thanks for the good wishes. I had thought of going native XML, just
>>> looked at the Apache XML-project Xindice thing yesterday. That is
>>> certainly a possibility, though we're talking about a very data-centric,
>>> fairly rigid system here, one that would probably not exploit more than
>>> in
>>> part the native-XML capabilities of a native XML-database.
>>
>> I was curious about this, as the Web page in your previous email did not
>> provide an XML Schema or DTD. As a general rule, if your schema has a
>> lot of mixed content or is semi-structured, you should strongly consider
>> a native XML database. However, if your data is (as you say) fairly
>> rigid and maps well to the relational model, an XML-enabled database (or
>> related middleware) should be sufficient as long as it performs well
>> enough.
>>
>> What I'm curious about here is that many of the biology DTDs I've seen
>> are quite semi-structured. Is there something about the SNP data that
>> makes it more rigid?
>>
>>> Also
>>> XML-relational setup, like the Oracle thing I mentioned, allows one to
>>> use
>>> existing SQL/RDMS know-how instead of going full-bore down the new XPath
>>> etc. route.
>>
>> A very good point.
>>
>>>    Additionally, I have concerns about the scalability and performance
>>> part
>>> of the equation. We're talking about genotyping up to 1.5 million SNPs
>>> in
>>>> 180 individuals, resulting in up to 400 million genotype records (i.e
>>> smallish XML complex types). That's a mighty big dataset where I come
>>> from
>>> anyway, something that I don't know if the emerging native XML-db's can
>>> handle at this time.
>>
>> This is largely an unknown. Xindice is apparently written to handle
>> large numbers of small XML documents, but I don't know what "large
>> numbers" is. Another open source native XML database to look at (if you
>> do look at them) is eXist, which is built on top of a relational
>> database.
>>
>> -- Ron
>>
>>
>> To post a message, send it to: xml-dbms@yahoogroups.com
>> To unsubscribe, send a blank message to: xml-dbms-
>> unsubscribe@yahoogroups.
>> com
>> Or join the newsgroup at: news://news.barkto.com/homeless.xml.xml-dbms
>>
>> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>>
>>
>>
>
>
>
> To post a message, send it to: xml-dbms@yahoogroups.com
> To unsubscribe, send a blank message to: xml-dbms-unsubscribe@yahoogroups.
> com
> Or join the newsgroup at: news://news.barkto.com/homeless.xml.xml-dbms
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>


[Non-text portions of this message have been removed]

You probably didn't forget to attach them. This list does not accept
attachments as a protection against viruses. You can either:

1) Send the entire files inline (if they are not too big),
2) Excerpt relevant pieces of the files and send them inline, or
3) Send the files to me privately (if they are too big or not of general
interest to the list).

-- Ron

Gudmundur Arni Thorisson wrote:
>
> Damn. Forgot to attach the documents!:
>
>        Mummi

Gudmundur Arni Thorisson wrote:

>    Thanks a lot, Ron. I agree with much of of what you say in your previous
> reply. The reason why I looked at Xindice (and still am) in the first
> place is that the records in question are A) fairly rigidly structured B)
> and relatively small, but C) large numbers of them. The 'large numbers' is
> in the hundreds-millions range. There is a chance that commercial
> offerings (i.e. cheaper than Oracle!) could be used, such as Tamino, given
> that they have the necessary scalability. But I'll take a look at eXist as
> well.

OK. Scalability is something you'll have to figure out on your own. In
the commercial RDBMS world, millions of rows is not that big. I don't
know about the native XML database world.

By the way, XML-DBMS doesn't use the most efficient algorithm when
retrieving data, so it might have scalability problems as well. Both
size of the database and depth of the XML document will affect
scalability, although with good indexing I suspect that database size
will be less of a factor than depth.

Has anybody used XML-DBMS with databases with millions of rows?

>    As for the schema-2-tables issues that you commented on in the other
> mail; the reason why our schema structure would be a lot more rigid and
> small scale than say BSML or AGAVE is that those are supposed to be
> general genome-annotation centric formats, something that can be loaded
> into e.g. compliant GUI browsers and viewed and manipulated. This stuff is
> supposed to be much more project-specific but extensible and/or easily
> transformable (using XML-technologies) into other formats.

1) I took a look at AGAVE (BSML's web site wasn't up) and it wouldn't be
wholly unreasonable to map it to a relational database. However, the
result will spread data across quite a few tables and I won't guarantee
it will scale well. With respect to automatically generating the
database schema from the AGAVE DTD, I don't think the lack of data
typing is really an issue. The DTD is small enough that a human could
hand-enter the data types in a day or less. (This assumes that the data
types are known, which is the case if an XML Schema is available.)

2) Your comment that "this stuff is supposed to be ... extensible" might
be a problem for any product (including XML-DBMS) that uses an
object-relational mapping. The reason is that these products require
design-time mapping. There are two ways to handle extensions:

a) Map them at design time. This may be a problem if lots of different
people have lots of different extensions.

b) Ignore them. This is more sensible than it sounds. The idea is that a
particular application is interested in a particular set of data and can
safely ignore the rest. The only time this is not true is when
extensions change the semantics of existing elements/attributes.
Although this is bad design, it is a possibility, so a human should be
involved at design time before accepting a document from a particular
source.

(I don't know how other products handle elements / attributes that are
not of interest. XML-DBMS simply ignores them. If a product requires all
elements/attributes to be mapped, you can use XSLT to remove the
elements / attributes you are not interested in before passing the
document to the product.)

One advantage of a native XML database here is that it can silently
store any extension (unless it is set up to validate the document
against a particular DTD or schema that doesn't allow the extension).

However, this just moves the problem from the database to the
application. That is, the application must now decide how to handle the
extension elements / attributes. As it turns out, the choices are
roughly the same as those above -- write queries that can handle all
possible extensions or write queries that ignore extension elements /
attributes you aren't interested in.

-- Ron

I took a very quick look at Eclipse. Looks like a good starting point to
me.

Also, what is SWT? Is it a GUI framework like Swing? Is it available on
all platforms?

-- Ron

Adam Flinton wrote:

> I've been going over the code I wrote for the map builder & it suddenly
> struck me that I was rewriting eclipse (www.eclipse.org) right down to
> plugins etc. Given how: (a) This seems silly as I use eclipse & does
> anyone really need another standalone tool...?
> (b) The number of plugins for eclipse is growing fast (esp the free
> ones) inc XML editors etc & frankly I see the pace of this picking up
> not slowing down.
> (c) eclipse seems to be steadily conquering all the other ide'es
> especially in the java world & as such an eclipse plugin might be more
> usefull to people anyway.
> (d) eclipse is both opensource & ported/being ported to just about every OS
>
> I am contemplating (re)doing the mapbuilder as an Eclipse plugin
>
> Obviously this will mean doing the GUI in SWT rather than swing however
> the underlying models etc don't need to change as they work on the
> JDBC/JDOM Level so....
>
> Does anyone have any massive objections? Does anyone prefer that it be
> done as an eclipse plugin?
>
> I have to say that I spent yesterday playing with eclipse/swt & maybe
> it's because it was written some time after Swing & maybe it's because
> of it's native peers however it seems to be to be a much better GUI toolkit.

OK, I see. Here are two or three inline examples of what we have so far,
   with some annotation:

#The root element defining the 400M instance class:
<xs:element name="genotype">
    <xs:annotation>
      <xs:documentation xml:lang="en">
        A single genotype for a sample. This applies to a particular SNP
        and an assay for that SNP. The values that the allele 'base'
attribute
        can take is restricted to the familiar A/C/T/G values plus 'N' for
        an unknown genotype (e.g. when the experiment yielded a call
        on one of the two chromosomes but not the other.
        Note also that we keep track of genotypes that failed, as the
genotyping
        labs send in ALL genotype results, not just the successful ones. This
        enables the DCC to compute success rate and similar stats.
      </xs:documentation>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:element name="snp_assay">
          <xs:complexType>
            <xs:attribute name="lsid" type="lsidType" use="required"/>
          </xs:complexType>
        </xs:element>
        <xs:element name="sample">
          <xs:complexType>
            <xs:attribute name="lsid" type="lsidType" use="required"/>
          </xs:complexType>
        </xs:element>
        <xs:element name="alleles" minOccurs="0">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="allele" minOccurs="2" maxOccurs="2">
                <xs:complexType>
                  <xs:attribute name="base"  type="alleleType" use="required"
/>
                </xs:complexType>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
      <xs:attribute name="lsid" type="lsidType" use="required"/>
    </xs:complexType>
</xs:element>

    And here is a made-up instance of the above:

      <genotype lsid="urn:LSID:genome.wi.mit.edu:HapMap/Genotype:883423434:1"
  >
        <snp_assay
lsid="urn:LSID:genome.wi.mit.edu:HapMap/SNPAssay:300004343:1"/>
        <sample lsid="urn:LSID:hapmap.org:HapMap/Sample:1004:1"/>
        <genotyping_protocol
lsid="urn:LSID:genome.wi.mit.edu:HapMap/Protocol:0034:1:1/>
        <alleles>
          <allele base="G"/>
          <allele base="T"/>
        </alleles>
      </genotype>

    There are more components in the schema that define either this sort of
small elements that store information on biologically or experimentally
relevant things, plus a few container things that will be used for things
like query results, batch data submission from collaborating laboratories,
   etc. But compared to what you have already looked at, AGAVE BSML and
similar, this schema of ours is NOT trying to do the same thing. These
formats are more general and thus may, as you noticed, be rather
unspecific in many regards. Our schema will be much more specific to the
project and more targeted towards the data exchange and internal handling
within our system. BUT, as I may have mentioned before, a major point with
using XML in the first place is that it will be sooo easy to write a
program that queries the database (of whatever kind) for, say, all
haplotype blocks on a chromosome and reformat them (via XSL possibly) into
AGAVE, BSML, GFF (Gene Finding Format), flatfile, summary report or other
kind of files that would be useful to the research community.

     I do however agree with you that the size of our schema will probably
not be unmanageble via manual XML-2-relational mappings. A hassle, but
doable. That is certainly an option, depending on how things go with
native XML db investigations and/or evaluation of XML-on-top-of-RDMS,
including your XML-DBMS.

     Thanks again for the useful remarks, Ron, I appreciate your help.


             Mummi, CSHL



On Friday, December 20, 2002, at 10:29 PM, Ronald Bourret wrote:

> You probably didn't forget to attach them. This list does not accept
> attachments as a protection against viruses. You can either:
>
> 1) Send the entire files inline (if they are not too big),
> 2) Excerpt relevant pieces of the files and send them inline, or
> 3) Send the files to me privately (if they are too big or not of general
> interest to the list).
>
> -- Ron
>
> Gudmundur Arni Thorisson wrote:
>>
>> Damn. Forgot to attach the documents!:
>>
>>        Mummi
>
>
> To post a message, send it to: xml-dbms@yahoogroups.com
> To unsubscribe, send a blank message to: xml-dbms-unsubscribe@yahoogroups.
> com
> Or join the newsgroup at: news://news.barkto.com/homeless.xml.xml-dbms
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>

If this is typical of your elements, I would strongly suggest looking at
using an XML-enabled database rather than a native XML database. The
reason for this is (1) that using an XML-enabled database means that
SQL-based applications can also look at the data (2) this element
flattens very nicely into the following two tables.

    genotypes:
    ---------
    genotype_lsid
    snp_assay_lsid
    sample_lsid
    protocol_lsid

    alleles
    -------
    genotype_lsid
    allele

This is a perfect example of XML having extra structure (the snp_assay,
sample, and genotyping_protocol elements) which is not present in the
database.

Depending on what product you use, you may have to use XSLT to remove
this structure when going to or recreate this structure when coming from
the database. XML-DBMS v2 supports this transform natively, and I
suspect some other products do as well.

-- Ron

Gudmundur Arni Thorisson wrote:
>
>    OK, I see. Here are two or three inline examples of what we have so far,
>   with some annotation:
>
> #The root element defining the 400M instance class:
> <xs:element name="genotype">
>    <xs:annotation>
>      <xs:documentation xml:lang="en">
>        A single genotype for a sample. This applies to a particular SNP
>        and an assay for that SNP. The values that the allele 'base'
> attribute
>        can take is restricted to the familiar A/C/T/G values plus 'N' for
>        an unknown genotype (e.g. when the experiment yielded a call
>        on one of the two chromosomes but not the other.
>        Note also that we keep track of genotypes that failed, as the
> genotyping
>        labs send in ALL genotype results, not just the successful ones. This
>        enables the DCC to compute success rate and similar stats.
>      </xs:documentation>
>    </xs:annotation>
>    <xs:complexType>
>      <xs:sequence>
>        <xs:element name="snp_assay">
>          <xs:complexType>
>            <xs:attribute name="lsid" type="lsidType" use="required"/>
>          </xs:complexType>
>        </xs:element>
>        <xs:element name="sample">
>          <xs:complexType>
>            <xs:attribute name="lsid" type="lsidType" use="required"/>
>          </xs:complexType>
>        </xs:element>
>        <xs:element name="alleles" minOccurs="0">
>          <xs:complexType>
>            <xs:sequence>
>              <xs:element name="allele" minOccurs="2" maxOccurs="2">
>                <xs:complexType>
>                  <xs:attribute name="base"  type="alleleType" use="required"
> />
>                </xs:complexType>
>              </xs:element>
>            </xs:sequence>
>          </xs:complexType>
>        </xs:element>
>      </xs:sequence>
>      <xs:attribute name="lsid" type="lsidType" use="required"/>
>    </xs:complexType>
> </xs:element>
>
>    And here is a made-up instance of the above:
>
>      <genotype lsid="urn:LSID:genome.wi.mit.edu:HapMap/Genotype:883423434:1"
>  >
>        <snp_assay
> lsid="urn:LSID:genome.wi.mit.edu:HapMap/SNPAssay:300004343:1"/>
>        <sample lsid="urn:LSID:hapmap.org:HapMap/Sample:1004:1"/>
>        <genotyping_protocol
> lsid="urn:LSID:genome.wi.mit.edu:HapMap/Protocol:0034:1:1/>
>        <alleles>
>          <allele base="G"/>
>          <allele base="T"/>
>        </alleles>
>      </genotype>
>
>    There are more components in the schema that define either this sort of
> small elements that store information on biologically or experimentally
> relevant things, plus a few container things that will be used for things
> like query results, batch data submission from collaborating laboratories,
>   etc. But compared to what you have already looked at, AGAVE BSML and
> similar, this schema of ours is NOT trying to do the same thing. These
> formats are more general and thus may, as you noticed, be rather
> unspecific in many regards. Our schema will be much more specific to the
> project and more targeted towards the data exchange and internal handling
> within our system. BUT, as I may have mentioned before, a major point with
> using XML in the first place is that it will be sooo easy to write a
> program that queries the database (of whatever kind) for, say, all
> haplotype blocks on a chromosome and reformat them (via XSL possibly) into
> AGAVE, BSML, GFF (Gene Finding Format), flatfile, summary report or other
> kind of files that would be useful to the research community.
>
>     I do however agree with you that the size of our schema will probably
> not be unmanageble via manual XML-2-relational mappings. A hassle, but
> doable. That is certainly an option, depending on how things go with
> native XML db investigations and/or evaluation of XML-on-top-of-RDMS,
> including your XML-DBMS.
>
>     Thanks again for the useful remarks, Ron, I appreciate your help.
>
>             Mummi, CSHL

I want to be able to store an activity element which in turn stores
an activity element.
<activities>
        <activity>
            ......
            ......
              <activity>
                   ......
                   ......
              </activity>
              <activity>
                  ......
                  ......
              </activity>
         </activity>
</activities>

I am able to map an activity type element fine but xmldbms seems to
ignore any child <activity> elements.It is mapped as follows:

<ClassMap>
       <ElementType Name="act:activity"/>
       <ToClassTable Name="ACTIVITY"/>
       <PropertyMap>
        ...............
       </PropertyMap>
       <RelatedClass KeyInParentTable="Unique">
       ..................
       </RelatedClass>
</ClassMap>

I think it what is happening is that because the activity element is
not explicitly mapped within the activity class map is is not being
recognised.Any suggestions would be greatly appreciated.

Thanks
Ruairi

My XML document is as follows:

<activity>
         <dr:dataKey>
                <dr:recordId>2</dr:recordId>
                <dr:applicationId>child</dr:applicationId>
          </dr:dataKey>
          <parentActivity>
               <dr:recordId>2</dr:recordId>
               <dr:applicationId>parent</dr:applicationId>
          </parentActivity>
          ...............
          ...............
          ...............
</activity>

The two tables I would like to be able to map these elements to are

Activity(recordId,applicationId)
primary key - recordId,applicationId


SubActivities
(childRecordId,childApplicationId,parentRecordId,parentApplicationId)
primary key - childRecordId,childApplicationId

foreign key(1) - childRecordId,childApplicationId references
Activity(recordId,applicationId)

foreign key (2)- parentRecordId,parentApplicationId references
Activity(recordId,applicationId)

My problem is that I am unable to map the relationship as to related
class map without removing foreign key(2).

I would like to be able to map <parentActivity> to the Activity class
and as a related class to the subActivities table.Any suggestions?

Thanks
Ruairi

To map an element recursively, you need two things:

1) The table to which the element is mapped must contain a foreign key pointing
to itself, and

2) The ClassMap for the element must contain a RelatedClass map pointing to the
ClassMap.

Suppose your ACTIVITY table looks like this

ACTIVITY (ActivityID, ParentActivityID, ...)

Then your map should look like this:

  <ClassMap>
        <ElementType Name="act:activity"/>
        <ToClassTable Name="ACTIVITY"/>
        <PropertyMap>
         ...............
        </PropertyMap>
        <RelatedClass KeyInParentTable="Unique">
           <ElementType Name="act:activity"/>
           <UseUniqueKey Name="name of key pointing to ActivityID">
           <UseForeignKey Name="name of key pointing to ParentActivityID">
        </RelatedClass>
  </ClassMap>

Also, I assume that you have namespaces set up correctly, as detailed in:

    http://groups.yahoo.com/group/xml-dbms/message/2530

-- Ron

> I want to be able to store an activity element which in turn stores
> an activity element.
> <activities>
>        <activity>
>            ......
>            ......
>              <activity>
>                   ......
>                   ......
>              </activity>
>              <activity>
>                  ......
>                  ......
>              </activity>
>         </activity>
> </activities>
>
> I am able to map an activity type element fine but xmldbms seems to
> ignore any child <activity> elements.It is mapped as follows:
>
> <ClassMap>
>       <ElementType Name="act:activity"/>
>       <ToClassTable Name="ACTIVITY"/>
>       <PropertyMap>
>        ...............
>       </PropertyMap>
>       <RelatedClass KeyInParentTable="Unique">
>       ..................
>       </RelatedClass>
> </ClassMap>
>
> I think it what is happening is that because the activity element is
> not explicitly mapped within the activity class map is is not being
> recognised.Any suggestions would be greatly
appreciated.