On 09/12/2009 16:41, sayajirao4 wrote:
> Hello,
>
> I am evaluating Jena and experimenting with the various ways of persistent
storage available with Jena.  I also came across a paper on clustered TDB:
>
> http://eprints.ecs.soton.ac.uk/16974/1/www2009fixedref.pdf
>
> Is the clustered version under development? Is it going to be available any
time soon?  Andy, since you're one of the authors, would you tell us more about
it?

A clustered version is under development.

It will not be ready any time soon. It's very early days and does not
even load data at the moment, let alone answer queries.

It's derived from the design in that paper and is a complete
implementation (the prototype experiments weren't for a complete
system).  It also turns out that the design of each of the node types
can be separated out from the design of the distribution layer so there
will be several possible backend designs to try out.

What are you experimenting with regards to persistent storage? Sounds
interesting ...

	 Andy

>
> Thanks,
> Sunil

Here's the output from the windows term...

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 5.0.51a-community-nt MySQL Community Edition (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> use ivhm;
Database changed
mysql> show index from nodes;
+-------+------------+----------+--------------+-------------+-----------+------
-------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardi
nality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+------
-------+----------+--------+------+------------+---------+
| nodes |          0 | PRIMARY  |            1 | hash        | A         |
   2821 |     NULL | NULL   |      | BTREE      |         |
+-------+------------+----------+--------------+-------------+-----------+------
-------+----------+--------+------+------------+---------+
1 row in set (0.00 sec)

mysql> show index from quads;
+-------+------------+-------------+--------------+-------------+-----------+---
----------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name    | Seq_in_index | Column_name | Collation | Ca
rdinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-------------+--------------+-------------+-----------+---
----------+----------+--------+------+------------+---------+
| quads |          0 | PRIMARY     |            1 | g           | A         |
        20 |     NULL | NULL   |      | BTREE      |         |
| quads |          0 | PRIMARY     |            2 | s           | A         |
      6253 |     NULL | NULL   |      | BTREE      |         |
| quads |          0 | PRIMARY     |            3 | p           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          0 | PRIMARY     |            4 | o           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | SubjPredObj |            1 | s           | A         |
      2084 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | SubjPredObj |            2 | p           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | SubjPredObj |            3 | o           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | PredObjSubj |            1 | p           | A         |
        12 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | PredObjSubj |            2 | o           | A         |
      6253 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | PredObjSubj |            3 | s           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | ObjSubjPred |            1 | o           | A         |
      4169 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | ObjSubjPred |            2 | s           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | ObjSubjPred |            3 | p           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | GraPredObj  |            1 | g           | A         |
        38 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | GraPredObj  |            2 | p           | A         |
       403 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | GraPredObj  |            3 | o           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | GraObjSubj  |            1 | g           | A         |
         2 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | GraObjSubj  |            2 | o           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
| quads |          1 | GraObjSubj  |            3 | s           | A         |
     12507 |     NULL | NULL   |      | BTREE      |         |
+-------+------------+-------------+--------------+-------------+-----------+---
----------+----------+--------+------+------------+---------+
19 rows in set (0.08 sec)

mysql>

--- In jena-dev@yahoogroups.com, Damian Steer <pldms@...> wrote:
>
> On 09/12/09 14:09, Earnest_T_Bass wrote:
> > Hello, I'm transitioning our app to use the SDB database, away from
> > the deprecated RDB.  It's been pretty straightforward to do, but I'm
> > experiencing much poorer SPARQL Query performance.  While I haven't
> > taken any specific timing measurements, I'm guessing it's on the
> > order of 3 to 5 times slower for select queries and construct queries
> > as well.  The models I'm using are small (<100,000 triples).
> >
> > Below is the my store config and store and model creation code, as
> > well as what is used to execute the select query.
> >
> > Note that I'm using the same database instance that I used for RDB,
> > so it has all the same configurations.
> >
> > Any ideas on how I might improve performance?  If you'd like more
> > specific on my queries and timing info I can provide.
>
> Hi Earnest,
>
> Could you log into the database and check the indexes on tables 'Nodes'
> and 'Quads'? SHOW INDEX is the command, iirc.
>
> There was an unfortunate bug around here in the past, but it is easily
> fixable :-/
>
> Damian
>

Hello,

I am evaluating Jena and experimenting with the various ways of persistent
storage available with Jena.  I also came across a paper on clustered TDB:

http://eprints.ecs.soton.ac.uk/16974/1/www2009fixedref.pdf

Is the clustered version under development? Is it going to be available any time
soon?  Andy, since you're one of the authors, would you tell us more about it?

Thanks,
Sunil

On 09/12/09 14:09, Earnest_T_Bass wrote:
> Hello, I'm transitioning our app to use the SDB database, away from
> the deprecated RDB.  It's been pretty straightforward to do, but I'm
> experiencing much poorer SPARQL Query performance.  While I haven't
> taken any specific timing measurements, I'm guessing it's on the
> order of 3 to 5 times slower for select queries and construct queries
> as well.  The models I'm using are small (<100,000 triples).
>
> Below is the my store config and store and model creation code, as
> well as what is used to execute the select query.
>
> Note that I'm using the same database instance that I used for RDB,
> so it has all the same configurations.
>
> Any ideas on how I might improve performance?  If you'd like more
> specific on my queries and timing info I can provide.

Hi Earnest,

Could you log into the database and check the indexes on tables 'Nodes'
and 'Quads'? SHOW INDEX is the command, iirc.

There was an unfortunate bug around here in the past, but it is easily
fixable :-/

Damian

On 09/12/2009 12:22, max.gorbunov wrote:
> Hi all,
>
> Look at this code fragment:
>
>          Model m = ModelFactory.createDefaultModel();
>          m.createResource("http://example.org/#-1", RDFS.Resource);
>          m.write(new FileWriter("test.ttl"), "TURTLE");
>          m.read(new FileReader("test.ttl"), null, "TURTLE");
>
>          m.setNsPrefix("eg", "http://example.org/#");
>          m.write(new FileWriter("test.ttl"), "TURTLE");
>          m.read(new FileReader("test.ttl"), null, "TURTLE");
>
> It contains the valid URI which can be written and read without prefixes.
> But if I try to add prefix, it is written as eg:-1, which is invalid prefixed
name and cannot be read. In case of<http://example.org/#@1>, it works fine since
the URI doesn't become prefixed name.
>
> Thanks,
> Max
>

Max,

Thanks for the report.

The Turtle writer wasn't being strict enough. I've fixed this and it's
now in Jena CVS.

	 Andy

On 09/12/2009 00:21, Gokhan Soydan wrote:
> Hello,
>
> A Jena assembler file supports defining stores, multiple graphs and where the
content of the graphs come from. The SDB documentation for loading shows
examples where the assembler files only define stores and don't define any
graphs. The sdbload command loads to one named graph, where the named graph and
the file location for the data are specified explicitly at the command line.
>
> Is it possible to define named graphs in an assembler file to be loaded? Are
there any examples?
>

This isn't possible.  If the content were loaded, it would be difficult
to know that on a restart so that it was not loaded again.  So the
assembler description is for connecting to a store, not loading it.

	 Andy

> Thanks,
> Gokhan
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________

stijn35811 wrote:
> Hey All,
>
> I have a problem parsing websites with GRDDL.
>
> I use this code:
>
>    Model m = ModelFactory.createDefaultModel();
>    RDFReader r = m.getReader("GRDDL");
>    r.setProperty("grddl.rdfa", true);
>    r.read(m, "some website...");
>
> And get the exception pasted at the end.
>
> Most of the times when I download the html file locally and delete the first
line about DOCTYPE it works just fine. But how can I overcome this?

I do not have an answer to your question, sorry.

But, if it is RDFa you are interested in, you might
have a look at: http://github.com/shellac/java-rdfa
(written by Damian).

Have fun.

Paolo

Hello,
    I'm transitioning our app to use the SDB database, away from the deprecated
RDB.  It's been pretty straightforward to do, but I'm experiencing much poorer
SPARQL Query performance.  While I haven't taken any specific timing
measurements, I'm guessing it's on the order of 3 to 5 times slower for select
queries and construct queries as well.  The models I'm using are small (<100,000
triples).

  Below is the my store config and store and model creation code, as well as what
is used to execute the select query.

Note that I'm using the same database instance that I used for RDB, so it has
all the same configurations.

Any ideas on how I might improve performance?  If you'd like more specific on my
queries and timing info I can provide.

@prefix sdb:     <http://jena.hpl.hp.com/2007/sdb#> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .
@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ja:      <http://jena.hpl.hp.com/2005/11/Assembler#> .

# MySQL - InnoDB


<#store> rdf:type sdb:Store ;
     sdb:layout     "layout2" ;
     sdb:connection <#conn> ;
     sdb:engine     "InnoDB" ;      # MySQL specific
.

<#conn> rdf:type sdb:SDBConnection ;
     sdb:jdbcURL       "jdbc:mysql://localhost/ivhm?autoReconnect=true";
     sdb:sdbType       "MySQL" ;    # Needed for JDBC URL
     sdb:sdbHost       "localhost" ;
     sdb:sdbName       "ivhm" ;
     sdb:sdbUser       "xxx";
     sdb:sdbPassword   "xxx";
     sdb:driver        "com.mysql.jdbc.Driver";
     .



    //Model Creation code
    store = SDBFactory.connectStore("sdb.ttl") ;


    _baseModel = SDBFactory.connectNamedModel(store, this.getURI());

    ontSpec = new OntModelSpec(OntModelSpec.OWL_MEM);
    _memModel = ModelFactory.createOntologyModel(ontSpec, _baseModel);



    // Query execution code
    qe = QueryExecutionFactory.create(selQuery, _memModel,
                                      queryMap);
    ResultSet results = qe.execSelect();

    //Process results...

    qe.close();

Hey All,

I have a problem parsing websites with GRDDL.

I use this code:

    Model m = ModelFactory.createDefaultModel();
    RDFReader r = m.getReader("GRDDL");
    r.setProperty("grddl.rdfa", true);
    r.read(m, "some website...");

And get the exception pasted at the end.

Most of the times when I download the html file locally and delete the first
line about DOCTYPE it works just fine. But how can I overcome this?

Thanks,
Stijn

----

Exception output:

null
ERROR [main] (RDFDefaultErrorHandler.java:40) - java.io.IOException: Server
returned HTTP response code: 503 for URL:
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
ERROR [main] (RDFDefaultErrorHandler.java:40) - java.io.IOException: Server
returned HTTP response code: 503 for URL:
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
Exception in thread "main" net.sf.saxon.trans.DynamicError: java.io.IOException:
Server returned HTTP response code: 503 for URL:
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
	 at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:313)
	 at net.sf.saxon.event.Sender.send(Sender.java:142)
	 at net.sf.saxon.IdentityTransformer.transform(IdentityTransformer.java:29)
	 at com.hp.hpl.jena.grddl.impl.GRDDL.initialParse(GRDDL.java:234)
	 at com.hp.hpl.jena.grddl.impl.GRDDL.go(GRDDL.java:199)
	 at com.hp.hpl.jena.grddl.GRDDLReader.read(GRDDLReader.java:47)
	 at sites.ParseSites.getModel(ParseSites.java:32)
	 at sites.ParseSites.queryWebSite(ParseSites.java:39)
	 at sites.ParseSites.main(ParseSites.java:20)
Caused by: java.io.IOException: Server returned HTTP response code: 503 for URL:
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
	 at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.jav\
a:1313)
	 at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLEntityManager.startDTDEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLDTDScannerImpl.setInputSource(Unknown Source)
	 at org.apache.xerces.impl.XMLDocumentScannerImpl$DTDDispatcher.dispatch(Unknown
Source)
	 at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
	 at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	 at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	 at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	 at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	 at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:300)
	 ... 8 more
---------
java.io.IOException: Server returned HTTP response code: 503 for URL:
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
	 at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.jav\
a:1313)
	 at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLEntityManager.startDTDEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLDTDScannerImpl.setInputSource(Unknown Source)
	 at org.apache.xerces.impl.XMLDocumentScannerImpl$DTDDispatcher.dispatch(Unknown
Source)
	 at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
	 at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	 at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	 at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	 at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	 at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:300)
	 at net.sf.saxon.event.Sender.send(Sender.java:142)
	 at net.sf.saxon.IdentityTransformer.transform(IdentityTransformer.java:29)
	 at com.hp.hpl.jena.grddl.impl.GRDDL.initialParse(GRDDL.java:234)
	 at com.hp.hpl.jena.grddl.impl.GRDDL.go(GRDDL.java:199)
	 at com.hp.hpl.jena.grddl.GRDDLReader.read(GRDDLReader.java:47)
	 at sites.ParseSites.getModel(ParseSites.java:32)
	 at sites.ParseSites.queryWebSite(ParseSites.java:39)
	 at sites.ParseSites.main(ParseSites.java:20)
com.hp.hpl.jena.shared.JenaException: rethrew: net.sf.saxon.trans.DynamicError:
java.io.IOException: Server returned HTTP response code: 503 for URL:
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
	 at com.hp.hpl.jena.grddl.impl.GRDDL.initialParse(GRDDL.java:254)
	 at com.hp.hpl.jena.grddl.impl.GRDDL.go(GRDDL.java:199)
	 at com.hp.hpl.jena.grddl.GRDDLReader.read(GRDDLReader.java:47)
	 at sites.ParseSites.getModel(ParseSites.java:32)
	 at sites.ParseSites.queryWebSite(ParseSites.java:39)
	 at sites.ParseSites.main(ParseSites.java:20)
Caused by: net.sf.saxon.trans.DynamicError: java.io.IOException: Server returned
HTTP response code: 503 for URL:
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
	 at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:313)
	 at net.sf.saxon.event.Sender.send(Sender.java:142)
	 at net.sf.saxon.IdentityTransformer.transform(IdentityTransformer.java:29)
	 at com.hp.hpl.jena.grddl.impl.GRDDL.initialParse(GRDDL.java:234)
	 ... 5 more
Caused by: java.io.IOException: Server returned HTTP response code: 503 for URL:
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
	 at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.jav\
a:1313)
	 at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLEntityManager.startDTDEntity(Unknown Source)
	 at org.apache.xerces.impl.XMLDTDScannerImpl.setInputSource(Unknown Source)
	 at org.apache.xerces.impl.XMLDocumentScannerImpl$DTDDispatcher.dispatch(Unknown
Source)
	 at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
	 at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	 at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	 at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	 at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	 at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:300)
	 ... 8 more

Hi all,

Look at this code fragment:

         Model m = ModelFactory.createDefaultModel();
         m.createResource("http://example.org/#-1", RDFS.Resource);
         m.write(new FileWriter("test.ttl"), "TURTLE");
         m.read(new FileReader("test.ttl"), null, "TURTLE");

         m.setNsPrefix("eg", "http://example.org/#");
         m.write(new FileWriter("test.ttl"), "TURTLE");
         m.read(new FileReader("test.ttl"), null, "TURTLE");

It contains the valid URI which can be written and read without prefixes.
But if I try to add prefix, it is written as eg:-1, which is invalid prefixed
name and cannot be read. In case of <http://example.org/#@1>, it works fine
since the URI doesn't become prefixed name.

Thanks,
Max

ktslwy wrote:
> I only saw the ways to create an ontology model in memory mode or at most
backed by RDB. Is it possible to make it backed by TDB?

Yes, an Ontology model is just a wrapper round a base model. You can
take an TDB model and wrap it as an OntModel quite happily.

> If no, does it mean I must assign a large memory block to the application
doing reasoning?

That's a different issue - reasoning and OntModels are orthogonal. All
the built-in reasoners keep their intermediate state in memory. Using a
database or TDB  model will not increase the scale of reasoning
possible, it will just slow things down.

Dave

I only saw the ways to create an ontology model in memory mode or at most backed
by RDB. Is it possible to make it backed by TDB?

If no, does it mean I must assign a large memory block to the application doing
reasoning?

Thanks

Hello,

A Jena assembler file supports defining stores, multiple graphs and where the
content of the graphs come from. The SDB documentation for loading shows
examples where the assembler files only define stores and don't define any
graphs. The sdbload command loads to one named graph, where the named graph and
the file location for the data are specified explicitly at the command line.

Is it possible to define named graphs in an assembler file to be loaded? Are
there any examples?

Thanks,
Gokhan

On 08/12/2009 18:37, Uday Subbarayan wrote:
>
>
> Hi All,
> Is Jena can be used inside a commercial product? It looks to me that
> it's ok but just want to confirm with the community. Any one used jena
> inside a shipping commercial product?
>
> Thanks,
> -Uday.

The Jena license is the definitive source:
    http://www.openjena.org/license.html
and if you want a cast-iron judgement, you need to take that to a
lawyer.  Only the words matter, not the intent of the license.

I am not a lawyer.

But that was the idea why we used that particular license.

The BSD-style license does mean you can use it in a commercial product.
   It is being used as such in both software products and service offerings.

My current employer (Talis) uses it.

There quite a few others but I'll let them talk about their offerings.

	 Andy

On 08/12/2009 13:58, Thorsten Möller wrote:
> Hi Andy,
>
> wow, in-depth information. Thanks again. Some comments inline ...
>
>
> On Saturday, December 05, 2009 6:49 PM [GMT+1=CET],
> Andy Seaborne<afs.lists@...>  wrote (with possible deletions):
>
>>> not sure if the the paper by Abadi et al [1] is known to you, but how
>>> does TDB compare conceptually to their approach? In other words, I
>>> was trying to find some information about TDB's approach to
>>> materialize triples on disk (column-wise vs. row-wise vs. ....). I
>>> couldn't get more out of the information on the web site than that
>>> it seems to make use a lot of (B-Tree?) indexing.
>>
>> I know the paper. Their design is quite similar to that for the RDB
>> database layout, which avoids joins by keeping the node details
>> directly in the triple table.  Their triple table is quite wide and
>> so the use of a column store makes some sense.  They show figures on
>> the Barton dataset for about 50 million triples.
>>
>> I'll write an entry on the TDB wiki to outline the design but teh key
>> points here are that TDB uses a dictionary for nodes.
> SWStore also use dictionary encoding; their two-attribute colums have
> fixed-length (of 8 bytes).
>
>>   It's a
>> normalized design.  The id for a Node is an 8 byte value and triples
>> are stored as 24 byte records (32 for named graphs) in three indexes
>> (6 for the quads).   Values are inlined (there's not node table entry
>> for the integer 3, say).  The fact here are lots of indexes for quads
>> does not matter a great deal for query because unused indexes simply
>> sit on disk and don't take RAM space and RAM which is a critical
>>    performance matter.
> I didn't get the last sentence.

Managing RAM is the critical factor.  Having indexes, and not using them
much, falls out as OK because of caching.

In fact, caching is everything and getting more so on modern machines.

This is an amusing look at modern disk usage:
http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf

>> The indexes are cluster indexes, and are custom threaded B+Trees.  A
>> "row" is 24 bytes - there is no row overhead unlike SQL databases
>> where there the extra bytes per row can be of the same amount.
> Also in SWStore the header is "small" because tuples have fixed length
> and never contain NULL. In addition, the header is stored in a separate
> column because it can be selectively ignored.

There have been several column-ish designs proposed for RDF stores in a
non-SQL world - not sure if any have ever made it to deployment.

The point for me is that because a row is 24 bytes, the complexity of a
column store needs to be weighed against it's advantages and the key
advantage of column stores is retrieving just the columns of interest.

In RDF two things emerge:

1/ The predicate is usually fixed so that's 8 bytes wasted. But it's 8
in 24, not the data warehousing scale of things where it might be a few
10s of bytes out of 100's, and you have to add back a coordination
column.  And 4 bytes is 4G which is too small :-)

2/ There are two common patterns for joins
     x P y . y Q z (i.e. property chains)
     x P y . x Q z (i.e. same subject)

TDB's default index choice emphasises the same subject access, although
I can think of a way to improve the join processing which is good at the
moment because of cache effects but not ideal; a slight tweak will get
property chains more efficient.


In the latest BSBM figures, queries 9 and 12 let TDB down but that looks
like it's not an indexing issue, more to do with the integration with
ARQ.  Or the GC is going mad.

FYI:
http://seaborne.blogspot.com/2009/10/bsbm-jena.html
has BSBM setup for Jena to run a store from a Jena assembler locally.

Running on EC2, we find that more CPU is a better improvement than more
memory for 100m and 200m (high CPU instances vs high memory instances).
   This is good - it's something that can be improved more easily.

>>> Another question would be if it is planned to extend TDB to a
>>> consistent multiple reader, multiple writer store, that is,
>>> extending it to ensure certain isolation properties. Is SDB the way
>>> to go when applications require isolation properties to be ensured,
>>> or can we expect that TDB will provide similar properties some day.
>>
>> There is a cost to providing additional support so if it's done it's
>> not going to be the single common design.
>>
>> TDB is modular in terms of the storage technology used for node
>> storage and indexing.
> Well, this is important news.

Have fun!

The modularity works, because I've done several store kinds, but if
there is anything to improve, then I'll look at patches.

>
>>   I have done a version of TDB that uses BDB-JE
>> 4.0 for the indexing and node storage.  BDB-JE 4.0 has transactions
>> and also fail-over capabilities so it is possible to enhance TDB with
>> some transactional capabilities using that storage layer.
> Interesting.
>
>
>> While I have made sure it works as a direct replacement for TDB custom
>> node and index storage, I have not integrated in the transactional
>> capabilities which is matter of making the overall code base reflect
>> the possibility on a per-dataset basis.
>>
>> TDB/BDB is reasonably fast (not as fast as custom but close enough),
>> and certainly fast enough to be serious.  It passes TDB's test suite
>> and also the BSBM benchmark sets I tried it with.
> Does this mean that it will be "released" as an alternative storage
> layer some day?

The code is available if anyone wants it. It tracks the TDB development
project so it will always be runnable.  I could split it into it's own
experimental project if someone is actively interested.

Releasing has a support cost on me - I have no plans for that at the
moment.   There is lots of advice on tuning BDB-JE (Project Voldemort
for example) via Google.

>> FYI: I've just added MRSW checking into TDB in SVN - misusing the
>> Multiple reader OR Single Writer policy will probably result in a
>> ConcurrentModificationExceptio.
> I Agree. The (unaware) user should see an exception rather than not
> seeing that data was probably damaged.
>
> -- Thorsten

	 Andy

Thanks!  So glad not to have to add a stand-alone utility to setup our webapp on
all our dev and deployment machines, figure out how to setup classpath and other
environment variables to run it (always painful in windows) and then have to add
all that info to our documentation just to initially setup the database.

--- In jena-dev@yahoogroups.com, Damian Steer <pldms@...> wrote:
>
> On 08/12/09 15:22, Earnest_T_Bass wrote:
> >
> > Is there anyway to first test in java to see if the SDB data base has been
formatted?  I guess I could try doing it in the exception handler if nothing
else.
> >
>
> Added relatively recently:
>
> com.hp.hpl.jena.sdb.util.StoreUtils.isFormatted(Store store)
>
> Don't pepper your code with it, unless you have reason to slow that code
> down :-) It checks for the existence of the tables and columns.
>
> Damian
>

On 08/12/09 15:22, Earnest_T_Bass wrote:
>
> Is there anyway to first test in java to see if the SDB data base has been
formatted?  I guess I could try doing it in the exception handler if nothing
else.
>

Added relatively recently:

com.hp.hpl.jena.sdb.util.StoreUtils.isFormatted(Store store)

Don't pepper your code with it, unless you have reason to slow that code
down :-) It checks for the existence of the tables and columns.

Damian

Is there anyway to first test in java to see if the SDB data base has been
formatted?  I guess I could try doing it in the exception handler if nothing
else.


--- In jena-dev@yahoogroups.com, "Seaborne, Andy" <andy_seaborne@...> wrote:
>
>
>
> > -----Original Message-----
> > From: jena-dev@yahoogroups.com [mailto:jena-dev@yahoogroups.com] On Behalf
> > Of dotmedia08
> > Sent: 24 July 2008 19:04
> > To: jena-dev@yahoogroups.com
> > Subject: [jena-dev] Re: SDB problem - Table '???.triples' doesn't exist
> >
> > Is there a way to call "sdbconfig" from the Java application?
>
> Store.getTableFormatter().create() ;
>
> But it will happen everytimne you run the program so usually one would create
the database (however the DB does that), write the sdb.ttl file, run sdbconfig
and then run the application.
>
> >
> > If not, how can I run it for Windows XP platform?
>
> Set the classpath and run
>
> java -cp .... sdb.sdbconfig .....
>
> In Java6, a classpath of <SDBROOT>/lib/* will work for suitable SDBROOT.
>
> I do my development on Windows and use the scripts in the SDB distribution.
> I install cygwin to get a usable command shell and lots of utilities :-)
>
>         Andy
>
> >
> > Sorry for my newbie question, I'm just starting to play Jena and SDB.
> >
> > Thanks a lot for your help.
> >
> > Karen
> >
> > --- In jena-dev@yahoogroups.com, "Seaborne, Andy" <andy_seaborne@>
> > wrote:
> > >
> > > The database exists but has not been configured (run sdbconfig)
> > >
> > > I ran MySQL and created database "foo":
> > >
> > > 18:22:39 WARN  SDBConnection             :: execQuery: SQLException
> > > Table 'foo.triples' doesn't exist
> > >
> > > As you get.
> > >
> > > Now run sdbconfig, rerun and I get:
> > >
> > > -------------
> > > | s | p | o |
> > > =============
> > > -------------
> > >
> > > So it is because the database is not formatted for SDB.
> > >
> > >
> > > If I call "bar", which does not exist,
> > > :
> > > com.hp.hpl.jena.sdb.SDBException: SQL Exception while connecting to
> > database: jdbc:mysql://localhost/bar : Unknown database 'bar'
> > >
> > >         Andy
> > >
> > > > -----Original Message-----
> > > > From: jena-dev@yahoogroups.com [mailto:jena-dev@yahoogroups.com]
> > On Behalf
> > > > Of dotmedia08
> > > > Sent: 24 July 2008 17:26
> > > > To: jena-dev@yahoogroups.com
> > > > Subject: [jena-dev] Re: SDB problem - Table '???.triples' doesn't
> > exist
> > > >
> > > > Thanks Andy for your swift reply.
> > > > Yes, I have multiple DB's in MySql, "census70k" is one of them, so
> > > > it's "census70k.triples", not "census.triples".
> > > > Sorry, I messed up with "census.triples".
> > > >
> > > > I ran the program again, and here is the results:
> > > >
> > > > =========================
> > > >
> > > > Populating CensusAllSDB from DB is DONE........
> > > >  WARN [main] (SDBConnection.java:299) - execQuery: SQLException
> > > > Table 'census70k.triples' doesn't exist
> > > > SELECT                                   -- V_1=?p V_2=?s V_3=?o
> > > >   R_1.lex AS V_1_lex, R_1.datatype AS V_1_datatype, R_1.lang AS
> > > > V_1_lang, R_1.ty
> > > > pe AS V_1_type,
> > > >   R_2.lex AS V_2_lex, R_2.datatype AS V_2_datatype, R_2.lang AS
> > > > V_2_lang, R_2.ty
> > > > pe AS V_2_type,
> > > >   R_3.lex AS V_3_lex, R_3.datatype AS V_3_datatype, R_3.lang AS
> > > > V_3_lang, R_3.ty
> > > > pe AS V_3_type
> > > > FROM
> > > >     Triples AS T_1                       -- ?s ?p ?o
> > > >   LEFT OUTER JOIN
> > > >     Nodes AS R_1                         -- Var: ?p
> > > >   ON ( T_1.p = R_1.hash )
> > > >   LEFT OUTER JOIN
> > > >     Nodes AS R_2                         -- Var: ?s
> > > >   ON ( T_1.s = R_2.hash )
> > > >   LEFT OUTER JOIN
> > > >     Nodes AS R_3                         -- Var: ?o
> > > >   ON ( T_1.o = R_3.hash )
> > > >
> > > >
> > > > Exception in thread "main"
> > > > com.mysql.jdbc.exceptions.MySQLSyntaxErrorException:
> > > > Table 'census70k.triples' doesn't exist
> > > >         at com.mysql.jdbc.SQLError.createSQLException
> > > > (SQLError.java:936)
> > > >         at com.mysql.jdbc.MysqlIO.checkErrorPacket
> > (MysqlIO.java:2822)
> > > >         at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1536)
> > > >         at com.mysql.jdbc.MysqlIO.sqlQueryDirect
> > (MysqlIO.java:1626)
> > > >         at com.mysql.jdbc.Connection.execSQL(Connection.java:3025)
> > > >         at com.mysql.jdbc.Connection.execSQL(Connection.java:2954)
> > > >         at com.mysql.jdbc.Statement.executeQuery
> > (Statement.java:1009)
> > > >         at com.hp.hpl.jena.sdb.sql.SDBConnection.execQuery
> > > > (SDBConnection.java:11
> > > > 9)
> > > >         at com.hp.hpl.jena.sdb.compiler.QC.exec(QC.java:64)
> > > >         at com.hp.hpl.jena.sdb.compiler.OpSQL.exec(OpSQL.java:53)
> > > >         at com.hp.hpl.jena.sdb.engine.QueryEngineSDB.eval
> > > > (QueryEngineSDB.java:97
> > > > )
> > > >         at
> > com.hp.hpl.jena.sparql.engine.QueryEngineBase.createPlan
> > > > (QueryEngineB
> > > > ase.java:86)
> > > >         at com.hp.hpl.jena.sparql.engine.QueryEngineBase.getPlan
> > > > (QueryEngineBase
> > > > .java:75)
> > > >         at
> > > >
> > com.hp.hpl.jena.sdb.engine.QueryEngineSDB$QueryEngineFactorySDB.creat
> > > > e(QueryEngineSDB.java:137)
> > > >         at com.hp.hpl.jena.sdb.graph.GraphSDB.graphBaseFind
> > > > (GraphSDB.java:165)
> > > >         at com.hp.hpl.jena.graph.impl.GraphBase.find
> > > > (GraphBase.java:240)
> > > >         at
> > com.hp.hpl.jena.graph.compose.DisjointUnion.graphBaseFind
> > > > (DisjointUni
> > > > on.java:27)
> > > >         at com.hp.hpl.jena.graph.impl.GraphBase.find
> > > > (GraphBase.java:240)
> > > >         at com.hp.hpl.jena.graph.impl.GraphBase.graphBaseFind
> > > > (GraphBase.java:260
> > > > )
> > > >         at com.hp.hpl.jena.graph.impl.GraphBase.find
> > > > (GraphBase.java:257)
> > > >         at
> > > > com.hp.hpl.jena.graph.query.SimpleQueryHandler.predicatesFor
> > (SimpleQu
> > > > eryHandler.java:68)
> > > >         at
> > > > com.hp.hpl.jena.graph.query.SimpleQueryHandler.predicatesFor
> > (SimpleQu
> > > > eryHandler.java:47)
> > > >         at com.hp.hpl.jena.rdf.model.impl.ModelCom.listPredicates
> > > > (ModelCom.java:
> > > > 761)
> > > >         at com.hp.hpl.jena.rdf.model.impl.ModelCom.listNameSpaces
> > > > (ModelCom.java:
> > > > 773)
> > > >         at
> > com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.addNameSpaces
> > > > (BaseXMLWri
> > > > ter.java:201)
> > > >         at
> > > > com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.setupNamespaces
> > (BaseXMLW
> > > > riter.java:439)
> > > >         at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write
> > > > (BaseXMLWriter.java
> > > > :421)
> > > >         at com.hp.hpl.jena.xmloutput.impl.Abbreviated.write
> > > > (Abbreviated.java:124
> > > > )
> > > >         at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write
> > > > (BaseXMLWriter.java
> > > > :410)
> > > >         at com.hp.hpl.jena.rdf.model.impl.ModelCom.write
> > > > (ModelCom.java:276)
> > > >         at LoadCensusAllSDB.main(LoadCensusAllSDB.java:40)
> > > > com.hp.hpl.jena.sdb.sql.SDBExceptionSQL: SQLException in executing
> > > > SQL statement
> > > >
> > > >         at com.hp.hpl.jena.sdb.compiler.QC.exec(QC.java:76)
> > > >         at com.hp.hpl.jena.sdb.compiler.OpSQL.exec(OpSQL.java:53)
> > > >         at com.hp.hpl.jena.sdb.engine.QueryEngineSDB.eval
> > > > (QueryEngineSDB.java:97
> > > > )
> > > >         at
> > com.hp.hpl.jena.sparql.engine.QueryEngineBase.createPlan
> > > > (QueryEngineB
> > > > ase.java:86)
> > > >         at com.hp.hpl.jena.sparql.engine.QueryEngineBase.getPlan
> > > > (QueryEngineBase
> > > > .java:75)
> > > >         at
> > > >
> > com.hp.hpl.jena.sdb.engine.QueryEngineSDB$QueryEngineFactorySDB.creat
> > > > e(QueryEngineSDB.java:137)
> > > >         at com.hp.hpl.jena.sdb.graph.GraphSDB.graphBaseFind
> > > > (GraphSDB.java:165)
> > > >         at com.hp.hpl.jena.graph.impl.GraphBase.find
> > > > (GraphBase.java:240)
> > > >         at
> > com.hp.hpl.jena.graph.compose.DisjointUnion.graphBaseFind
> > > > (DisjointUni
> > > > on.java:27)
> > > >         at com.hp.hpl.jena.graph.impl.GraphBase.find
> > > > (GraphBase.java:240)
> > > >         at com.hp.hpl.jena.graph.impl.GraphBase.graphBaseFind
> > > > (GraphBase.java:260
> > > > )
> > > >         at com.hp.hpl.jena.graph.impl.GraphBase.find
> > > > (GraphBase.java:257)
> > > >         at
> > > > com.hp.hpl.jena.graph.query.SimpleQueryHandler.predicatesFor
> > (SimpleQu
> > > > eryHandler.java:68)
> > > >         at
> > > > com.hp.hpl.jena.graph.query.SimpleQueryHandler.predicatesFor
> > (SimpleQu
> > > > eryHandler.java:47)
> > > >         at com.hp.hpl.jena.rdf.model.impl.ModelCom.listPredicates
> > > > (ModelCom.java:
> > > > 761)
> > > >         at com.hp.hpl.jena.rdf.model.impl.ModelCom.listNameSpaces
> > > > (ModelCom.java:
> > > > 773)
> > > >         at
> > com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.addNameSpaces
> > > > (BaseXMLWri
> > > > ter.java:201)
> > > >         at
> > > > com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.setupNamespaces
> > (BaseXMLW
> > > > riter.java:439)
> > > >         at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write
> > > > (BaseXMLWriter.java
> > > > :421)
> > > >         at com.hp.hpl.jena.xmloutput.impl.Abbreviated.write
> > > > (Abbreviated.java:124
> > > > )
> > > >         at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write
> > > > (BaseXMLWriter.java
> > > > :410)
> > > >         at com.hp.hpl.jena.rdf.model.impl.ModelCom.write
> > > > (ModelCom.java:276)
> > > >         at LoadCensusAllSDB.main(LoadCensusAllSDB.java:40)
> > > > Caused by: com.mysql.jdbc.exceptions.MySQLSyntaxErrorException:
> > > > Table 'census70k
> > > > .triples' doesn't exist
> > > >         at com.mysql.jdbc.SQLError.createSQLException
> > > > (SQLError.java:936)
> > > >         at com.mysql.jdbc.MysqlIO.checkErrorPacket
> > (MysqlIO.java:2822)
> > > >         at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1536)
> > > >         at com.mysql.jdbc.MysqlIO.sqlQueryDirect
> > (MysqlIO.java:1626)
> > > >         at com.mysql.jdbc.Connection.execSQL(Connection.java:3025)
> > > >         at com.mysql.jdbc.Connection.execSQL(Connection.java:2954)
> > > >         at com.mysql.jdbc.Statement.executeQuery
> > (Statement.java:1009)
> > > >         at com.hp.hpl.jena.sdb.sql.SDBConnection.execQuery
> > > > (SDBConnection.java:11
> > > > 9)
> > > >         at com.hp.hpl.jena.sdb.compiler.QC.exec(QC.java:64)
> > > >         ... 22 more
> > > > ===========================
> > > >
> > > > Thanks again,
> > > >
> > > > Karen
> > > >
> > > >
> > > >
> > > > --- In jena-dev@yahoogroups.com, "Seaborne, Andy" <andy_seaborne@>
> > > > wrote:
> > > > >
> > > > > Karen,
> > > > >
> > > > > From the stacktrace, it looks like the database "census70k" has
> > not
> > > > been created or formatted.  But the WARN seems to be from a
> > different
> > > > database.
> > > > >
> > > > > > WARN [main] (SDBConnection.java:299) - execQuery: SQLException
> > > > > > Table 'census.triples' doesn't exist
> > > > >
> > > > > Says: 'census.triples'
> > > > >
> > > > > And later:
> > > > >
> > > > > > Exception in thread "main"
> > > > > > com.mysql.jdbc.exceptions.MySQLSyntaxErrorException:
> > > > > > Table 'census70k.triples' doesn't exist
> > > > >
> > > > > Says 'census70k.triples'
> > > > >
> > > > > That's inconsistent.  What databases do you have?  And are there
> > > > different SDB connections to different databases in the MySQL
> > server?
> > > > >
> > > > >         Andy
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: jena-dev@yahoogroups.com [mailto:jena-
> > dev@yahoogroups.com]
> > > > On Behalf
> > > > > > Of dotmedia08
> > > > > > Sent: 24 July 2008 16:46
> > > > > > To: jena-dev@yahoogroups.com
> > > > > > Subject: [jena-dev] SDB problem - Table '???.triples' doesn't
> > > > exist
> > > > > >
> > > > > > Hi,
> > > > > >   I'm testing SDB from a Java application, when I work with a
> > > > small
> > > > > > data (around 500 triples), the program works fine; persists
> > data
> > > > in
> > > > > > DB MySql, and populates it back into memory.
> > > > > >
> > > > > > But when I tried the same program with large date like US
> > census
> > > > (50K
> > > > > > triples and more), I got the following error:
> > > > > >
> > > > > > ====================
> > > > > >
> > > > > > WARN [main] (SDBConnection.java:299) - execQuery: SQLException
> > > > > > Table 'census.triples' doesn't exist
> > > > > > SELECT                                   -- V_1=?p V_2=?s
> > V_3=?o
> > > > > >   R_1.lex AS V_1_lex, R_1.datatype AS V_1_datatype, R_1.lang
> > AS
> > > > > > V_1_lang, R_1.ty
> > > > > > pe AS V_1_type,
> > > > > >   R_2.lex AS V_2_lex, R_2.datatype AS V_2_datatype, R_2.lang
> > AS
> > > > > > V_2_lang, R_2.ty
> > > > > > pe AS V_2_type,
> > > > > >   R_3.lex AS V_3_lex, R_3.datatype AS V_3_datatype, R_3.lang
> > AS
> > > > > > V_3_lang, R_3.ty
> > > > > > pe AS V_3_type
> > > > > > FROM
> > > > > >     Triples AS T_1                       -- ?s ?p ?o
> > > > > >   LEFT OUTER JOIN
> > > > > >     Nodes AS R_1                         -- Var: ?p
> > > > > >   ON ( T_1.p = R_1.hash )
> > > > > >   LEFT OUTER JOIN
> > > > > >     Nodes AS R_2                         -- Var: ?s
> > > > > >   ON ( T_1.s = R_2.hash )
> > > > > >   LEFT OUTER JOIN
> > > > > >     Nodes AS R_3                         -- Var: ?o
> > > > > >   ON ( T_1.o = R_3.hash )
> > > > > >
> > > > > >
> > > > > > Exception in thread "main"
> > > > > > com.mysql.jdbc.exceptions.MySQLSyntaxErrorException:
> > > > > > Table 'census70k.triples' doesn't exist
> > > > > >         at com.mysql.jdbc.SQLError.createSQLException
> > > > > > (SQLError.java:936)
> > > > > >         at com.mysql.jdbc.MysqlIO.checkErrorPacket
> > > > (MysqlIO.java:2822)
> > > > > >         at com.mysql.jdbc.MysqlIO.sendCommand
> > (MysqlIO.java:1536)
> > > > > >         at com.mysql.jdbc.MysqlIO.sqlQueryDirect
> > > > (MysqlIO.java:1626)
> > > > > >         at com.mysql.jdbc.Connection.execSQL
> > (Connection.java:3025)
> > > > > >         at com.mysql.jdbc.Connection.execSQL
> > (Connection.java:2954)
> > > > > >         at com.mysql.jdbc.Statement.executeQuery
> > > > (Statement.java:1009)
> > > > > >         at com.hp.hpl.jena.sdb.sql.SDBConnection.execQuery
> > > > > > (SDBConnection.java:11
> > > > > > 9)
> > > > > >         at com.hp.hpl.jena.sdb.compiler.QC.exec(QC.java:64)
> > > > > >         at com.hp.hpl.jena.sdb.compiler.OpSQL.exec
> > (OpSQL.java:53)
> > > > > >         at com.hp.hpl.jena.sdb.engine.QueryEngineSDB.eval
> > > > > > (QueryEngineSDB.java:97
> > > > > > )
> > > > > >         at
> > > > com.hp.hpl.jena.sparql.engine.QueryEngineBase.createPlan
> > > > > > (QueryEngineB
> > > > > > ase.java:86)
> > > > > >         at
> > com.hp.hpl.jena.sparql.engine.QueryEngineBase.getPlan
> > > > > > (QueryEngineBase
> > > > > > .java:75)
> > > > > >         at
> > > > > >
> > > >
> > com.hp.hpl.jena.sdb.engine.QueryEngineSDB$QueryEngineFactorySDB.creat
> > > > > > e(QueryEngineSDB.java:137)
> > > > > >         at com.hp.hpl.jena.sdb.graph.GraphSDB.graphBaseFind
> > > > > > (GraphSDB.java:165)
> > > > > >         at com.hp.hpl.jena.graph.impl.GraphBase.find
> > > > > > (GraphBase.java:240)
> > > > > >         at com.hp.hpl.jena.graph.GraphUtil.findAll
> > > > (GraphUtil.java:31)
> > > > > >         at
> > com.hp.hpl.jena.rdf.model.impl.ModelCom.listStatements
> > > > > > (ModelCom.java:
> > > > > > 869)
> > > > > >         at LoadCensusAllSDB.main(LoadCensusAllSDB.java:35)
> > > > > > com.hp.hpl.jena.sdb.sql.SDBExceptionSQL: SQLException in
> > executing
> > > > > > SQL statement
> > > > > >
> > > > > >         at com.hp.hpl.jena.sdb.compiler.QC.exec(QC.java:76)
> > > > > >         at com.hp.hpl.jena.sdb.compiler.OpSQL.exec
> > (OpSQL.java:53)
> > > > > >         at com.hp.hpl.jena.sdb.engine.QueryEngineSDB.eval
> > > > > > (QueryEngineSDB.java:97
> > > > > > )
> > > > > >         at
> > > > com.hp.hpl.jena.sparql.engine.QueryEngineBase.createPlan
> > > > > > (QueryEngineB
> > > > > > ase.java:86)
> > > > > >         at
> > com.hp.hpl.jena.sparql.engine.QueryEngineBase.getPlan
> > > > > > (QueryEngineBase
> > > > > > .java:75)
> > > > > >         at
> > > > > >
> > > >
> > com.hp.hpl.jena.sdb.engine.QueryEngineSDB$QueryEngineFactorySDB.creat
> > > > > > e(QueryEngineSDB.java:137)
> > > > > >         at com.hp.hpl.jena.sdb.graph.GraphSDB.graphBaseFind
> > > > > > (GraphSDB.java:165)
> > > > > >         at com.hp.hpl.jena.graph.impl.GraphBase.find
> > > > > > (GraphBase.java:240)
> > > > > >         at com.hp.hpl.jena.graph.GraphUtil.findAll
> > > > (GraphUtil.java:31)
> > > > > >         at
> > com.hp.hpl.jena.rdf.model.impl.ModelCom.listStatements
> > > > > > (ModelCom.java:
> > > > > > 869)
> > > > > >         at LoadCensusAllSDB.main(LoadCensusAllSDB.java:35)
> > > > > > Caused by:
> > com.mysql.jdbc.exceptions.MySQLSyntaxErrorException:
> > > > > > Table 'census70k
> > > > > > .triples' doesn't exist
> > > > > >         at com.mysql.jdbc.SQLError.createSQLException
> > > > > > (SQLError.java:936)
> > > > > >         at com.mysql.jdbc.MysqlIO.checkErrorPacket
> > > > (MysqlIO.java:2822)
> > > > > >         at com.mysql.jdbc.MysqlIO.sendCommand
> > (MysqlIO.java:1536)
> > > > > >         at com.mysql.jdbc.MysqlIO.sqlQueryDirect
> > > > (MysqlIO.java:1626)
> > > > > >         at com.mysql.jdbc.Connection.execSQL
> > (Connection.java:3025)
> > > > > >         at com.mysql.jdbc.Connection.execSQL
> > (Connection.java:2954)
> > > > > >         at com.mysql.jdbc.Statement.executeQuery
> > > > (Statement.java:1009)
> > > > > >         at com.hp.hpl.jena.sdb.sql.SDBConnection.execQuery
> > > > > > (SDBConnection.java:11
> > > > > > 9)
> > > > > >         at com.hp.hpl.jena.sdb.compiler.QC.exec(QC.java:64)
> > > > > >         ... 10 more
> > > > > >
> > > > > > =======================
> > > > > >
> > > > > >
> > > > > > I appreciate any help.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Karen
> > > > >
> > > >
> > > >
> > > >
> > > > ------------------------------------
> > > >
> > > > Yahoo! Groups Links
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
>

Hi Andy,

wow, in-depth information. Thanks again. Some comments inline ...


On Saturday, December 05, 2009 6:49 PM [GMT+1=CET],
Andy Seaborne <afs.lists@...> wrote (with possible deletions):

>> not sure if the the paper by Abadi et al [1] is known to you, but how
>> does TDB compare conceptually to their approach? In other words, I
>> was trying to find some information about TDB's approach to
>> materialize triples on disk (column-wise vs. row-wise vs. ....). I
>> couldn't get more out of the information on the web site than that
>> it seems to make use a lot of (B-Tree?) indexing.
>
> I know the paper. Their design is quite similar to that for the RDB
> database layout, which avoids joins by keeping the node details
> directly in the triple table.  Their triple table is quite wide and
> so the use of a column store makes some sense.  They show figures on
> the Barton dataset for about 50 million triples.
>
> I'll write an entry on the TDB wiki to outline the design but teh key
> points here are that TDB uses a dictionary for nodes.
SWStore also use dictionary encoding; their two-attribute colums have
fixed-length (of 8 bytes).

>  It's a
> normalized design.  The id for a Node is an 8 byte value and triples
> are stored as 24 byte records (32 for named graphs) in three indexes
> (6 for the quads).   Values are inlined (there's not node table entry
> for the integer 3, say).  The fact here are lots of indexes for quads
> does not matter a great deal for query because unused indexes simply
> sit on disk and don't take RAM space and RAM which is a critical
>   performance matter.
I didn't get the last sentence.


> The indexes are cluster indexes, and are custom threaded B+Trees.  A
> "row" is 24 bytes - there is no row overhead unlike SQL databases
> where there the extra bytes per row can be of the same amount.
Also in SWStore the header is "small" because tuples have fixed length
and never contain NULL. In addition, the header is stored in a separate
column because it can be selectively ignored.


>> Another question would be if it is planned to extend TDB to a
>> consistent multiple reader, multiple writer store, that is,
>> extending it to ensure certain isolation properties. Is SDB the way
>> to go when applications require isolation properties to be ensured,
>> or can we expect that TDB will provide similar properties some day.
>
> There is a cost to providing additional support so if it's done it's
> not going to be the single common design.
>
> TDB is modular in terms of the storage technology used for node
> storage and indexing.
Well, this is important news.


>  I have done a version of TDB that uses BDB-JE
> 4.0 for the indexing and node storage.  BDB-JE 4.0 has transactions
> and also fail-over capabilities so it is possible to enhance TDB with
> some transactional capabilities using that storage layer.
Interesting.


> While I have made sure it works as a direct replacement for TDB custom
> node and index storage, I have not integrated in the transactional
> capabilities which is matter of making the overall code base reflect
> the possibility on a per-dataset basis.
>
> TDB/BDB is reasonably fast (not as fast as custom but close enough),
> and certainly fast enough to be serious.  It passes TDB's test suite
> and also the BSBM benchmark sets I tried it with.
Does this mean that it will be "released" as an alternative storage
layer some day?


> FYI: I've just added MRSW checking into TDB in SVN - misusing the
> Multiple reader OR Single Writer policy will probably result in a
> ConcurrentModificationExceptio.
I Agree. The (unaware) user should see an exception rather than not
seeing that data was probably damaged.

-- Thorsten

On Monday 07 December 2009 22:18:35 harshal patni wrote:
> Hey I have specified a mapping for xsd namespace in N3. Its
> still does not pick it up.That is why i feel that this is wrong.

As James says: show code.

Minimal complete examples -- ones we can paste into Eclipse and
run -- are really, really helpful.

--
"It does not need to take events in their correct order."       - DWJ /Hexwood/

Epimorphics Ltd
Registered address: C/O Robson Taylor, Froomsgate House, Bristol
Registered number:  7016688

We need sample, reproducible, minimal code that produces this output,
otherwise we really have no way to help.

On Dec 7, 2009, at 17:18, harshal patni wrote:

> Hey I have specified a mapping for xsd namespace in N3. Its still
> does not pick it up.That is why i feel that this is wrong.
> Thanks,
> Harshal
>
> --- On Mon, 12/7/09, Rob Vesse <rav08r@...> wrote:
>
> From: Rob Vesse <rav08r@...>
> Subject: RE: {Disarmed} [jena-dev] issues with typedLiteral
> To: jena-dev@yahoogroups.com
> Date: Monday, December 7, 2009, 12:53 PM
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> The two are identical, the first which you are getting as output
> is just the full URI for xsd:dateTime
>
> i.e. they mean exactly the same thing, you’ve not done anything
> wrong in creating your typed literal
>
>
>
> Unless you have actually defined the xsd prefix to map to the
> XML Schema namespace in your model the N3 writer won’t know about it
> and
> compress the URI to the QName form – assuming the N3 writer performs
> such
> compressions at all – hopefully someone else can answer that.
>
>
>
> Rob Vesse
>
>
>
>
>
>
>
> From: jena-dev@yahoogroup s.com
> [mailto:jena- dev@yahoogroups. com] On Behalf Of harshal patni
>
> Sent: 07 December 2009 17:48
>
> To: jena-dev@yahoogroup s.com
>
> Subject: {Disarmed} [jena-dev] issues with typedLiteral
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>  Hello All,
>
>
>  I am trying to use typedliteral for xsd:dateTime datatype. The java
> code
>  snippet can be found below -
>
>
>
>  Literal c = model.createTypedLi teral(get_
>
>  date(date), XSDDatatype. XSDdateTime) ;
>
>
>
>  This should basically create a literal of type xsd:dateTime
>
>
>
>  The code generates RDF - N3
>
>
>
>  However when I run the code the output for dateTime is follows -
>
>
>
>  "2008-8-24T20: 45:00^^http://www.w3. org/2001/ XMLSchema# dateTime" .
>
>
>
>  when it should actually be
>
>
>
>  "2008-8-24T20: 45:00"^^xsd:dateTime
>
>
>
>  Is this not the right way to specify a typed literal
>
>  Thanks,
>
>  Harshal
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Dave, you are indeed correct. AddLiteral(Property, Literal) should have
overloaded AddLiteral(Property, Object), but it did not. So I explicitly called
addProperty(Property, RFNode). And that generated the desired output.

Thank you,

--- In jena-dev@yahoogroups.com, Dave Reynolds <dave.e.reynolds@...> wrote:
>
> michma_1999 wrote:
> > Thanks Dave,
> >
> > When I explicitly state the type of literal and add it to a resource
> >
> > model.createTypedLiteral("mytoken", XSDDatatype.XSDtoken);
> > resource.addLiteral(Property, Object);
>
> > I got XML-ABBREV output like
> >
> > <j.0:method>GET^^http://www.w3.org/2001/XMLSchema#token</j.0:method>
> >
> > I'm just wondering is the above output the identical to
> >
> > <j.0:method
rdf:datatype="http://www.w3.org/2001/XMLSchema#token">GET</j.0:method>
> >
> > ?
>
> No it's not.
>
> What has happened is that your addLiteral call didn't recognize the type
> of the object and so defaulted to treating it as a generic java Object
> and using it's toString operation, which is not helpful in this case.
>
> This is odd (unless you are using an old version of Jena) because the
> createTypedLiteral creates a Literal instance and the type overloading
> should call addLiteral(Property, Literal) which will do the right thing.
>
> Since you have not shown your real code I suspect it is a problem with
> how you are treating the return type from createTypedLiteral and the
> java compiler doesn't know you have a Literal and so resolves the
> overloaded method call incorrectly.
>
> Since you have having trouble with the type overloading I'd recommend
> using the more explicit:
>
>   resource.addProperty(Property,
>        model.createTypedLiteral("mytoken", XSDDatatype.XSDtoken));
>
> Dave
>
> >
> > --- In jena-dev@yahoogroups.com, Dave Reynolds <dave.e.reynolds@> wrote:
> >> michma_1999 wrote:
> >>> I suspect this is probably a generic JAVA question. In my RDF file, the
data is a token like
> >>>
> >>> rdf:datatype="http://www.w3.org/2001/XMLSchema#token"
> >>>
> >>> If I use a String type, the datatype will become string. What class do I
need to declare my variable so that it will be a token?
> >> There is no Java type mapping for xsd:token (since it would a sub-type
> >> of string). You'll have to explicitly state the type when you create the
> >> literal:
> >>
> >>   model.createTypedLiteral("mytoken", XSDDatatype.XSDtoken);
> >>
> >> Dave
> >>
> >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
>

michma_1999 wrote:
> Thanks Dave,
>
> When I explicitly state the type of literal and add it to a resource
>
> model.createTypedLiteral("mytoken", XSDDatatype.XSDtoken);
> resource.addLiteral(Property, Object);

> I got XML-ABBREV output like
>
> <j.0:method>GET^^http://www.w3.org/2001/XMLSchema#token</j.0:method>
>
> I'm just wondering is the above output the identical to
>
> <j.0:method
rdf:datatype="http://www.w3.org/2001/XMLSchema#token">GET</j.0:method>
>
> ?

No it's not.

What has happened is that your addLiteral call didn't recognize the type
of the object and so defaulted to treating it as a generic java Object
and using it's toString operation, which is not helpful in this case.

This is odd (unless you are using an old version of Jena) because the
createTypedLiteral creates a Literal instance and the type overloading
should call addLiteral(Property, Literal) which will do the right thing.

Since you have not shown your real code I suspect it is a problem with
how you are treating the return type from createTypedLiteral and the
java compiler doesn't know you have a Literal and so resolves the
overloaded method call incorrectly.

Since you have having trouble with the type overloading I'd recommend
using the more explicit:

   resource.addProperty(Property,
        model.createTypedLiteral("mytoken", XSDDatatype.XSDtoken));

Dave

>
> --- In jena-dev@yahoogroups.com, Dave Reynolds <dave.e.reynolds@...> wrote:
>> michma_1999 wrote:
>>> I suspect this is probably a generic JAVA question. In my RDF file, the data
is a token like
>>>
>>> rdf:datatype="http://www.w3.org/2001/XMLSchema#token"
>>>
>>> If I use a String type, the datatype will become string. What class do I
need to declare my variable so that it will be a token?
>> There is no Java type mapping for xsd:token (since it would a sub-type
>> of string). You'll have to explicitly state the type when you create the
>> literal:
>>
>>   model.createTypedLiteral("mytoken", XSDDatatype.XSDtoken);
>>
>> Dave
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Thanks Dave,

When I explicitly state the type of literal and add it to a resource

model.createTypedLiteral("mytoken", XSDDatatype.XSDtoken);
resource.addLiteral(Property, Object);

I got XML-ABBREV output like

<j.0:method>GET^^http://www.w3.org/2001/XMLSchema#token</j.0:method>

I'm just wondering is the above output the identical to

<j.0:method
rdf:datatype="http://www.w3.org/2001/XMLSchema#token">GET</j.0:method>

?

--- In jena-dev@yahoogroups.com, Dave Reynolds <dave.e.reynolds@...> wrote:
>
> michma_1999 wrote:
> > I suspect this is probably a generic JAVA question. In my RDF file, the data
is a token like
> >
> > rdf:datatype="http://www.w3.org/2001/XMLSchema#token"
> >
> > If I use a String type, the datatype will become string. What class do I
need to declare my variable so that it will be a token?
>
> There is no Java type mapping for xsd:token (since it would a sub-type
> of string). You'll have to explicitly state the type when you create the
> literal:
>
>   model.createTypedLiteral("mytoken", XSDDatatype.XSDtoken);
>
> Dave
>

Alex Genadinik wrote:
> Any ideas why that is still lingering?

Alex, why don't you try to use Maven to manage your project?
You would solve all these issues and avoid them in the future.

Installing and using Maven is very easy:
http://maven.apache.org/download.html
See "Installation Instructions" at the bottom of the page.

Once you have Maven and a basic pom.xml for your project,
you can specify a dependency on Jena or ARQ:

    <dependency>
      <groupId>com.hp.hpl.jena</groupId>
      <artifactId>jena</artifactId>
      <version>2.6.2</version>
    </dependency>

    <dependency>
      <groupId>com.hp.hpl.jena</groupId>
      <artifactId>arq</artifactId>
      <version>2.8.2</version>
    </dependency>

Maven will do all the rest for you.

You can try these commands:

    mvn -Declipse.workspace=/opt/workspace eclipse:add-maven-repo
    mvn eclipse:clean eclipse:eclipse -DdownloadSources=true
-DdownloadJavadocs=true
    mvn dependency:resolve
    mvn compile
    mvn test
    mvn package
    mvn site

Have fun!

Paolo

On 07/12/2009 19:00, Alex Genadinik wrote:
>
>
> Hmmm,
>
> I just changed the jena.jar and it seems to have solved the first
> problem, but this one is still there:
>
> java.lang.NoClassDefFoundError: Could not initialize class
> com.hp.hpl.jena.sparql.expr.NodeValue

java.lang.NoClassDefFoundError

"""
Thrown if the Java Virtual Machine or a ClassLoader instance tries to
load in the definition of a class (as part of a normal method call or as
part of creating a new instance using the new expression) and no
definition of the class could be found.

The searched-for class definition existed when the currently executing
class was compiled, but the definition can no longer be found.
"""

Insufficient information to identify further.

Complete, mininal example please!

NodeValue does quite a lot of initialization during classloading, so
another broken jar may be the cause.  Use a set of jars from a single
distribution or from maven/ivy.

	 Andy

>
> Any ideas why that is still lingering?
>
> Thanks,
> Alex
>
> --- On *Mon, 12/7/09, Andy Seaborne /<afs.lists@...>/* wrote:
>
>
>     From: Andy Seaborne <afs.lists@...>
>     Subject: Re: [jena-dev] Exceptions from SPRQL
>     To: jena-dev@yahoogroups.com
>     Cc: "Alex Genadinik" <genadinik@...>
>     Date: Monday, December 7, 2009, 9:43 AM
>
>
>
>     On 07/12/2009 15:59, Alex Genadinik wrote:
>      > Hello,
>      >
>      > I am having some strange exception like this:
>      > java.lang.Incompati bleClassChangeEr ror: Found class
>     com.hp.hpl.jena. graph.impl. LiteralLabel, but interface was expected
>
>     You have an inconsistent set of jars on the classpath - here, the wrong
>     jena.jar.
>
>      >
>      > Followed later in the exception stack by this:
>      > java.lang.NoClassDe fFoundError: Could not initialize class
>     com.hp.hpl.jena. sparql.expr. NodeValue
>
>     Probably related to the first exception.
>
>      >
>      >
>      > I suspect it was caused by something having to do with the Filter
>     clause where I deal with numbers. Here is my query:
>
>     Unlikely. NodeValue is pretty fundamental to all filter processing.
>     What's more, the ARQ jar was found to get this far (it parsed).
>
>     Andy
>
>      >
>      > select ?individual ?price ?itemUrl where {
>      > ?individual<http://www.w3. org/1999/ 02/22-rdf- syntax-ns# type
>     <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>>
>     <http://www.semantic valley.com/ ontologies/ 2009/shopping.
>     owl#RedColor
>     <http://www.semanticvalley.com/ontologies/2009/shopping.owl#RedColor>> .
>      > ?individual<http://www.semantic valley.com/ ontologies/
>     2009/shopping. owl#itemPrice
>     <http://www.semanticvalley.com/ontologies/2009/shopping.owl#itemPrice>>
>     ?price .
>      > Filter (<http://www.w3. org/2001/ XMLSchema# float
>     <http://www.w3.org/2001/XMLSchema#float>>(?price)> 100.0 ) Filter
>     (<http://www.w3. org/2001/ XMLSchema# float
>     <http://www.w3.org/2001/XMLSchema#float>>(?price)< 500.0 )
>      > ?individual<http://www.semantic valley.com/ ontologies/
>     2009/shopping. owl#itemUrl
>     <http://www.semanticvalley.com/ontologies/2009/shopping.owl#itemUrl>>
?itemUrl
>      > }
>      >
>      > This works locally, but not on my server, so I think I just have
>     to add another jar file to on the server, but I am not sure which.
>     Any suggestions which jar and where to download?
>      >
>      > Thanks,
>      > Alex
>      >
>      >
>      >
>      >
>      >
>      > ------------ --------- --------- ------
>      >
>      > Yahoo! Groups Links
>      >
>      >
>      >
>      >
>      > ____________ _________ _________ _________ _________ _________ _
>      > This email has been scanned by the MessageLabs Email Security System.
>      > For more information please visit http://www.messagel
>     abs.com/email <http://www.messagelabs.com/email>
>      > ____________ _________ _________ _________ _________ _________ _
>
>
>
>
>
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________