Search the web
Sign In
New User? Sign Up
xml-dbms
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Real people. Real stories. See how Yahoo! Groups impacts members worldwide.

Best of Y! Groups

   Check them out and nominate your group.
Having problems with message search? Fill out this form to ensure your group is one of the first to be migrated to the new message search system.

Messages

  Messages Help
Advanced
Out of memory error while importing large xml files   Message List  
Reply | Forward Message #3860 of 3864 |
Re: [xml-dbms] Out of memory error while importing large xml files

Hello,

XML-DBMS does not support SAX because linear processing of documents is
possible only in specific cases. (It is very easy to create an XML
document whose data cannot be transferred to the database in a linear
manner.)

Because of this, you have several choices:

1) Increase the amount of memory Java uses using the -Xmx switch on the
java command. For details, see:

http://java.sun.com/j2se/1.3/docs/tooldocs/solaris/java.html

Because DOM trees are 5-10 times as large as XML documents, setting
memory usage to 50MB should solve your problem. This is the easiest
solution to your problem.

2) Cut the document into smaller pieces with SAX and pass these pieces
to XML-DBMS as DOM trees. This only works with certain types of
documents. For details, see:

http://www.rpbourret.com/xmldbms/faqs/large.htm

3) Use a different tool. For example, you could use DataDirect XQuery,
which can store XML documents in a relational database and also
implements XQuery over relational data. Other options are HiT Allora and
XQuare Bridge. (I don't know how well these products work with large XML
documents.)

-- Ron

Ruchika Israni wrote:
> Hi all,
>
> I am using a xml which is 3.08 MB in size. I am running the transfer tool and
am getting the following error. I read somewhere in the previous posts that DOM
parser creates the entire tree from the XML which poses a problem for large xml
files. Is it possible to perhaps use the SAX parser in place of the DOM parser ?
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at org.apache.xerces.dom.DeferredDocumentImpl.getNodeObject(Unknown
Sour
> ce)
> at
org.apache.xerces.dom.DeferredDocumentImpl.synchronizeChildren(Unknow
> n Source)
> at
org.apache.xerces.dom.DeferredElementNSImpl.synchronizeChildren(Unkno
> wn Source)
> at org.apache.xerces.dom.ParentNode.getFirstChild(Unknown Source)
> at
org.xmlmiddleware.xmlutils.DOMNormalizer.getFirstChild(DOMNormalizer.
> java:123)
> at
org.xmlmiddleware.xmldbms.DOMToDBMS.processChildren(DOMToDBMS.java:60
> 5)
> at
org.xmlmiddleware.xmldbms.DOMToDBMS.processClassRow(DOMToDBMS.java:54
> 4)
> at org.xmlmiddleware.xmldbms.DOMToDBMS.processRow(DOMToDBMS.java:881)
> at
org.xmlmiddleware.xmldbms.DOMToDBMS.processFKNodes(DOMToDBMS.java:946
> )
> at
org.xmlmiddleware.xmldbms.DOMToDBMS.processClassRow(DOMToDBMS.java:56
> 6)
> at org.xmlmiddleware.xmldbms.DOMToDBMS.processRoot(DOMToDBMS.java:447)
> at
org.xmlmiddleware.xmldbms.DOMToDBMS.storeDocument(DOMToDBMS.java:368)
>
> at
org.xmlmiddleware.xmldbms.DOMToDBMS.storeDocument(DOMToDBMS.java:317)
>
> at
org.xmlmiddleware.xmldbms.tools.Transfer.storeDocumentInternal(Transf
> er.java:840)
> at
org.xmlmiddleware.xmldbms.tools.Transfer.storeDocument(Transfer.java:
> 479)
> at
org.xmlmiddleware.xmldbms.tools.Transfer.dispatchStoreDocument(Transf
> er.java:696)
> at
org.xmlmiddleware.xmldbms.tools.Transfer.dispatch(Transfer.java:434)
> at org.xmlmiddleware.xmldbms.tools.Transfer.main(Transfer.java:353)
>
> Many thanks in anticipation.
> Ruchika Israni



Wed Jun 3, 2009 8:33 pm

xmldbms
Offline Offline
Send Email Send Email

Forward
Message #3860 of 3864 |
Expand Messages Author Sort by Date

Hi all, I am using a xml which is 3.08 MB in size. I am running the transfer tool and am getting the following error. I read somewhere in the previous posts...
Ruchika Israni
israniruchika
Offline Send Email
Jun 3, 2009
7:14 pm

Hello, XML-DBMS does not support SAX because linear processing of documents is possible only in specific cases. (It is very easy to create an XML document...
Ronald Bourret
xmldbms
Offline Send Email
Jun 3, 2009
8:32 pm

Thank You Ron. I included -Xmx option into the command and it worked. Thanks once again. Ruchika Israni...
Ruchika Israni
israniruchika
Offline Send Email
Jun 4, 2009
4:02 am
Advanced

Copyright © 2009 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines - Help