Search the web
Sign In
New User? Sign Up
bixo-dev · Bixo Web Mining Toolkit
  • Members Only
  • Post
  • Photos
  • Groups Labs (Beta)

Group Information

  • Members: 79
  • Category: Open Source
  • Founded: May 17, 2009
  • Language: English
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Hear how Yahoo! Groups has changed the lives of others. Take me there.

Messages

   Messages Help
Message #
Search:
Advanced
JDBC Tap   Message List  
Reply Message #890 of 916 |
Re: [bixo-dev] Re: JDBC Tap


On Jan 18, 2012, at 2:43pm, crazytora wrote:

 

Do I need to write my own implementation of Writable to serialize java.sql.Timestamp?

As per the Cascading page, you need to provide a class that implements the Hadoop Serialization<T> interface, and have it return a Serializer and Deserializer that handles Timestamp.


Though I think you could just tell Hadoop that it should use Java serialization that it's got, since Timestamp implements Serializable.

You'd need to set the io.serializations property to include (in addition to all of the default serializations) the org.apache.hadoop.io.serializer.JavaSerialization classname.

-- Ken


--- In bixo-dev@yahoogroups.com, Ken Krugler <KKrugler_lists@...> wrote:
>
>
> On Jan 18, 2012, at 12:29pm, crazytora wrote:
>
> > I am using my own database to load my crawl urls and the associated information. I am using mysql and my data has a DateTime fields. I am getting the following serialization error:
> >
> > cascading.CascadingException: unable to load serializer for: java.sql.Timestamp from: org.apache.hadoop.io.serializer.SerializationFactory
> > at cascading.tuple.hadoop.TupleSerialization.getNewSerializer(TupleSerialization.java:309)
> > at cascading.tuple.hadoop.SerializationElementWriter.write(SerializationElementWriter.java:75)
> > at cascading.tuple.TupleOutputStream.write(TupleOutputStream.java:221)
> > at cascading.tuple.TupleOutputStream.writeTuple(TupleOutputStream.java:179)
> > at cascading.tuple.hadoop.TupleSerializer.serialize(TupleSerializer.java:37)
> > at cascading.tuple.hadoop.TupleSerializer.serialize(TupleSerializer.java:28)
> >
> > Any workaround where I can use my original datatype and don't need to modify it?
> >
> You'd need to specify a serialization class to use with Timestamp.
>
> See http://www.cascading.org/1.2/userguide/html/ch06s08.html for more details.
>
> -- Ken
>
> --------------------------
> Ken Krugler
> http://www.scaleunlimited.com
> custom big data solutions & training
> Hadoop, Cascading, Mahout & Solr
>


--------------------------------------------
+1 530-210-6378




--------------------------
Ken Krugler
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr
Wed Jan 18, 2012 11:52 pm

Show Message Option
kkrugler
Offline Offline
Send Email Send Email

Message #890 of 916 |
Expand Messages Author Sort by Date

I am using my own database to load my crawl urls and the associated information. I am using mysql and my data has a DateTime fields. I am getting the following... 		crazytora Offline Jan 18, 2012
8:29 pm

... You'd need to specify a serialization class to use with Timestamp. See http://www.cascading.org/1.2/userguide/html/ch06s08.html for more details. -- Ken ...
Ken Krugler
kkrugler Offline Send Email 		Jan 18, 2012
8:52 pm

Do I need to write my own implementation of Writable to serialize java.sql.Timestamp?...
crazytora Offline Jan 18, 2012
10:43 pm

... As per the Cascading page, you need to provide a class that implements the Hadoop Serialization<T> interface, and have it return a Serializer and...
Ken Krugler
kkrugler Offline Send Email 		Jan 18, 2012
11:52 pm
< Prev Topic  |  Next Topic >
Message #
Search:
Advanced
Copyright © 2010 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines NEW - Help