Search the web
Sign In
New User? Sign Up
CEP-Interest · Event Processing Technologies
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Want to share photos of your group with the world? Add a group photo to Flickr.

Best of Y! Groups

   Check them out and nominate your group.
Having problems with message search? Fill out this form to ensure your group is one of the first to be migrated to the new message search system.

Messages

  Messages Help
Advanced
How would you solve this simple problem?   Message List  
Reply | Forward Message #2005 of 2137 |
Re: How would you solve this simple problem?

Hi Luis: congrats on tasking the CEP community with another apparent
conundrum! I drafted a reply yesterday but thought it too obvious -
today I see the debate continues so here are my thoughts...

This is basically a query on an operational data (or event) store of
stock prices. It's time based (ie a temporal query). You could use EP
to monitor the events but (as I think has been discussed / you
surmise) there is an initial query part too.

1. What was the stock price at time t-5? Was this above $70?
2. Were there any events that changed that in t to t-4.999999...? Was
this above $70?

I think Ophers blog on this topic is quite correct (albeit long!) -
http://epthinking.blogspot.com/2009/01/on-event-processing-and-some.html .

In providing a solution for this: probably I would want to know if
(a) this was a generic problem:
e.g. for any period p I want to know if stock S had a f(value)
relative to some other value
(b) whether this was a continuous query or a static one (in the
business sense)
e.g. do I need to know the result for now to now-5 for all now's?
(b) whether the volumes require data grid storage or not
e.g. whether I need to consider data access considerations.
(c) how often these queries would be required and their performance
requirements
e.g. is this a non-performance-critical occasional report, or a key
application?

For example, it might make sense to relate all events (ie use a linked
list) so I can (a) query / find the set s of all price events in time
period p and (b) find the preceding event for the earliest member of s.

In TIBCO, of course we can define whatever we want to store (as the
derived event object), including references to other events. Typically
this sort of problem would require a data grid storage, but of course
you could also make use of some other storage mechanism for the
historic query. But essentially it is 2 queries... with some care
needed for query/store latency and time.

Cheers
Paul Vincent for Tibco

--- In CEP-Interest@yahoogroups.com, "pureza_l" <pureza_l@...> wrote:
>
> Hi,
>
> I have the good old StockTrades stream with two fields: symbol and
> price. I'm trying to answer the following question "Were Company X
> stock actions priced above $70 during any moment of the last 5
minutes?".
>
> Unfortunately, using a simple time-based sliding window won't work. To
> see why, imagine there were only two price updates: the first, at
> 10:54 am stated that stocks were at $71. The second, 2 minutes later,
> notified that the stocks went down to $69. Now imagine that the above
> question was posed at 11:00 am (of the same day). We, humans, know
> that the answer is "yes" because the price was $71 between 10:54 and
> 10:56. But the first event is outside the 5 minutes window and will
> thus be ignored by the system.
>
> How would you solve this?
>
> Thanks,
>
> Luís Pureza
>





Tue Jan 6, 2009 2:04 pm

isvana321
Offline Offline
Send Email Send Email

Forward
Message #2005 of 2137 |
Expand Messages Author Sort by Date

Esper actually indexes the named window contents based on the predefined query. Thereby predefined queries *can* become much less expensive to execute. Below...
Thomas Bernhardt
bernhardttom
Offline Send Email
Jan 6, 2009
12:38 pm

Hi Luis: congrats on tasking the CEP community with another apparent conundrum! I drafted a reply yesterday but thought it too obvious - today I see the debate...
isvana321
Offline Send Email
Jan 6, 2009
2:04 pm

Hi, I'll try to answer your questions. ... Yes. ... I purposely described the query as a static, ad-hoc one, to simplify. But you may as well want to perform...
Luis Pureza
pureza_l
Offline Send Email
Jan 6, 2009
3:43 pm

Hi Luis, Of course this is an event processing problem. You just did not word it in a way to make it sound like one for the EP crowd, LOL: Event A: "Company X...
Tim Bass
tim_tibco
Offline Send Email
Jan 6, 2009
5:31 pm

Tim, ... Is this just an obtuse way of agreeing that the problem as posed isn't an event processing problem without seeming to agree? :-) ... Being pedantic,...
Brian Connell
cep_ws_bam
Offline Send Email
Jan 6, 2009
5:56 pm

I'm enjoying this discussion. anyone else notice that a slight change in the semantics of the patterns ends up changing the problem? Phrased one way, the...
Peter Lin
woolfel
Offline Send Email
Jan 6, 2009
6:08 pm

So in all this, there seems to be exactly one piece classification that comes with any kind of real theory on why it matters. On one hand, we might interpret...
hansgilde
Offline Send Email
Jan 6, 2009
9:17 pm

If I read the response correctly, I think we are mostly in agreement. from my perspective, classifying a problem as EP or not isn't important. what matters...
Peter Lin
woolfel
Offline Send Email
Jan 6, 2009
9:45 pm

... "Was IP address X logged in anytime during the 3 weeks before IP address Y logged in". Brian...
Brian Connell
cep_ws_bam
Offline Send Email
Jan 6, 2009
6:00 pm

Hello all, very interesting discussion clearly showing the limitations of (timed) windows and data streaming/continuous queries approaches to CEP. I am with...
PatternStorm
claudi_p
Offline Send Email
Jan 7, 2009
1:22 am

I can see some benefits to modeling this way and in this particular example, your sequence technique looks to be complimentary with the streaming SQL approach....
hansgilde
Offline Send Email
Jan 7, 2009
3:50 am

Well, not really ... First: When you reduce the problem to simple 5 minute sliding windows, yes it looks like a streaming SQL approach. However, the vast ...
Tim Bass
tim_tibco
Offline Send Email
Jan 7, 2009
5:53 pm

Yes, in real-world things does not happen in neat-little 5 minute windows. You'll get minutes of in-activity and then thousands of ticks in a fraction of a...
Fatih Ildiz
fildiz57
Offline Send Email
Jan 7, 2009
6:10 pm
 First  |  |  Next > Last 
Advanced

Copyright © 2009 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines - Help