[R-SIG-Finance] How to know when an XTS row is at a particular interval

[Brian G. Peterson]

On Sat, 2011-07-09 at 12:37 -0700, Noah Silverman wrote:
> Hi,
> 
> Endpoints won't always work for me either.  In order to use endpoints, I would need to know the whole series ahead of time.  
> 
> If I have transactions streaming in (Like from an IBrokers account) then I can't compute "endpoints".

I'll note that you didn't say that you wanted to apply to streaming
data.  Your original post said:

"I have a big dataset of tick data.  (Actual transactions at sub-second
frequency)"

though you did say:

"Iterating through the series to simulate a trading day."

You didn't say that you ultimately wanted to apply the same algorithm to
streaming data.  All backtesting (trading simulation) involves some
amount of iteration through the data, assuming that there is some path
dependent logic (and real trading is always path dependent, though
path-independent assumptions may serve for simple things).

Even when 'iterating' through historical data, endpoints is still the
most efficient mechanism in R that I'm aware of.  Streaming data is
necessarily a different problem.

R doesn't have any native support for streaming data types that I am
aware of.  You need to use some sort of sampling or event loop
algorithm.


> I suppose that I could calculate the minutes and seconds of each transaction and then compare to the bar frequency I want, but that seems computationally excessive.
>
> I guess what I want to do is that when a transaction arrives, ask, "Is this on the mark, or close enough to treat it as on the mark".


Well, messages from an exchange are timestamped.  Use those.  Keep your
'last' transaction in a buffer, and then pop it onto a stack as soon as
you have an observation (transaction or not) that is at or past the
mark.

Jeff has previously reported being able to handle several thougsand
messages per second from IB in an event loop in R, which is probably
sufficient for most single-instrument modeling.

Regards,

   - Brian

> On Jul 7, 2011, at 1:42 PM, Brian G. Peterson wrote:
> 
> > On Thu, 2011-07-07 at 13:27 -0700, Noah Silverman wrote:
> >> Hi,
> >> 
> >> 
> >> I have a big dataset of tick data.  (Actual transactions at sub-second frequency)
> >> 
> >> I've used R to convert this to an xts object.
> >> 
> >> Iterating through the series to simulate a trading day.
> >> 
> >> I'd like to know when a particular transaction is "on the minute" or "on the 5 minute" mark.  Of course, I can use all the great functions of xts to convert the series to bars of any frequency, but that's not what I want.  I'd like to iterate over all the transaction and then somehow know when I've hit a bar boundary.
> > 
> > With actual ticks, there will be few or no trades 'on the mark'. This is
> > why most tick-level analysis relies on the prevailing bid and offer,
> > since there is always *some* real price at any given time, even though
> > trades happen at discrete times.
> > 
> > Anyway, the function you need is 'endpoints', which will give you the
> > closest stamp to the mark, at whatever periodicity you want.
> > 
> > Then you can filter for the almost nonexistant transactions 'on the
> > mark'.
> > 
> > 
> >> One possible issue is if I don't have a transaction exactly at the bar.  (More of a theoretical question.)  For example, If I have a transaction at 11:59:59, and then the next transaction is at 12:00:01.  Then no transaction occurred exactly on the 1 minute boundary.  I'm not sure the best way to handle this.
> >> 
> >> The general "big picture' concept is that a trading strategy might only want to trigger on 5 minute bars, but I'd like to keep updating some of my indicators as individual trades flow in.  
> > 
> > Use bids and offers.
> > 
> >> Thoughts?
> >> 
> >> 
> >> --
> >> Noah Silverman
> >> UCLA Department of Statistics
> >> 8117 Math Sciences Building
> >> Los Angeles, CA 90095
> >> 
> >> _______________________________________________
> >> R-SIG-Finance at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> >> -- Subscriber-posting only. If you want to post, subscribe first.
> >> -- Also note that this is not the r-help list where general R questions should go.
> > 
> > -- 
> > Brian G. Peterson
> > http://braverock.com/brian/
> > Ph: 773-459-4973
> > IM: bgpbraverock
> > 
> 
> _______________________________________________
> R-SIG-Finance at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock