Online aggregation

Almost all database operations are performed in batch mode, i.e. the user issues a query and waits till the database has finished processing the entire query. On the contrary, using online aggregation, the user gets estimates of an aggregate query in an online fashion as soon as the query is issued. For example, if the final answer is 1000, after k seconds, the user gets the estimates in form of a confidence interval like [990, 1020] with 95% probability. This confidence keeps on shrinking as the system gets more and more samples.

Online Aggregation (OLA) was proposed in 1997 by Hellerstein, Haas and Wang^[1] for a group-by aggregate query over single table. Later, the authors showed how to evaluate joins in online fashion^[2]. In 2007, Jermaine et al. designed and implemented a prototype database system called Database-Online (or DBO) that computes group-by aggregate query over multiple tables in an online and more importantly in a scalable fashion^[3]. All the approaches for OLA use random sampling, which is non-trivial in distributed environment due to inspection paradox of renewal reward theory. In 2011, Pansare et al. proposed a bayesian model to deal with the inspection paradox and implemented Online Aggregation for MapReduce-like environment^[4].

References

1. ^ Hellerstein, Joseph M.; Haas, Peter J. and Wang, Helen J. (June 1997). "Online aggregation". SIGMOD Rec. 26: 171–182. doi:http://doi.acm.org/10.1145/253262.253291. http://doi.acm.org/10.1145/253262.253291. Retrieved 30 September 2011. 
2. ^ Haas, Peter; Hellerstein, Joseph (June 1999). "Ripple joins for online aggregation". SIGMOD Rec. 28 (2): 287–298. doi:http://doi.acm.org/10.1145/304181.304208. http://doi.acm.org/10.1145/304181.304208. Retrieved 30 September 2011. 
3. ^ Jermaine, Chris; Arumugam, Subramanian and Pol, Abhijit and Dobra, Alin (2007). "Scalable approximate query processing with the DBO engine". SIGMOD: 725–736. doi:http://doi.acm.org/10.1145/1247480.1247560. http://doi.acm.org/10.1145/1247480.1247560. Retrieved 30 September 2011. 
4. ^ Pansare, Niketan; Borkar Vinayak, Jermaine Chris, Condie Tyson (August 2011). "Online Aggregation for Large MapReduce Jobs". VLDB. http://www.cs.rice.edu/~np6/Papers/OnlineAggregationForLargeMapReduceJobs.pdf. Retrieved 30 September 2011.