2013 Poster Sessions : HicampDB - an In-memory Column Database with Hardware Snapshot Isolation and Deduplication

Student Name : Bo Wang
Advisor : David Cheriton
Research Areas: Computer Systems
Nowadays real-time analytics is becoming a hot topic in database community. Some new databases (e.g. SAP HANA, HYRISE, Vertica, CStore) attempt to process both OLTP and OLAP workloads in the same system to shorten the latency for updates to be visible by analytics. However, the modifications introduced by OLTP transactions can frequently disrupt the analytical workloads. To tackle this problem, most of these databases keep a separate write-optimized delta partition and a read-optimized full partition. Changes are cached in the delta partition and merged into the full partition regularly. However, this approach has significant merging overhead and non-neglectable latency. The caching and merging processes also require high extra memory space.

HicampDB is an in-memory column database built on top of the HICAMP memory system to handle both analytical and transactional workloads. HicampDB takes advantage of the hardware-enforced immutability and deduplication provided by HICAMP to create light-weight snapshot isolations for concurrent transactions and avoid the overhead of keeping separate partitions. Updates are visible as soon as the transactions are committed. No significant spatial or temporal overhead exists for caching and merging. The column-oriented format also exposes more deduplication opportunities for hardware and reduces the footprint of analytical workloads.

Bo Wang is a third year PhD student working with Professor David Cheriton. He is interested in parallel and distributed systems and big data management technologies.