2013 Poster Sessions : A New Lossy Compressor for Quality Scores Based on Rate Distortion Theory

Student Name : Idoia Ochoa
Advisor : Tsachy Weissman
Research Areas: Information Systems
Next Generation Sequencing technologies have revolutionized many fields in biology by reducing the time and cost required for sequencing. As a result, large amounts of sequencing data are being generated. A typical sequencing data file may occupy tens or even hundreds of gigabytes of disk space, prohibitively large for many users. This data consists of both the nucleotide sequences and per-base quality scores that indicate the level of confidence in the readout of these sequences. Quality scores account for about half of the required disk space in the commonly used FASTQ format (before compression), and therefore the compression of the quality scores can significantly reduce storage requirements and speed up analysis and transmission of sequencing data.

We present a new scheme for the lossy compression of the quality scores, to address the problem of storage. Our framework allows the user to specify the rate (bits per quality score) prior to compression, independent of the data to be compressed. Given a model for the quality scores, we use rate-distortion results to optimally allocate the available bits in order to minimize the Mean Squared Error (MSE). This metric allows to compare different lossy compression algorithms for quality scores without depending on downstream applications that may use the quality scores in very different ways. We envisage our algorithm as being part of a more general compression scheme that works with the entire FASTQ file. Numerical experiments show that we can achieve a better MSE for small rates (bits per quality score) than other lossy compression schemes. For the organism emph{PhiX}, whose assembled genome is known and assumed to be correct, we show that it is possible to achieve a significant reduction in size with little compromise in performance on downstream applications.

Idoia Ochoa received the B.Eng. degree in Electrical Engineering from University of Navarra, Donostia, Spain, in 2009, and the Masters degree in Electrical Engineering from Stanford University in 2012. She is currently working towards the Ph.D. degree in the Department of Electrical Engineering, Stanford University. Her research interests include information theory, signal processing and coding, with applications in bioinformatics, data compression and communications. Idoia was a recipient of La Caixa fellowship and the Basque Government fellowship.