Remove consensus annotation if out of memory