Annotation gradient will use direct colours if no graph exists