());
}
return sb;
}
/**
* Parse one line of interleaved data e.g.
*
*
* #TheSeqId CGATCGCATGCA
*
*
* @param dataLine
* @param seqId
* @throws FileFormatException
*/
protected void parseInterleavedDataLine(String dataLine, String seqId)
throws FileFormatException
{
/*
* New sequence found in second or later data block - error.
*/
if (this.firstDataBlockRead && !seqData.containsKey(seqId))
{
throw new FileFormatException(
"Parse error: misplaced new sequence starting at " + dataLine);
}
String data = dataLine.substring(seqId.length() + 1).trim();
/*
* Do nothing if this line is _only_ a sequence id with no data following.
*/
if (data != null && data.length() > 0)
{
data = addSequenceData(seqId, data);
setPositionsPerLine(Math.max(positionsPerLine, data.length()));
assertInterleaved(true, dataLine);
}
}
/**
* Remove spaces, and replace identity symbol, before appending the sequence
* data to the buffer for the sequence id. Returns the reformatted added data.
* Also updates a count of residues read for the sequence.
*
* @param seqId
* @param data
* @return
*/
protected String addSequenceData(String seqId, String data)
{
StringBuilder sb = getSequenceDataBuffer(seqId);
int len = sb.length();
String formatted = data.replace(SPACE, "");
/*
* If sequence contains '.' or other identity symbol; replace these with the
* same position from the first (reference) sequence
*/
int nonGapped = 0;
StringBuilder referenceSequence = seqData.values().iterator().next();
StringBuilder sb1 = new StringBuilder(formatted.length());
for (int i = 0; i < formatted.length(); i++)
{
char nextChar = formatted.charAt(i);
if (nextChar != gapCharacter)
{
nonGapped++;
}
if (nextChar == identityCharacter
&& len + i < referenceSequence.length())
{
sb1.append(referenceSequence.charAt(len + i));
}
else
{
sb1.append(nextChar);
}
}
formatted = sb1.toString();
data = formatted;
sb.append(data);
/*
* increment residue count for the sequence
*/
if (nonGapped > 0)
{
Integer residueCount = residuesRead.get(seqId);
residuesRead.put(seqId, nonGapped
+ (residueCount == null ? 0 : residueCount));
}
return data;
}
/**
* If the line begins with (e.g.) "#abcde " then returns "abcde" as the
* identifier. Else returns null.
*
* @param dataLine
* @return
*/
public static String getSequenceId(String dataLine)
{
// TODO refactor to a StringUtils type class
if (dataLine != null)
{
if (dataLine.startsWith(HASHSIGN))
{
int spacePos = dataLine.indexOf(" ");
return (spacePos == -1 ? dataLine.substring(1) : dataLine
.substring(1, spacePos));
}
}
return null;
}
/**
* Read the #MEGA and Title/Format/Description header lines (if present).
*
* Save as alignment properties in case useful.
*
* @return the next non-blank line following the header lines.
* @throws IOException
*/
protected String parseHeaderLines() throws IOException
{
String inputLine = null;
while ((inputLine = nextNonCommentLine()) != null)
{
inputLine = inputLine.trim();
/*
* skip blank lines
*/
if (inputLine.length() == 0)
{
continue;
}
if (inputLine.toUpperCase().startsWith(MEGA_ID))
{
continue;
}
if (isTitle(inputLine))
{
this.title = getValue(inputLine);
setAlignmentProperty(PROP_TITLE, title);
}
else if (inputLine.startsWith(BANG + DESCRIPTION))
{
parseDescription(inputLine);
}
else if (inputLine.startsWith(BANG + FORMAT))
{
parseFormat(inputLine);
}
else if (!inputLine.toUpperCase().startsWith(MEGA_ID))
{
/*
* Return the first 'data line' i.e. one that is not blank, #MEGA or
* TITLE:
*/
break;
}
}
return inputLine;
}
/**
* Parse a !Format statement. This may be multiline, and is ended by a
* semicolon.
*
* @param inputLine
* @throws IOException
*/
protected void parseFormat(String inputLine) throws IOException
{
while (inputLine != null)
{
parseFormatLine(inputLine);
if (inputLine.endsWith(SEMICOLON))
{
break;
}
inputLine = nextNonCommentLine();
}
}
/**
* Parse one line of a !Format statement. This may contain one or more
* keyword=value pairs.
*
* @param inputLine
* @throws FileFormatException
*/
protected void parseFormatLine(String inputLine)
throws FileFormatException
{
if (inputLine.startsWith(BANG + FORMAT))
{
inputLine = inputLine.substring((BANG + FORMAT).length());
}
if (inputLine.endsWith(SEMICOLON))
{
inputLine = inputLine.substring(0, inputLine.length() - 1);
}
if (inputLine.length() == 0)
{
return;
}
String[] tokens = inputLine.trim().split(WHITESPACE);
for (String token : tokens)
{
parseFormatKeyword(token);
}
}
/**
* Parse a Keyword=Value token. Possible keywords are
*
* - DataType= DNA, RNA, Nucleotide, Protein
* - DataFormat= Interleaved, ?
* - NSeqs= number of sequences (synonym NTaxa)
* - NSites= number of bases / residues
* - Property= Exon (or Coding), Intron (or Noncoding), End (of domain)
* - Indel= gap character
* - Identical= identity character (to first sequence) (synonym MatchChar)
* - Missing= missing data character
* - CodeTable= Standard, other (MEGA supports various)
*
*
* @param token
* @throws FileFormatException
* if an unrecognised keyword or value is encountered
*/
protected void parseFormatKeyword(String token)
throws FileFormatException
{
String msg = "Unrecognised Format command: " + token;
String[] bits = token.split(EQUALS);
if (bits.length != 2)
{
throw new FileFormatException(msg);
}
String keyword = bits[0];
String value = bits[1];
/*
* Jalview will work out whether nucleotide or not anyway
*/
if (keyword.equalsIgnoreCase(DATATYPE))
{
if (value.equalsIgnoreCase("DNA") || value.equalsIgnoreCase("RNA")
|| value.equalsIgnoreCase("Nucleotide"))
{
this.nucleotide = true;
// alignment computes whether or not it is nucleotide when created
}
else if (value.equalsIgnoreCase(PROTEIN))
{
this.nucleotide = false;
}
else
{
throw new FileFormatException(msg);
}
setAlignmentProperty(PROP_DATATYPE, value);
}
/*
* accept non-Standard code table but save in case we want to disable
* 'translate as cDNA'
*/
else if (keyword.equalsIgnoreCase(CODETABLE))
{
setAlignmentProperty(PROP_CODETABLE, value);
}
/*
* save gap char to set later on alignment once created
*/
else if (keyword.equalsIgnoreCase(INDEL))
{
this.gapCharacter = value.charAt(0);
}
else if (keyword.equalsIgnoreCase(IDENTICAL)
|| keyword.equalsIgnoreCase("MatchChar"))
{
setAlignmentProperty(PROP_IDENTITY, value);
this.identityCharacter = value.charAt(0);
if (!".".equals(value))
{
System.err.println("Warning: " + token
+ " not supported, Jalview uses '.' for identity");
}
}
else if (keyword.equalsIgnoreCase(MISSING))
{
setAlignmentProperty(PROP_MISSING, value);
System.err.println("Warning: " + token + " not supported");
}
else if (keyword.equalsIgnoreCase(PROPERTY))
{
// TODO: can Property appear in a Format command?
// suspect this is a mistake in the manual
}
else if (!keyword.equalsIgnoreCase(N_SEQS)
&& !keyword.equalsIgnoreCase("NTaxa")
&& !keyword.equalsIgnoreCase(N_SITES))
{
System.err.println("Warning: " + msg);
}
}
/**
* Returns the trimmed data on the line following either whitespace or '=',
* with any trailing semi-colon removed
* So
*
* - Hello World
* - !Hello: \tWorld;
* - !Hello=World
*
* should all return "World"
*
* @param inputLine
* @return
*/
protected static String getValue(String inputLine)
{
if (inputLine == null)
{
return null;
}
String value = null;
String s = inputLine.replaceAll("\t", " ").trim();
/*
* KEYWORD = VALUE should return VALUE
*/
int equalsPos = s.indexOf("=");
if (equalsPos >= 0)
{
value = s.substring(equalsPos + 1);
}
else
{
int spacePos = s.indexOf(' ');
value = spacePos == -1 ? "" : s.substring(spacePos + 1);
}
value = value.trim();
if (value.endsWith(SEMICOLON))
{
value = value.substring(0, value.length() - 1).trim();
}
return value;
}
/**
* Returns true if the input line starts with "TITLE" or "!TITLE" (not case
* sensitive). The latter is the official format, some older data file
* examples have it without the !.
*
* @param inputLine
* @return
*/
protected static boolean isTitle(String inputLine)
{
if (inputLine == null)
{
return false;
}
String upper = inputLine.toUpperCase();
return (upper.startsWith(TITLE.toUpperCase()) || upper.startsWith(BANG
+ TITLE.toUpperCase()));
}
/**
* Reads lines until terminated by semicolon, appending each to the
* Description property value.
*
* @throws IOException
*/
protected void parseDescription(String firstDescriptionLine)
throws IOException
{
StringBuilder desc = new StringBuilder(256);
desc.append(getValue(firstDescriptionLine));
if (!firstDescriptionLine.endsWith(SEMICOLON))
{
String line = nextNonCommentLine();
while (line != null)
{
if (line.endsWith(SEMICOLON))
{
desc.append(line.substring(0, line.length() - 1));
break;
}
else if (line.length() > 0)
{
desc.append(line).append(newline);
}
line = nextNonCommentLine();
}
}
setAlignmentProperty(PROP_DESCRIPTION, desc.toString());
}
/**
* Returns the alignment sequences in Mega format.
*/
@Override
public String print()
{
return MEGA_ID + newline + print(getSeqsAsArray());
}
/**
* Write out the alignment sequences in Mega format - interleaved unless
* explicitly noninterleaved.
*/
protected String print(SequenceI[] s)
{
String result;
if (this.interleaved != null && !this.interleaved)
{
result = printNonInterleaved(s);
}
else
{
result = printInterleaved(s);
}
return result;
}
/**
* Print to string in Interleaved format - blocks of next N characters of each
* sequence in turn.
*
* @param s
*/
protected String printInterleaved(SequenceI[] s)
{
int maxIdLength = getMaxIdLength(s);
int maxSequenceLength = getMaxSequenceLength(s);
int numLines = maxSequenceLength / positionsPerLine + 3; // approx
int numDataBlocks = (maxSequenceLength - 1) / positionsPerLine + 1;
int spaceEvery = this.nucleotide != null && this.nucleotide ? 3 : 10;
int chunksPerLine = (positionsPerLine + spaceEvery - 1) / spaceEvery;
/*
* Roughly size a buffer to hold the whole output
*/
StringBuilder sb = new StringBuilder(numLines
* (maxIdLength + positionsPerLine + chunksPerLine + 10));
/*
* Output as: #Seqid CGT AGC ACT ... or blocks of 10 for peptide
*/
int from = 0;
for (int i = 0; i < numDataBlocks; i++)
{
sb.append(newline);
boolean first = true;
int advancedBy = 0;
for (SequenceI seq : s)
{
int seqFrom = from;
String seqId = String.format("#%-" + maxIdLength + "s",
seq.getName());
/*
* output next line for this sequence
*/
sb.append(seqId);
int lastPos = seqFrom + positionsPerLine; // exclusive
for (int j = 0; j < chunksPerLine; j++)
{
char[] subSequence = seq.getSequence(seqFrom,
Math.min(lastPos, seqFrom + spaceEvery));
if (subSequence.length > 0)
{
sb.append(SPACE).append(subSequence);
}
seqFrom += subSequence.length;
if (first)
{
// all sequences should be the same length in MEGA
advancedBy += subSequence.length;
}
}
// write last position as a comment
if (writePositionNumbers)
{
sb.append(SPACE).append(COMMENT_START).append(from + advancedBy)
.append(COMMENT_END);
}
sb.append(newline);
first = false;
}
from += advancedBy;
}
return new String(sb);
}
/**
* Outputs to string the MEGA header and any other known and relevant
* alignment properties
*
* @param al
*/
protected String printHeaders(AlignmentI al)
{
StringBuilder sb = new StringBuilder(128);
sb.append(MEGA_ID).append(newline);
String propertyValue = (String) al.getProperty(PROP_TITLE);
if (propertyValue != null)
{
sb.append(BANG).append(TITLE).append(SPACE).append(propertyValue)
.append(SEMICOLON).append(newline);
}
propertyValue = (String) al.getProperty(PROP_DESCRIPTION);
if (propertyValue != null)
{
sb.append(BANG).append(DESCRIPTION).append(newline)
.append(propertyValue).append(SEMICOLON)
.append(newline);
}
/*
* !Format DataType CodeTable
*/
sb.append(BANG).append(FORMAT).append(newline);
String dataType = (String) al.getProperty(PROP_DATATYPE);
if (dataType == null)
{
dataType = al.isNucleotide() ? NUCLEOTIDE : PROTEIN;
}
sb.append(INDENT).append(DATATYPE).append(EQUALS).append(dataType);
String codeTable = (String) al.getProperty(PROP_CODETABLE);
sb.append(SPACE).append(CODETABLE).append(EQUALS)
.append(codeTable == null ? "Standard" : codeTable)
.append(newline);
/*
* !Format NSeqs NSites (the length of sequences - they should all be the
* same - including gaps)
*/
sb.append(INDENT).append(N_SEQS).append(EQUALS).append(al.getHeight());
sb.append(SPACE).append(N_SITES).append(EQUALS)
.append(String.valueOf(al.getWidth()));
sb.append(newline);
/*
* !Format Indel Identical Missing
*/
sb.append(INDENT);
sb.append(INDEL).append(EQUALS).append(al.getGapCharacter());
String identity = (String) al.getProperty(PROP_IDENTITY);
if (identity != null)
{
sb.append(SPACE).append(IDENTICAL).append(EQUALS).append(identity);
}
String missing = (String) al.getProperty(PROP_MISSING);
if (missing != null)
{
sb.append(SPACE).append(MISSING).append(EQUALS).append(missing);
}
sb.append(SEMICOLON).append(newline);
return sb.toString();
}
/**
* Get the longest sequence id (to allow aligned printout).
*
* @param s
* @return
*/
protected static int getMaxIdLength(SequenceI[] s)
{
// TODO pull up for reuse
int maxLength = 0;
for (SequenceI seq : s)
{
int len = seq.getName().length();
if (len > maxLength)
{
maxLength = len;
}
}
return maxLength;
}
/**
* Get the longest sequence length
*
* @param s
* @return
*/
protected static int getMaxSequenceLength(SequenceI[] s)
{
// TODO pull up for reuse
int maxLength = 0;
for (SequenceI seq : s)
{
int len = seq.getLength();
if (len > maxLength)
{
maxLength = len;
}
}
return maxLength;
}
/**
* Print to string in noninterleaved format - all of each sequence in turn, in
* blocks of 50 characters.
*
* @param s
* @return
*/
protected String printNonInterleaved(SequenceI[] s)
{
int maxSequenceLength = getMaxSequenceLength(s);
// approx
int numLines = maxSequenceLength / positionsPerLine + 2 + s.length;
/*
* Roughly size a buffer to hold the whole output
*/
StringBuilder sb = new StringBuilder(numLines * positionsPerLine);
int spaceEvery = this.nucleotide != null && this.nucleotide ? 3 : 10;
int chunksPerLine = positionsPerLine / spaceEvery;
for (SequenceI seq : s)
{
sb.append(newline);
sb.append(HASHSIGN + seq.getName()).append(newline);
int startPos = 0;
while (startPos < seq.getLength())
{
boolean firstChunk = true;
/*
* print next line for this sequence
*/
int lastPos = startPos + positionsPerLine; // exclusive
for (int j = 0; j < chunksPerLine; j++)
{
char[] subSequence = seq.getSequence(startPos,
Math.min(lastPos, startPos + positionsPerLine));
if (subSequence.length > 0)
{
if (!firstChunk)
{
sb.append(SPACE);
}
sb.append(subSequence);
firstChunk = false;
}
startPos += subSequence.length;
}
sb.append(newline);
}
}
return new String(sb);
}
/**
* Flag this file as interleaved or not, based on data format. Throws an
* exception if has previously been determined to be otherwise.
*
* @param isIt
* @param dataLine
* @throws IOException
*/
protected void assertInterleaved(boolean isIt, String dataLine)
throws FileFormatException
{
if (this.interleaved != null && isIt != this.interleaved.booleanValue())
{
throw new FileFormatException(
"Parse error: mix of interleaved and noninterleaved detected, at line: "
+ dataLine);
}
this.interleaved = new Boolean(isIt);
setAlignmentProperty(PROP_INTERLEAVED, interleaved.toString());
}
public boolean isInterleaved()
{
return this.interleaved == null ? false : this.interleaved
.booleanValue();
}
/**
* Adds saved parsed values either as alignment properties, or (in some cases)
* as specific member fields of the alignment
*/
@Override
public void addProperties(AlignmentI al)
{
super.addProperties(al);
al.setGapCharacter(gapCharacter);
/*
* warn if e.g. DataType=DNA but data is protein (or vice versa)
*/
if (this.nucleotide != null && this.nucleotide != al.isNucleotide()) {
System.err.println("Warning: " + this.title + " declared "
+ (nucleotide ? "" : " not ") + "nucleotide but it is"
+ (nucleotide ? " not" : ""));
}
}
/**
* Print the given alignment in MEGA format. If the alignment was created by
* parsing a MEGA file, it should have properties set (e.g. Title) which can
* influence the output.
*/
@Override
public String print(AlignmentI al)
{
this.nucleotide = al.isNucleotide();
String lineLength = (String) al.getProperty(PROP_LINELENGTH);
this.positionsPerLine = lineLength == null ? DEFAULT_LINE_LENGTH : Integer
.parseInt(lineLength);
/*
* round down to a multiple of 3 positions per line for nucleotide
*/
if (nucleotide)
{
positionsPerLine = positionsPerLine - (positionsPerLine % 3);
}
String interleave = (String) al.getProperty(PROP_INTERLEAVED);
if (interleave != null)
{
this.interleaved = Boolean.valueOf(interleave);
}
String headers = printHeaders(al);
return headers + print(al.getSequencesArray());
}
/**
* Returns the number of sequence positions output per line
*
* @return
*/
public int getPositionsPerLine()
{
return positionsPerLine;
}
/**
* Sets the number of sequence positions output per line. Note these will be
* formatted in blocks of 3 (nucleotide) or 10 (peptide).
*
* @param p
*/
public void setPositionsPerLine(int p)
{
this.positionsPerLine = p;
}
}