The \eslmod{sq} module provides \Easel's object for single biological
sequences: an \ccode{ESL\_SQ}.

Sequence objects invariably become complicated even when their
designer intends them to be simple. There's many things we want to do
with a sequence, and useful features naturally accrete over time. If a
library isn't careful to balance creeping featuritis against having an
easy way to start using the object in simple applications, then the
sequence object - possibly the most fundamental object of a
biosequence library - can become a barrier to anyone else actually
using the library! All those useful features won't matter much if you
can't figure out how to turn your sequence data into an object, or get
it back out. \Easel\ expects you to have your own preferred way of
dealing with sequence data that's not necessarily \Easel's way, so it
provides simple ways to create sequence objects from elemental (C
string) data, and simple ways to get elemental C strings back out.
This lets you minimize your exposure to \Easel's more advanced (and
complicated) capabilities if you like.

The most basic use of an \ccode{ESL\_SQ} object is to hold one
complete sequence, simply as a plain C string. A sequence may also
have a name, an accession, and a description line.  This is called a
\esldef{text mode} sequence. In text mode, \Easel\ doesn't know
whether the sequence is DNA, RNA, protein, or something else; it's
just an ASCII character string. This limits some of \Easel's more
powerful abilities, such as the ability to check the sequence for
errors, or to automatically deal with degenerate residue codes; but
it's a simple mode that's easy to start using.

Alternatively, a sequence may be in \esldef{digital mode}. In digital
mode, sequences are predigested and encoded into \Easel's internal
format, which makes many sequence routines more robust, efficient, and
powerful. Digital mode requires \eslmod{alphabet} augmentation.

In addition to storing a complete sequence, an \ccode{ESL\_SQ} is
designed to be used in three other situations:

\begin{itemize}
\item to hold a \esldef{subsequence} of a larger source sequence. The
  object maintains source and coordinate information necessary for
  crossreferencing the subsequence's coordinate system to the original
  source coordinate system.

\item to hold a \esldef{window} of a larger source sequence. This is
  like a subsequence, but is more specifically intended for reading a
  sequence from a file in overlapping windows. This avoids having to
  suck an entire chromosome (for example) into memory at any one
  time. The stored subsequence is composed of two segments, a
  \esldef{previous context} that gets saved from the previous window,
  and a \esldef{new window} of fresh residues. The size of both the
  context and the window are configurable at the time each new window
  is read.

\item to hold only \esldef{information} about a sequence, such as its
  name, its length, and its position in a file, excluding the sequence
  (and optional secondary structure annotation) itself. This is handy
  for example when indexing a sequence file, when we'd rather not read
  any (possibly prohibitively large) sequence into memory until after
  we've mapped out how big it is.
\end{itemize}

To keep all this straight, the object contains a bunch of internal
bookkeeping data.

Sequence objects are growable and reusable, for efficiency in memory
allocation. If you're going to go through many different sequences
sequentially, you would typically just allocate a single
\ccode{ESL\_SQ} object and \ccode{esl\_sq\_Reuse()} it for each new
sequence, rather than creating and destroying a lot of objects.

A sequence object can also store a secondary structure annotation line
for the sequence, one character per residue.

When augmented with \eslmod{msa}, an interface to the \ccode{ESL\_MSA}
multiple alignment object provides the ability to extract single
unaligned sequences from a multiple alignment.

You would often use the \eslmod{sq} module in conjunction with
\eslmod{sqio}, which provides the ability to read and write
\ccode{ESL\_SQ} objects from and to files.

Table~\ref{tbl:sq_api} lists the functions in the \eslmod{sq} API.

% Table generated by autodoc -t esl_sq.c (so don't edit here, edit esl_sq.c:)
\begin{table}[hbp]
\begin{center}
{\small
\begin{tabular}{|ll|}\hline
\apisubhead{Text version of the \ccode{ESL\_SQ} object.}\\
\hyperlink{func:esl_sq_Create()}{\ccode{esl\_sq\_Create()}} & Create a new, empty \ccode{ESL\_SQ}.\\
\hyperlink{func:esl_sq_CreateFrom()}{\ccode{esl\_sq\_CreateFrom()}} & Create a new \ccode{ESL\_SQ} from text information.\\
\hyperlink{func:esl_sq_Grow()}{\ccode{esl\_sq\_Grow()}} & Assure that a \ccode{ESL\_SQ} has space to add more residues.\\
\hyperlink{func:esl_sq_GrowTo()}{\ccode{esl\_sq\_GrowTo()}} & Grows an \ccode{ESL\_SQ} to hold a seq of at least \ccode{n} residues.\\
\hyperlink{func:esl_sq_Copy()}{\ccode{esl\_sq\_Copy()}} & Make a copy of an \ccode{ESL\_SQ}.\\
\hyperlink{func:esl_sq_Reuse()}{\ccode{esl\_sq\_Reuse()}} & Reinitialize an \ccode{ESL\_SQ} for re-use.\\
\hyperlink{func:esl_sq_Destroy()}{\ccode{esl\_sq\_Destroy()}} & Frees an \ccode{ESL\_SQ}.\\
\apisubhead{Digitized version of the \ccode{ESL\_SQ} object. (Requires \ccode{alphabet})}\\
\hyperlink{func:esl_sq_CreateDigital()}{\ccode{esl\_sq\_CreateDigital()}} & Create a new, empty \ccode{ESL\_SQ} in digital mode.\\
\hyperlink{func:esl_sq_CreateDigitalFrom()}{\ccode{esl\_sq\_CreateDigitalFrom()}} & Create a new digital \ccode{ESL\_SQ} from text info.\\
\hyperlink{func:esl_sq_Digitize()}{\ccode{esl\_sq\_Digitize()}} & Convert an \ccode{ESL\_SQ} to digital mode.\\
\hyperlink{func:esl_sq_Textize()}{\ccode{esl\_sq\_Textize()}} & Convert an \ccode{ESL\_SQ} to text mode.\\
\hyperlink{func:esl_sq_GuessAlphabet()}{\ccode{esl\_sq\_GuessAlphabet()}} & Guess alphabet type of a single sequence.\\
\apisubhead{Other functions that operate on sequences.}\\
\hyperlink{func:esl_sq_SetName()}{\ccode{esl\_sq\_SetName()}} & Format and set a name of a sequence.\\
\hyperlink{func:esl_sq_SetAccession()}{\ccode{esl\_sq\_SetAccession()}} & Format and set the accession field in a sequence.\\
\hyperlink{func:esl_sq_SetDesc()}{\ccode{esl\_sq\_SetDesc()}} & Format and set the description field in a sequence.\\
\hyperlink{func:esl_sq_SetSource()}{\ccode{esl\_sq\_SetSource()}} & Format and set the source name field in a sequence.\\
\hyperlink{func:esl_sq_CAddResidue()}{\ccode{esl\_sq\_CAddResidue()}} & Add one residue (or terminal NUL) to a text seq.\\
\hyperlink{func:esl_sq_XAddResidue()}{\ccode{esl\_sq\_XAddResidue()}} & Add one residue (or terminal sentinel) to digital seq.\\
\hyperlink{func:esl_sq_GetFromMSA()}{\ccode{esl\_sq\_GetFromMSA()}} & Get a single sequence from an MSA.\\
\hyperlink{func:esl_sq_FetchFromMSA()}{\ccode{esl\_sq\_FetchFromMSA()}} & Fetch a single sequence from an MSA.\\
\hline
\end{tabular}
}
\end{center}
\caption{The \eslmod{sq} API.}
\label{tbl:sq_api}
\end{table}


\subsection{Example of getting data in and out of an \ccode{ESL\_SQ}}

The easiest way to create a new \ccode{ESL\_SQ} object is with the
\ccode{esl\_sq\_CreateFrom()} function, which just takes character
strings for a sequence and its name (and also, optionally, an
accession, description, and/or secondary structure annotation string).

You can also build up (and/or change and manipulate) the contents of
an \ccode{ESL\_SQ} object by accessing the name, accession,
description, sequence, and structure annotation line more directly.

This code shows examples of both approaches:

\input{cexcerpts/sq_example}

A few things to notice about that code:

\begin{itemize}
\item Every sequence has a name and a sequence. If we didn't want to
  add the optional accession, description, or structure annotation
  line, we'd pass \ccode{NULL} for those arguments to
  \ccode{esl\_sq\_CreateFrom()}.

\item An RNA secondary structure annotation line is shown here as part
  of the example, but it's really sort of a more advanced
  feature. It's good to know it's there (see the \eslmod{wuss} module
  for more information about how \Easel\ annotates RNA structure) but
  you can ignore it if you're getting started.

\item The \ccode{esl\_sq\_Set*} functions use the same syntax as C's
  \ccode{*printf()} family, which gives you a flexible way to create 
  new sequence names, accessions, and descriptions automatically.

\item The sequence in \ccode{sq->seq} is just a C string. (Here it's a
  copy of the \ccode{testseq} string.) That has a couple of
  implications. One is that it's a verbatim copy of what you provided;
  \Easel\ doesn't know (or care) whether it's DNA or protein sequence,
  upper case or lower case, or if it contains illegal non-sequence
  characters. With a text mode sequence, that's \emph{your} problem!
  For more robustness and defined biosequence alphabets, read on below
  about digital mode sequences. The second implication is that, as a C
  string, the \ccode{n} residues are indexed \ccode{0..sq->n-1}, not
  \ccode{1..sq->n}.

\item If you're going to directly copy a sequence of length \ccode{n}
  into a \ccode{sq->seq} field, note the \ccode{esl\_sq\_GrowTo()}
  call, which makes sure the sequence object is allocated with enough
  space for \ccode{n} residues; and don't forget to set \ccode{sq->n}.

\item The structure annotation \ccode{sq->ss} is also a C string,
  indexed identically to \ccode{sq->seq}, but it's optional, and isn't
  allocated by default; \ccode{esl\_sq\_GrowTo()} calls will only
  reallocate for the structure annotation string after it's been
  allocated at least once.  Hence the \ccode{esl\_strdup} call in the
  example, which duplicates (allocates and copies) the annotation into
  \ccode{sq->ss}.
\end{itemize}



To get simple character strings back out of an \ccode{ESL\_SQ} object,
you're encouraged to peek inside the object. (Yeah, I know, object
oriented design says that there should be methods for this,
independent of the object's implementation; but I balance that against
simplicity, and here, simplicity wins.) The object is defined and
documented in \ccode{esl\_sq.h}. It contains various information; the
stuff you need to know is:

\input{cexcerpts/sq_sq}

Ignore the \ccode{dsq} field for now; we're about to get to it, when
we talk about digital mode sequences.

The \ccode{ESL\_SQ} object itself doesn't particularly care about the
contents of these text fields, so long as they're C strings, and so
long as \ccode{n} is the length of the \ccode{seq} (and optional
\ccode{ss}, if it's non-\ccode{NULL}) strings. However, sequence file
formats do impose some expectations on the annotation strings, and it
would be a Good Idea to adhere to them:

\begin{sreitems} {\emcode{desc}}
\item [\emcode{name}] A sequence name is almost always expected to be
  a single ``word'' (no whitespace), like \ccode{SNRPA\_HUMAN}. 

\item [\emcode{acc}] An accession is also usually expected to be a
  single ``word'' with no whitespace, like \ccode{P09012}. Database
  accessions only make sense if you know what database they're for, so
  when sequences might be from different databases, you'll sometimes
  see accessions prefixed with a code indicating the source database,
  as in something like \ccode{UniProt:P09012}. Again, \Easel\ itself
  isn't enforcing the format of this string, so your application is
  free to create its own accession/version/database format as needed.

\item [\emcode{desc}] A description line is something like \ccode{U1
small nuclear ribonucleoprotein A (U1 snRNP protein A) (U1A protein)
(U1-A).}; a one-line summary of what the sequence is. You can expect
the description line to show up in the tabular output of sequence
analysis applications, so ideally you want it to be short and sweet
(so it fits on one line with a name, accession, score, coords, and
other information from an analysis app). You also don't want the
description line to end in a newline (\verb+\n+) character, or the
description line will introduce unexpected line breaks in these
tabular output files.
\end{sreitems}

You can reach into a \ccode{ESL\_SQ} and copy or modify any of these
strings, but don't try to overwrite them with a larger string unless
You Know What You're Doing. Their memory allocations are managed by
the \ccode{ESL\_SQ} object. Instead, use the appropriate
\ccode{esl\_sq\_Set*} function to overwrite an annotation field.


The \eslmod{sq} module isn't much use by itself; it's a building block
for several other modules. For example, one of the main things you'll
want to do with sequences is to read them from a file. For examples
and documentation of sequence input, see the \eslmod{sqio} module.


\subsection{Example of using a digital \ccode{ESL\_SQ}}

What follows might make more sense if you've read about the
\eslmod{alphabet} module first. \eslmod{alphabet}'s documentation
explains how \Easel uses an internal digital biosequence ``alphabet'',
where residues are encoded as small integers, suitable for direct use
as array indices. But here's an example anyway, of creating and
accessing a digital mode sequence:

\input{cexcerpts/sq_example2}

Things to notice about this code:

\begin{itemize}
\item An \ccode{ESL\_SQ} object has a \ccode{sq->seq} if it's in text
mode, and \ccode{sq->dsq} if its in digital mode. These two fields are
mutually exclusive; one of them is \ccode{NULL}.

\item If you looked at the contents of \ccode{sq->dsq} in either of
  the objects, you'd see that each residue is encoded as a value
  \ccode{0..3}, representing (for an RNA alphabet) the residues
  \ccode{ACGU}. 

\item That representation is defined by the digital RNA alphabet
  \ccode{abc}, which was the first thing we created. 

\item In digital mode, both the sequence residues and the optional
  secondary structure characters are indexed \ccode{1..n}.

\item To make the digital sequence in the first sequence object, we
      created a digital sequence \ccode{dsq} by encoding the
      \ccode{testseq} using \ccode{esl\_abc\_CreateDsq()}; this
      function allocated new memory for \ccode{dsq}, so we have to
      free it. An \ccode{ESL\_DSQ *} is just a special character array;
      it's not a full-fledged \Easel\ object, and so there's no
      conventional \ccode{Create()},\ccode{Destroy()} function pair.

\item In the second sequence object, we used
  \ccode{esl\_abc\_Digitize()} to encode the \ccode{testseq} directly
  into space that the \ccode{sq2} object already had allocated, saving
  us the temporary allocation of another \ccode{dsq}, because we
  created it in digital mode (\ccode{esl\_sq\_CreateDigital()}) and
  made it big enough to hold \ccode{n} digital residues with
  \ccode{esl\_sq\_GrowTo()}. Notice that \ccode{esl\_sq\_GrowTo()} is
  smart enough to know whether to grow the digital or the text mode
  sequence field.

\item By convention, when using digital sequences, we usually keep
  track of (and pass as arguments) both a digital sequence \ccode{dsq}
  and its length \ccode{n}, and we also need to have the digital
  alphabet itself \ccode{abc} available to know what the \ccode{dsq}
  means; with text mode sequences, we usually just pass the string
  pointer. Thus the \ccode{esl\_sq\_CreateDigitalFrom()} function
  takes \ccode{abc}, \ccode{dsq}, and \ccode{n} as arguments, whereas
  the counterpart text mode \ccode{esl\_sq\_CreateDigitalFrom()} only
  took a C string \ccode{seq}. This is solely a convention - digital
  sequences begin and end with a special sentinel character, so we
  could always count the length of a \ccode{dsq} if we had to (using
  \ccode{esl\_abc\_dsqlen()}, for example), much as we can use ANSI
  C's \ccode{strlen()} to count the number of chars in a C string up
  to the sentinel \verb+\0+ \ccode{NUL} character at the end.

\item To get the structure annotation to be indexed \ccode{1..n} for
  consistency with the \ccode{dsq}, even though the annotation string
  is still just an ASCII string, it's offset by one, and the leading
  character is set by convention to a \verb+\0+. Therefore to access
  the whole structure string (for printing, for instance), you want to
  access \ccode{sq->ss+1}. This is a hack, but it's a simple one, so
  long as you don't forget about the convention.

\item Because the original sequence has been encoded, you may not get
  the original sequence back out when you decode the digital values as
  alphabet symbols. \ccode{abc->sym[sq2->dsq[3]]}, for example, takes
  the third digital residue and looks it up in the alphabet's symbol
  table, returning the canonical character it's
  representing. Upper/lower case distinctions are lost, for example;
  digital alphabet symbol tables are uniformly upper case. And this
  example shows another example, where the input \ccode{testseq}
  contains T's, but since the digital alphabet was declared as RNA,
  the symbol table represents those residues as U's when you access
  them.

\item In that respect, a more careful example should have checked the
  return status of the \ccode{esl\_abc\_CreateDsq()} and
  \ccode{esl\_abc\_Digitize()} calls. These have a normal failure
  mode, when the input text sequence contains one or more ASCII
  characters that are unrecognized and therefore invalid in the
  digital alphabet. If this had happened, these functions would have
  returned \ccode{eslEINVAL} instead of \ccode{eslOK}. We can get away
  without checking, however, because the functions just replace any
  invalid character with an ``any'' character (representing \ccode{N}
  for DNA or RNA, \ccode{X} for protein).
\end{itemize}

For more information about how digital sequence alphabets work, see
the \eslmod{alphabet} module.
