How to install and run pbalign ============================== pbalign is a tool for aligning PacBio reads to reference sequences. It is part of the PacBio Bioinformatics tools, and will be bundled in the 2.1 release of SMRTanalysis. You may also follow the instructions below to install pbalign. *Note: the pseudo namespace pbtools has been removed in version 0.2.0.* *Note: program name has been changed from `pbalign.py` in version 0.1.0 to `pbalign` in version 0.2.0.* *Note: please install this software on an isolated machine that does not have SMRTanalysis installed.* Background ---------- **pbalign** aligns PacBio reads to reference sequences, filters aligned reads according to user-specific filtering criteria, and converts the output to either the SAM format or PacBio Compare HDF5 (e.g., .cmp.h5) format. The output Compare HDF5 file will be compatible with Quiver if ``--forQuiver`` option is specified. Required software ----------------- pbalign is available through the ``pbalign`` script from the ``pbalign`` package. To use pbalign, the following PacBio software is required, - ``pbalign``, containing ``pbalign`` - ``pbcore``, a package providing access to PacBio data files - ``blasr``, a package of PacBio aligner blasr, containing c++ executables for processing PacBio data, such as ``blasr``, ``pls2fasta``, ``samFilter``, ``samtoh5`` and ``loadPulses`` The following software is optionally required if ``--forQuiver`` option will be used to convert the output Compare HDF5 file to be compatible with Quiver. - ``pbh5tools.cmph5tools``, a PacBio Bioinformatics tools that manipulates Compare HDF5 files. - ``h5repack``, a HDF5 tool to compress and repack HDF5 files. The default aligner that pbalign uses is ``blasr``. If you want to use bowtie2 as aligner, then the bowtie2 package also needs to be installed. Required libraries and tools ---------------------------- - Python 2.7.3 - virtualenv (builds isolated Python environments) - numpy 1.6.1 (required by pbcore) - h5py 2.0.1 (required by pbcore) If you are within PacBio, these requirements are already installed within the cluster environment. Otherwise, you will need to install them yourself. Data file requirements ---------------------- pbalign distinguishes input and output file formats by file extensions. The input PacBio reads can be in FASTA, Base HDF5, Pulse HDF5, Circular Consensus Sequence (CCS) HDF5 or file or file names (FOFN). The supported input file extensions are as follows. - FASTA : .fa or .fasta - PacBio BASE HDF5 : .bas.h5 or .bax.h5 - PacBio PULSE HDF5 : .pls.h5 or .plx.h5 - PacBio CCS HDF5 : .ccs.h5 - File of file names : .fofn The input reference sequences can be in a FASTA file or a reference deposit directory created by referenceUploader (a PacBio tool for uploading references to the server and data preprocessing). The output file can either be a SAM file or a Compare HDF5 file. The output Compare HDF5 file cannot be consumed by Quiver directly unleis ``--forQuiver`` option is specified. The supported output file extensions are as follows. - SAM : .sam - PacBio Compare HDF5: .cmp.h5 Manual installation instructions -------------------------------- Step 1: Set up your Python virtual environment ``````````````````````````````````````````````````` To install ``Python 2.7``, please visit :: http://www.python.org/ , or if you have root permission on Ubuntu, execute :: sudo apt-get install python To install ``pip``, please visit :: https://pypi.python.org/pypi/pip , or if you have root permission using Ubuntu, execute :: sudo apt-get install python-pip To install ``virtualenv``, please visit :: https://pypi.python.org/pypi/virtualenv , or execute :: pip install virtualenv To set up a new virtualenv, do :: $ cd; virtualenv -p python2.7 --no-site-packages my_env , and activate the virtualenv using :: $ source ~/my_env/bin/activate To install ``git``, please visit :: http://git-scm.com/. Step 2: Install required software and library ````````````````````````````````````````````` To install blasr, please execute :: $ git clone https://github.com/PacificBiosciences/blasr , and follow instructions at :: https://github.com/PacificBiosciences/blasr/blob/master/README.md Before installing pbcore, you may need to install numpy and h5py from :: http://www.numpy.org/ https://code.google.com/p/h5py/ , or if you have root permission on Ubuntu, do :: $ pip install numpy $ sudo apt-get install libhdf5-serial-dev $ pip install h5py To install pbcore, execute :: $ pip install git+https://github.com/PacificBiosciences/pbcore Step 3: Install optionally required software and library ```````````````````````````````````````````````````````` To install pbh5tools, execute :: $ pip install git+https://github.com/PacificBiosciences/pbh5tools To install ``HDF5 tools``, visit :: http://www.hdfgroup.org/products/hdf5_tools/ , or if you have root permission on Ubuntu, do :: $ sudo apt-get install hdf5-tools Step 4: Install pbalign ``````````````````````` To *uninstall* pbalign, execute :: $ pip uninstall pbalign To install pbalign, execute :: $ pip install git+https://github.com/PacificBiosciences/pbalign , or to download the whole pbalign package with examples :: $ git clone https://github.com/PacificBiosciences/pbalign.git $ cd pbalign $ pip install . Examples -------- (1) Basic usage of pbalign. - Example (1.1) :: $ pbalign tests/data/example_read.fasta \ tests/data/example_ref.fasta \ example.sam - Example (1.2) :: $ pbalign tests/data/example_read.fasta \ tests/data/example_ref.fasta \ example.cmp.h5 - Example (1.3) - with optional arguments :: $ pbalign --maxHits 10 --hitPolicy all \ tests/data/example_read.fasta \ tests/data/example_ref.fasta \ example.sam (2) Advanced usage of pbalign. - Example (2.1) - Import pre-defined options from a config File :: $ pbalign --configFile=tests/data/1.config \ tests/data/example_read.fasta \ tests/data/example_ref.fasta \ example.sam - Example (2.2) - Pass options through to aligner :: $ pbalign --algorithmOptions='-nCandidates 10 -sdpTupleSize 12' \ tests/data/example_read.fasta \ tests/data/example_ref.fasta \ example.sam - Example (2.3) - Create a cmp.h5 file with --forQuiver option :: # The output cmp.h5 file will loaded with quality values (pulses) # from the input bas/bax.h5 file, sorted and repacked, and therefore # can be consumed by Quiver directly, (Note that in order to use # --forQuiver option, cmph5tools and h5repack are required.) $ pbalign --forQuiver your_movie.bas.h5 your_reference.fasta out.cmp.h5 (3) Use pbalign as a library through Python API. - Example (3.1) :: $ python >>> from pbalign.pbalignrunner import PBAlignRunner >>> # Specify arguments in a list. >>> args = ['--maxHits', '20', 'tests/data/example_read.fasta',\ ... 'tests/data/example_ref.fasta', 'example.sam'] >>> # Create a PBAlignRunner object. >>> a = PBAlignRunner(args) >>> # Execute. >>> exitCode = a.start() >>> # Show all files used. >>> print a.fileNames Usage ----- :: usage: pbalign [-h] [--verbose] [--version] [--profile] [--debug] [--regionTable REGIONTABLE] [--configFile CONFIGFILE] [--algorithm {blasr,bowtie}] [--maxHits MAXHITS] [--minAnchorSize MINANCHORSIZE] [--useccs {useccs,useccsall,useccsdenovo}] [--noSplitSubreads] [--nproc NPROC] [--algorithmOptions ALGORITHMOPTIONS] [--maxDivergence MAXDIVERGENCE] [--minAccuracy MINACCURACY] [--minLength MINLENGTH] [--scoreFunction {alignerscore,editdist,blasrscore}] [--scoreCutoff SCORECUTOFF] [--hitPolicy {randombest,allbest,random,all}] [--forQuiver] [--seed SEED] [--tmpDir TMPDIR] inputFileName referencePath outputFileName Mapping PacBio sequences to references using an algorithm selected from a selection of supported command-line alignment algorithms. Input can be a fasta, pls.h5, bas.h5 or ccs.h5 file or a fofn (file of file names). Output is in either cmp.h5 or sam format. positional arguments: inputFileName The input file can be a fasta, pls.h5, bas.h5, ccs.h5 file or a fofn. referencePath Either a reference fasta file or a reference repository. outputFileName The output cmp.h5 or sam file. optional arguments: -h, --help show this help message and exit --verbose, -v Set the verbosity level --version show program's version number and exit --profile Print runtime profile at exit --debug Run within a debugger session --regionTable REGIONTABLE Specify a region table for filtering reads. --configFile CONFIGFILE Specify a set of user-defined argument values. --algorithm {blasr,bowtie} Select an aligorithm from ('blasr', 'bowtie'). Default algorithm is blasr. --maxHits MAXHITS The maximum number of matches of each read to the reference sequence that will be evaluated. Default value is 10. --minAnchorSize MINANCHORSIZE The minimum anchor size defines the length of the read that must match against the reference sequence. Default value is 12. --useccs {useccs,useccsall,useccsdenovo} Map the ccsSequence to the genome first, then align subreads to the interval that the CCS reads mapped to. useccs: only maps subreads that span the length of the template. useccsall: maps all subreads. useccsdenovo: maps ccs only. --noSplitSubreads Do not split reads into subreads even if subread regions are available. Default value is False. --nproc NPROC Number of threads. Default value is 8. --algorithmOptions ALGORITHMOPTIONS Pass alignment options through. --maxDivergence MAXDIVERGENCE The maximum allowed percentage divergence of a read from the reference sequence. Default value is 30. --minAccuracy MINACCURACY The minimum percentage accuracy of alignments that will be evaluated. Default value is 70. --minLength MINLENGTH The minimum aligned read length of alignments that will be evaluated. Default value is 50. --scoreFunction {alignerscore,editdist,blasrscore} Specify a score function for evaluating alignments. alignerscore : aligner's score in the SAM tag 'as'. editdist : edit distance between read and reference. blasrscore : blasr's default score function. Default value is alignerscore. --scoreCutoff SCORECUTOFF The worst score to output an alignment. --hitPolicy {randombest,allbest,random,all} Specify a policy for how to treat multiple hit random : selects a random hit. all : selects all hits. allbest : selects all the best score hits. randombest: selects a random hit from all best alignment score hits. Default value is randombest. --forQuiver The output cmp.h5 file which will be sorted, loaded with pulse information, and repacked, so that it can be consumed by quiver directly. This requires the input file to be in PacBio bas/pls.h5 format. Default value is False. --seed SEED Initialize the random number generator with a none-zero integer. Zero means that current system time is used. Default value is 1. --tmpDir TMPDIR Specify a directory for saving temporary files. Default is /scratch.