Development manual for vsearch 2.32.0, built from the dev branch: it describes changes that are not released yet. The manual for the current release is published separately.

NAME

vsearch --sintax — classify sequences using the SINTAX algorithm

SYNOPSIS

vsearch --sintax fastxfile --db filename --tabbedout filename [--sintax_cutoff real] [options]

DESCRIPTION

The vsearch command --sintax classifies query sequences from a fasta or fastq file using the SINTAX algorithm (Edgar 2016, doi:10.1101/074161), a non-Bayesian method for taxonomic classification. The reference database is specified with --db. Results are written to --tabbedout.

Classification works by finding the database sequence with the most shared k-mers with each query across 100 bootstrap resamples of the query’s k-mers. Each replicate draws 32 of the query’s unique k-mers (queries with fewer than 32 unique k-mers are never classified), and only counts when the best-matching database sequence shares at least 2 of the drawn k-mers with the query; the taxonomy of that sequence is then recorded. A query is classified when at least half of the 100 replicates counted; the frequency of agreement at each taxonomic rank, divided by the number of counted replicates, is reported as a bootstrap confidence value. Unclassified queries produce an output line with empty taxonomy, strand and cutoff columns. Use --sintax_cutoff to filter ranks below a confidence threshold.

The reference database must contain taxonomic annotations in the sequence headers. Each header must include a ;tax= field followed by a comma-separated list of taxonomic identifiers. Each identifier starts with a rank letter (d domain, k kingdom, p phylum, c class, o order, f family, g genus, s species, t strain), a colon, and the taxon name. Commas and semicolons are not allowed in taxon names. Example:

>X80725;tax=d:Bacteria,p:Proteobacteria,c:Gammaproteobacteria,o:Enterobacteriales,f:Enterobacteriaceae,g:Escherichia,s:Escherichia_coli
ACGT...

The --notrunclabels option is on by default for --sintax, allowing spaces in taxonomic identifiers.

When ties occur between database sequences with equally many k-mer matches, the shortest (and then earliest) sequence is chosen by default. The option --sintax_random is strongly recommended instead, as it breaks ties by a random draw and avoids a bias towards shorter reference sequences.

For reproducible results, set a fixed seed with --randseed. The classification is then identical regardless of the number of threads and reproducible across platforms. (Output lines are written in processing order, which can vary with --threads; sort the output if a fixed line order is also required.)

Both strands can be searched with --strand both. Databases in UDB format are supported (see vsearch-udb(5)). This command is multi-threaded.

OPTIONS

mandatory options

--db filename

Read reference sequences from filename, in fasta, fastq, or UDB format (see vsearch-udb(5)). Reference sequence headers must carry taxonomic annotations (see DESCRIPTION). --db accepts - to read the database from standard input, as well as an explicit stream path such as /dev/stdin, a named pipe, or a process substitution. The query and the database cannot both be -, however, as they would compete for the same standard input; give at least one of them an explicit path.

Note: a UDB database is not interchangeable with the fasta file it was built from. vsearch-makeudb_usearch(1) masks with dust by default and stores the masked sequences, whereas --sintax never runs the DUST algorithm on a fasta database (see --dbmask below). The same reference therefore yields different classifications in its two forms. Build the UDB with vsearch-makeudb_usearch(1) --dbmask none to obtain results identical to the fasta database.

--tabbedout filename

Write classification results to filename as a tab-separated file. Each query produces one row with the following columns:

query label;
predicted taxonomy with bootstrap confidence in parentheses after each rank (e.g., d:Bacteria(1.00),p:Proteobacteria(0.95));
strand (+ or -);
predicted taxonomy filtered by --sintax_cutoff, showing only ranks with bootstrap support at or above the threshold and omitting the values (e.g., d:Bacteria,p:Proteobacteria). Absent if --sintax_cutoff is not specified.

core options

--dbmask none|dust|soft: Mask regions in the database sequences using the dust method or the soft method, or none to suppress masking. See vsearch-fastx_mask(1) for more details. Warning, when using soft masking, search commands become case sensitive: masking excludes masked regions from the k-mer index used to select candidate targets (the pairwise alignment itself always ignores case). A database sequence with no unmasked stretch of at least the word length contributes no k-mer to the index and is never selected as a candidate target; it can still be reported when the query itself samples no k-mer, as such a query is compared against every database sequence (see --qmask). vsearch warns when there are such sequences, giving how many. The default is to mask using dust.; Note: --sintax does not run the DUST algorithm on the database: masking only controls whether lowercase regions of the reference sequences are ignored when indexing, so dust (the default) behaves like soft. This option has no effect on a UDB database, whose sequences were masked when the UDB was built (see --db).
--randseed integer: Set the seed for the pseudo-random generator. A given seed always produces the same results — reproducibly across platforms and independently of the number of threads — which is useful for replicability. By default, vsearch uses a pseudo-random seed if --randseed is not set, or set to the special value zero (0).
--sintax_cutoff real: Set the minimum bootstrap support required for a taxonomic rank to be included in column 4 of the output (e.g., 0.8 for 80%). Without this option, column 4 is absent.
--sintax_random: Break ties between database sequences with equally many k-mer matches by a random draw, rather than preferring the shortest or earliest sequence in the database. This option is strongly recommended as it avoids a bias towards shorter reference sequences.
--strand plus|both: Check the plus strand only (default), or check both strands when comparing sequences. Keywords are case-insensitive.
--threads positive integer: Set the number of computation threads to use, from 1 to 1024. The number of threads should not exceed the number of available CPU cores. The value 0 is also accepted and, like the default, uses all available cores; decimal values are truncated to their integer part. On Linux, “available” accounts for the CPU affinity mask and the cgroup CPU quota of the running process, so a job confined by taskset(1), Slurm, Docker or Kubernetes launches one thread per core it was actually granted, rather than one per core the machine has.

secondary options

--bzip2_decompress: Specify that the input pipe is streaming data compressed using Huffman coding. See bzip2(1) for more details. This option is required when compressed data arrives on standard input through a pipe (‘-’), where the format cannot be detected without consuming the stream. It is not needed when reading from a regular file compressed with bzip2, nor when such a file is redirected to standard input: compression is then detected automatically, and a contradicting option is ignored (with a warning when the input is standard input). Pipes other than standard input, such as shell process substitutions and named FIFOs, are always read as uncompressed data; compressed data must arrive on standard input or as a named file.
--fastq_ascii 33|64: Specify the offset used as the basis for the fastq quality score when reading fastq files. For example, an offset of 33 means that a quality value of 41 is represented by the 74th ASCII symbol (33 + 41 = 74), which is ‘J’. See ascii(7) for a view of the ASCII character set. The offset value is either 33 or 64, default is 33.

The offset matters even to a command that never decodes a quality score: it is what the reader compares the observed quality symbols against before warning that the file may use the other encoding, it sets the default of --fastq_qmax (the highest score the offset can represent), and the sum rules on --fastq_qmin and --fastq_qmax are stated in terms of it.

--fastq_qmax positive integer

Option is ignored: quality scores are not checked when reading fastq sequences, so no score is rejected for being too high. The argument itself is still validated, as the offset (see --fastq_ascii) plus the maximal score may not exceed 126, the last printable ASCII character.

--fastq_qmin positive integer

Option is ignored: quality scores are not checked when reading fastq sequences, so no score is rejected for being too low. The argument itself is still validated, as the offset (see --fastq_ascii) plus the minimal score must be at least 33, the first printable ASCII character.

--gzip_decompress

Specify that the input pipe is streaming data compressed using Lempel-Ziv coding. See gzip(1) for more details. This option is required when compressed data arrives on standard input through a pipe (‘-’), where the format cannot be detected without consuming the stream. It is not needed when reading from a regular file compressed with gzip, nor when such a file is redirected to standard input: compression is then detected automatically, and a contradicting option is ignored (with a warning when the input is standard input). Pipes other than standard input, such as shell process substitutions and named FIFOs, are always read as uncompressed data; compressed data must arrive on standard input or as a named file.

--label_suffix string

Add the suffix string to sequence headers when writing fasta or fastq files. For example, with --label_suffix ";status=healthy", sequence header ‘>seq1’ becomes ‘>seq1;status=healthy’.

--log filename

Write messages to filename. Messages include program version, start and finish times, elapsed time, amount of memory available, maximum amount of memory consumed, number of cores and command line options, and if need be, command-specific informational messages, warnings, and errors. Messages are written in addition to the standard error, not instead of it; use --quiet to silence the standard error copy (see the --quiet entry for what it spares).

--maxseqlength positive integer

Discard database sequences longer than positive integer (50,000 nucleotides by default).

--minseqlength positive integer

Discard database sequences shorter than positive integer (32 nucleotides by default).

--no_progress

Suppress the gradually increasing progress indicator normally written to the standard error stderr(3).

--notrunclabels

Retain whole sequence headers in output files. By default, vsearch truncates sequence headers at first space or tabulation. This option suppresses truncation.

Always implied with --sintax (headers are never truncated, so that taxonomy annotations may contain spaces); it cannot be disabled.

--quiet

Suppress messages to the standard output stdout(3) and standard error stderr(3), except for warnings and error messages.

--wordlength positive integer

Set the length of words (i.e. k-mers) used for sequence indexing and comparisons. Valid values range from 3 to 15. The default is 8. Note that the default --minwordmatches is derived from this value, so changing one changes both (see --minwordmatches).

Longer words make the k-mer index more selective, so fewer targets are offered as candidates and the search itself gets faster: from word length 5 to 11 the search phase shrank by a factor of 2.4 on 130-nucleotide amplicons and 5.8 on full-length reference sequences. Working against that, the index has 4^wordlength slots, so the memory it needs and the time spent building it both quadruple with each added nucleotide. On a 400 000-sequence database the whole run needed 0.3 GB at word length 10, 1.3 GB at 12 and 16 GB at 15.

The best setting balances the two, and depends on how many queries are searched against a given database: with few queries the index build dominates and a shorter word is cheaper overall, while with many queries the search dominates and a longer word repays its index. Changing the word length is not output-neutral, so it should be chosen for a workload rather than tuned per run.

EXAMPLES

Classify query sequences against a taxonomy-annotated reference database, writing all confidence values to the output:

vsearch \
    --sintax queries.fasta \
    --db reference.fasta \
    --tabbedout classification.tsv

Classify and report only ranks with at least 80% bootstrap support (--sintax_cutoff 0.8), using the recommended tie-breaking option and searching both strands:

vsearch \
    --sintax queries.fasta \
    --db reference.fasta \
    --sintax_cutoff 0.8 \
    --sintax_random \
    --strand both \
    --tabbedout classification.tsv

Produce reproducible results using a fixed random seed:

vsearch \
    --sintax queries.fasta \
    --db reference.fasta \
    --sintax_cutoff 0.8 \
    --sintax_random \
    --randseed 42 \
    --tabbedout classification.tsv

CITATION

Rognes T, Flouri T, Nichols B, Quince C, Mahé F. (2016) VSEARCH: a versatile open source tool for metagenomics. PeerJ 4:e2584 doi: 10.7717/peerj.2584

REPORTING BUGS

Submit suggestions and bug-reports at https://github.com/torognes/vsearch/issues, send a pull request on https://github.com/torognes/vsearch, or compose a friendly or curmudgeont e-mail to Torbjørn Rognes (torognes@ifi.uio.no).

AVAILABILITY

Source code and binaries are available at https://github.com/torognes/vsearch.

These manual pages are also published online at https://torognes.github.io/vsearch/.

COPYRIGHT

Contact: Torbjørn Rognes torognes@ifi.uio.no, Department of Informatics, University of Oslo, PO Box 1080 Blindern, NO-0316 Oslo, Norway

This software is dual-licensed and available under a choice of one of two licenses, either under the terms of the GNU General Public License version 3 or the BSD 2-Clause License.

GNU General Public License version 3

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

The BSD 2-Clause License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

ACKNOWLEDGMENTS

We would like to thank the authors of the following projects for making their source code available:

vsearch includes code from Google’s CityHash project by Geoff Pike and Jyrki Alakuijala, providing some excellent hash functions available under a MIT license.
vsearch includes code derived from Tatusov and Lipman’s DUST program that is in the public domain.
vsearch includes public domain code written by Alexander Peslyak for the MD5 message digest algorithm.
vsearch includes public domain code written by Steve Reid and others for the SHA1 message digest algorithm.
vsearch binaries may include code from the zlib library, copyright Jean-Loup Gailly and Mark Adler.
vsearch binaries may include code from the bzip2 library, copyright Julian R. Seward.