Development manual for vsearch 2.32.0, built from the dev branch: it describes changes that are not released yet. The manual for the current release is published separately.

NAME

vsearch --fastx_uniques — merge strictly identical fasta or fastq sequences

SYNOPSIS

vsearch --fastx_uniques fastxfile (--fastaout | --fastqout) filename [options]

DESCRIPTION

The vsearch command --fastx_uniques groups strictly identical sequences from a fasta or fastq file, like --derep_fulllength (see vsearch-derep_fulllength(1)). The unique sequences are written to --fastaout and/or --fastqout, sorted by decreasing abundance (ties are broken by header, in alphanumerical order, then by input order). --fastqout requires fastq input; --maxuniquesize, --minuniquesize and --topn filter the fasta and fastq outputs only, while the --uc and --tabbedout outputs always list all clusters.

When the input is a fastq file, quality scores in the output correspond by default to the average error probability at each position across all grouped sequences (see vsearch-expected_error(7)). Use --fastq_qout_max to use the best (highest) quality score observed at each position instead. Output quality scores are encoded with the --fastq_asciiout offset and clamped to the range defined by --fastq_qminout and --fastq_qmaxout; a quality score of 0 is written as 1, since both stand for the same uninformative error probability (0.75).

The average is accumulated one grouped sequence at a time, and the running value is converted back to a quality score at every step. A score is therefore a poor estimator of the group’s mean quality once the group is large: because a quality score stands for the highest error probability it can represent, a grouped sequence that disagrees downward at a position lowers the group’s score there, while one that agrees upward leaves it unchanged as soon as the group carries much more abundance than the incoming sequence. A high-abundance group’s scores thus reflect how many of its members disagreed downward at each position rather than the average of their scores. --fastq_qout_max is not affected, being independent of order and abundance.

This command is not multithreaded. The inverse operation is --rereplicate (see vsearch-rereplicate(1)).

See vsearch-fasta(5) and vsearch-fastq(5) for a description of the input formats.

OPTIONS

mandatory options

At least one of --fastaout, --fastqout, --tabbedout, or --uc must be specified.

core options

--fastaout filename
Write the dereplicated sequences to filename, in fasta format, sorted by decreasing abundance. Each unique sequence retains the header of the first occurrence in the input.
--fastqout filename
Write the dereplicated sequences to filename, in fastq format (see vsearch-fastq(5)), sorted by decreasing abundance. Quality scores correspond to the average error probability at each position across all grouped sequences, or to the best quality score if --fastq_qout_max is set.
--fastq_qout_max
Use the best (highest) quality score observed at each position when computing quality scores for fastq output, instead of averaging error probabilities.
--maxuniquesize positive integer

Discard sequences with a post-dereplication abundance greater than positive integer. When that value is smaller than the --minuniquesize value, no sequence can satisfy the abundance filter: vsearch issues a warning and writes no sequence.

The filter applies to the sequence output only. A cluster summary written with --uc, where the command offers it, still lists every cluster, discarded ones included. This matches usearch.

--minuniquesize positive integer

Discard sequences with a post-dereplication abundance smaller than positive integer. When that value is larger than the --maxuniquesize value, no sequence can satisfy the abundance filter: vsearch issues a warning and writes no sequence.

The filter applies to the sequence output only. A cluster summary written with --uc, where the command offers it, still lists every cluster, discarded ones included. This matches usearch.

--sizein
Use the abundance annotations present in sequence headers when reading fasta or fastq file. Search for the pattern [>@;]size=integer[;]. Entries without abundance annotations are silently assumed to be of size=1.
--sizeout
Add abundance annotations to sequence headers when writing fasta or fastq files. Add the pattern ;size=integer, where the integer is the total abundance of the cluster: the sum of the members’ abundance annotations when --sizein is used, or the number of merged sequences otherwise.
--strand plus|both
Check the plus strand only (default), or check both strands when comparing sequences. Keywords are case-insensitive.
--tabbedout filename

Write dereplication details to filename as a tab-separated file with one row per input sequence and 6 columns:

  1. original sequence label;
  2. output label (label of the first sequence in the cluster, possibly relabelled);
  3. cluster number (zero-based);
  4. sequence number within the cluster (zero-based);
  5. cluster size (the number of sequences merged into the cluster, whether or not --sizein is used);
  6. original label of the first sequence in the cluster (before any relabelling).
--topn positive non-null integer
Write only the n first entries. The parameter n must be greater than zero, and is silently ignored if it is greater than the total number of entries.
--uc filename

Write results to filename in a tab-separated uclust-like format with 10 columns. Three record types are used per row: cluster seeds (S), hits (H), and cluster summaries (C). Columns are:

  1. record type (S, H, or C);
  2. cluster number (zero-based);
  3. centroid length (S), query length (H), or cluster size (C): the number of sequences in the cluster, or their total abundance when --sizein is used;
  4. percent identity with centroid (H), or * (S, C);
  5. match orientation + or - (H), or * (S, C);
  6. not used; always 0 (H) or * (S, C);
  7. not used; always 0 (H) or * (S, C);
  8. CIGAR alignment string (H), or * (S, C); = denotes strictly identical sequences (with --cluster_fast, identical ignoring terminal gaps); the dereplication commands always write * here; see vsearch-cigar(5);
  9. query label (H), or centroid label (S, C);
  10. centroid label (H), or * (S, C).

secondary options

--bzip2_decompress
Specify that the input pipe is streaming data compressed using Huffman coding. See bzip2(1) for more details. This option is required when compressed data arrives on standard input through a pipe (‘-’), where the format cannot be detected without consuming the stream. It is not needed when reading from a regular file compressed with bzip2, nor when such a file is redirected to standard input: compression is then detected automatically, and a contradicting option is ignored (with a warning when the input is standard input). Pipes other than standard input, such as shell process substitutions and named FIFOs, are always read as uncompressed data; compressed data must arrive on standard input or as a named file.
--fasta_width positive integer
Set the maximal width of sequences when writing fasta files. Longer sequences are folded and written on several lines. Default width is 80 nucleotides. Set to zero (0) to suppress folding.
--fastq_ascii 33|64
Specify the offset used as the basis for the fastq quality score when reading fastq files. For example, an offset of 33 means that a quality value of 41 is represented by the 74th ASCII symbol (33 + 41 = 74), which is ‘J’. See ascii(7) for a view of the ASCII character set. The offset value is either 33 or 64, default is 33.

The offset matters even to a command that never decodes a quality score: it is what the reader compares the observed quality symbols against before warning that the file may use the other encoding, it sets the default of --fastq_qmax (the highest score the offset can represent), and the sum rules on --fastq_qmin and --fastq_qmax are stated in terms of it.

--fastq_asciiout 33|64
Specify the offset used as the basis for the fastq quality score when writing fastq output files. For example, an offset of 33 means that a quality value of 41 is represented by the 74th ASCII symbol (33 + 41 = 74), which is ‘J’. See ascii(7) for a view of the ASCII character set. The offset value is either 33 or 64, default is 33.
--fastq_qmax integer
Specify the maximal quality score accepted when reading fastq sequences. Stop with an error message if a quality score higher than the specified value is read. The offset (see --fastq_ascii) plus the maximal score may not exceed 126, the last printable ASCII character: scores up to 93 with offset 33, up to 62 with offset 64. The default is the highest score the offset can represent (93 with offset 33, 62 with offset 64), so no quality score is rejected unless this option is lowered. Before version 2.32.0 the default was 41, the usual maximum for Sanger/Illumina 1.8+ files, which rejected PacBio HiFi and nanopore files outright.
--fastq_qmaxout integer
Specify the maximum quality score used when writing fastq files. The default is the highest score the output offset can represent (93 with --fastq_asciiout 33, 62 with offset 64), so scores read from the input are written back unchanged. Before version 2.32.0 the default was 41, the usual maximum for Sanger/Illumina 1.8+ files, which silently reduced any higher score. Older formats may use a maximum quality score of 40. Two commands are exceptions and keep the old default of 41, because they generate the score they clamp instead of passing one through: --fasta2fastq, which has no input quality and uses this option as the value to write, and --fastq_mergepairs, which caps the computed posterior quality of a merged base.

For --fastq_mergepairs the offset in question is --fastq_ascii, not --fastq_asciiout: that command writes fastq but does not accept --fastq_asciiout, so a merged quality symbol carries the same offset the input was read with. The sum rule is stated against --fastq_ascii there, and against --fastq_asciiout everywhere else.

--fastq_qmin integer
Specify the minimal quality score accepted when reading fastq sequences. Stop with an error message if a quality score lower than the specified value is read. The offset (see --fastq_ascii) plus the minimal score must be at least 33, the first printable ASCII character: scores down to 0 with offset 33, down to -31 with offset 64. The value may therefore be negative, which is what the negative scores of older formats require, but note that those formats are not supported (see vsearch-fastq(5)). The default is 0, which is usual for recent Sanger/Illumina 1.8+ files.
--fastq_qminout integer
Specify the minimum quality score used when writing fastq files. The offset (see --fastq_asciiout) plus the minimum score must be at least 33, so the value may be negative when the output offset is 64. The default is 0, which is usual for recent Sanger/Illumina 1.8+ files. Older formats may use scores between -5 and 2.

For --fastq_mergepairs the offset in question is --fastq_ascii, not --fastq_asciiout: that command writes fastq but does not accept --fastq_asciiout, so a merged quality symbol carries the same offset the input was read with, and the bound follows it.

A negative minimum matters where the score passes through vsearch and may itself be negative: --fastq_convert on an offset-64 file read with a lowered --fastq_qmin writes the score back unchanged with --fastq_qminout -5, where the default 0 would raise it. It has no effect where vsearch computes the score it clamps, because such a score is derived from an error probability and is therefore never negative: --fastq_mergepairs, and --fastq_convert --fastq_solexa, which converts to the Phred scale before any output clamp applies.

--gzip_decompress
Specify that the input pipe is streaming data compressed using Lempel-Ziv coding. See gzip(1) for more details. This option is required when compressed data arrives on standard input through a pipe (‘-’), where the format cannot be detected without consuming the stream. It is not needed when reading from a regular file compressed with gzip, nor when such a file is redirected to standard input: compression is then detected automatically, and a contradicting option is ignored (with a warning when the input is standard input). Pipes other than standard input, such as shell process substitutions and named FIFOs, are always read as uncompressed data; compressed data must arrive on standard input or as a named file.
--label_suffix string
Add the suffix string to sequence headers when writing fasta or fastq files. For example, with --label_suffix ";status=healthy", sequence header ‘>seq1’ becomes ‘>seq1;status=healthy’.
--lengthout
Add a sequence length annotation (;length=integer) to each sequence header when writing fasta or fastq files.
--log filename
Write messages to filename. Messages include program version, start and finish times, elapsed time, amount of memory available, maximum amount of memory consumed, number of cores and command line options, and if need be, command-specific informational messages, warnings, and errors. Messages are written in addition to the standard error, not instead of it; use --quiet to silence the standard error copy (see the --quiet entry for what it spares).
--maxseqlength positive integer
Discard sequences longer than positive integer (50,000 nucleotides by default). The value must not exceed 2,147,481,646 (INT_MAX minus 2,001). When that value is smaller than the effective --minseqlength value (whose default is command-specific), no sequence can pass the length filter: vsearch issues a warning.
--minseqlength non-negative integer
Discard sequences shorter than non-negative integer (1 nucleotide by default). A value of 0 retains empty sequences. When that value is larger than the --maxseqlength value, no sequence can pass the length filter: vsearch issues a warning.
--no_progress
Suppress the gradually increasing progress indicator normally written to the standard error stderr(3).
--notrunclabels
Retain whole sequence headers in output files. By default, vsearch truncates sequence headers at first space or tabulation. This option suppresses truncation.
--quiet
Suppress messages to the standard output stdout(3) and standard error stderr(3), except for warnings and error messages.
--relabel string
Replace sequence headers with the prefix string and a ticker (1, 2, 3, etc.). For example, with --relabel "cluster:", the first sequence header becomes ‘>cluster:1’, the second sequence header becomes ‘>cluster:2’, and so on. To retain annotations, use their corresponding options (--lengthout, --eeout, and --sizeout). Use --relabel_keep to also retain old sequence identifiers.
--relabel_keep
Retain old sequence identifiers by including them at the end of the new headers, after a space.
--relabel_md5
Replace each sequence header with the MD5 digest derived from the sequence itself. The sequence is converted to upper case, and each ‘U’ is replaced with a ‘T’ before computation of the digest. The MD5 digest is a 128-bit value (16 bytes), represented using a string of 32 ASCII characters. Each pair of characters encodes an hexadecimal value, ranging from x00 to xff. See md5(3) for more details, and --relabel_sha1 for an alternative hashing algorithm. To retain annotations, use their corresponding options (--lengthout, --eeout, and --sizeout). Use --relabel_keep to also retain old sequence identifiers.
--relabel_self
Replace each sequence header with the sequence itself. To retain annotations, use their corresponding options (--lengthout, --eeout, and --sizeout). Use --relabel_keep to also retain old sequence identifiers.
--relabel_sha1
Replace each sequence header with the SHA1 digest derived from the sequence itself. The sequence is converted to upper case, and each ‘U’ is replaced with a ‘T’ before computation of the digest. The SHA1 digest is a 160-bit value (20 bytes), represented using a string of 40 ASCII characters. Each pair of characters encodes an hexadecimal value, ranging from x00 to xff. See sha1(3) for more details, and --relabel_md5 for an alternative hashing algorithm. To retain annotations, use their corresponding options (--lengthout, --eeout, and --sizeout). Use --relabel_keep to also retain old sequence identifiers.
--sample string
Add the given sample identifier string to sequence headers when writing fasta or fastq files. For instance, if string is ‘ABC’, the text ;sample=ABC will be added to the headers. string is silently truncated at the first ‘;’ or whitespace character (space, tab, newline, carriage return, vertical tab or form feed), so such characters should not be used in string. Other characters (alphabetical, numerical and punctuations) are accepted. When nothing is left after truncation — an empty string, or one starting with ‘;’ or a blank character — vsearch issues a warning and writes a bare ;sample= annotation.
--xee
Strip expected error (ee) annotations from sequence headers when writing fasta or fastq files. Search for the pattern [>@;]ee=float[;]. Expected error annotations are added by the synonymous options --fastq_eeout and --eeout described in vsearch-fastx_filter(1).
--xlength
Strip sequence length annotations from sequence headers when writing fasta or fastq files. Search for the pattern [>@;]length=integer[;]. Sequence length annotations are added by the --lengthout option.
--xsize
Strip abundance annotations from sequence headers when writing fasta or fastq files. Search for the pattern [>@;]size=integer[;]. Abundance annotations are added by the --sizeout option.

ignored options

--threads positive non-null integer
Command is not multithreaded, option has no effect (a warning is printed when more than one thread is requested).

EXAMPLES

Dereplicate sequences in input.fasta, annotate with abundance, write to derep.fasta, and record clustering details in derep.uc:

vsearch \
    --fastx_uniques input.fasta \
    --sizeout \
    --fastaout derep.fasta \
    --uc derep.uc

Dereplicate a fastq file, keeping only the best quality score at each position:

vsearch \
    --fastx_uniques input.fastq \
    --fastq_qout_max \
    --sizeout \
    --fastqout derep.fastq

SEE ALSO

vsearch-derep_fulllength(1), vsearch-derep_id(1), vsearch-derep_prefix(1), vsearch-derep_smallmem(1), vsearch-rereplicate(1), vsearch-fasta(5), vsearch-fastq(5), vsearch-expected_error(7)

CITATION

Rognes T, Flouri T, Nichols B, Quince C, Mahé F. (2016) VSEARCH: a versatile open source tool for metagenomics. PeerJ 4:e2584 doi: 10.7717/peerj.2584

REPORTING BUGS

Submit suggestions and bug-reports at https://github.com/torognes/vsearch/issues, send a pull request on https://github.com/torognes/vsearch, or compose a friendly or curmudgeont e-mail to Torbjørn Rognes (torognes@ifi.uio.no).

AVAILABILITY

Source code and binaries are available at https://github.com/torognes/vsearch.

These manual pages are also published online at https://torognes.github.io/vsearch/.

COPYRIGHT

Copyright (C) 2014-2026, Torbjørn Rognes, Frédéric Mahé and Tomás Flouri

All rights reserved.

Contact: Torbjørn Rognes torognes@ifi.uio.no, Department of Informatics, University of Oslo, PO Box 1080 Blindern, NO-0316 Oslo, Norway

This software is dual-licensed and available under a choice of one of two licenses, either under the terms of the GNU General Public License version 3 or the BSD 2-Clause License.

GNU General Public License version 3

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

The BSD 2-Clause License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

ACKNOWLEDGMENTS

We would like to thank the authors of the following projects for making their source code available:

  • vsearch includes code from Google’s CityHash project by Geoff Pike and Jyrki Alakuijala, providing some excellent hash functions available under a MIT license.
  • vsearch includes code derived from Tatusov and Lipman’s DUST program that is in the public domain.
  • vsearch includes public domain code written by Alexander Peslyak for the MD5 message digest algorithm.
  • vsearch includes public domain code written by Steve Reid and others for the SHA1 message digest algorithm.
  • vsearch binaries may include code from the zlib library, copyright Jean-Loup Gailly and Mark Adler.
  • vsearch binaries may include code from the bzip2 library, copyright Julian R. Seward.
This site uses Just the Docs, a documentation theme for Jekyll.