vsearch

NAME

vsearch \-\-fastq_stats — analyze fastq sequences and output detailed statistics

SYNOPSIS

vsearch \-\-fastq_stats fastqfile \-\-log outputfile [options]

DESCRIPTION

The vsearch command --fastq_stats analyzes fastq sequences and outputs detailed statistics, including projections of the impact different filtering thresholds would have. Results are written to --log outputfile. The quality encoding of the input fastq and the range of accepted quality values can be specified with --fastq_ascii, --fastq_qmin and --fastq_qmax.

The five different statistics tables reported by --fastq_stats are described below, with an example for each.

Read length distribution

Observed read lengths are sorted in decreasing order. The largest length value is marked with ‘>=’.

1. L: read length
2. N: number of reads
3. Pct: fraction of reads with this length
4. AccPct: fraction of reads with this length or longer

Read length distribution
      L           N      Pct   AccPct
-------  ----------  -------  -------
>=  251       19795    69.2%    69.2%
    250        6847    23.9%    93.2%
    249        1626     5.7%    98.9%
    248         260     0.9%    99.8%
    247          24     0.1%    99.8%
[...]
     75           5     0.0%   100.0%
     74           2     0.0%   100.0%
     66           2     0.0%   100.0%
     65           4     0.0%   100.0%
     64           1     0.0%   100.0%

Quality score distribution

Observed quality values are sorted in decreasing order.

1. ASCII: character encoding the quality score
2. Q: Phred quality score
3. Pe: probability of error associated with the quality score
4. N: number of bases with this quality score
5. Pct: fraction of bases with this quality score
6. AccPct: fraction of bases with this quality score or higher

Q score distribution
ASCII    Q       Pe           N      Pct   AccPct
-----  ---  -------  ----------  -------  -------
    I   40  0.00010           2     0.0%     0.0%
    H   39  0.00013     1009552    14.1%    14.1%
    G   38  0.00016     1236861    17.3%    31.4%
    F   37  0.00020     1231882    17.2%    48.6%
    E   36  0.00025      330094     4.6%    53.2%
    D   35  0.00032      143134     2.0%    55.2%
    C   34  0.00040      250511     3.5%    58.7%
    B   33  0.00050      451696     6.3%    65.0%
    A   32  0.00063      273153     3.8%    68.8%
    @   31  0.00079      171356     2.4%    71.2%
    ?   30  0.00100      131957     1.8%    73.0%
    >   29  0.00126       59308     0.8%    73.9%
    =   28  0.00158       25174     0.4%    74.2%
    <   27  0.00200       44282     0.6%    74.8%
    ;   26  0.00251      204896     2.9%    77.7%
    :   25  0.00316       49600     0.7%    78.4%
    9   24  0.00398      251427     3.5%    81.9%
    6   21  0.00794          32     0.0%    81.9%
    5   20  0.01000       11170     0.2%    82.0%
    4   19  0.01259        9188     0.1%    82.2%
    3   18  0.01585       34528     0.5%    82.7%
    2   17  0.01995       25452     0.4%    83.0%
    1   16  0.02512      100936     1.4%    84.4%
    0   15  0.03162       96675     1.3%    85.8%
    /   14  0.03981      387620     5.4%    91.2%
    .   13  0.05012      162798     2.3%    93.5%
    -   12  0.06310      468864     6.5%   100.0%

Length vs. quality distribution

Positions in reads are sorted in increasing order.

1. L: position in reads (starting from position 2)
2. PctRecs: fraction of reads with at least this length
3. AvgQ: average quality at this position
4. P(AvgQ): error probability corresponding to AvgQ
5. AvgP: average error probability at this position
6. AvgEE: average expected error over all reads up to this position
7. Rate: growth rate of AvgEE between the current and previous position
8. RatePct: Rate (as explained above) expressed as a percentage

    L  PctRecs  AvgQ  P(AvgQ)      AvgP  AvgEE       Rate   RatePct
-----  -------  ----  -------  --------  -----  ---------  --------
    2   100.0%  28.2  0.00153  0.005655   0.01   0.005053    0.505%
    3   100.0%  30.1  0.00097  0.002893   0.01   0.004333    0.433%
    4   100.0%  27.7  0.00171  0.005935   0.02   0.004734    0.473%
    5   100.0%  28.8  0.00131  0.004416   0.02   0.004670    0.467%
    6   100.0%  29.5  0.00112  0.005633   0.03   0.004830    0.483%
[...]
  247    99.8%  24.4  0.00360  0.019798   2.14   0.008668    0.867%
  248    99.8%  24.0  0.00394  0.020644   2.16   0.008711    0.871%
  249    98.9%  23.9  0.00407  0.020943   2.17   0.008716    0.872%
  250    93.2%  23.8  0.00417  0.021239   2.22   0.008877    0.888%
  251    69.2%  22.3  0.00589  0.023948   2.27   0.009048    0.905%

Effect of expected error and length filtering

Positions in reads are sorted in decreasing order, starting with the first read length (L) with a cummulated expected error smaller or equal to 1.0. The next four columns indicate the number of reads that would be retained by the command vsearch-fastq_filter(1) if the reads were truncated at length L with the option --fastq_trunclen, and filtered to have a maximum expected error equal to or lesser than 1.0, 0.5, 0.25 or 0.1 with the option --fastq_maxee. The last four columns indicate the fraction of reads that would be retained by the command vsearch-fastq_filter(1) using the same length L and maximum expected error parameters.

1. L: read length
2. number of reads if truncating at L and filtering at EE =< 1.0
3. number of reads if truncating at L and filtering at EE =< 0.5
4. number of reads if truncating at L and filtering at EE =< 0.25
5. number of reads if truncating at L and filtering at EE =< 0.1
6. fraction of reads if truncating at L and filtering at EE =< 1.0
7. fraction of reads if truncating at L and filtering at EE =< 0.5
8. fraction of reads if truncating at L and filtering at EE =< 0.25
9. fraction of reads if truncating at L and filtering at EE =< 0.1

    L   1.0000   0.5000   0.2500   0.1000   1.0000   0.5000   0.2500   0.1000
-----  -------  -------  -------  -------  -------  -------  -------  -------
  251     6250     3022     1305      354   21.86%   10.57%    4.56%    1.24%
  250     8894     4467     2080      637   31.10%   15.62%    7.27%    2.23%
  249     9700     4930     2291      693   33.92%   17.24%    8.01%    2.42%
  248     9854     5046     2348      712   34.46%   17.65%    8.21%    2.49%
  247     9962     5127     2390      727   34.84%   17.93%    8.36%    2.54%
[...]
    5    28595    28595    28595    28157  100.00%  100.00%  100.00%   98.47%
    4    28595    28595    28595    28461  100.00%  100.00%  100.00%   99.53%
    3    28595    28595    28595    28595  100.00%  100.00%  100.00%  100.00%
    2    28595    28595    28595    28595  100.00%  100.00%  100.00%  100.00%
    1    28595    28595    28595    28595  100.00%  100.00%  100.00%  100.00%

Effect of minimum quality and length filtering

Positions in reads are sorted in decreasing order, starting with the largest read length (Len), and limited to the top n lengths, where n = floor(largest read length / 2) + 1. The next four columns indicate the fraction of reads that would be retained by the command vsearch-fastq_filter(1) if the reads were truncated at length Len with the option --fastq_trunclen, or at the first position with a quality Q equal to or lesser than 5, 10, 15 or 20 with the option --fastq_truncqual.

1. Len: read length
2. fraction of reads if truncating at Len or at first Q =< 5
3. fraction of reads if truncating at Len or at first Q =< 10
4. fraction of reads if truncating at Len or at first Q =< 15
5. fraction of reads if truncating at Len or at first Q =< 20

Truncate at first Q
  Len     Q=5    Q=10    Q=15    Q=20
-----  ------  ------  ------  ------
  251   69.2%   69.2%    0.8%    0.6%
  250   93.2%   93.2%    1.6%    1.1%
  249   98.9%   98.9%    1.7%    1.2%
  248   99.8%   99.8%    1.8%    1.2%
  247   99.8%   99.8%    1.8%    1.2%
[...]
  129   99.9%   99.9%   30.3%   10.3%
  128   99.9%   99.9%   30.7%   10.4%
  127   99.9%   99.9%   31.1%   10.5%
  126   99.9%   99.9%   31.4%   10.5%
  125   99.9%   99.9%   32.1%   10.7%

OPTIONS

mandatory options

• --log filename
  Write messages to filename. Messages include program version, start and finish times, elapsed time, amount of memory available, maximum amount of memory consumed, number of cores and command line options, and if need be, command-specific informational messages, warnings, and errors.

core options

• --fastq_ascii 33|64
  Specify the offset used as the basis for the fastq quality score when reading fastq files. For example, an offset of 33 means that a quality value of 41 is represented by the 74th ASCII symbol (33 + 41 = 74), which is ‘J’. See ascii(7) for a view of the ASCII character set. The offset value is either 33 or 64, default is 33.
• --fastq_qmax positive integer
  Specify the maximal quality score accepted when reading fastq sequences. Stop with an error message if a quality score higher than the specified value is read. Accepted values range from 0 to 93 if the offset is 33 (see --fastq_ascii), or range from 0 to 62 if the offset is 64. The default is 41, which is usual for recent Sanger/Illumina 1.8+ files.
• --fastq_qmin positive integer
  Specify the minimal quality score accepted when reading fastq sequences. Stop with an error message if a quality score lower than the specified value is read. Accepted values range from 0 to 93 if the offset is 33 (see --fastq_ascii), or range from 0 to 62 if the offset is 64. The default is 0, which is usual for recent Sanger/Illumina 1.8+ files. Older formats may use scores between -5 and 2.

secondary options

• --bzip2_decompress
  Specify that the input pipe is streaming data compressed using Huffman coding (see bzip2(1)). This option is not needed when reading from a regular file compressed with bzip2.
• --gzip_decompress
  Specify that the input pipe is streaming data compressed using Lempel-Ziv coding (see gzip(1)). This option is not needed when reading from a regular file compressed with gzip.
• --no_progress
  Suppress the gradually increasing progress indicator normally written to the standard error stderr(3).
• --quiet
  Suppress messages to the standard output stdout(3) and standard error stderr(3), except for warnings and error messages.

ignored options

• --threads positive non-null integer
  Command is not multithreaded, option has no effect.

EXAMPLES

Read from input.fastq.gz, do not write to the standard error (--quiet), and write results to output.log (--log):

vsearch \
    --fastq_stats input.fastq.gz \
    --quiet \
    --log output.log

SEE ALSO

vsearch-fastq_chars(1), vsearch-fastq(5)

CITATION

Rognes T, Flouri T, Nichols B, Quince C, Mahé F. (2016) VSEARCH: a versatile open source tool for metagenomics. PeerJ 4:e2584 doi: 10.7717/peerj.2584

REPORTING BUGS

Submit suggestions and bug-reports at https://github.com/torognes/vsearch/issues, send a pull request on https://github.com/torognes/vsearch, or compose a friendly or curmudgeont e-mail to Torbjørn Rognes (torognes@ifi.uio.no).

AVAILABILITY

Source code and binaries are available at https://github.com/torognes/vsearch.

COPYRIGHT

Copyright (C) 2014-2024, Torbjørn Rognes, Frédéric Mahé and Tomás Flouri

All rights reserved.

Contact: Torbjørn Rognes torognes@ifi.uio.no, Department of Informatics, University of Oslo, PO Box 1080 Blindern, NO-0316 Oslo, Norway

This software is dual-licensed and available under a choice of one of two licenses, either under the terms of the GNU General Public License version 3 or the BSD 2-Clause License.

GNU General Public License version 3

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

The BSD 2-Clause License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

ACKNOWLEDGMENTS

We would like to thank the authors of the following projects for making their source code available:

• vsearch includes code from Google’s CityHash project by Geoff Pike and Jyrki Alakuijala, providing some excellent hash functions available under a MIT license.
• vsearch includes code derived from Tatusov and Lipman’s DUST program that is in the public domain.
• vsearch includes public domain code written by Alexander Peslyak for the MD5 message digest algorithm.
• vsearch includes public domain code written by Steve Reid and others for the SHA1 message digest algorithm.
• vsearch binaries may include code from the zlib library, copyright Jean-Loup Gailly and Mark Adler.
• vsearch binaries may include code from the bzip2 library, copyright Julian R. Seward.

This site is open source. Improve this page.