0s autopkgtest [09:49:34]: starting date and time: 2025-11-01 09:49:34+0000 0s autopkgtest [09:49:34]: git checkout: 4b346b80 nova: make wait_reboot return success even when a no-op 0s autopkgtest [09:49:34]: host juju-7f2275-prod-proposed-migration-environment-20; command line: /home/ubuntu/autopkgtest/runner/autopkgtest --output-dir /tmp/autopkgtest-work.02ln6uhf/out --timeout-copy=6000 --setup-commands /home/ubuntu/autopkgtest-cloud/worker-config-production/setup-canonical.sh --apt-pocket=proposed=src:openjdk-26,src:jtreg8 --apt-upgrade bbmap --timeout-short=300 --timeout-copy=20000 --timeout-test=20000 --timeout-build=20000 '--env=ADT_TEST_TRIGGERS=openjdk-26/26~22ea-2 jtreg8/8.1+1+ds1-1' -- ssh -s /home/ubuntu/autopkgtest/ssh-setup/nova -- --flavor autopkgtest-big --security-groups autopkgtest-juju-7f2275-prod-proposed-migration-environment-20@bos03-arm64-1.secgroup --name adt-resolute-arm64-bbmap-20251101-094934-juju-7f2275-prod-proposed-migration-environment-20-92c60822-3e6c-4d33-a8ce-94f3718c16dd --image adt/ubuntu-resolute-arm64-server --keyname testbed-juju-7f2275-prod-proposed-migration-environment-20 --net-id=net_prod-proposed-migration -e TERM=linux --mirror=http://ftpmaster.internal/ubuntu/ 3s Creating nova instance adt-resolute-arm64-bbmap-20251101-094934-juju-7f2275-prod-proposed-migration-environment-20-92c60822-3e6c-4d33-a8ce-94f3718c16dd from image adt/ubuntu-resolute-arm64-server-20251101.img (UUID 895f8d05-65a0-4cf8-81a2-1a04d79b538c)... 60s autopkgtest [09:50:34]: testbed dpkg architecture: arm64 60s autopkgtest [09:50:34]: testbed apt version: 3.1.11 61s autopkgtest [09:50:35]: @@@@@@@@@@@@@@@@@@@@ test bed setup 61s autopkgtest [09:50:35]: testbed release detected to be: None 62s autopkgtest [09:50:36]: updating testbed package index (apt update) 62s Get:1 http://ftpmaster.internal/ubuntu resolute-proposed InRelease [87.8 kB] 62s Hit:2 http://ftpmaster.internal/ubuntu resolute InRelease 63s Hit:3 http://ftpmaster.internal/ubuntu resolute-updates InRelease 63s Hit:4 http://ftpmaster.internal/ubuntu resolute-security InRelease 63s Get:5 http://ftpmaster.internal/ubuntu resolute-proposed/main Sources [138 kB] 63s Get:6 http://ftpmaster.internal/ubuntu resolute-proposed/restricted Sources [9848 B] 63s Get:7 http://ftpmaster.internal/ubuntu resolute-proposed/universe Sources [2408 kB] 63s Get:8 http://ftpmaster.internal/ubuntu resolute-proposed/multiverse Sources [50.0 kB] 63s Get:9 http://ftpmaster.internal/ubuntu resolute-proposed/main arm64 Packages [207 kB] 63s Get:10 http://ftpmaster.internal/ubuntu resolute-proposed/main arm64 c-n-f Metadata [5072 B] 63s Get:11 http://ftpmaster.internal/ubuntu resolute-proposed/restricted arm64 Packages [107 kB] 63s Get:12 http://ftpmaster.internal/ubuntu resolute-proposed/restricted arm64 c-n-f Metadata [324 B] 63s Get:13 http://ftpmaster.internal/ubuntu resolute-proposed/universe arm64 Packages [1756 kB] 63s Get:14 http://ftpmaster.internal/ubuntu resolute-proposed/universe arm64 c-n-f Metadata [49.6 kB] 63s Get:15 http://ftpmaster.internal/ubuntu resolute-proposed/multiverse arm64 Packages [35.5 kB] 63s Get:16 http://ftpmaster.internal/ubuntu resolute-proposed/multiverse arm64 c-n-f Metadata [1700 B] 65s Fetched 4856 kB in 2s (3212 kB/s) 66s Reading package lists... 67s Hit:1 http://ftpmaster.internal/ubuntu resolute-proposed InRelease 67s Hit:2 http://ftpmaster.internal/ubuntu resolute InRelease 67s Hit:3 http://ftpmaster.internal/ubuntu resolute-updates InRelease 67s Hit:4 http://ftpmaster.internal/ubuntu resolute-security InRelease 69s Reading package lists... 69s Reading package lists... 69s Building dependency tree... 69s Reading state information... 69s Calculating upgrade... 70s The following packages will be upgraded: 70s gcc-15-base libatomic1 libgcc-s1 libstdc++6 publicsuffix 70s 5 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 70s Need to get 1027 kB of archives. 70s After this operation, 14.3 kB of additional disk space will be used. 70s Get:1 http://ftpmaster.internal/ubuntu resolute/main arm64 libatomic1 arm64 15.2.0-7ubuntu1 [11.2 kB] 70s Get:2 http://ftpmaster.internal/ubuntu resolute/main arm64 libstdc++6 arm64 15.2.0-7ubuntu1 [760 kB] 71s Get:3 http://ftpmaster.internal/ubuntu resolute/main arm64 gcc-15-base arm64 15.2.0-7ubuntu1 [58.4 kB] 71s Get:4 http://ftpmaster.internal/ubuntu resolute/main arm64 libgcc-s1 arm64 15.2.0-7ubuntu1 [61.1 kB] 71s Get:5 http://ftpmaster.internal/ubuntu resolute/main arm64 publicsuffix all 20251016.1743-0.1 [136 kB] 71s dpkg-preconfigure: unable to re-open stdin: No such file or directory 71s Fetched 1027 kB in 1s (1332 kB/s) 71s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 83460 files and directories currently installed.) 71s Preparing to unpack .../libatomic1_15.2.0-7ubuntu1_arm64.deb ... 71s Unpacking libatomic1:arm64 (15.2.0-7ubuntu1) over (15.2.0-5ubuntu1) ... 72s Preparing to unpack .../libstdc++6_15.2.0-7ubuntu1_arm64.deb ... 72s Unpacking libstdc++6:arm64 (15.2.0-7ubuntu1) over (15.2.0-5ubuntu1) ... 72s Preparing to unpack .../gcc-15-base_15.2.0-7ubuntu1_arm64.deb ... 72s Unpacking gcc-15-base:arm64 (15.2.0-7ubuntu1) over (15.2.0-5ubuntu1) ... 72s Setting up gcc-15-base:arm64 (15.2.0-7ubuntu1) ... 72s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 83460 files and directories currently installed.) 72s Preparing to unpack .../libgcc-s1_15.2.0-7ubuntu1_arm64.deb ... 72s Unpacking libgcc-s1:arm64 (15.2.0-7ubuntu1) over (15.2.0-5ubuntu1) ... 72s Setting up libgcc-s1:arm64 (15.2.0-7ubuntu1) ... 72s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 83460 files and directories currently installed.) 72s Preparing to unpack .../publicsuffix_20251016.1743-0.1_all.deb ... 72s Unpacking publicsuffix (20251016.1743-0.1) over (20250328.1952-0.1) ... 72s Setting up libatomic1:arm64 (15.2.0-7ubuntu1) ... 72s Setting up publicsuffix (20251016.1743-0.1) ... 72s Setting up libstdc++6:arm64 (15.2.0-7ubuntu1) ... 72s Processing triggers for libc-bin (2.42-0ubuntu3) ... 72s autopkgtest [09:50:46]: upgrading testbed (apt dist-upgrade and autopurge) 73s Reading package lists... 73s Building dependency tree... 73s Reading state information... 73s Calculating upgrade... 74s 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 74s Reading package lists... 74s Building dependency tree... 74s Reading state information... 75s Solving dependencies... 75s 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 78s autopkgtest [09:50:52]: testbed running kernel: Linux 6.17.0-5-generic #5-Ubuntu SMP PREEMPT_DYNAMIC Mon Sep 22 09:50:31 UTC 2025 78s autopkgtest [09:50:52]: @@@@@@@@@@@@@@@@@@@@ apt-source bbmap 82s Get:1 http://ftpmaster.internal/ubuntu resolute/universe bbmap 39.20+dfsg-2 (dsc) [2286 B] 82s Get:2 http://ftpmaster.internal/ubuntu resolute/universe bbmap 39.20+dfsg-2 (tar) [5748 kB] 82s Get:3 http://ftpmaster.internal/ubuntu resolute/universe bbmap 39.20+dfsg-2 (diff) [26.8 kB] 83s gpgv: Signature made Sun Apr 6 17:54:51 2025 UTC 83s gpgv: using RSA key 8F91B227C7D6F2B1948C8236793CF67E8F0D11DA 83s gpgv: issuer "emollier@debian.org" 83s gpgv: Can't check signature: No public key 83s dpkg-source: warning: cannot verify inline signature for ./bbmap_39.20+dfsg-2.dsc: no acceptable signature found 83s autopkgtest [09:50:57]: testing package bbmap version 39.20+dfsg-2 83s autopkgtest [09:50:57]: build not needed 85s autopkgtest [09:50:59]: test run-unit-test: preparing testbed 85s Reading package lists... 86s Building dependency tree... 86s Reading state information... 86s Solving dependencies... 87s The following NEW packages will be installed: 87s bbmap bbmap-jni ca-certificates-java default-jre-headless java-common 87s libapache-pom-java libcommons-cli-java libcommons-codec-java 87s libcommons-io-java libcommons-parent-java libjpeg-turbo8 libjpeg8 87s libjs-jquery libjs-sphinxdoc libjs-underscore liblcms2-2 liblog4j1.2-java 87s libmpj-java libnspr4 libnss3 libpcsclite1 openjdk-21-jre-headless 87s python-biopython-doc 87s 0 upgraded, 23 newly installed, 0 to remove and 0 not upgraded. 87s Need to get 78.9 MB of archives. 87s After this operation, 265 MB of additional disk space will be used. 87s Get:1 http://ftpmaster.internal/ubuntu resolute/universe arm64 libcommons-cli-java all 1.6.0-1 [59.9 kB] 87s Get:2 http://ftpmaster.internal/ubuntu resolute/universe arm64 libapache-pom-java all 33-2 [5874 B] 87s Get:3 http://ftpmaster.internal/ubuntu resolute/universe arm64 libcommons-parent-java all 56-1 [10.7 kB] 87s Get:4 http://ftpmaster.internal/ubuntu resolute/universe arm64 libcommons-codec-java all 1.18.0-1 [303 kB] 87s Get:5 http://ftpmaster.internal/ubuntu resolute/universe arm64 libcommons-io-java all 2.19.0-1 [491 kB] 88s Get:6 http://ftpmaster.internal/ubuntu resolute/universe arm64 liblog4j1.2-java all 1.2.17-11 [439 kB] 88s Get:7 http://ftpmaster.internal/ubuntu resolute/universe arm64 libmpj-java all 0.44+dfsg-4 [443 kB] 88s Get:8 http://ftpmaster.internal/ubuntu resolute/main arm64 ca-certificates-java all 20240118 [11.6 kB] 88s Get:9 http://ftpmaster.internal/ubuntu resolute/main arm64 java-common all 0.76 [6852 B] 88s Get:10 http://ftpmaster.internal/ubuntu resolute/main arm64 liblcms2-2 arm64 2.16-2 [170 kB] 88s Get:11 http://ftpmaster.internal/ubuntu resolute/main arm64 libjpeg-turbo8 arm64 2.1.5-4ubuntu2 [165 kB] 88s Get:12 http://ftpmaster.internal/ubuntu resolute/main arm64 libjpeg8 arm64 8c-2ubuntu11 [2148 B] 88s Get:13 http://ftpmaster.internal/ubuntu resolute/main arm64 libnspr4 arm64 2:4.36-1ubuntu2 [115 kB] 88s Get:14 http://ftpmaster.internal/ubuntu resolute/main arm64 libnss3 arm64 2:3.114-1 [1453 kB] 89s Get:15 http://ftpmaster.internal/ubuntu resolute/main arm64 libpcsclite1 arm64 2.3.3-1 [24.7 kB] 89s Get:16 http://ftpmaster.internal/ubuntu resolute/main arm64 openjdk-21-jre-headless arm64 21.0.9+10-1 [45.9 MB] 101s Get:17 http://ftpmaster.internal/ubuntu resolute/main arm64 default-jre-headless arm64 2:1.21-76 [3176 B] 101s Get:18 http://ftpmaster.internal/ubuntu resolute/universe arm64 bbmap all 39.20+dfsg-2 [10.6 MB] 103s Get:19 http://ftpmaster.internal/ubuntu resolute/universe arm64 bbmap-jni arm64 39.20+dfsg-2 [28.6 kB] 103s Get:20 http://ftpmaster.internal/ubuntu resolute/main arm64 libjs-jquery all 3.6.1+dfsg+~3.5.14-1 [328 kB] 103s Get:21 http://ftpmaster.internal/ubuntu resolute/main arm64 libjs-underscore all 1.13.4~dfsg+~1.11.4-3 [118 kB] 103s Get:22 http://ftpmaster.internal/ubuntu resolute/main arm64 libjs-sphinxdoc all 8.2.3-1ubuntu2 [28.0 kB] 103s Get:23 http://ftpmaster.internal/ubuntu resolute/universe arm64 python-biopython-doc all 1.85+dfsg-4 [18.2 MB] 105s Fetched 78.9 MB in 18s (4367 kB/s) 105s Selecting previously unselected package libcommons-cli-java. 105s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 83460 files and directories currently installed.) 105s Preparing to unpack .../00-libcommons-cli-java_1.6.0-1_all.deb ... 105s Unpacking libcommons-cli-java (1.6.0-1) ... 105s Selecting previously unselected package libapache-pom-java. 105s Preparing to unpack .../01-libapache-pom-java_33-2_all.deb ... 105s Unpacking libapache-pom-java (33-2) ... 105s Selecting previously unselected package libcommons-parent-java. 106s Preparing to unpack .../02-libcommons-parent-java_56-1_all.deb ... 106s Unpacking libcommons-parent-java (56-1) ... 106s Selecting previously unselected package libcommons-codec-java. 106s Preparing to unpack .../03-libcommons-codec-java_1.18.0-1_all.deb ... 106s Unpacking libcommons-codec-java (1.18.0-1) ... 106s Selecting previously unselected package libcommons-io-java. 106s Preparing to unpack .../04-libcommons-io-java_2.19.0-1_all.deb ... 106s Unpacking libcommons-io-java (2.19.0-1) ... 106s Selecting previously unselected package liblog4j1.2-java. 106s Preparing to unpack .../05-liblog4j1.2-java_1.2.17-11_all.deb ... 106s Unpacking liblog4j1.2-java (1.2.17-11) ... 106s Selecting previously unselected package libmpj-java. 106s Preparing to unpack .../06-libmpj-java_0.44+dfsg-4_all.deb ... 106s Unpacking libmpj-java (0.44+dfsg-4) ... 106s Selecting previously unselected package ca-certificates-java. 106s Preparing to unpack .../07-ca-certificates-java_20240118_all.deb ... 106s Unpacking ca-certificates-java (20240118) ... 106s Selecting previously unselected package java-common. 106s Preparing to unpack .../08-java-common_0.76_all.deb ... 106s Unpacking java-common (0.76) ... 106s Selecting previously unselected package liblcms2-2:arm64. 106s Preparing to unpack .../09-liblcms2-2_2.16-2_arm64.deb ... 106s Unpacking liblcms2-2:arm64 (2.16-2) ... 106s Selecting previously unselected package libjpeg-turbo8:arm64. 106s Preparing to unpack .../10-libjpeg-turbo8_2.1.5-4ubuntu2_arm64.deb ... 106s Unpacking libjpeg-turbo8:arm64 (2.1.5-4ubuntu2) ... 106s Selecting previously unselected package libjpeg8:arm64. 106s Preparing to unpack .../11-libjpeg8_8c-2ubuntu11_arm64.deb ... 106s Unpacking libjpeg8:arm64 (8c-2ubuntu11) ... 106s Selecting previously unselected package libnspr4:arm64. 106s Preparing to unpack .../12-libnspr4_2%3a4.36-1ubuntu2_arm64.deb ... 106s Unpacking libnspr4:arm64 (2:4.36-1ubuntu2) ... 106s Selecting previously unselected package libnss3:arm64. 106s Preparing to unpack .../13-libnss3_2%3a3.114-1_arm64.deb ... 106s Unpacking libnss3:arm64 (2:3.114-1) ... 106s Selecting previously unselected package libpcsclite1:arm64. 106s Preparing to unpack .../14-libpcsclite1_2.3.3-1_arm64.deb ... 106s Unpacking libpcsclite1:arm64 (2.3.3-1) ... 106s Selecting previously unselected package openjdk-21-jre-headless:arm64. 106s Preparing to unpack .../15-openjdk-21-jre-headless_21.0.9+10-1_arm64.deb ... 106s Unpacking openjdk-21-jre-headless:arm64 (21.0.9+10-1) ... 108s Selecting previously unselected package default-jre-headless. 108s Preparing to unpack .../16-default-jre-headless_2%3a1.21-76_arm64.deb ... 108s Unpacking default-jre-headless (2:1.21-76) ... 108s Selecting previously unselected package bbmap. 108s Preparing to unpack .../17-bbmap_39.20+dfsg-2_all.deb ... 108s Unpacking bbmap (39.20+dfsg-2) ... 108s Selecting previously unselected package bbmap-jni. 108s Preparing to unpack .../18-bbmap-jni_39.20+dfsg-2_arm64.deb ... 108s Unpacking bbmap-jni (39.20+dfsg-2) ... 108s Selecting previously unselected package libjs-jquery. 108s Preparing to unpack .../19-libjs-jquery_3.6.1+dfsg+~3.5.14-1_all.deb ... 108s Unpacking libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... 108s Selecting previously unselected package libjs-underscore. 108s Preparing to unpack .../20-libjs-underscore_1.13.4~dfsg+~1.11.4-3_all.deb ... 108s Unpacking libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... 108s Selecting previously unselected package libjs-sphinxdoc. 108s Preparing to unpack .../21-libjs-sphinxdoc_8.2.3-1ubuntu2_all.deb ... 108s Unpacking libjs-sphinxdoc (8.2.3-1ubuntu2) ... 108s Selecting previously unselected package python-biopython-doc. 108s Preparing to unpack .../22-python-biopython-doc_1.85+dfsg-4_all.deb ... 108s Unpacking python-biopython-doc (1.85+dfsg-4) ... 109s Setting up liblcms2-2:arm64 (2.16-2) ... 109s Setting up java-common (0.76) ... 109s Setting up libcommons-cli-java (1.6.0-1) ... 109s Setting up liblog4j1.2-java (1.2.17-11) ... 109s Setting up libnspr4:arm64 (2:4.36-1ubuntu2) ... 109s Setting up libapache-pom-java (33-2) ... 109s Setting up libpcsclite1:arm64 (2.3.3-1) ... 109s Setting up libjpeg-turbo8:arm64 (2.1.5-4ubuntu2) ... 109s Setting up bbmap-jni (39.20+dfsg-2) ... 109s Setting up libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... 109s Setting up ca-certificates-java (20240118) ... 109s No JRE found. Skipping Java certificates setup. 109s Setting up libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... 109s Setting up libjpeg8:arm64 (8c-2ubuntu11) ... 109s Setting up libcommons-parent-java (56-1) ... 109s Setting up libnss3:arm64 (2:3.114-1) ... 109s Setting up libjs-sphinxdoc (8.2.3-1ubuntu2) ... 109s Setting up python-biopython-doc (1.85+dfsg-4) ... 109s Setting up libcommons-codec-java (1.18.0-1) ... 109s Setting up openjdk-21-jre-headless:arm64 (21.0.9+10-1) ... 109s update-alternatives: using /usr/lib/jvm/java-21-openjdk-arm64/bin/java to provide /usr/bin/java (java) in auto mode 109s update-alternatives: using /usr/lib/jvm/java-21-openjdk-arm64/bin/jpackage to provide /usr/bin/jpackage (jpackage) in auto mode 109s update-alternatives: using /usr/lib/jvm/java-21-openjdk-arm64/bin/keytool to provide /usr/bin/keytool (keytool) in auto mode 109s update-alternatives: using /usr/lib/jvm/java-21-openjdk-arm64/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregistry) in auto mode 109s update-alternatives: using /usr/lib/jvm/java-21-openjdk-arm64/lib/jexec to provide /usr/bin/jexec (jexec) in auto mode 109s Setting up libcommons-io-java (2.19.0-1) ... 109s Setting up libmpj-java (0.44+dfsg-4) ... 109s Processing triggers for libc-bin (2.42-0ubuntu3) ... 109s Processing triggers for man-db (2.13.1-1) ... 110s Processing triggers for ca-certificates-java (20240118) ... 111s Adding debian:ACCVRAIZ1.pem 111s Adding debian:AC_RAIZ_FNMT-RCM.pem 111s Adding debian:AC_RAIZ_FNMT-RCM_SERVIDORES_SEGUROS.pem 111s Adding debian:ANF_Secure_Server_Root_CA.pem 111s Adding debian:Actalis_Authentication_Root_CA.pem 111s Adding debian:AffirmTrust_Commercial.pem 111s Adding debian:AffirmTrust_Networking.pem 111s Adding debian:AffirmTrust_Premium.pem 111s Adding debian:AffirmTrust_Premium_ECC.pem 111s Adding debian:Amazon_Root_CA_1.pem 111s Adding debian:Amazon_Root_CA_2.pem 111s Adding debian:Amazon_Root_CA_3.pem 111s Adding debian:Amazon_Root_CA_4.pem 111s Adding debian:Atos_TrustedRoot_2011.pem 111s Adding debian:Atos_TrustedRoot_Root_CA_ECC_TLS_2021.pem 111s Adding debian:Atos_TrustedRoot_Root_CA_RSA_TLS_2021.pem 111s Adding debian:Autoridad_de_Certificacion_Firmaprofesional_CIF_A62634068.pem 111s Adding debian:BJCA_Global_Root_CA1.pem 111s Adding debian:BJCA_Global_Root_CA2.pem 111s Adding debian:Baltimore_CyberTrust_Root.pem 111s Adding debian:Buypass_Class_2_Root_CA.pem 111s Adding debian:Buypass_Class_3_Root_CA.pem 111s Adding debian:CA_Disig_Root_R2.pem 111s Adding debian:CFCA_EV_ROOT.pem 111s Adding debian:COMODO_Certification_Authority.pem 111s Adding debian:COMODO_ECC_Certification_Authority.pem 111s Adding debian:COMODO_RSA_Certification_Authority.pem 111s Adding debian:Certainly_Root_E1.pem 111s Adding debian:Certainly_Root_R1.pem 111s Adding debian:Certigna.pem 111s Adding debian:Certigna_Root_CA.pem 111s Adding debian:Certum_EC-384_CA.pem 111s Adding debian:Certum_Trusted_Network_CA.pem 111s Adding debian:Certum_Trusted_Network_CA_2.pem 111s Adding debian:Certum_Trusted_Root_CA.pem 111s Adding debian:CommScope_Public_Trust_ECC_Root-01.pem 111s Adding debian:CommScope_Public_Trust_ECC_Root-02.pem 111s Adding debian:CommScope_Public_Trust_RSA_Root-01.pem 111s Adding debian:CommScope_Public_Trust_RSA_Root-02.pem 111s Adding debian:Comodo_AAA_Services_root.pem 111s Adding debian:D-TRUST_BR_Root_CA_1_2020.pem 111s Adding debian:D-TRUST_BR_Root_CA_2_2023.pem 111s Adding debian:D-TRUST_EV_Root_CA_1_2020.pem 111s Adding debian:D-TRUST_EV_Root_CA_2_2023.pem 111s Adding debian:D-TRUST_Root_Class_3_CA_2_2009.pem 111s Adding debian:D-TRUST_Root_Class_3_CA_2_EV_2009.pem 111s Adding debian:DigiCert_Assured_ID_Root_CA.pem 111s Adding debian:DigiCert_Assured_ID_Root_G2.pem 111s Adding debian:DigiCert_Assured_ID_Root_G3.pem 111s Adding debian:DigiCert_Global_Root_CA.pem 111s Adding debian:DigiCert_Global_Root_G2.pem 111s Adding debian:DigiCert_Global_Root_G3.pem 111s Adding debian:DigiCert_High_Assurance_EV_Root_CA.pem 111s Adding debian:DigiCert_TLS_ECC_P384_Root_G5.pem 111s Adding debian:DigiCert_TLS_RSA4096_Root_G5.pem 111s Adding debian:DigiCert_Trusted_Root_G4.pem 111s Adding debian:Entrust.net_Premium_2048_Secure_Server_CA.pem 111s Adding debian:Entrust_Root_Certification_Authority.pem 111s Adding debian:Entrust_Root_Certification_Authority_-_EC1.pem 111s Adding debian:Entrust_Root_Certification_Authority_-_G2.pem 111s Adding debian:FIRMAPROFESIONAL_CA_ROOT-A_WEB.pem 111s Adding debian:GDCA_TrustAUTH_R5_ROOT.pem 111s Adding debian:GLOBALTRUST_2020.pem 111s Adding debian:GTS_Root_R1.pem 111s Adding debian:GTS_Root_R2.pem 111s Adding debian:GTS_Root_R3.pem 111s Adding debian:GTS_Root_R4.pem 111s Adding debian:GlobalSign_ECC_Root_CA_-_R4.pem 111s Adding debian:GlobalSign_ECC_Root_CA_-_R5.pem 111s Adding debian:GlobalSign_Root_CA.pem 111s Adding debian:GlobalSign_Root_CA_-_R3.pem 111s Adding debian:GlobalSign_Root_CA_-_R6.pem 111s Adding debian:GlobalSign_Root_E46.pem 111s Adding debian:GlobalSign_Root_R46.pem 111s Adding debian:Go_Daddy_Class_2_CA.pem 111s Adding debian:Go_Daddy_Root_Certificate_Authority_-_G2.pem 111s Adding debian:HARICA_TLS_ECC_Root_CA_2021.pem 111s Adding debian:HARICA_TLS_RSA_Root_CA_2021.pem 111s Adding debian:Hellenic_Academic_and_Research_Institutions_ECC_RootCA_2015.pem 111s Adding debian:Hellenic_Academic_and_Research_Institutions_RootCA_2015.pem 111s Adding debian:HiPKI_Root_CA_-_G1.pem 111s Adding debian:Hongkong_Post_Root_CA_3.pem 111s Adding debian:ISRG_Root_X1.pem 111s Adding debian:ISRG_Root_X2.pem 111s Adding debian:IdenTrust_Commercial_Root_CA_1.pem 111s Adding debian:IdenTrust_Public_Sector_Root_CA_1.pem 111s Adding debian:Izenpe.com.pem 111s Adding debian:Microsec_e-Szigno_Root_CA_2009.pem 111s Adding debian:Microsoft_ECC_Root_Certificate_Authority_2017.pem 111s Adding debian:Microsoft_RSA_Root_Certificate_Authority_2017.pem 111s Adding debian:NAVER_Global_Root_Certification_Authority.pem 111s Adding debian:NetLock_Arany_=Class_Gold=_Főtanúsítvány.pem 111s Adding debian:OISTE_WISeKey_Global_Root_GB_CA.pem 111s Adding debian:OISTE_WISeKey_Global_Root_GC_CA.pem 111s Adding debian:QuoVadis_Root_CA_1_G3.pem 111s Adding debian:QuoVadis_Root_CA_2.pem 111s Adding debian:QuoVadis_Root_CA_2_G3.pem 111s Adding debian:QuoVadis_Root_CA_3.pem 111s Adding debian:QuoVadis_Root_CA_3_G3.pem 111s Adding debian:SSL.com_EV_Root_Certification_Authority_ECC.pem 111s Adding debian:SSL.com_EV_Root_Certification_Authority_RSA_R2.pem 111s Adding debian:SSL.com_Root_Certification_Authority_ECC.pem 111s Adding debian:SSL.com_Root_Certification_Authority_RSA.pem 111s Adding debian:SSL.com_TLS_ECC_Root_CA_2022.pem 111s Adding debian:SSL.com_TLS_RSA_Root_CA_2022.pem 111s Adding debian:SZAFIR_ROOT_CA2.pem 111s Adding debian:Sectigo_Public_Server_Authentication_Root_E46.pem 111s Adding debian:Sectigo_Public_Server_Authentication_Root_R46.pem 111s Adding debian:SecureSign_Root_CA12.pem 111s Adding debian:SecureSign_Root_CA14.pem 111s Adding debian:SecureSign_Root_CA15.pem 111s Adding debian:SecureTrust_CA.pem 111s Adding debian:Secure_Global_CA.pem 111s Adding debian:Security_Communication_ECC_RootCA1.pem 111s Adding debian:Security_Communication_RootCA2.pem 111s Adding debian:Starfield_Class_2_CA.pem 111s Adding debian:Starfield_Root_Certificate_Authority_-_G2.pem 111s Adding debian:Starfield_Services_Root_Certificate_Authority_-_G2.pem 111s Adding debian:SwissSign_Gold_CA_-_G2.pem 111s Adding debian:T-TeleSec_GlobalRoot_Class_2.pem 111s Adding debian:T-TeleSec_GlobalRoot_Class_3.pem 111s Adding debian:TUBITAK_Kamu_SM_SSL_Kok_Sertifikasi_-_Surum_1.pem 111s Adding debian:TWCA_CYBER_Root_CA.pem 111s Adding debian:TWCA_Global_Root_CA.pem 111s Adding debian:TWCA_Root_Certification_Authority.pem 111s Adding debian:Telekom_Security_TLS_ECC_Root_2020.pem 111s Adding debian:Telekom_Security_TLS_RSA_Root_2023.pem 111s Adding debian:TeliaSonera_Root_CA_v1.pem 111s Adding debian:Telia_Root_CA_v2.pem 111s Adding debian:TrustAsia_Global_Root_CA_G3.pem 111s Adding debian:TrustAsia_Global_Root_CA_G4.pem 111s Adding debian:Trustwave_Global_Certification_Authority.pem 111s Adding debian:Trustwave_Global_ECC_P256_Certification_Authority.pem 111s Adding debian:Trustwave_Global_ECC_P384_Certification_Authority.pem 111s Adding debian:TunTrust_Root_CA.pem 111s Adding debian:UCA_Extended_Validation_Root.pem 111s Adding debian:UCA_Global_G2_Root.pem 111s Adding debian:USERTrust_ECC_Certification_Authority.pem 111s Adding debian:USERTrust_RSA_Certification_Authority.pem 111s Adding debian:XRamp_Global_CA_Root.pem 111s Adding debian:certSIGN_ROOT_CA.pem 111s Adding debian:certSIGN_Root_CA_G2.pem 111s Adding debian:e-Szigno_Root_CA_2017.pem 111s Adding debian:ePKI_Root_Certification_Authority.pem 111s Adding debian:emSign_ECC_Root_CA_-_C3.pem 111s Adding debian:emSign_ECC_Root_CA_-_G3.pem 111s Adding debian:emSign_Root_CA_-_C1.pem 111s Adding debian:emSign_Root_CA_-_G1.pem 111s Adding debian:vTrus_ECC_Root_CA.pem 111s Adding debian:vTrus_Root_CA.pem 111s done. 111s Setting up bbmap (39.20+dfsg-2) ... 111s Setting up default-jre-headless (2:1.21-76) ... 113s autopkgtest [09:51:27]: test run-unit-test: [----------------------- 113s 113s # bbmap 113s bbmap.sh --version 113s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 113s java -ea -Xmx2832m -Xms2832m -cp /usr/share/java/bbmap.jar align2.BBMap build=1 overwrite=true fastareadlen=500 --version 113s BBTools version 39.20 113s For help, please run the shellscript with no parameters, or look in /docs/. 113s bbmap.sh --help 113s 113s BBMap 113s Written by Brian Bushnell, from Dec. 2010 - present 113s Last modified September 15, 2022 113s 113s Description: Fast and accurate splice-aware read aligner. 113s Please read bbmap/docs/guides/BBMapGuide.txt for more information. 113s 113s To index: bbmap.sh ref= 113s To map: bbmap.sh in= out= 113s To map without writing an index: 113s bbmap.sh ref= in= out= nodisk 113s 113s in=stdin will accept reads from standard in, and out=stdout will write to 113s standard out, but file extensions are still needed to specify the format of the 113s input and output files e.g. in=stdin.fa.gz will read gzipped fasta from 113s standard in; out=stdout.sam.gz will write gzipped sam. 113s 113s Indexing Parameters (required when building the index): 113s nodisk=f Set to true to build index in memory and write nothing 113s to disk except output. 113s ref= Specify the reference sequence. Only do this ONCE, 113s when building the index (unless using 'nodisk'). 113s build=1 If multiple references are indexed in the same directory, 113s each needs a unique numeric ID (unless using 'nodisk'). 113s k=13 Kmer length, range 8-15. Longer is faster but uses 113s more memory. Shorter is more sensitive. 113s If indexing and mapping are done in two steps, K should 113s be specified each time. 113s path=<.> Specify the location to write the index, if you don't 113s want it in the current working directory. 113s usemodulo=f Throw away ~80% of kmers based on remainder modulo a 113s number (reduces RAM by 50% and sensitivity slightly). 113s Should be enabled both when building the index AND 113s when mapping. 113s rebuild=f Force a rebuild of the index (ref= should be set). 113s 113s Input Parameters: 113s build=1 Designate index to use. Corresponds to the number 113s specified when building the index. 113s in= Primary reads input; required parameter. 113s in2= For paired reads in two files. 113s interleaved=auto True forces paired/interleaved input; false forces 113s single-ended mapping. If not specified, interleaved 113s status will be autodetected from read names. 113s fastareadlen=500 Break up FASTA reads longer than this. Max is 500 for 113s BBMap and 6000 for BBMapPacBio. Only works for FASTA 113s input (use 'maxlen' for FASTQ input). The default for 113s bbmap.sh is 500, and for mapPacBio.sh is 6000. 113s unpigz=f Spawn a pigz (parallel gzip) process for faster 113s decompression than using Java. 113s Requires pigz to be installed. 113s touppercase=t (tuc) Convert lowercase letters in reads to upper case 113s (otherwise they will not match the reference). 113s 113s Sampling Parameters: 113s 113s reads=-1 Set to a positive number N to only process the first N 113s reads (or pairs), then quit. -1 means use all reads. 113s samplerate=1 Set to a number from 0 to 1 to randomly select that 113s fraction of reads for mapping. 1 uses all reads. 113s skipreads=0 Set to a number N to skip the first N reads (or pairs), 113s then map the rest. 113s 113s Mapping Parameters: 113s fast=f This flag is a macro which sets other paramters to run 113s faster, at reduced sensitivity. Bad for RNA-seq. 113s slow=f This flag is a macro which sets other paramters to run 113s slower, at greater sensitivity. 'vslow' is even slower. 113s maxindel=16000 Don't look for indels longer than this. Lower is faster. 113s Set to >=100k for RNAseq with long introns like mammals. 113s strictmaxindel=f When enabled, do not allow indels longer than 'maxindel'. 113s By default these are not sought, but may be found anyway. 113s tipsearch=100 Look this far for read-end deletions with anchors 113s shorter than K, using brute force. 113s minid=0.76 Approximate minimum alignment identity to look for. 113s Higher is faster and less sensitive. 113s minhits=1 Minimum number of seed hits required for candidate sites. 113s Higher is faster. 113s local=f Set to true to use local, rather than global, alignments. 113s This will soft-clip ugly ends of poor alignments. 113s perfectmode=f Allow only perfect mappings when set to true (very fast). 113s semiperfectmode=f Allow only perfect and semiperfect (perfect except for 113s N's in the reference) mappings. 113s threads=auto (t) Set to number of threads desired. By default, uses 113s all cores available. 113s ambiguous=best (ambig) Set behavior on ambiguously-mapped reads (with 113s multiple top-scoring mapping locations). 113s best (use the first best site) 113s toss (consider unmapped) 113s random (select one top-scoring site randomly) 113s all (retain all top-scoring sites) 113s samestrandpairs=f (ssp) Specify whether paired reads should map to the 113s same strand or opposite strands. 113s requirecorrectstrand=t (rcs) Forbid pairing of reads without correct strand 113s orientation. Set to false for long-mate-pair libraries. 113s killbadpairs=f (kbp) If a read pair is mapped with an inappropriate 113s insert size or orientation, the read with the lower 113s mapping quality is marked unmapped. 113s pairedonly=f (po) Treat unpaired reads as unmapped. Thus they will 113s be sent to 'outu' but not 'outm'. 113s rcomp=f Reverse complement both reads prior to mapping (for LMP 113s outward-facing libraries). 113s rcompmate=f Reverse complement read2 prior to mapping. 113s pairlen=32000 Set max allowed distance between paired reads. 113s (insert size)=(pairlen)+(read1 length)+(read2 length) 113s rescuedist=1200 Don't try to rescue paired reads if avg. insert size 113s greater than this. Lower is faster. 113s rescuemismatches=32 Maximum mismatches allowed in a rescued read. Lower 113s is faster. 113s averagepairdist=100 (apd) Initial average distance between paired reads. 113s Varies dynamically; does not need to be specified. 113s deterministic=f Run in deterministic mode. In this case it is good 113s to set averagepairdist. BBMap is deterministic 113s without this flag if using single-ended reads, 113s or run singlethreaded. 113s bandwidthratio=0 (bwr) If above zero, restrict alignment band to this 113s fraction of read length. Faster but less accurate. 113s bandwidth=0 (bw) Set the bandwidth directly. 113s fraction of read length. Faster but less accurate. 113s usejni=f (jni) Do alignments faster, in C code. Requires 113s compiling the C code; details are in /jni/README.txt. 113s maxsites2=800 Don't analyze (or print) more than this many alignments 113s per read. 113s ignorefrequentkmers=t (ifk) Discard low-information kmers that occur often. 113s excludefraction=0.03 (ef) Fraction of kmers to ignore. For example, 0.03 113s will ignore the most common 3% of kmers. 113s greedy=t Use a greedy algorithm to discard the least-useful 113s kmers on a per-read basis. 113s kfilter=0 If positive, potential mapping sites must have at 113s least this many consecutive exact matches. 113s 113s 113s Quality and Trimming Parameters: 113s qin=auto Set to 33 or 64 to specify input quality value ASCII 113s offset. 33 is Sanger, 64 is old Solexa. 113s qout=auto Set to 33 or 64 to specify output quality value ASCII 113s offset (only if output format is fastq). 113s qtrim=f Quality-trim ends before mapping. Options are: 113s 'f' (false), 'l' (left), 'r' (right), and 'lr' (both). 113s untrim=f Undo trimming after mapping. Untrimmed bases will be 113s soft-clipped in cigar strings. 113s trimq=6 Trim regions with average quality below this 113s (phred algorithm). 113s mintrimlength=60 (mintl) Don't trim reads to be shorter than this. 113s fakefastaquality=-1 (ffq) Set to a positive number 1-50 to generate fake 113s quality strings for fasta input reads. 113s ignorebadquality=f (ibq) Keep going, rather than crashing, if a read has 113s out-of-range quality values. 113s usequality=t Use quality scores when determining which read kmers 113s to use as seeds. 113s minaveragequality=0 (maq) Do not map reads with average quality below this. 113s maqb=0 If positive, calculate maq from this many initial bases. 113s 113s Output Parameters: 113s out= Write all reads to this file. 113s outu= Write only unmapped reads to this file. Does not 113s include unmapped paired reads with a mapped mate. 113s outm= Write only mapped reads to this file. Includes 113s unmapped paired reads with a mapped mate. 113s mappedonly=f If true, treats 'out' like 'outm'. 113s bamscript= (bs) Write a shell script to that will turn 113s the sam output into a sorted, indexed bam file. 113s ordered=f Set to true to output reads in same order as input. 113s Slower and uses more memory. 113s overwrite=f (ow) Allow process to overwrite existing files. 113s secondary=f Print secondary alignments. 113s sssr=0.95 (secondarysitescoreratio) Print only secondary alignments 113s with score of at least this fraction of primary. 113s ssao=f (secondarysiteasambiguousonly) Only print secondary 113s alignments for ambiguously-mapped reads. 113s maxsites=5 Maximum number of total alignments to print per read. 113s Only relevant when secondary=t. 113s quickmatch=f Generate cigar strings more quickly. 113s trimreaddescriptions=f (trd) Truncate read and ref names at the first whitespace, 113s assuming that the remainder is a comment or description. 113s ziplevel=2 (zl) Compression level for zip or gzip output. 113s pigz=f Spawn a pigz (parallel gzip) process for faster 113s compression than Java. Requires pigz to be installed. 113s machineout=f Set to true to output statistics in machine-friendly 113s 'key=value' format. 113s printunmappedcount=f Print the total number of unmapped reads and bases. 113s If input is paired, the number will be of pairs 113s for which both reads are unmapped. 113s showprogress=0 If positive, print a '.' every X reads. 113s showprogress2=0 If positive, print the number of seconds since the 113s last progress update (instead of a '.'). 113s renamebyinsert=f Renames reads based on their mapped insert size. 113s 113s Bloom-Filtering Parameters (bloomfilter.sh is the standalone version). 113s bloom=f Use a Bloom filter to ignore reads not sharing kmers 113s with the reference. This uses more memory, but speeds 113s mapping when most reads don't match the reference. 113s bloomhashes=2 Number of hash functions. 113s bloomminhits=3 Number of consecutive hits to be considered matched. 113s bloomk=31 Bloom filter kmer length. 113s bloomserial=t Use the serialized Bloom filter for greater loading 113s speed, if available. If not, generate and write one. 113s 113s Post-Filtering Parameters: 113s idfilter=0 Independant of minid; sets exact minimum identity 113s allowed for alignments to be printed. Range 0 to 1. 113s subfilter=-1 Ban alignments with more than this many substitutions. 113s insfilter=-1 Ban alignments with more than this many insertions. 113s delfilter=-1 Ban alignments with more than this many deletions. 113s indelfilter=-1 Ban alignments with more than this many indels. 113s editfilter=-1 Ban alignments with more than this many edits. 113s inslenfilter=-1 Ban alignments with an insertion longer than this. 113s dellenfilter=-1 Ban alignments with a deletion longer than this. 113s nfilter=-1 Ban alignments with more than this many ns. This 113s includes nocall, noref, and off scaffold ends. 113s 113s Sam flags and settings: 113s noheader=f Disable generation of header lines. 113s sam=1.4 Set to 1.4 to write Sam version 1.4 cigar strings, 113s with = and X, or 1.3 to use M. 113s saa=t (secondaryalignmentasterisks) Use asterisks instead of 113s bases for sam secondary alignments. 113s cigar=t Set to 'f' to skip generation of cigar strings (faster). 113s keepnames=f Keep original names of paired reads, rather than 113s ensuring both reads have the same name. 113s intronlen=999999999 Set to a lower number like 10 to change 'D' to 'N' in 113s cigar strings for deletions of at least that length. 113s rgid= Set readgroup ID. All other readgroup fields 113s can be set similarly, with the flag rgXX= 113s If you set a readgroup flag to the word 'filename', 113s e.g. rgid=filename, the input file name will be used. 113s mdtag=f Write MD tags. 113s nhtag=f Write NH tags. 113s xmtag=f Write XM tags (may only work correctly with ambig=all). 113s amtag=f Write AM tags. 113s nmtag=f Write NM tags. 113s xstag=f Set to 'xs=fs', 'xs=ss', or 'xs=us' to write XS tags 113s for RNAseq using firststrand, secondstrand, or 113s unstranded libraries. Needed by Cufflinks. 113s JGI mainly uses 'firststrand'. 113s stoptag=f Write a tag indicating read stop location, prefixed by YS:i: 113s lengthtag=f Write a tag indicating (query,ref) alignment lengths, 113s prefixed by YL:Z: 113s idtag=f Write a tag indicating percent identity, prefixed by YI:f: 113s inserttag=f Write a tag indicating insert size, prefixed by X8:Z: 113s scoretag=f Write a tag indicating BBMap's raw score, prefixed by YR:i: 113s timetag=f Write a tag indicating this read's mapping time, prefixed by X0:i: 113s boundstag=f Write a tag indicating whether either read in the pair 113s goes off the end of the reference, prefixed by XB:Z: 113s notags=f Turn off all optional tags. 113s 113s Histogram and statistics output parameters: 113s scafstats= Statistics on how many reads mapped to which scaffold. 113s refstats= Statistics on how many reads mapped to which reference 113s file; only for BBSplit. 113s sortscafs=t Sort scaffolds or references by read count. 113s bhist= Base composition histogram by position. 113s qhist= Quality histogram by position. 113s aqhist= Histogram of average read quality. 113s bqhist= Quality histogram designed for box plots. 113s lhist= Read length histogram. 113s ihist= Write histogram of insert sizes (for paired reads). 113s ehist= Errors-per-read histogram. 113s qahist= Quality accuracy histogram of error rates versus 113s quality score. 113s indelhist= Indel length histogram. 113s mhist= Histogram of match, sub, del, and ins rates by 113s read location. 113s gchist= Read GC content histogram. 113s gcbins=100 Number gchist bins. Set to 'auto' to use read length. 113s gcpairs=t Use average GC of paired reads. 113s idhist= Histogram of read count versus percent identity. 113s idbins=100 Number idhist bins. Set to 'auto' to use read length. 113s statsfile=stderr Mapping statistics are printed here. 113s 113s Coverage output parameters (these may reduce speed and use more RAM): 113s covstats= Per-scaffold coverage info. 113s rpkm= Per-scaffold RPKM/FPKM counts. 113s covhist= Histogram of # occurrences of each depth level. 113s basecov= Coverage per base location. 113s bincov= Print binned coverage per location (one line per X bases). 113s covbinsize=1000 Set the binsize for binned coverage output. 113s nzo=t Only print scaffolds with nonzero coverage. 113s twocolumn=f Change to true to print only ID and Avg_fold instead of 113s all 6 columns to the 'out=' file. 113s 32bit=f Set to true if you need per-base coverage over 64k. 113s strandedcov=f Track coverage for plus and minus strand independently. 113s startcov=f Only track start positions of reads. 113s secondarycov=t Include coverage of secondary alignments. 113s physcov=f Calculate physical coverage for paired reads. 113s This includes the unsequenced bases. 113s delcoverage=t (delcov) Count bases covered by deletions as covered. 113s True is faster than false. 113s covk=0 If positive, calculate kmer coverage statistics. 113s 113s Java Parameters: 113s -Xmx This will set Java's memory usage, 113s overriding autodetection. 113s -Xmx20g will specify 20 gigs of RAM, and -Xmx800m 113s will specify 800 megs. The max is typically 85% of 113s physical memory. The human genome requires around 24g, 113s or 12g with the 'usemodulo' flag. The index uses 113s roughly 6 bytes per reference base. 113s -eoom This flag will cause the process to exit if an 113s out-of-memory exception occurs. Requires Java 8u92+. 113s -da Disable assertions. 113s 113s Please contact Brian Bushnell at bbushnell@lbl.gov if you encounter 113s any problems, or post at: http://seqanswers.com/forums/showthread.php?t=41057 113s 113s bbmap.sh ref=example.fasta 114s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 114s java -ea -Xmx2832m -Xms2832m -cp /usr/share/java/bbmap.jar align2.BBMap build=1 overwrite=true fastareadlen=500 ref=example.fasta 114s Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, ref=example.fasta] 114s Version 39.20 114s 114s No output file. 114s Writing reference. 114s Executing dna.FastaToChromArrays2 [example.fasta, 1, writeinthread=false, genscaffoldinfo=true, retain, waitforwriting=false, gz=true, maxlen=536670912, writechroms=true, minscaf=1, midpad=300, startpad=8000, stoppad=8000, nodisk=false] 114s 114s Set genScaffoldInfo=true 114s Writing chunk 1 114s Set genome to 1 114s 114s Loaded Reference: 0.012 seconds. 114s Loading index for chunk 1-1, build 1 114s No index available; generating from reference genome: /tmp/autopkgtest.LJIMfH/autopkgtest_tmp/ref/index/1/chr1_index_k13_c16_b1.block 114s Indexing threads started for block 0-1 114s Indexing threads finished for block 0-1 115s Generated Index: 0.872 seconds. 115s No reads to process; quitting. 115s 115s Total time: 1.193 seconds. 115s test -d ref 115s bbmap.sh in=misc_dna_as_sanger.fastq out=out.sam 115s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 115s java -ea -Xmx2832m -Xms2832m -cp /usr/share/java/bbmap.jar align2.BBMap build=1 overwrite=true fastareadlen=500 in=misc_dna_as_sanger.fastq out=out.sam 115s Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, in=misc_dna_as_sanger.fastq, out=out.sam] 115s Version 39.20 115s 115s Retaining first best site only for ambiguous mappings. 115s Set genome to 1 115s 115s Loaded Reference: 0.044 seconds. 115s Loading index for chunk 1-1, build 1 116s Generated Index: 0.338 seconds. 119s Analyzed Index: 3.283 seconds. 119s Started output stream: 0.044 seconds. 119s Cleared Memory: 0.125 seconds. 119s Processing reads in single-ended mode. 119s Started read stream. 119s Started 4 mapping threads. 119s Detecting finished threads: 0, 1, 2, 3 119s 119s ------------------ Results ------------------ 119s 119s Genome: 1 119s Key Length: 13 119s Max Indel: 16000 119s Minimum Score Ratio: 0.56 119s Mapping Mode: normal 119s Reads Used: 4 (153 bases) 119s 119s Mapping: 0.096 seconds. 119s Reads/sec: 41.57 119s kBases/sec: 1.59 119s 119s 119s Read 1 data: pct reads num reads pct bases num bases 119s 119s mapped: 0.0000% 0 0.0000% 0 119s unambiguous: 0.0000% 0 0.0000% 0 119s ambiguous: 0.0000% 0 0.0000% 0 119s low-Q discards: 25.0000% 1 19.6078% 30 119s 119s perfect best site: 0.0000% 0 0.0000% 0 119s semiperfect site: 0.0000% 0 0.0000% 0 119s 119s Match Rate: NA NA NaN% 0 119s Error Rate: NaN% 0 NaN% 0 119s Sub Rate: NaN% 0 NaN% 0 119s Del Rate: NaN% 0 NaN% 0 119s Ins Rate: NaN% 0 NaN% 0 119s N Rate: NaN% 0 NaN% 0 119s 119s Total time: 4.101 seconds. 119s test -r out.sam 119s bbmap.sh ref=example.fasta in=example.fastq out=out2.sam nodisk 119s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 119s java -ea -Xmx2832m -Xms2832m -cp /usr/share/java/bbmap.jar align2.BBMap build=1 overwrite=true fastareadlen=500 ref=example.fasta in=example.fastq out=out2.sam nodisk 119s Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, ref=example.fasta, in=example.fastq, out=out2.sam, nodisk] 119s Version 39.20 119s 120s Retaining first best site only for ambiguous mappings. 120s Executing dna.FastaToChromArrays2 [example.fasta, 1, writeinthread=false, genscaffoldinfo=true, retain, waitforwriting=false, gz=true, maxlen=536670912, writechroms=false, minscaf=1, midpad=300, startpad=8000, stoppad=8000, nodisk=true] 120s 120s Set genScaffoldInfo=true 120s Set genome to 1 120s 120s Loaded Reference: 0.002 seconds. 120s Loading index for chunk 1-1, build 1 120s Indexing threads started for block 0-1 120s Indexing threads finished for block 0-1 120s Generated Index: 0.246 seconds. 123s Analyzed Index: 3.536 seconds. 123s Started output stream: 0.048 seconds. 124s Cleared Memory: 0.123 seconds. 124s Processing reads in single-ended mode. 124s Started read stream. 124s Started 4 mapping threads. 124s Detecting finished threads: 0, 1, 2, 3 124s 124s ------------------ Results ------------------ 124s 124s Genome: 1 124s Key Length: 13 124s Max Indel: 16000 124s Minimum Score Ratio: 0.56 124s Mapping Mode: normal 124s Reads Used: 3 (75 bases) 124s 124s Mapping: 0.100 seconds. 124s Reads/sec: 30.03 124s kBases/sec: 0.75 124s 124s 124s Read 1 data: pct reads num reads pct bases num bases 124s 124s mapped: 100.0000% 3 100.0000% 75 124s unambiguous: 100.0000% 3 100.0000% 75 124s ambiguous: 0.0000% 0 0.0000% 0 124s low-Q discards: 0.0000% 0 0.0000% 0 124s 124s perfect best site: 100.0000% 3 100.0000% 75 124s semiperfect site: 100.0000% 3 100.0000% 75 124s 124s Match Rate: NA NA 100.0000% 75 124s Error Rate: 0.0000% 0 0.0000% 0 124s Sub Rate: 0.0000% 0 0.0000% 0 124s Del Rate: 0.0000% 0 0.0000% 0 124s Ins Rate: 0.0000% 0 0.0000% 0 124s N Rate: 0.0000% 0 0.0000% 0 124s 124s Total time: 4.240 seconds. 124s test -r out2.sam 124s 124s # bbduk 124s bbduk.sh --version 124s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 124s java -ea -Xmx1416m -Xms1416m -cp /usr/share/java/bbmap.jar jgi.BBDuk --version 124s BBTools version 39.20 124s For help, please run the shellscript with no parameters, or look in /docs/. 124s bbduk.sh --help 124s 124s Written by Brian Bushnell 124s Last modified November 18, 2024 124s 124s Description: Compares reads to the kmers in a reference dataset, optionally 124s allowing an edit distance. Splits the reads into two outputs - those that 124s match the reference, and those that don't. Can also trim (remove) the matching 124s parts of the reads rather than binning the reads. 124s Please read bbmap/docs/guides/BBDukGuide.txt for more information. 124s 124s Usage: bbduk.sh in= out= ref= 124s 124s Input may be stdin or a fasta or fastq file, compressed or uncompressed. 124s If you pipe via stdin/stdout, please include the file type; e.g. for gzipped 124s fasta input, set in=stdin.fa.gz 124s 124s Input parameters: 124s in= Main input. in=stdin.fq will pipe from stdin. 124s in2= Input for 2nd read of pairs in a different file. 124s ref= Comma-delimited list of reference files. 124s In addition to filenames, you may also use the keywords: 124s adapters, artifacts, phix, lambda, pjet, mtst, kapa 124s literal= Comma-delimited list of literal reference sequences. 124s Polymers are also allowed with the 'poly' prefix; 124s for example, 'literal=ATGGT,polyGC' will add both ATGGT 124s and GCGCGCGCGCGCGCGCGCGCGCGCGCGCGCGCGCGC - 32+ of them, 124s enough replicates to ensure that all kmers are present. 124s touppercase=f (tuc) Change all bases upper-case. 124s interleaved=auto (int) t/f overrides interleaved autodetection. 124s Must be set mainually when streaming fastq input. 124s qin=auto Input quality offset: 33 (Sanger), 64, or auto. 124s reads=-1 If positive, quit after processing X reads or pairs. 124s copyundefined=f (cu) Process non-AGCT IUPAC reference bases by making all 124s possible unambiguous copies. Intended for short motifs 124s or adapter barcodes, as time/memory use is exponential. 124s samplerate=1 Set lower to only process a fraction of input reads. 124s samref= Optional reference fasta for processing sam files. 124s 124s Output parameters: 124s out= (outnonmatch) Write reads here that do not contain 124s kmers matching the database. 'out=stdout.fq' will pipe 124s to standard out. 124s out2= (outnonmatch2) Use this to write 2nd read of pairs to a 124s different file. 124s outm= (outmatch) Write reads here that fail filters. In default 124s kfilter mode, this means any read with a matching kmer. 124s In any mode, it also includes reads that fail filters such 124s as minlength, mingc, maxgc, entropy, etc. In other words, 124s it includes all reads that do not go to 'out'. 124s outm2= (outmatch2) Use this to write 2nd read of pairs to a 124s different file. 124s outs= (outsingle) Use this to write singleton reads whose mate 124s was trimmed shorter than minlen. 124s stats= Write statistics about which contamininants were detected. 124s refstats= Write statistics on a per-reference-file basis. 124s rpkm= Write RPKM for each reference sequence (for RNA-seq). 124s dump= Dump kmer tables to a file, in fasta format. 124s duk= Write statistics in duk's format. *DEPRECATED* 124s nzo=t Only write statistics about ref sequences with nonzero hits. 124s overwrite=t (ow) Grant permission to overwrite files. 124s showspeed=t (ss) 'f' suppresses display of processing speed. 124s ziplevel=2 (zl) Compression level; 1 (min) through 9 (max). 124s fastawrap=70 Length of lines in fasta output. 124s qout=auto Output quality offset: 33 (Sanger), 64, or auto. 124s statscolumns=3 (cols) Number of columns for stats output, 3 or 5. 124s 5 includes base counts. 124s rename=f Rename reads to indicate which sequences they matched. 124s refnames=f Use names of reference files rather than scaffold IDs. 124s trd=f Truncate read and ref names at the first whitespace. 124s ordered=f Set to true to output reads in same order as input. 124s maxbasesout=-1 If positive, quit after writing approximately this many 124s bases to out (outu/outnonmatch). 124s maxbasesoutm=-1 If positive, quit after writing approximately this many 124s bases to outm (outmatch). 124s json=f Print to screen in json format. 124s 124s Histogram output parameters: 124s bhist= Base composition histogram by position. 124s qhist= Quality histogram by position. 124s qchist= Count of bases with each quality value. 124s aqhist= Histogram of average read quality. 124s bqhist= Quality histogram designed for box plots. 124s lhist= Read length histogram. 124s phist= Polymer length histogram. 124s gchist= Read GC content histogram. 124s enthist= Read entropy histogram. 124s ihist= Insert size histogram, for paired reads in mapped sam. 124s gcbins=100 Number gchist bins. Set to 'auto' to use read length. 124s maxhistlen=6000 Set an upper bound for histogram lengths; higher uses 124s more memory. The default is 6000 for some histograms 124s and 80000 for others. 124s 124s Histograms for mapped sam/bam files only: 124s histbefore=t Calculate histograms from reads before processing. 124s ehist= Errors-per-read histogram. 124s qahist= Quality accuracy histogram of error rates versus quality 124s score. 124s indelhist= Indel length histogram. 124s mhist= Histogram of match, sub, del, and ins rates by position. 124s idhist= Histogram of read count versus percent identity. 124s idbins=100 Number idhist bins. Set to 'auto' to use read length. 124s varfile= Ignore substitution errors listed in this file when 124s calculating error rates. Can be generated with 124s CallVariants. 124s vcf= Ignore substitution errors listed in this VCF file 124s when calculating error rates. 124s ignorevcfindels=t Also ignore indels listed in the VCF. 124s 124s Processing parameters: 124s k=27 Kmer length used for finding contaminants. Contaminants 124s shorter than k will not be found. k must be at least 1. 124s rcomp=t Look for reverse-complements of kmers in addition to 124s forward kmers. 124s maskmiddle=t (mm) Treat the middle base of a kmer as a wildcard, to 124s increase sensitivity in the presence of errors. This may 124s also be set to a number, e.g. mm=3, to mask that many bp. 124s The default mm=t corresponds to mm=1 for odd-length kmers 124s and mm=2 for even-length kmers (as of v39.04), while 124s mm=f is always equivalent to mm=0. 124s minkmerhits=1 (mkh) Reads need at least this many matching kmers 124s to be considered as matching the reference. 124s minkmerfraction=0.0 (mkf) A reads needs at least this fraction of its total 124s kmers to hit a ref, in order to be considered a match. 124s If this and minkmerhits are set, the greater is used. 124s mincovfraction=0.0 (mcf) A reads needs at least this fraction of its total 124s bases to be covered by ref kmers to be considered a match. 124s If specified, mcf overrides mkh and mkf. 124s hammingdistance=0 (hdist) Maximum Hamming distance for ref kmers (subs only). 124s Memory use is proportional to (3*K)^hdist. 124s qhdist=0 Hamming distance for query kmers; impacts speed, not memory. 124s editdistance=0 (edist) Maximum edit distance from ref kmers (subs 124s and indels). Memory use is proportional to (8*K)^edist. 124s hammingdistance2=0 (hdist2) Sets hdist for short kmers, when using mink. 124s qhdist2=0 Sets qhdist for short kmers, when using mink. 124s editdistance2=0 (edist2) Sets edist for short kmers, when using mink. 124s forbidn=f (fn) Forbids matching of read kmers containing N. 124s By default, these will match a reference 'A' if 124s hdist>0 or edist>0, to increase sensitivity. 124s removeifeitherbad=t (rieb) Paired reads get sent to 'outmatch' if either is 124s match (or either is trimmed shorter than minlen). 124s Set to false to require both. 124s trimfailures=f Instead of discarding failed reads, trim them to 1bp. 124s This makes the statistics a bit odd. 124s findbestmatch=f (fbm) If multiple matches, associate read with sequence 124s sharing most kmers. Reduces speed. 124s skipr1=f Don't do kmer-based operations on read 1. 124s skipr2=f Don't do kmer-based operations on read 2. 124s ecco=f For overlapping paired reads only. Performs error- 124s correction with BBMerge prior to kmer operations. 124s recalibrate=f (recal) Recalibrate quality scores. Requires calibration 124s matrices generated by CalcTrueQuality. 124s sam= If recalibration is desired, and matrices have not already 124s been generated, BBDuk will create them from the sam file. 124s amino=f Run in amino acid mode. Some features have not been 124s tested, but kmer-matching works fine. Maximum k is 12. 124s 124s Speed and Memory parameters: 124s threads=auto (t) Set number of threads to use; default is number of 124s logical processors. 124s bbduk.sh in1=misc_dna_as_illumina.fastq in2=misc_rna_as_illumina.fastq \ 124s qtrim=rl trimq=15 minlen=75 out=out.fastq.gz 124s prealloc=f Preallocate memory in table. Allows faster table loading 124s and more efficient memory usage, for a large reference. 124s monitor=f Kill this process if it crashes. monitor=600,0.01 would 124s kill after 600 seconds under 1% usage. 124s minrskip=1 (mns) Force minimal skip interval when indexing reference 124s kmers. 1 means use all, 2 means use every other kmer, etc. 124s maxrskip=1 (mxs) Restrict maximal skip interval when indexing 124s reference kmers. Normally all are used for scaffolds<100kb, 124s but with longer scaffolds, up to maxrskip-1 are skipped. 124s rskip= Set both minrskip and maxrskip to the same value. 124s If not set, rskip will vary based on sequence length. 124s qskip=1 Skip query kmers to increase speed. 1 means use all. 124s speed=0 Ignore this fraction of kmer space (0-15 out of 16) in both 124s reads and reference. Increases speed and reduces memory. 124s Note: Do not use more than one of 'speed', 'qskip', and 'rskip'. 124s 124s Trimming/Filtering/Masking parameters: 124s Note - if ktrim, kmask, and ksplit are unset, the default behavior is kfilter. 124s All kmer processing modes are mutually exclusive. 124s Reads only get sent to 'outm' purely based on kmer matches in kfilter mode. 124s 124s ktrim=f Trim reads to remove bases matching reference kmers, plus 124s all bases to the left or right. 124s Values: 124s f (don't trim), 124s r (trim to the right), 124s l (trim to the left) 124s ktrimtips=0 Set this to a positive number to perform ktrim on both 124s ends, examining only the outermost X bases. 124s kmask= Replace bases matching ref kmers with another symbol. 124s Allows any non-whitespace character, and processes short 124s kmers on both ends if mink is set. 'kmask=lc' will 124s convert masked bases to lowercase. 124s maskfullycovered=f (mfc) Only mask bases that are fully covered by kmers. 124s ksplit=f For single-ended reads only. Reads will be split into 124s pairs around the kmer. If the kmer is at the end of the 124s read, it will be trimmed instead. Singletons will go to 124s out, and pairs will go to outm. Do not use ksplit with 124s other operations such as quality-trimming or filtering. 124s mink=0 Look for shorter kmers at read tips down to this length, 124s when k-trimming or masking. 0 means disabled. Enabling 124s this will disable maskmiddle. 124s qtrim=f Trim read ends to remove bases with quality below trimq. 124s Performed AFTER looking for kmers. Values: 124s rl (trim both ends), 124s f (neither end), 124s r (right end only), 124s l (left end only), 124s w (sliding window). 124s trimq=6 Regions with average quality BELOW this will be trimmed, 124s if qtrim is set to something other than f. Can be a 124s floating-point number like 7.3. 124s quantize Bin quality scores to reduce file size. quantize=2 will 124s eliminate all odd quality scores, while quantize=0,10,37 124s will only allow qualty scores of 0, 10, or 37. 124s trimclip=f Trim soft-clipped bases from sam files. 124s minlength=10 (ml) Reads shorter than this after trimming will be 124s discarded. Pairs will be discarded if both are shorter. 124s mlf=0 (minlengthfraction) Reads shorter than this fraction of 124s original length after trimming will be discarded. 124s maxlength= Reads longer than this after trimming will be discarded. 124s minavgquality=0 (maq) Reads with average quality (after trimming) below 124s this will be discarded. 124s maqb=0 If positive, calculate maq from this many initial bases. 124s minbasequality=0 (mbq) Reads with any base below this quality (after 124s trimming) will be discarded. 124s maxns=-1 If non-negative, reads with more Ns than this 124s (after trimming) will be discarded. 124s mcb=0 (minconsecutivebases) Discard reads without at least 124s this many consecutive called bases. 124s ottm=f (outputtrimmedtomatch) Output reads trimmed to shorter 124s than minlength to outm rather than discarding. 124s tp=0 (trimpad) Trim this much extra around matching kmers. 124s tbo=f (trimbyoverlap) Trim adapters based on where paired 124s reads overlap. 124s strictoverlap=t Adjust sensitivity for trimbyoverlap mode. 124s minoverlap=14 Require this many bases of overlap for detection. 124s mininsert=40 Require insert size of at least this for overlap. 124s Should be reduced to 16 for small RNA sequencing. 124s tpe=f (trimpairsevenly) When kmer right-trimming, trim both 124s reads to the minimum length of either. 124s forcetrimleft=0 (ftl) If positive, trim bases to the left of this position 124s (exclusive, 0-based). 124s forcetrimright=0 (ftr) If positive, trim bases to the right of this position 124s (exclusive, 0-based). 124s forcetrimright2=0 (ftr2) If positive, trim this many bases on the right end. 124s forcetrimmod=0 (ftm) If positive, right-trim length to be equal to zero, 124s modulo this number. 124s restrictleft=0 If positive, only look for kmer matches in the 124s leftmost X bases. 124s restrictright=0 If positive, only look for kmer matches in the 124s rightmost X bases. 124s NOTE: restrictleft and restrictright are mutually exclusive. If trimming 124s both ends is desired, use ktrimtips. 124s mingc=0 Discard reads with GC content below this. 124s maxgc=1 Discard reads with GC content above this. 124s gcpairs=t Use average GC of paired reads. 124s Also affects gchist. 124s tossjunk=f Discard reads with invalid characters as bases. 124s swift=f Trim Swift sequences: Trailing C/T/N R1, leading G/A/N R2. 124s 124s Header-parsing parameters - these require Illumina headers: 124s chastityfilter=f (cf) Discard reads with id containing ' 1:Y:' or ' 2:Y:'. 124s barcodefilter=f Remove reads with unexpected barcodes if barcodes is set, 124s or barcodes containing 'N' otherwise. A barcode must be 124s the last part of the read header. Values: 124s t: Remove reads with bad barcodes. 124s f: Ignore barcodes. 124s crash: Crash upon encountering bad barcodes. 124s barcodes= Comma-delimited list of barcodes or files of barcodes. 124s xmin=-1 If positive, discard reads with a lesser X coordinate. 124s ymin=-1 If positive, discard reads with a lesser Y coordinate. 124s xmax=-1 If positive, discard reads with a greater X coordinate. 124s ymax=-1 If positive, discard reads with a greater Y coordinate. 124s 124s Polymer trimming: 124s trimpolya=0 If greater than 0, trim poly-A or poly-T tails of 124s at least this length on either end of reads. 124s trimpolygleft=0 If greater than 0, trim poly-G prefixes of at least this 124s length on the left end of reads. Does not trim poly-C. 124s trimpolygright=0 If greater than 0, trim poly-G tails of at least this 124s length on the right end of reads. Does not trim poly-C. 124s trimpolyg=0 This sets both left and right at once. 124s filterpolyg=0 If greater than 0, remove reads with a poly-G prefix of 124s at least this length (on the left). 124s Note: there are also equivalent poly-C flags. 124s 124s Polymer tracking: 124s pratio=base,base 'pratio=G,C' will print the ratio of G to C polymers. 124s plen=20 Length of homopolymers to count. 124s 124s Entropy/Complexity parameters: 124s entropy=-1 Set between 0 and 1 to filter reads with entropy below 124s that value. Higher is more stringent. 124s entropywindow=50 Calculate entropy using a sliding window of this length. 124s entropyk=5 Calculate entropy using kmers of this length. 124s minbasefrequency=0 Discard reads with a minimum base frequency below this. 124s entropytrim=f Values: 124s f: (false) Do not entropy-trim. 124s r: (right) Trim low entropy on the right end only. 124s l: (left) Trim low entropy on the left end only. 124s rl: (both) Trim low entropy on both ends. 124s entropymask=f Values: 124s f: (filter) Discard low-entropy sequences. 124s t: (true) Mask low-entropy parts of sequences with N. 124s lc: Change low-entropy parts of sequences to lowercase. 124s entropymark=f Mark each base with its entropy value. This is on a scale 124s of 0-41 and is reported as quality scores, so the output 124s should be fastq or fasta+qual. 124s NOTE: If set, entropytrim overrides entropymask. 124s 124s Cardinality estimation: 124s cardinality=f (loglog) Count unique kmers using the LogLog algorithm. 124s cardinalityout=f (loglogout) Count unique kmers in output reads. 124s loglogk=31 Use this kmer length for counting. 124s loglogbuckets=2048 Use this many buckets for counting. 124s khist= Kmer frequency histogram; plots number of kmers versus 124s kmer depth. This is approximate. 124s khistout= Kmer frequency histogram for output reads. 124s 124s Side Channel: 124s sideout= Output for aligned reads. 124s sideref=phix Reference for side-channel alignment; must be a single 124s sequence and virtually repeat-free at selected k. 124s sidek1=17 Kmer length for seeding alignment to reference. 124s sidek2=13 Kmer length for seeding alignment of unaligned reads 124s with an aligned mate. 124s sideminid1=0.66 Minimum identity to accept individual alignments. 124s sideminid2=0.58 Minimum identity for aligning reads with aligned mates. 124s sidemm1=1 Middle mask length for sidek1. 124s sidemm2=1 Middle mask length for sidek2. 124s Note: The side channel is a special additional output that allows alignment 124s to a secondary reference while also doing trimming. Alignment does not affect 124s whether reads go to the normal outputs (out, outm). The main purpose is to 124s simplify pipelines that need trimmed, aligned phiX reads for recalibration. 124s 124s 124s Java Parameters: 124s 124s -Xmx This will set Java's memory usage, overriding autodetection. 124s -Xmx20g will 124s specify 20 gigs of RAM, and -Xmx200m will specify 200 megs. 124s The max is typically 85% of physical memory. 124s -eoom This flag will cause the process to exit if an 124s out-of-memory exception occurs. Requires Java 8u92+. 124s -da Disable assertions. 124s 124s Please contact Brian Bushnell at bbushnell@lbl.gov if you encounter any problems. 124s 124s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 124s java -ea -Xmx1416m -Xms1416m -cp /usr/share/java/bbmap.jar jgi.BBDuk in1=misc_dna_as_illumina.fastq in2=misc_rna_as_illumina.fastq qtrim=rl trimq=15 minlen=75 out=out.fastq.gz 124s Executing jgi.BBDuk [in1=misc_dna_as_illumina.fastq, in2=misc_rna_as_illumina.fastq, qtrim=rl, trimq=15, minlen=75, out=out.fastq.gz] 124s Version 39.20 124s 124s Changed from ASCII-33 to ASCII-64 on input quality X (Q55) for base A at lines 1 and 3, position 24 while prescanning. 124s Changed from ASCII-33 to ASCII-64 on input quality X (Q55) for base A at lines 1 and 3, position 24 while prescanning. 124s Allocating kmer table: 0.033 seconds. 124s Initial: 124s Memory: max=1484m, total=1484m, free=1453m, used=31m 124s 125s Input is being processed as paired 125s Changed from ASCII-33 to ASCII-64 on input 7: 55 -> 24 125s Changed from ASCII-33 to ASCII-64 on input 7: 55 -> 24 125s Started output streams: 0.058 seconds. 125s Processing time: 0.008 seconds. 125s 125s Input: 8 reads 306 bases. 125s QTrimmed: 0 reads (0.00%) 0 bases (0.00%) 125s Total Removed: 8 reads (100.00%) 306 bases (100.00%) 125s Result: 0 reads (0.00%) 0 bases (0.00%) 125s 125s Time: 0.071 seconds. 125s Reads Processed: 8 0.11k reads/sec 125s Bases Processed: 306 0.00m bases/sec 125s test -r out.fastq.gz 125s 125s # bbnorm 125s bbnorm.sh --version 125s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 125s java -ea -Xmx2832m -Xms2832m -cp /usr/share/java/bbmap.jar jgi.KmerNormalize bits=32 --version 125s BBTools version 39.20 125s For help, please run the shellscript with no parameters, or look in /docs/. 125s bbnorm.sh --help 125s 125s Written by Brian Bushnell 125s Last modified October 19, 2017 125s 125s Description: Normalizes read depth based on kmer counts. 125s Can also error-correct, bin reads by kmer depth, and generate a kmer depth histogram. 125s However, Tadpole has superior error-correction to BBNorm. 125s Please read bbmap/docs/guides/BBNormGuide.txt for more information. 125s 125s Usage: bbnorm.sh in= out= outt= hist= 125s 125s Input parameters: 125s in=null Primary input. Use in2 for paired reads in a second file 125s in2=null Second input file for paired reads in two files 125s extra=null Additional files to use for input (generating hash table) but not for output 125s fastareadlen=2^31 Break up FASTA reads longer than this. Can be useful when processing scaffolded genomes 125s tablereads=-1 Use at most this many reads when building the hashtable (-1 means all) 125s kmersample=1 Process every nth kmer, and skip the rest 125s readsample=1 Process every nth read, and skip the rest 125s interleaved=auto May be set to true or false to force the input read file to ovverride autodetection of the input file as paired interleaved. 125s qin=auto ASCII offset for input quality. May be 33 (Sanger), 64 (Illumina), or auto. 125s 125s Output parameters: 125s out= File for normalized or corrected reads. Use out2 for paired reads in a second file 125s outt= (outtoss) File for reads that were excluded from primary output 125s reads=-1 Only process this number of reads, then quit (-1 means all) 125s sampleoutput=t Use sampling on output as well as input (not used if sample rates are 1) 125s keepall=f Set to true to keep all reads (e.g. if you just want error correction). 125s zerobin=f Set to true if you want kmers with a count of 0 to go in the 0 bin instead of the 1 bin in histograms. 125s Default is false, to prevent confusion about how there can be 0-count kmers. 125s The reason is that based on the 'minq' and 'minprob' settings, some kmers may be excluded from the bloom filter. 125s tmpdir= This will specify a directory for temp files (only needed for multipass runs). If null, they will be written to the output directory. 125s usetempdir=t Allows enabling/disabling of temporary directory; if disabled, temp files will be written to the output directory. 125s qout=auto ASCII offset for output quality. May be 33 (Sanger), 64 (Illumina), or auto (same as input). 125s rename=f Rename reads based on their kmer depth. 125s 125s Hashing parameters: 125s k=31 Kmer length (values under 32 are most efficient, but arbitrarily high values are supported) 125s bits=32 Bits per cell in bloom filter; must be 2, 4, 8, 16, or 32. Maximum kmer depth recorded is 2^cbits. Automatically reduced to 16 in 2-pass. 125s Large values decrease accuracy for a fixed amount of memory, so use the lowest number you can that will still capture highest-depth kmers. 125s hashes=3 Number of times each kmer is hashed and stored. Higher is slower. 125s Higher is MORE accurate if there is enough memory, and LESS accurate if there is not enough memory. 125s prefilter=f True is slower, but generally more accurate; filters out low-depth kmers from the main hashtable. The prefilter is more memory-efficient because it uses 2-bit cells. 125s prehashes=2 Number of hashes for prefilter. 125s prefilterbits=2 (pbits) Bits per cell in prefilter. 125s prefiltersize=0.35 Fraction of memory to allocate to prefilter. 125s buildpasses=1 More passes can sometimes increase accuracy by iteratively removing low-depth kmers 125s minq=6 Ignore kmers containing bases with quality below this 125s minprob=0.5 Ignore kmers with overall probability of correctness below this 125s threads=auto (t) Spawn exactly X hashing threads (default is number of logical processors). Total active threads may exceed X due to I/O threads. 125s rdk=t (removeduplicatekmers) When true, a kmer's count will only be incremented once per read pair, even if that kmer occurs more than once. 125s 125s Normalization parameters: 125s fixspikes=f (fs) Do a slower, high-precision bloom filter lookup of kmers that appear to have an abnormally high depth due to collisions. 125s target=100 (tgt) Target normalization depth. NOTE: All depth parameters control kmer depth, not read depth. 125s For kmer depth Dk, read depth Dr, read length R, and kmer size K: Dr=Dk*(R/(R-K+1)) 125s maxdepth=-1 (max) Reads will not be downsampled when below this depth, even if they are above the target depth. 125s mindepth=5 (min) Kmers with depth below this number will not be included when calculating the depth of a read. 125s minkmers=15 (mgkpr) Reads must have at least this many kmers over min depth to be retained. Aka 'mingoodkmersperread'. 125s percentile=54.0 (dp) Read depth is by default inferred from the 54th percentile of kmer depth, but this may be changed to any number 1-100. 125s uselowerdepth=t (uld) For pairs, use the depth of the lower read as the depth proxy. 125s deterministic=t (dr) Generate random numbers deterministically to ensure identical output between multiple runs. May decrease speed with a huge number of threads. 125s passes=2 (p) 1 pass is the basic mode. 2 passes (default) allows greater accuracy, error detection, better contol of output depth. 125s 125s Error detection parameters: 125s hdp=90.0 (highdepthpercentile) Position in sorted kmer depth array used as proxy of a read's high kmer depth. 125s ldp=25.0 (lowdepthpercentile) Position in sorted kmer depth array used as proxy of a read's low kmer depth. 125s tossbadreads=f (tbr) Throw away reads detected as containing errors. 125s requirebothbad=f (rbb) Only toss bad pairs if both reads are bad. 125s errordetectratio=125 (edr) Reads with a ratio of at least this much between their high and low depth kmers will be classified as error reads. 125s highthresh=12 (ht) Threshold for high kmer. A high kmer at this or above are considered non-error. 125s lowthresh=3 (lt) Threshold for low kmer. Kmers at this and below are always considered errors. 125s 125s Error correction parameters: 125s ecc=f Set to true to correct errors. NOTE: Tadpole is now preferred for ecc as it does a better job. 125s ecclimit=3 Correct up to this many errors per read. If more are detected, the read will remain unchanged. 125s errorcorrectratio=140 (ecr) Adjacent kmers with a depth ratio of at least this much between will be classified as an error. 125s echighthresh=22 (echt) Threshold for high kmer. A kmer at this or above may be considered non-error. 125s eclowthresh=2 (eclt) Threshold for low kmer. Kmers at this and below are considered errors. 125s eccmaxqual=127 Do not correct bases with quality above this value. 125s aec=f (aggressiveErrorCorrection) Sets more aggressive values of ecr=100, ecclimit=7, echt=16, eclt=3. 125s cec=f (conservativeErrorCorrection) Sets more conservative values of ecr=180, ecclimit=2, echt=30, eclt=1, sl=4, pl=4. 125s meo=f (markErrorsOnly) Marks errors by reducing quality value of suspected errors; does not correct anything. 125s mue=t (markUncorrectableErrors) Marks errors only on uncorrectable reads; requires 'ecc=t'. 125s overlap=f (ecco) Error correct by read overlap. 125s 125s Depth binning parameters: 125s lowbindepth=10 (lbd) Cutoff for low depth bin. 125s highbindepth=80 (hbd) Cutoff for high depth bin. 125s outlow= Pairs in which both reads have a median below lbd go into this file. 125s outhigh= Pairs in which both reads have a median above hbd go into this file. 125s outmid= All other pairs go into this file. 125s 125s Histogram parameters: 125s hist= Specify a file to write the input kmer depth histogram. 125s histout= Specify a file to write the output kmer depth histogram. 125s histcol=3 (histogramcolumns) Number of histogram columns, 2 or 3. 125s pzc=f (printzerocoverage) Print lines in the histogram with zero coverage. 125s histlen=1048576 Max kmer depth displayed in histogram. Also affects statistics displayed, but does not affect normalization. 125s 125s Peak calling parameters: 125s peaks= Write the peaks to this file. Default is stdout. 125s minHeight=2 (h) Ignore peaks shorter than this. 125s minVolume=5 (v) Ignore peaks with less area than this. 125s minWidth=3 (w) Ignore peaks narrower than this. 125s minPeak=2 (minp) Ignore peaks with an X-value below this. 125s maxPeak=BIG (maxp) Ignore peaks with an X-value above this. 125s maxPeakCount=8 (maxpc) Print up to this many peaks (prioritizing height). 125s 125s Java Parameters: 125s -Xmx This will set Java's memory usage, overriding autodetection. 125s -Xmx20g will specify 20 gigs of RAM, and -Xmx200m will specify 200 megs. 125s The max is typically 85% of physical memory. 125s -eoom This flag will cause the process to exit if an 125s out-of-memory exception occurs. Requires Java 8u92+. 125s -da Disable assertions. 125s 125s Please contact Brian Bushnell at bbushnell@lbl.gov if you encounter any problems. 125s 125s bbnorm.sh in=sanger_full_range_original_sanger.fastq qin=33 \ 125s out=out3.fastq outt=outt.fastq hist=hist.plt 125s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 125s java -ea -Xmx2832m -Xms2832m -cp /usr/share/java/bbmap.jar jgi.KmerNormalize bits=32 in=sanger_full_range_original_sanger.fastq qin=33 out=out3.fastq outt=outt.fastq hist=hist.plt 125s Executing jgi.KmerNormalize [bits=32, in=sanger_full_range_original_sanger.fastq, qin=33, out=out3.fastq, outt=outt.fastq, hist=hist.plt] 125s 125s 125s *********** Pass 1 ********** 125s 125s 125s Settings: 125s threads: 4 125s k: 31 125s deterministic: true 125s toss error reads: false 125s passes: 1 125s bits per cell: 16 125s cells: 1048.33M 125s hashes: 3 125s base min quality: 5 125s kmer min prob: 0.5 125s 125s target depth: 400 125s min depth: 3 125s max depth: 500 125s min good kmers: 15 125s depth percentile: 64.8 125s ignore dupe kmers: true 125s fix spikes: false 125s histogram length: 65536 125s print zero cov: false 125s 126s Made hash table: hashes = 3 mem = 1.95 GB cells = 1047.34M used = 0.000% 126s 126s Estimated unique kmers: 4 126s 126s Table creation time: 1.102 seconds. 126s Started output threads. 126s Started output threads. 126s Table read time: 0.008 seconds. 23.79 kb/sec 126s Total reads in: 2 0.000% Kept 126s Total bases in: 188 0.000% Kept 126s Error reads in: 2 100.000% 126s Error type 1: 2 100.000% 126s Error type 2: 0 0.000% 126s Error type 3: 0 0.000% 126s 126s Wrote histogram to hist.plt 126s Total kmers counted: 128 126s Total unique kmer count: 128 126s Includes forward kmers only. 126s The unique kmer estimate can be more accurate than the unique count, if the tables are very full. 126s The most accurate value is the greater of the two. 126s 126s Percent unique: 100.00% 126s Depth average: 1.00 (unique kmers) 126s Depth median: 1 (unique kmers) 126s Depth standard deviation: 0.00 (unique kmers) 126s Corrected depth average: 0.00 126s 126s Depth average: 1.00 (all kmers) 126s Depth median: 1 (all kmers) 126s Depth standard deviation: 0.00 (all kmers) 126s 126s Approx. read depth median: 1.47 126s 126s *********** Pass 2 ********** 126s 126s 126s Settings: 126s threads: 4 126s k: 31 126s deterministic: true 126s toss error reads: false 126s passes: 1 126s bits per cell: 16 126s cells: 1048.33M 126s hashes: 3 126s base min quality: 5 126s kmer min prob: 0.5 126s 126s target depth: 100 126s min depth: 5 126s max depth: 100 126s min good kmers: 15 126s depth percentile: 54.0 126s ignore dupe kmers: true 126s fix spikes: false 126s histogram length: 65536 126s 127s Made hash table: hashes = 3 mem = 1.95 GB cells = 1047.34M used = 0.000% 127s 127s Estimated unique kmers: 0 127s 127s Table creation time: 0.504 seconds. 127s Started output threads. 127s Started output threads. 127s Table read time: 0.003 seconds. 0.00 kb/sec 127s Total reads in: 0 NaN% Kept 127s Total bases in: 0 NaN% Kept 127s Error reads in: 0 NaN% 127s Error type 1: 0 NaN% 127s Error type 2: 0 NaN% 127s Error type 3: 0 NaN% 127s Total kmers counted: 0 127s Total unique kmer count: 0 127s Includes forward kmers only. 127s The unique kmer estimate can be more accurate than the unique count, if the tables are very full. 127s The most accurate value is the greater of the two. 127s 127s Percent unique: NaN% 127s Depth average: NaN (unique kmers) 127s Depth median: 0 (unique kmers) 127s Depth standard deviation: NaN (unique kmers) 127s Corrected depth average: NaN 127s 127s Depth average: NaN (all kmers) 127s Depth median: 0 (all kmers) 127s Depth standard deviation: 0.00 (all kmers) 127s 127s Approx. read depth median: NaN 127s 127s Removing temp files. 127s 127s Total time: 1.713 seconds. 0.11 kb/sec 127s test -r out3.fastq 127s test -r outt.fastq 127s test -r hist.plt 127s bbnorm.sh in=illumina_full_range_original_illumina.fastq qin=64 \ 127s out=out4.fastq outt=outt2.fastq hist=hist2.plt 127s dpkg-architecture: warning: cannot determine CC system type, falling back to default (native compilation) 127s java -ea -Xmx2832m -Xms2832m -cp /usr/share/java/bbmap.jar jgi.KmerNormalize bits=32 in=illumina_full_range_original_illumina.fastq qin=64 out=out4.fastq outt=outt2.fastq hist=hist2.plt 127s Executing jgi.KmerNormalize [bits=32, in=illumina_full_range_original_illumina.fastq, qin=64, out=out4.fastq, outt=outt2.fastq, hist=hist2.plt] 127s 127s 127s *********** Pass 1 ********** 127s 127s 127s Settings: 127s threads: 4 127s k: 31 127s deterministic: true 127s toss error reads: false 127s passes: 1 127s bits per cell: 16 127s cells: 1048.33M 127s hashes: 3 127s base min quality: 5 127s kmer min prob: 0.5 127s 127s target depth: 400 127s min depth: 3 127s max depth: 500 127s min good kmers: 15 127s depth percentile: 64.8 127s ignore dupe kmers: true 127s fix spikes: false 127s histogram length: 65536 127s print zero cov: false 127s 128s Made hash table: hashes = 3 mem = 1.95 GB cells = 1047.34M used = 0.000% 128s 128s Estimated unique kmers: 4 128s 128s Table creation time: 1.026 seconds. 128s Started output threads. 128s Started output threads. 128s Table read time: 0.007 seconds. 18.13 kb/sec 128s Total reads in: 2 0.000% Kept 128s Total bases in: 126 0.000% Kept 128s Error reads in: 2 100.000% 128s Error type 1: 2 100.000% 128s Error type 2: 0 0.000% 128s Error type 3: 0 0.000% 128s 128s Wrote histogram to hist2.plt 128s Total kmers counted: 66 128s Total unique kmer count: 66 128s Includes forward kmers only. 128s The unique kmer estimate can be more accurate than the unique count, if the tables are very full. 128s The most accurate value is the greater of the two. 128s 128s Percent unique: 100.00% 128s Depth average: 1.00 (unique kmers) 128s Depth median: 1 (unique kmers) 128s Depth standard deviation: 0.00 (unique kmers) 128s Corrected depth average: 0.00 128s 128s Depth average: 1.00 (all kmers) 128s Depth median: 1 (all kmers) 128s Depth standard deviation: 0.00 (all kmers) 128s 128s Approx. read depth median: 1.91 128s 128s *********** Pass 2 ********** 128s 128s 128s Settings: 128s threads: 4 128s k: 31 128s deterministic: true 128s toss error reads: false 128s passes: 1 128s bits per cell: 16 128s cells: 1048.33M 128s hashes: 3 128s base min quality: 5 128s kmer min prob: 0.5 128s 128s target depth: 100 128s min depth: 5 128s max depth: 100 128s min good kmers: 15 128s depth percentile: 54.0 128s ignore dupe kmers: true 128s fix spikes: false 128s histogram length: 65536 128s 129s Made hash table: hashes = 3 mem = 1.95 GB cells = 1047.34M used = 0.000% 129s 129s Estimated unique kmers: 0 129s 129s Table creation time: 0.558 seconds. 129s Started output threads. 129s Started output threads. 129s Table read time: 0.002 seconds. 0.00 kb/sec 129s Total reads in: 0 NaN% Kept 129s Total bases in: 0 NaN% Kept 129s Error reads in: 0 NaN% 129s Error type 1: 0 NaN% 129s Error type 2: 0 NaN% 129s Error type 3: 0 NaN% 129s Total kmers counted: 0 129s Total unique kmer count: 0 129s Includes forward kmers only. 129s The unique kmer estimate can be more accurate than the unique count, if the tables are very full. 129s The most accurate value is the greater of the two. 129s 129s Percent unique: NaN% 129s Depth average: NaN (unique kmers) 129s Depth median: 0 (unique kmers) 129s Depth standard deviation: NaN (unique kmers) 129s Corrected depth average: NaN 129s 129s Depth average: NaN (all kmers) 129s Depth median: 0 (all kmers) 129s Depth standard deviation: 0.00 (all kmers) 129s 129s Approx. read depth median: NaN 129s 129s Removing temp files. 129s 129s Total time: 1.682 seconds. 0.07 kb/sec 129s test -r out4.fastq 129s test -r outt2.fastq 129s test -r hist2.plt 129s autopkgtest [09:51:43]: test run-unit-test: -----------------------] 130s run-unit-test PASS 130s autopkgtest [09:51:44]: test run-unit-test: - - - - - - - - - - results - - - - - - - - - - 131s autopkgtest [09:51:45]: @@@@@@@@@@@@@@@@@@@@ summary 131s run-unit-test PASS