GATK MarkDuplicates报错：

02:20:40.492 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/share/work/biosoft/GATK/gatk-4.4.0.0/gatk-package-4.4.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so

[Thu Sep 17 02:20:40 CST 2026] MarkDuplicates --INPUT sort.bam --OUTPUT sort.dedup.bam --METRICS_FILE sort.dedup.metrics --MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP 50000 --MAX_FILE_HANDLES_FOR_READ_ENDS_MAP 8000 --SORTING_COLLECTION_SIZE_RATIO 0.25 --TAG_DUPLICATE_SET_MEMBERS false --REMOVE_SEQUENCING_DUPLICATES false --TAGGING_POLICY DontTag --CLEAR_DT true --DUPLEX_UMI false --FLOW_MODE false --FLOW_QUALITY_SUM_STRATEGY false --USE_END_IN_UNPAIRED_READS false --USE_UNPAIRED_CLIPPED_END false --UNPAIRED_END_UNCERTAINTY 0 --FLOW_SKIP_FIRST_N_FLOWS 0 --FLOW_Q_IS_KNOWN_END false --FLOW_EFFECTIVE_QUALITY_THRESHOLD 15 --ADD_PG_TAG_TO_READS true --REMOVE_DUPLICATES false --ASSUME_SORTED false --DUPLICATE_SCORING_STRATEGY SUM_OF_BASE_QUALITIES --PROGRAM_RECORD_ID MarkDuplicates --PROGRAM_GROUP_NAME MarkDuplicates --READ_NAME_REGEX <optimized capture of last three ':' separated fields as numeric values> --OPTICAL_DUPLICATE_PIXEL_DISTANCE 100 --MAX_OPTICAL_DUPLICATE_SET_SIZE 300000 --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 2 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false

[Thu Sep 17 02:20:40 CST 2026] Executing as liuyf@compute-0-2.local on Linux 3.10.0-693.5.2.el7.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 19.0.1+10-21; Deflater: Intel; Inflater: Intel; Provider GCS is available; Picard version: Version:4.4.0.0

INFO    2026-09-17 02:20:40     MarkDuplicates  Start of doWork freeMemory: 401469536; totalMemory: 469762048; maxMemory: 53687091200

INFO    2026-09-17 02:20:40     MarkDuplicates  Reading input file and constructing read end information.

INFO    2026-09-17 02:20:40     MarkDuplicates  Will retain up to 194518446 data points before spilling to disk.

WARNING 2026-09-17 02:20:41     AbstractOpticalDuplicateFinderCommandLineProgram        A field field parsed out of a read name was expected to contain an integer and did not. Read name: ML150028039L1C007R02407936068. Cause: String 'ML150028039L1C007R02407936068' did not start with a parsable number.

INFO    2026-09-17 02:20:49     MarkDuplicates  Read     1,000,000 records.  Elapsed time: 00:00:08s.  Time for last 1,000,000:    8s.  Last read position: GWHBJCQ00000001:7,302,261

INFO    2026-09-17 02:20:49     MarkDuplicates  Tracking 23137 as yet unmatched pairs. 210 records in RAM.

[Thu Sep 17 02:20:50 CST 2026] picard.sam.markduplicates.MarkDuplicates done. Elapsed time: 0.17 minutes.

Runtime.totalMemory()=13220446208

To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp

htsjdk.samtools.SAMException: tmp/CSPI.tmp982590807608139993/3455.tmpnot found

        at htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.makeValue(FileAppendStreamLRUCache.java:64)

        at htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.makeValue(FileAppendStreamLRUCache.java:49)

        at htsjdk.samtools.util.ResourceLimitedMap.get(ResourceLimitedMap.java:76)

        at htsjdk.samtools.CoordinateSortedPairInfoMap.getOutputStreamForSequence(CoordinateSortedPairInfoMap.java:180)

        at htsjdk.samtools.CoordinateSortedPairInfoMap.put(CoordinateSortedPairInfoMap.java:164)

        at picard.sam.markduplicates.util.DiskBasedReadEndsForMarkDuplicatesMap.put(DiskBasedReadEndsForMarkDuplicatesMap.java:65)

        at picard.sam.markduplicates.MarkDuplicates.buildSortedReadEndLists(MarkDuplicates.java:567)

        at picard.sam.markduplicates.MarkDuplicates.doWork(MarkDuplicates.java:270)

        at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:289)

        at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgramExecutor.instanceMain(PicardCommandLineProgramExecutor.java:37)

        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)

        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)

        at org.broadinstitute.hellbender.Main.main(Main.java:289)

Caused by: java.io.FileNotFoundException:tmp/CSPI.tmp982590807608139993/3455.tmp (Too many open files)

        at java.base/java.io.FileOutputStream.open0(Native Method)

        at java.base/java.io.FileOutputStream.open(FileOutputStream.java:295)

        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:236)

        at htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.makeValue(FileAppendStreamLRUCache.java:61)

        ... 12 more

原因：

Caused by: java.io.FileNotFoundException: .../tmp/CSPI.tmp.../3455.tmp (Too many open files)

MarkDuplicates 在处理大量 contig 分组时，会把中间数据 spill 到磁盘临时文件，同时打开很多文件句柄。

环境里当前用户的 open files 限制太小，导致：

临时目录能创建，但打开第 N 个 .tmp 文件时失败

解决办法：

查看当前限制：

ulimit -n

根据当前限制修改参数：

命令修改为：

/share/work/biosoft/GATK/gatk-4.4.0.0/gatk \
--java-options '-Xmx50G -XX:ConcGCThreads=2 -XX:ParallelGCThreads=2 -Djava.io.tmpdir=tmp' \
MarkDuplicates \
  -I sort.bam \
  -O sort.dedup.bam \
  -M sort.dedup.metrics \
  --MAX_FILE_HANDLES_FOR_READ_ENDS_MAP 800 \
  --MAX_RECORDS_IN_RAM 50000