Skip to main content
Version: Next

Persistent Cache Task

It designs to provide persistent caching for tasks. This tool can import file and export file in P2P network. The solution is specifically engineered for high-speed read and write operations. This makes it particularly advantageous for scenarios involving large files, such as machine learning model checkpoints, where rapid, reliable access and distribution across the network are critical for training and inference workflows. By leveraging P2P distribution and persistent caching, dfcache significantly reduces I/O bottlenecks and accelerates the lifecycle of large data assets.

persistent-cache-task-architecture

Dfcache

Use dfcache to import files, please refer to dfcache.

$ dfcache import /tmp/file.txt
⣷ Done: 852606796

Use dfcache to export files.

$ dfcache export 852606796 -O /tmp/file.txt
[00:00:00] [############################################################] 8.73 KiB/8.73 KiB (7.30 MiB/s, 0.0s)

Console

View persistent cache task details or delete a persistent cache task, please refer to persistent cache task.