@ronomon/deduplication

Fast multi-threaded content-dependent chunking deduplication for Buffers in C++ with a reference implementation in Javascript. Ships with extensive tests, a fuzz test and a benchmark.
OpenBrowse Files
asyncbackupbandwidthblock sizeboundary shiftcascdccontent-addressable storagecontent-defined chunkingcontent-dependent chunkingcryptographic hash calculationcut-pointdeduplicationdifferential compressioneliminationfingerprintgear hashidenticalmaximumminimumnormalized chunkingpatternrabin karpratioreferencereduceredundancyredundantrsyncsliding blocksliding windowstoragetabulation hashvariable-sized chunking