Parallel Compression

How to use multi-core CPU to acclerate the tar

1
2
3
tar cf - paths-to-archive | pigz > archive.tar.gz
tar -c --use-compress-program=pigz -f tar.file dir_to_zip 

for each dir create a tar file

1
2
~/VRust$ for i in *; do tar -c --use-compress-program=pigz -f ../VRust_Compress/$i.tar.gz $i; done

Ref: https://stackoverflow.com/questions/15936003/for-each-dir-create-a-tar-file

78

The script that you wrote will not work if you have some spaces in a directory name, because the name will be split, and also it will tar files if they exist on this level.

You can use this command to list directories not recursively:

1
find . -maxdepth 1 -mindepth 1 -type d

and this one to perform a tar on each one:

1
2
find . -maxdepth 1 -mindepth 1 -type d -exec tar cvf {}.tar {}  \;