How to Delete A Large Directory with Thousands of Files in Linux

Deleting a large directory with thousands of files in Linux can be slow or even fail with basic commands. Whether you’re cleaning up old logs, project files, or datasets, knowing the right commands makes the process faster and more reliable.

In this guide, we’ll show you how to delete directories with thousands of files in Linux using several methods – from rm and find to faster alternatives like rsync. You’ll learn which approach is best for different scenarios, how to avoid common errors, and how to free up disk space safely.

Why Would You Need to Delete Thousands of Files at Once in Linux?

If you use a computer professionally, you may have stored many files on your hard drive over time. These files can include documents, photos, videos, music, and more. Some of these files may be important and useful, while others may be outdated and unnecessary.

Here are some possible scenarios:

• You are a web developer and you have a project folder that contains thousands of files, such as HTML, CSS, JavaScript, images, etc. You want to delete the project folder because you no longer need it or you want to start from scratch.
• You are a data analyst and you have a directory that contains thousands of CSV files, each containing data that you have processed or analyzed. You want to delete the directory because you have finished your analysis, and you want to free up some disk space.
• You are a system administrator and you have a directory that contains thousands of log files, each recording system events or errors. You want to delete the directory because you have resolved the issues. If you want to clear the logs periodically then you should also read our article on log-rotation.
• You are a photographer and you have a directory that contains thousands of RAW images, each capturing moments that you have taken with your camera. You want to delete the directory because you have edited or backed up the images, or you want to make room for new photos.

Having too many files can affect your computer’s performance and disk space. Therefore, it is advisable to periodically delete the files that you no longer need or want.

How Do You Delete a Directory in Linux?

There are multiple ways to delete a file or folder in Linux. Let’s take a look at some of these ways.

However, be careful when using these commands, as they are irreversible and may delete important files if used incorrectly.

Always double-check the path and the options before executing them.

How to Delete a Large Directory in Linux with rm

The rm command is one of the most common and basic commands for deleting files and directories in Linux. To delete a directory and all its contents, you can use the -r option, which stands for recursive.

For example, if you want to delete a directory named project, you can use the command:

rm -r project

However, this command may take a long time and generate a lot of output if there are too many files in the directory. To speed up the process and suppress the output, you can use the -f option, which stands for force.

This option will delete the files and directories without prompting for confirmation or showing any messages.

rm -rf project

How to Delete Files and Directories in Linux with find

Alternatively, you can use the find command, which is a powerful and flexible command for finding and manipulating files and directories in Linux.

To delete a directory and all its contents, you can use the -delete option, which will find and delete all the files and directories under a given path. For example:

find project -delete

The find command also has many other options that allow you to selectively delete files based on different criteria, such as pattern, date, time, or size.

How to Use rsync to Delete a Directory in Linux

The rsync command is generally used to transfer and synchronize files between local and remote devices in an efficient way. It uses a special algorithm that only sends the differences between the source and destination files, which reduces the network usage and speeds up the transfers.

This can be used for various scenarios, such as backup, mirroring, updating, copying, or even deleting files and directories. To delete a folder using rsync, you can use the following simple command:

rsync -a --delete source/ destination/

This command will sync the contents of the source folder to the destination folder, and delete any files or subfolders in the destination that do not exist in the source. The options used in this command are:

• -a archive mode, which preserves almost everything (such as symbolic links, file permissions, user & group ownership, and timestamps).
• --delete delete mode, which deletes extraneous files from the destination location.

Note that you need to add a trailing slash (/) after the source and destination folder names, otherwise rsync will treat them as file names and create a subfolder in the destination.

For example, if you use rsync -a --delete source destination, rsync will create a subfolder named source in the destination and sync the contents of the source folder to it.

How to Securely Delete Files in Linux with shred

Shred is used to permanently erase or destroy files so that no one can recover them. It is better than deleting normally, because a normal deletion only removes the reference to the file in the file system, but the actual data remains on the disk until it is overwritten by other data.

A deleted file can be recovered using special software or hardware tools that can scan the disk for traces of data. Shred prevents this by overwriting the file multiple times with random data, making it impossible to reconstruct the original data.

Shred also adds a final overwrite with zeros to hide the fact that the file was shredded. Therefore, shred is more secure and reliable than deleting normally, especially for sensitive or confidential files. 

To shred a document in Linux, you can use the following simple command:

shred -uvfz document

This command will overwrite the data in the document file several times, making it harder for third party software and hardware probing to recover the data. The options used in this command are:

• -u flag will remove the file or directory after overwriting it with random data. 
• -v verbose mode, which shows information on shredded files.
• -f force mode, which changes permissions to allow writing if necessary.
• -z zero mode, which adds a final overwrite with zeros to hide shredding.

The shred command can only be used to delete files and not directories. However, it is possible to use it in conjunction with other commands to create automated scripts that shred entire directories.

For example, you can use the ‘find’ command with the -exec option to execute the shred command on each file found by ‘find’. Here is an example of how to shred and delete all the files in a directory named ‘secret’:

find secret -type f -exec shred -u {} ;

This command will find all the files in the secret directory and execute the shred command with the -u option on each of them.

However, be aware that the shred command relies on the assumption that the underlying file system overwrites the same physical block when writing new data. Many newer file systems do not follow this assumption, and may use techniques such as journaling, copy-on-write, or wear leveling, which may prevent the shred command from effectively destroying the data. Therefore, shred may not work as expected on some file systems, such as ext3, ext4, btrfs, xfs, etc. 

How to Selectively Delete Files in Linux

There are also a number of different ways in which it is possible to selectively delete files in Linux, depending on the criteria that you want to use. Here are some examples:

How to Delete Files by Pattern in Linux

You can use the rm command with a wildcard character (*), which matches any number of characters. This way, you can delete files that match a certain pattern, such as files with a specific extension, name, or prefix.

For example, if you want to delete all the files that have the .txt extension in the current directory, you can use the command:

rm *.txt

If you want to delete all the files that start with the prefix test in the current directory, you can use the command:

rm test*

How to Delete Files by Date or Time in Linux

You can use the find command with various options that locate files based on their modification (-mtime or -mmin), change (-ctime or -cmin), or access (-atime or -amin) time.

These options take a number as an argument, which can be preceded by either a plus sign (+) or a minus sign (-). A plus sign means more than the given number, while a minus sign means less than the given number.

The number can be either days (-mtime or -ctime) or minutes (-mmin, -cmin, or -amin).

For example, if you want to delete all the files that were modified more than 10 days ago in the current directory, you can use the command:

find . -mtime +10 -delete

If you want to delete all the files that were accessed less than 30 minutes ago in the current directory, you can use the command:

find . -amin -30 -delete

How to Delete Files by Size in Linux

You can use the find command with the -size option, which finds files based on their size.

This option takes a number as an argument, which can be followed by a suffix that indicates the unit of measurement. The suffix can be either bytes (b), kilobytes (k), megabytes (M), gigabytes (G), or blocks (c).

A plus sign (+) or a minus sign (-) before the number means larger than or smaller than respectively.

For example, if you want to delete all the files that are larger than 100 MB in the current directory, you can use the command:

find . -size +100M -delete

If you want to delete all the files that are smaller than 1 KB in the current directory, you can use the command:

find . -size -1k -delete

Key Takeaways on Deleting Large Directories in Linux

Deleting files in Linux may seem like a simple action, but there are lots of hidden nuances and challenges that you may encounter. In this article, we have explained how to delete directories with a large number of files in Linux using a variety of different commands and options.

We have also shown you how to selectively delete files based on different criteria, such as pattern, date, time, or size. However, you need to be careful when using these commands, as they are irreversible and may delete important files if used incorrectly. Always double-check the path and the options before executing them. 

If you are struggling with the Linux command line, then you should take a look at RunCloud. RunCloud allows you to manage your sites effectively without needing to become a Linux system administrator. You can easily create, deploy, and update your websites with just a few clicks.

RunCloud supports various web servers, such as Apache, Nginx, and LiteSpeed. You can also switch between different PHP versions with the click of a button. RunCloud works with any cloud provider, such as AWS, Google Cloud, DigitalOcean, and more. Sign up to RunCloud today and forget all the hassle!

FAQs on Deleting Large Directories and Files in Linux