A smarter cleanup process on a working directory

Published on: 5th July 2024

Overview

It is common for a system to generate some temporary file, keep it in a working directory and finally send it to the user. These temporary files might be alive for 10 minutes before we delete it.In such a case, we need to clean up the working directory periodically by having a timer to run the cleanup process every 10 minutes. But, there is a concern - the cleanup process will be called periodically regardless if the temporary file exists or not. This is wasting the server resource by running a process that does not have any effect. A better solution is to create the timer to cleanup the temporary file and stop the timer when there are no more user activities.

Let's write the code

This solution is simple and straightforward. First, we need to include the file system package in this cleanup process.

const fs = require('fs');

And then we need some variables to keep track of the smarter cleanup process.

const my_var = {
	last_access_on: null,
	cleanup_timer: null,
	watch_tmp_dir_timer: null,
};

The usage of the variables:

• The last last_access_on - is to record down the latest time of creating the temporary file.
• The cleanup_timer is the one responsible for deleting the temporary file that has been alive for more than 10 minutes (or whatever duration that you like).
• The watch_tmp_dir_timer - we need another timer to monitor the last access to the working directory. If there are no changes for more than x minutes, it will shut down the cleanup_timer and itself.

Here's the code that watches the working directory and creates cleanup_timer and watch_tmp_dir_timer. In this code, we will run the cleanup process every 3 seconds so that you can see the effect when you are debugging the program.

fs.watch('d:\\temp\\',
	function(eventType, filename) {
    	my_var.last_access_on = new Date();

    	if (!my_var.watch_tmp_dir_timer) {
        	console.log('cleanup process start');

        	//activate scheduled cleanup job
        	my_var.cleanup_timer = setInterval(() => {
            	// you need to write the file deletion logic here.
            	console.log('delete the old files..', new Date());
        	}, 3000);

        	my_var.watch_tmp_dir_timer = setInterval(() => {
            	const diff_ms = (new Date()).getTime() - my_var.last_access_on.getTime();

            	// stop the cleanup timer after 10 seconds if there is no new file that needs to be cleaned.
            	if (diff_ms > 10000) {
                	//deactivated scheduled cleanup job
                	clearInterval(my_var.cleanup_timer);
                	my_var.cleanup_timer = null;

                	// disable the file watcher
                	clearInterval(my_var.watch_tmp_dir_timer);
                	my_var.watch_tmp_dir_timer = null;

                	console.log('shut down cleanup process');
            	}

        	}, 1000);
    	}
	});

Here's the cleanup process that is running on an interval of 3 seconds and stops after 10 seconds when no new file has been generated.

/*
output:

cleanup process start
delete the old files.. 2023-10-20T07:26:46.789Z
delete the old files.. 2023-10-20T07:26:49.791Z
delete the old files.. 2023-10-20T07:26:52.791Z
shut down cleanup process

*/

Conclusion

You may not see much savings on a not so busy server. But, just a small tweak to the cleanup process will definitely save some of the server resources and reduce the unnecessary disk IO. This is helpful if you are running your program on the same server as your database which generates a huge amount of disk IO.

Jump to #NODEJS blog

Author

Lau Hon Wan, software developer.