Download remote URL files using Node JS, wget

Last updated on November 21, 2022

In this post, I would like to show you downloading files using node js and wget. We gonna use URL, child_process and path modules to achieve this. Just go through the comments for a better understanding.

You can download files from remote sources in several ways, but I prefer using wget the utility because it’s efficient and reliable. wget is a free utility for the non-interactive download of files from the web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

Below is the function to download remote files by using the HTTP protocol.

const url = require('url');
const child_process = require('child_process');
const path = require('path')


function download_file_with_wget(file_url, DOWNLOAD_DIR, DOWNLOADABLE_EXTENTIONS) {


    return new Promise((resolve, reject) => {

        DOWNLOAD_DIR = DOWNLOAD_DIR || '.';

        DOWNLOADABLE_EXTENTIONS = DOWNLOADABLE_EXTENTIONS || ['.zip'];

        // extract the file name
        const file_name = url.parse(file_url).pathname.split('/').pop()

        // get the file extention
        const file_extention = path.extname(file_name)

        // allow only .zip files to download
        if (!DOWNLOADABLE_EXTENTIONS.includes(file_extention.toLowerCase())) {
            return reject('Invalid file..')
        }

        // compose the wget command
        const wget = 'wget -P ' + DOWNLOAD_DIR + ' ' + file_url

        // excute wget using child_process' exec function
        child_process.exec(wget, function (err, stdout, stderr) {
            return err ? reject(err) : resolve(file_name)
        });
    })

};

How to use

download_file_with_wget('https://github.com/bcit-ci/CodeIgniter/archive/develop.zip').then(file=> console.log(file)).catch(e => console.log(e));