Node.js Async Programming: Callbacks and Promises

Key takeaways

Learn Node.js async I/O: callbacks, error-first style, Promises, async/await, the event loop, streams, and patterns for APIs and file pipelines—essential for Express and production Node apps.

Introduction

What is asynchronous programming?

Asynchronous code does not block the caller while waiting for I/O; other work can proceed.

console.log('1');
setTimeout(() => console.log('2'), 0);
console.log('3');
// 1, 3, then 2

Why Node uses async:

  • Non-blocking I/O keeps the server responsive under load
  • High concurrency on a single thread for I/O-bound workloads
  • Less CPU wasted waiting on disks and networks Other ecosystems: compare with C++ Asio async I/O, Rust concurrency, and browser async JavaScript.

The setTimeout(..., 0) example shows the core rule: a callback never interrupts running JavaScript. It is queued, and it runs only after the current synchronous code — the whole call stack — has finished. That is also Node’s biggest limitation. Your JavaScript runs on one thread, so a CPU-heavy loop (parsing a 200 MB JSON string, a synchronous bcrypt hash, a badly written regex) blocks every other request on that process until it finishes; the “async” part only helps while waiting for I/O. Offload real CPU work to worker_threads, a separate process, or a native module that runs off the main thread.

Behind the scenes, network sockets are handled by the operating system’s readiness mechanisms (epoll, kqueue, IOCP), while file system calls, dns.lookup, some crypto functions, and zlib run on libuv’s thread pool — four threads by default (UV_THREADPOOL_SIZE). A service that does many slow file reads or crypto.pbkdf2 calls in parallel can therefore queue up behind those four threads even though the main thread is idle, a bottleneck that is invisible in CPU graphs.


Callbacks

Error-first callbacks

const fs = require('fs');
fs.readFile('file.txt', 'utf8', (err, data) => {
    if (err) {
        console.error(err.message);
        return;
    }
    console.log(data);
});

Callback hell

Chaining multiple async steps nests callbacks and duplicates error handling:

const fs = require('fs');
fs.readFile('file1.txt', 'utf8', (err1, data1) => {
    if (err1) {
        console.error(err1);
        return;
    }
    fs.readFile('file2.txt', 'utf8', (err2, data2) => {
        if (err2) {
            console.error(err2);
            return;
        }
        fs.readFile('file3.txt', 'utf8', (err3, data3) => {
            if (err3) {
                console.error(err3);
                return;
            }
            console.log(data1, data2, data3);
        });
    });
});

Issues: readability, repeated if (err), hard debugging. Fix: Promises or async/await.

The error-first convention exists because exceptions do not work across asynchronous boundaries. By the time readFile’s callback runs, the function that called readFile has long returned, so a try/catch around the readFile(...) call cannot catch anything that goes wrong later. Errors therefore travel as the first argument, and every callback must check it — forgetting the return after handling an error is a classic bug, because the code then continues with data === undefined. The other rule callback APIs must follow is to be consistently asynchronous: a function that sometimes calls its callback synchronously (for example, from a cache) and sometimes later makes callers’ code run in an unpredictable order, a problem Node’s documentation calls “releasing Zalgo”.


Promises

Creating a Promise

function delay(ms) {
    return new Promise((resolve, reject) => {
        if (ms < 0) {
            reject(new Error('ms must be non-negative'));
            return;
        }
        setTimeout(() => resolve(`${ms}ms done`), ms);
    });
}
delay(1000)
    .then((result) => console.log(result))
    .catch((err) => console.error(err.message));

Chaining

const fs = require('fs').promises;
fs.readFile('file1.txt', 'utf8')
    .then((data1) => {
        console.log('file1:', data1);
        return fs.readFile('file2.txt', 'utf8');
    })
    .then((data2) => {
        console.log('file2:', data2);
        return fs.readFile('file3.txt', 'utf8');
    })
    .then((data3) => console.log('file3:', data3))
    .catch((err) => console.error('Error:', err.message))
    .finally(() => console.log('done'));

A Promise is an object representing a value that will exist later, and it settles exactly once — fulfilled or rejected. Chaining works because .then() always returns a new Promise: returning a value from the callback fulfills it, returning a Promise makes the chain wait for that Promise, and throwing rejects it. That is why a single .catch() at the end handles a failure in any step. The most frequent chaining bug is forgetting the return inside a .then() callback: the inner readFile still runs, but the chain does not wait for it, the next step receives undefined, and an error in the inner Promise is not caught by the final .catch().

Promise callbacks also run asynchronously even when the Promise is already resolved. Promise.resolve(1).then(f) schedules f as a microtask rather than calling it immediately, which gives Promises the consistent ordering that callback APIs had to enforce by convention.

Static helpers

Promise.all:

const fs = require('fs').promises;
Promise.all([
    fs.readFile('file1.txt', 'utf8'),
    fs.readFile('file2.txt', 'utf8'),
    fs.readFile('file3.txt', 'utf8')
])
    .then(([a, b, c]) => console.log(a, b, c))
    .catch((err) => console.error(err.message));

Promise.allSettled:

Promise.allSettled([
    fs.readFile('file1.txt', 'utf8'),
    fs.readFile('file2.txt', 'utf8'),
    fs.readFile('missing.txt', 'utf8')
]).then((results) => {
    results.forEach((r, i) => {
        if (r.status === 'fulfilled') console.log(i, r.value);
        else console.log(i, r.reason.message);
    });
});

Promise.race:

Promise.race([
    delay(1000).then(() => '1s'),
    delay(2000).then(() => '2s'),
    delay(500).then(() => '0.5s')
]).then((result) => console.log('Fastest:', result));

Promise.any:

Promise.any([
    Promise.reject('e1'),
    Promise.reject('e2'),
    Promise.resolve('ok')
]).then(console.log).catch(console.error);

The four combinators differ in when they settle. Promise.all rejects as soon as any input rejects (fail-fast) and otherwise resolves with all results in input order; allSettled always waits for everything and reports each outcome; race settles with whichever input settles first, success or failure; any resolves with the first success and rejects only if all inputs reject, with an AggregateError whose .errors array holds every reason. A detail that matters for all: when one input rejects, the others keep running — nothing is cancelled — and their later results or errors are simply ignored. If those operations have side effects (writes, payments), fail-fast does not mean “nothing else happened”.


Async/await

async functions always return a Promise. await pauses only the async function until the Promise settles; the event loop can still run other work.

const fs = require('fs').promises;
async function readFiles() {
    try {
        const data1 = await fs.readFile('file1.txt', 'utf8');
        const data2 = await fs.readFile('file2.txt', 'utf8');
        const data3 = await fs.readFile('file3.txt', 'utf8');
        console.log(data1, data2, data3);
        return 'all read';
    } catch (err) {
        console.error('Error:', err.message);
        throw err;
    }
}
readFiles()
    .then((msg) => console.log(msg))
    .catch((err) => console.error('Final error:', err.message));

Sequential vs parallel

Sequential (slower):

async function sequential() {
    const start = Date.now();
    await delay(1000);
    await delay(1000);
    await delay(1000);
    console.log(Date.now() - start); // ~3000ms
}

Parallel (faster):

async function parallel() {
    const start = Date.now();
    await Promise.all([delay(1000), delay(1000), delay(1000)]);
    console.log(Date.now() - start); // ~1000ms
}

The difference is when each operation starts. A Promise-returning function starts its work immediately when called; await only waits for the result. In sequential, each delay is called only after the previous await finished. In parallel, all three calls happen first and the await waits for the group. The same effect is available without Promise.all by starting operations before awaiting them (const a = delay(1000), b = delay(1000); await a; await b;), but that form has a trap: if b rejects while the code is still awaiting a, Node sees a rejection with no handler yet and may report it as unhandled. Promise.all attaches handlers to all inputs at once, which is why it is the safer way to express “run these in parallel”.

Sequential awaits are correct when each step depends on the previous one (log in, then fetch the profile with the token). A common performance bug in code review is the unintentional version: independent database queries awaited one after another in a request handler, turning three 20 ms queries into a 60 ms response.


Event loop (overview)

   ┌───────────────────────────┐
┌─>│           timers          │  setTimeout, setInterval
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │     pending callbacks     │  I/O callbacks
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │       idle, prepare       │
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │           poll            │
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │           check           │  setImmediate
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
└──│      close callbacks      │
   └───────────────────────────┘

Typical ordering example:

console.log('1. sync');
setTimeout(() => console.log('4. setTimeout'), 0);
setImmediate(() => console.log('5. setImmediate'));
Promise.resolve().then(() => console.log('3. Promise microtask'));
console.log('2. sync');
process.nextTick(() => console.log('3b. nextTick'));

Rule of thumb: run sync code first, then nextTick, then other microtasks (Promises), then timers / I/O / setImmediate depending on context. Avoid starving the loop with excessive nextTick.

Run as a CommonJS script, this prints 1. sync, 2. sync, 3b. nextTick, 3. Promise microtask, and then 4. setTimeout and 5. setImmediate — in that order usually, but not guaranteed. When both are scheduled from the main module, whether the 0 ms timer is already due on the first loop iteration depends on how long process startup took, so the two can swap between runs. Inside an I/O callback (for example in a readFile callback), setImmediate always runs first, because the check phase directly follows the poll phase. The labels “3b” and “3” reflect that process.nextTick callbacks drain before Promise microtasks, even though nextTick was called later in the code. (In an ES module, the whole module body already runs inside a microtask-like context, and Promise callbacks can come before nextTick ones — another reason not to depend on this ordering.)

The practical lessons are about starvation. Microtasks and nextTick callbacks are drained completely before the loop moves on, so a Promise chain or nextTick recursion that keeps scheduling more work never lets timers or I/O run: the server stops answering requests while the CPU sits at 100%. Breaking long CPU work into chunks with setImmediate yields to the event loop between chunks; nextTick and await Promise.resolve() do not.


util.promisify

Wrap callback-style APIs:

const fs = require('fs');
const util = require('util');
const readFile = util.promisify(fs.readFile);
async function main() {
    const data = await readFile('file.txt', 'utf8');
    console.log(data);
}
main();

promisify works for any function that follows the error-first convention with the callback as the last argument. For Node’s own modules it is rarely needed now: require('fs/promises'), require('timers/promises') (await setTimeout(100)), require('stream/promises'), and dns.promises expose Promise-based versions directly. Watch out for callbacks that pass more than one result value, such as the legacy dns.lookup(host, cb(err, address, family)) — plain promisify keeps only the first value unless the function defines a custom util.promisify.custom implementation, which Node’s built-ins do in those cases. Also, main() at the bottom is called without .catch(); if readFile fails, the rejection is unhandled, which on current Node versions terminates the process (see the pitfalls section).


Streams

Streams process data in chunks instead of loading entire files into memory.

const fs = require('fs');
const readStream = fs.createReadStream('large-file.txt', {
    encoding: 'utf8',
    highWaterMark: 64 * 1024
});
readStream.on('data', (chunk) => console.log('chunk', chunk.length));
readStream.on('end', () => console.log('done'));
readStream.on('error', (err) => console.error(err.message));
const writeStream = fs.createWriteStream('output.txt');
writeStream.write('line1\n');
writeStream.end('last\n');
writeStream.on('finish', () => console.log('written'));

Pipe and gzip:

const fs = require('fs');
const zlib = require('zlib');
fs.createReadStream('input.txt')
    .pipe(zlib.createGzip())
    .pipe(fs.createWriteStream('input.txt.gz'));

Streams exist for two reasons: memory and backpressure. Reading a 5 GB log with readFile needs 5 GB of memory; a stream holds only highWaterMark bytes at a time. Backpressure is what happens when the consumer is slower than the producer. writeStream.write() returns false when its internal buffer is full, and a well-behaved producer should stop writing until the 'drain' event. .pipe() handles this for you — it pauses the source when the destination is full — which is why piping is preferred over hand-written 'data' handlers that call write() and ignore its return value, a pattern that works in testing and exhausts memory when the destination (a slow disk, a slow client) cannot keep up.

.pipe() has one serious flaw: it does not forward errors. If the input file does not exist, the read stream emits 'error', the gzip and write streams are never closed, and an unhandled 'error' event crashes the process. stream.pipeline fixes both problems — it propagates errors, destroys every stream in the chain on failure, and tells you when everything finished:

const { pipeline } = require('stream/promises');

await pipeline(
    fs.createReadStream('input.txt'),
    zlib.createGzip(),
    fs.createWriteStream('input.txt.gz')
);   // throws if any stage fails; all streams are cleaned up

Transform:

const fs = require('fs');
const { Transform } = require('stream');
class UpperCaseTransform extends Transform {
    _transform(chunk, encoding, callback) {
        this.push(chunk.toString().toUpperCase());
        callback();
    }
}
fs.createReadStream('input.txt')
    .pipe(new UpperCaseTransform())
    .pipe(fs.createWriteStream('output.txt'));

_transform receives raw Buffer chunks cut at arbitrary byte boundaries (64 KB by default for file streams). chunk.toString() on each chunk is therefore subtly wrong for UTF-8 text: a multi-byte character such as é or any Korean syllable can be split across two chunks, and each half decodes to the replacement character �. The bug appears only in large files with non-ASCII text, exactly where it is hardest to notice. Call setEncoding('utf8') on the source stream (the stream then decodes with a StringDecoder that carries partial characters over), or use string_decoder inside the transform. Line-oriented processing has the same issue with lines split across chunks; readline.createInterface({ input }) handles that for text files.


Practical examples

Fetch with retry and timeout

Combining Promise.race (for a timeout) with a retry loop is a pattern worth having ready for any network call that talks to a flaky or rate-limited service:

function timeout(ms) {
    return new Promise((_, reject) =>
        setTimeout(() => reject(new Error('timeout')), ms)
    );
}

async function fetchWithRetry(url, { retries = 3, timeoutMs = 2000 } = {}) {
    for (let attempt = 1; attempt <= retries; attempt++) {
        try {
            return await Promise.race([fetch(url), timeout(timeoutMs)]);
        } catch (err) {
            if (attempt === retries) throw err;
            await delay(2 ** attempt * 100); // exponential backoff
        }
    }
}

Promise.race here doesn’t actually cancel the underlying fetch when the timeout wins — the real request keeps running in the background even after this function has thrown a timeout error and moved on to a retry. That’s a real limitation worth knowing: for genuinely cancellable requests, pair fetch with an AbortController instead of relying on Promise.race alone, so the abandoned request is actually torn down rather than left to complete unobserved.

On Node 18+ the built-in way is a single option: fetch(url, { signal: AbortSignal.timeout(timeoutMs) }) aborts the request and rejects with a TimeoutError DOMException. It also fixes a second, quieter problem in the version above: the setTimeout inside timeout() is never cleared, so even when fetch wins, a pending timer stays alive for the full timeoutMs, which delays process exit in scripts and tests. Two more details matter for retries. fetch resolves normally for HTTP errors — a 503 is a successful Promise with response.ok === false — so this loop never retries server errors unless you throw when !response.ok. And only retry idempotent requests: retrying a POST that timed out may create the order twice, because the first request may have succeeded after all. Adding random jitter to the backoff (delay(2 ** attempt * 100 * (0.5 + Math.random()))) keeps many clients from retrying in lockstep after an outage.

Limiting concurrency

Firing off Promise.all over a thousand URLs at once can overwhelm the target server or your own outbound connection limit. A simple concurrency-limited runner processes a fixed-size window of work at a time:

async function mapWithConcurrency(items, limit, fn) {
    const results = [];
    let index = 0;

    async function worker() {
        while (index < items.length) {
            const current = index++;
            results[current] = await fn(items[current]);
        }
    }

    await Promise.all(Array.from({ length: limit }, worker));
    return results;
}

Each worker shares the same index counter, pulling the next unclaimed item as soon as it finishes its current one — this is the same shape a thread pool uses, just implemented with async/await instead of OS threads, and it keeps exactly limit requests in flight at any moment regardless of how many total items there are.

The shared index++ is safe without any lock precisely because JavaScript is single-threaded: the read-and-increment runs synchronously between two awaits, so no other worker can interleave with it. The same code with real threads would need an atomic counter. Results land at results[current], so they come back in input order even though items finish out of order. Error behavior follows Promise.all: the first failing fn rejects the whole call, while the other workers keep going with their current items. If partial results are useful, catch inside worker and store an error marker instead. Libraries such as p-limit implement the same idea with a slightly different API.


Common pitfalls

Unhandled rejections

Always await inside try/catch or attach .catch(), and consider:

process.on('unhandledRejection', (reason) => {
    console.error('Unhandled rejection:', reason);
});

Since Node.js 15, an unhandled rejection terminates the process by default (--unhandled-rejections=throw), printing the error and exiting with a non-zero code. Older versions only printed a warning, so code upgraded from Node 12 or 14 sometimes starts crashing on errors that were silently ignored for years. Registering the handler above suppresses the crash, which can hide real bugs; in servers, the usual policy is to log the error with context and then exit deliberately (letting the process manager restart it), since the application may be in an inconsistent state. The real fix is at the source: every Promise needs someone who awaits it or attaches .catch(), including “fire-and-forget” calls like sendAnalytics() that nobody awaits.

Missing await

// Wrong: data is a Promise
const data = fetchData();
// Right
const data = await fetchData();

A missing await rarely throws. The code continues with a pending Promise object, so data.length is undefined, if (user) is always true (a Promise is truthy), and JSON.stringify(data) produces {}. TypeScript and the ESLint rules @typescript-eslint/no-floating-promises and require-await catch most of these at build time, which is the cheapest place to find them.

forEach does not await

Use for...of with await, or Promise.all with map.

array.forEach(async (item) => { await save(item); }) starts every save call at once and returns immediately; forEach ignores the Promises its callback returns. The code after the loop runs before any save has finished, and a failing save becomes an unhandled rejection. for (const item of items) await save(item); processes items one at a time; await Promise.all(items.map(save)) runs them all in parallel and waits; for large arrays, the concurrency-limited runner above sits between the two.