Node.js File System (fs): Promises API, Streams, Atomic Writes, and Watching Files

By 0xmote · Jun 28, 2026 · 25 min read · Updated Sep 27, 2026 Intermediate Tutorial

Key takeaways

Node.js fs module guide: sync vs async APIs, fs.promises, read/write JSON, directories, streams, watch, chokidar, errors, and performance—essential for Express and CLI tools.

Introduction

What is the `fs` module?

fs is Node’s built-in API for files and directories. Capabilities:

Read/write files
Create/remove directories
stat metadata
Watch for changes
Stream large data Three styles: synchronous (*Sync), callback, and Promise (fs.promises) — prefer Promises with async/await for server code.

Why three styles for the same operations? The sync functions block the whole process until the disk answers, which is fine in a CLI script or at startup but stalls every other request in a server, since Node runs your JavaScript on a single thread. The callback and promise styles hand the work to libuv’s thread pool (four threads by default), so the event loop keeps serving other requests. The promise API, importable as require('fs/promises') or import fs from 'node:fs/promises', is the same functionality as the callbacks with better error handling through try/catch. One consequence of the thread pool is worth remembering: many slow file operations at once (for example on a network drive) can occupy all four threads, and then even DNS lookups and crypto calls queue behind them.

Reading files

Synchronous (blocking)

const fs = require('fs');
try {
    const data = fs.readFileSync('file.txt', 'utf8');
    console.log(data);
} catch (err) {
    console.error(err.message);
}

Avoid on request handlers—it blocks the event loop.

Callback

fs.readFile('file.txt', 'utf8', (err, data) => {
    if (err) return console.error(err.message);
    console.log(data);
});

Promises (recommended)

const fs = require('fs').promises;
async function readFile() {
    try {
        const data = await fs.readFile('file.txt', 'utf8');
        console.log(data);
    } catch (err) {
        if (err.code === 'ENOENT') console.error('Missing file');
        else if (err.code === 'EACCES') console.error('Permission denied');
        else console.error(err.message);
    }
}
readFile();

Encoding and buffers

const text = await fs.readFile('text.txt', 'utf8');
const buf = await fs.readFile('image.png'); // Buffer
const asString = buf.toString('utf8');
const base64 = buf.toString('base64');

Writing files

const fs = require('fs').promises;
await fs.writeFile('out.txt', 'Hello, Node.js!', 'utf8');
await fs.appendFile('out.txt', '\nMore', 'utf8');

JSON helpers

async function readJSON(file) {
    try {
        const data = await fs.readFile(file, 'utf8');
        return JSON.parse(data);
    } catch (err) {
        if (err.code === 'ENOENT') return null;
        throw err;
    }
}
async function writeJSON(file, obj) {
    await fs.writeFile(file, JSON.stringify(obj, null, 2), 'utf8');
}

writeFile is not atomic. It truncates the file and then writes the new content, so if the process crashes, is killed, or the disk fills up in between, the file is left empty or half-written, and the next readJSON fails with a JSON parse error. For configuration and state files, write to a temporary file in the same directory and rename it over the original. A rename within one filesystem is atomic, so readers always see either the complete old file or the complete new one:

const path = require('path');
async function writeJSONAtomic(file, obj) {
    const tmp = path.join(path.dirname(file), `.${path.basename(file)}.${process.pid}.tmp`);
    const handle = await fs.open(tmp, 'w');
    try {
        await handle.writeFile(JSON.stringify(obj, null, 2), 'utf8');
        await handle.sync();              // flush to disk before the rename
    } finally {
        await handle.close();
    }
    await fs.rename(tmp, file);           // atomic replace on the same filesystem
}

The temp file must be in the same directory (not /tmp), because rename across filesystems fails with EXDEV. The sync() call matters for durability after a power loss: without it, the operating system may write the rename to disk before the file contents, and you can still end up with an empty file after a crash. This is the file equivalent of a database transaction, and it is the pattern the “safe write” note in section 8 refers to.

File metadata and operations

const stats = await fs.stat('file.txt');
console.log(stats.size, stats.isFile(), stats.isDirectory());
await fs.copyFile('a.txt', 'b.txt');
await fs.rename('b.txt', 'c.txt');
await fs.unlink('c.txt');
await fs.chmod('file.txt', 0o755); // Unix

Existence check

async function exists(file) {
    try {
        await fs.access(file);
        return true;
    } catch {
        return false;
    }
}

Use this for reporting (“the config file is missing”), not as a guard before opening. Code like if (await exists(f)) data = await fs.readFile(f) has a race: the file can be deleted, created, or replaced between the check and the read, especially with other processes or a watcher involved. It is also two system calls where one would do. Just attempt the operation and handle the error code, as readJSON above does with ENOENT. For creating a file only if it does not exist yet, open it with the 'wx' flag, which fails with EEXIST atomically instead of silently overwriting. The old fs.exists() function was deprecated for exactly this reason.

Directories

await fs.mkdir('nested/path', { recursive: true });
const names = await fs.readdir('.');
const entries = await fs.readdir('.', { withFileTypes: true });
await fs.rm('dir', { recursive: true, force: true });

Recursive directory walk and findJSFiles patterns follow the idioms below.

Streams

readFile loads the whole file into memory, which is fine for configuration files and a problem for a 5 GB log. Streams process a file in chunks (64 KB by default for file streams):

const fs = require('fs');
const zlib = require('zlib');
const { pipeline } = require('stream/promises');

// Compress a large file without loading it into memory
await pipeline(
    fs.createReadStream('access.log'),
    zlib.createGzip(),
    fs.createWriteStream('access.log.gz')
);

Use pipeline rather than chaining .pipe() calls. With .pipe(), an error in one stream does not close the others, so a failed write leaves the read stream open and leaks a file descriptor, and you have to attach an error handler to every stream yourself. pipeline propagates errors, destroys every stream on failure, and returns a promise you can await. Streams also handle backpressure: if the write side (a slow disk, a network response) cannot keep up, the read side pauses instead of buffering everything in memory. See Node.js async programming for transform streams and the event model behind this.

Watching files

`fs.watch`

const watcher = fs.watch('dir', { recursive: true }, (event, filename) => {
    console.log(event, filename);
});

chokidar

npm install chokidar

const chokidar = require('chokidar');
// chokidar v4+ no longer accepts globs: watch a directory and filter instead
chokidar.watch('src', { ignored: (p, stats) => stats?.isFile() && !p.endsWith('.js') })
    .on('change', (path) => console.log('changed', path));

fs.watch is a thin layer over each operating system’s notification API (inotify on Linux, FSEvents on macOS, ReadDirectoryChangesW on Windows), and it inherits their differences. Events can arrive twice for one save, editors that save by writing a temp file and renaming it produce rename events instead of change, the filename argument can be missing on some platforms, and the recursive option only became available on Linux in Node 20. Watching files on network drives or inside Docker bind mounts from macOS or Windows often produces no events at all, because the change happens outside the kernel that is watching.

chokidar smooths over most of this: it normalizes events, handles editors’ atomic saves, and falls back to polling (usePolling: true) where native events do not work. Note that chokidar 4 (2024) removed glob support to cut dependencies, so tutorials using chokidar.watch('src/**/*.js') need the directory-plus-filter form above, or chokidar 3 if you pin the older version. Whatever you use, debounce the handler: one “save” can trigger several events within a few milliseconds.

Practical examples

JSON read/write (full flow)

const fs = require('fs').promises;
async function readJSON(filename) {
    try {
        const data = await fs.readFile(filename, 'utf8');
        return JSON.parse(data);
    } catch (err) {
        if (err.code === 'ENOENT') return null;
        throw err;
    }
}
async function writeJSON(filename, data) {
    await fs.writeFile(filename, JSON.stringify(data, null, 2), 'utf8');
}
async function main() {
    const users = [
        { id: 1, name: 'Alice', age: 25 },
        { id: 2, name: 'Bob', age: 30 }
    ];
    await writeJSON('users.json', users);
    const loaded = await readJSON('users.json');
    loaded[0].age = 26;
    await writeJSON('users.json', loaded);
}
main().catch(console.error);

Recursive directory walk

const fs = require('fs').promises;
const path = require('path');
async function walk(directory, onFile) {
    const entries = await fs.readdir(directory, { withFileTypes: true });
    for (const entry of entries) {
        const full = path.join(directory, entry.name);
        if (entry.isDirectory()) await walk(full, onFile);
        else await onFile(full);
    }
}
async function findJsFiles(root) {
    const out = [];
    await walk(root, async (file) => {
        if (path.extname(file) === '.js') out.push(file);
    });
    return out;
}

Backup helper (simplified)

class BackupManager {
    constructor(sourceDir, backupDir) {
        this.sourceDir = sourceDir;
        this.backupDir = backupDir;
    }
    async backup() {
        const fs = require('fs').promises;
        const path = require('path');
        await fs.mkdir(this.backupDir, { recursive: true });
        const stamp = new Date().toISOString().replace(/:/g, '-');
        const destDir = path.join(this.backupDir, `backup-${stamp}`);
        await fs.mkdir(destDir);
        const files = await fs.readdir(this.sourceDir);
        for (const name of files) {
            const src = path.join(this.sourceDir, name);
            const stat = await fs.stat(src);
            if (stat.isFile()) {
                await fs.copyFile(src, path.join(destDir, name));
            }
        }
        return destDir;
    }
}

CSV via readline (sketch)

const fs = require('fs');
const readline = require('readline');
async function parseCsv(filename) {
    const stream = fs.createReadStream(filename);
    const rl = readline.createInterface({ input: stream, crlfDelay: Infinity });
    const rows = [];
    let headers = [];
    let first = true;
    for await (const line of rl) {
        if (first) {
            headers = line.split(',');
            first = false;
            continue;
        }
        const cols = line.split(',');
        const obj = {};
        headers.forEach((h, i) => {
            obj[h.trim()] = (cols[i] || '').trim();
        });
        rows.push(obj);
    }
    return rows;
}

The split(',') here is a sketch, not a CSV parser. Real CSV allows commas and newlines inside quoted fields ("Smith, John"), escaped quotes (""), and a byte-order mark at the start of files exported from Excel, all of which this code gets wrong without any error. For anything beyond a controlled internal format, use a parser such as csv-parse, which also streams. The readline plus for await pattern is still the right tool for line-oriented formats like logs or JSON Lines.

The backup helper has a limitation worth noting too: it copies only the top level of the directory (stat.isFile() skips subdirectories). Node 16.7+ provides fs.cp(src, dest, { recursive: true }) for recursive copies, which is simpler and handles nested folders.

Error codes

Errors from fs carry a string code from the operating system, and checking it is how you decide whether a failure is expected:

Code	Meaning	Typical handling
`ENOENT`	No such file or directory	Missing input: return a default, or report a clear message
`EACCES` / `EPERM`	Permission denied / operation not permitted	Configuration problem: fail with the path in the message
`EEXIST`	Already exists	Expected with `'wx'` or `mkdir` without `recursive`
`EISDIR`	Expected a file, got a directory	Usually a path bug
`ENOTDIR`	A path component is not a directory	Usually a path bug
`EMFILE`	Too many open files	Too much concurrency, or leaked file handles
`ENOSPC`	No space left on device	Disk full: important to surface, not swallow
`EXDEV`	Cross-device link	`rename` across filesystems: copy and delete instead

EMFILE deserves a special mention. Processing thousands of files with Promise.all(files.map(readFile)) opens them all at once and hits the per-process file descriptor limit (often 1024 on Linux). Limit concurrency with a small pool (for example p-limit) instead. The graceful-fs package, which npm itself uses, retries on EMFILE automatically, but bounding concurrency is the real fix.

Safe write pattern (write temp then rename)

See writeJSONAtomic in section 2: write to a temporary file in the same directory, sync() it, and rename it over the target. That is the standard way to avoid half-written files after a crash.

In my experience, the file bugs that reach production are rarely about the API itself. They are a config file that becomes empty after a server was killed mid-write, a Promise.all over a directory that works on a laptop and fails with EMFILE on the full dataset, and watch-based reload that never fires inside a container. All three have simple, well-known fixes (atomic writes, bounded concurrency, polling or a proper watcher), which is why they are worth knowing before they happen.

Performance

Tune highWaterMark on streams
Prefer Promise.all for independent reads
Stream line counts instead of loading multi-GB files fully

Common pitfalls

Use path.join(__dirname, 'file.txt') instead of fragile relative paths: relative paths resolve against the process’s working directory, not the script’s location. In ES modules, __dirname does not exist; use import.meta.dirname (Node 20.11+)
Pass 'utf8' when you expect a string
Avoid readFileSync in Express handlers
Close streams or use stream.pipeline / util.promisify(pipeline)

Next in the series

The API reference pages for what this post used are fs, path and stream.