C++ Segmentation Fault & Core Dump: GDB/LLDB Debugging Guide [#49-1]

Introduction: When you see “Segmentation fault (core dumped)“

For readers who landed here from a search

A segmentation fault means the OS terminated your process after invalid memory access: null dereference, dangling pointers, stack overflow, buffer overrun, etc. This article walks through enabling core dumps, loading them in GDB/LLDB, reading backtraces, and using AddressSanitizer (ASan) to catch issues earlier.

Topics covered:

• Core dumps: ulimit, systemd/kernel settings
• Post-mortem analysis: load core in GDB/LLDB, bt, frame navigation, variables
• Common causes: null deref, use-after-free, stack overflow
• ASan for proactive detection
• Production: automated core collection and analysis

See also: GDB/LLDB, AddressSanitizer.

---

Experience from real projects

On large C++ codebases, book theory rarely matches production crashes. Profiling and code review helped turn vague failures into concrete fixes. The patterns below are meant to shorten your own trial-and-error loop.

---

Problem scenarios

Scenario 1: Occasional crashes in production

Logs show only Segmentation fault (core dumped) and no core file.

Cause: ulimit -c is 0, core_pattern discards cores, or systemd limits cores.

Fix: Enable cores → collect with coredumpctl or core_pattern → gdb ./program core and bt.

Scenario 2: Crash inside a third-party library

Cause: Bad pointer arguments, size/type mismatch, use-after-free across the FFI boundary.

Fix: bt full, move to caller frames with frame N, inspect arguments; info sharedlibrary for symbols.

Scenario 3: Recursion or huge stack objects

Cause: Default stack size (~8 MB) exceeded.

Fix: Repeated frames in bt → recursion; ulimit -s or heap allocation.

Scenario 4: Buffer overrun only on certain inputs

Fix: Re-run under ASan or Valgrind for exact line and size.

Summary

Scenario	Hint	Approach
Intermittent prod crash	No core	ulimit, coredumpctl
Crash in .so	Trace callers	bt full, inspect args
Stack overflow	Deep bt	ulimit -s or heap
Input-dependent	Bounds	ASan / Valgrind

---

Segfault debugging flow

flowchart TB
    subgraph Detect["Segfault"]
        A[Segmentation fault] --> B{Core file?}
    end

    subgraph NoCore["No core"]
        B -->|No| C[ulimit -c unlimited]
        C --> D[Check kernel.core_pattern]
        D --> E[Reproduce]
    end

    subgraph WithCore["Core available"]
        B -->|Yes| F[gdb ./program core]
        F --> G[bt backtrace]
        G --> H[frame 0]
        H --> I[print variables]
        I --> J{Root cause?}
    end

    subgraph Prevent["Prevention"]
        K[ASan build] --> L[Run tests]
        L --> M[Fix on first report]
    end

    E --> F
    J -->|Reproducible| K

---

Table of contents

1. Enabling core dumps
2. Analyzing cores with GDB/LLDB
3. Complete debugging examples
4. Common causes
5. Debugging strategies
6. Defensive coding
7. ASan
8. Production patterns
9. Common tool errors
10. FAQ

---

1. Enabling core dumps

ulimit

• ulimit -c: if 0, no core file is written.
• ulimit -c unlimited: in the shell session that runs the binary.
• For persistence: /etc/security/limits.conf or shell rc files.

ulimit -c
ulimit -c unlimited
echo "ulimit -c unlimited" >> ~/.zshrc

Location and naming

• Often core or core.<pid> in the process CWD.
• /proc/sys/kernel/core_pattern: e.g. core.%e.%p.%t.

cat /proc/sys/kernel/core_pattern
sysctl kernel.core_pattern
echo "core.%e.%p.%t" | sudo tee /proc/sys/kernel/core_pattern

systemd

• Set DefaultLimitCORE=infinity (and service LimitCORE= as needed).
• coredumpctl list, coredumpctl debug <pid> or time range.

# /etc/systemd/system/myapp.service
[Service]
DefaultLimitCORE=infinity

---

2. GDB/LLDB core analysis

Loading a core

Use the same binary build that produced the crash, preferably with -g.

gdb ./your_program core
lldb -c core ./your_program

Backtrace

• bt: call stack; #0 is the faulting frame.
• bt full: locals per frame (GDB); bt -f (LLDB).

Frames and variables

• frame 0, list, print ptr
• print *ptr may fail if the address is invalid in the dump.

Task	GDB	LLDB
Backtrace	bt	bt
Locals	bt full	bt -f
Select frame	frame N	frame select N
Print	print x	frame variable x
Disassembly	disas	disassemble

---

3. Complete examples

Example 1: Null pointer dereference

// g++ -g -o segfault_demo segfault_demo.cpp && ./segfault_demo
#include <iostream>

int main() {
    int* p = nullptr;
    std::cout << *p << "\n";  // null deref → segfault
    return 0;
}

gdb ./segfault_demo core
(gdb) bt
(gdb) print p
$1 = (int *) 0x0

Example 2: Use-after-free

int* p = new int(42);
delete p;
std::cout << *p << "\n";  // UAF

Rebuild with -fsanitize=address and run; ASan prints heap-use-after-free with free and use sites.

Example 3: Stack overflow (deep recursion)

bt shows the same function repeated many times → reduce recursion or increase stack / use heap.

Example 4: Buffer overflow

Use ASan: stack-buffer-overflow with file and line.

Example 5: Double free

ASan: attempting double-free with allocation and both frees.

Example 6: LLDB on macOS

lldb -c core ./myapp
(lldb) bt
(lldb) bt -f
(lldb) frame select 0

Enable cores: ulimit -c unlimited; cores may appear under ~/Library/Logs/DiagnosticReports/.

---

4. Common causes

Cause	GDB / clues	ASan
Null deref	print ptr → 0x0	Crash at site
UAF	mappings / ASan	heap-use-after-free
Stack overflow	Repeated frames	—
Buffer overrun	Hard to see	stack/heap-buffer-overflow
Double free	—	attempting double-free

---

5. Strategies

1. Ensure cores or ASan repro.
2. bt → frame 0 → print.
3. For library crashes, inspect caller frames and arguments.
4. CI with ASan tests; Valgrind where ASan is awkward.

flowchart LR
    A[Core?] --> B[GDB / print]
    A --> C[ASan run]
    B --> D[Fix]
    C --> D

---

6. Defensive patterns

• Check pointers before use; assert in debug.
• delete p; p = nullptr;
• std::unique_ptr / std::shared_ptr
• Prefer strncpy / snprintf / std::string over unchecked C APIs.
• RAII for files, locks, memory.

---

7. AddressSanitizer

g++ -g -fsanitize=address -fno-omit-frame-pointer -o myapp myapp.cpp

CMake: target_compile_options / target_link_options with -fsanitize=address.

---

8. Production patterns

• systemd: DefaultLimitCORE=infinity, LimitCORE=infinity
• Docker: ulimit / core pattern volume
• Scripts: gdb -batch -ex "bt full" -ex quit on program + core
• coredumpctl: coredumpctl debug, dump -o
• CI: ASan build + ctest
• Split debug info: objcopy --only-keep-debug, debuglink

---

Common tool errors

1. Core format mismatch: Same arch, same build machine where possible; file core vs file ./app.
2. No symbols: Rebuild with -g, matching binary.
3. Cannot access memory: In dumps, some addresses are unmapped—avoid dereferencing in GDB if it fails.
4. ulimit cannot change: systemd / limits.conf / Docker --ulimit core=-1

---

FAQ

Q. No core file?
A. Set ulimit -c unlimited, fix core_pattern, systemd limits, and use coredumpctl on systemd.

Q. No source in GDB?
A. Debug or RelWithDebInfo, -g, same build as the crashing binary.

Q. ASan in production?
A. Typically no—use ASan in test/CI; production uses cores + symbols.

Q. When is this guide useful?
A. Whenever you need a repeatable workflow from segfault → core → bt → fix, plus ASan for earlier detection.

Q. What to read next?
A. Follow Previous/Next links at the bottom or the C++ series index.

Q. Learn more?
A. cppreference, sanitizer docs, GDB/LLDB manuals.

---

Related posts

• C++ Segmentation fault: causes and GDB
• C++ GDB/LLDB
• C++ debugging basics

---

Keywords

Segmentation fault, core dumped, GDB, LLDB, core dump, debugging, use-after-free, null pointer, AddressSanitizer.

---

Practical tips

• Reproduce with minimal tests; fix compiler warnings first.
• Measure before optimizing unrelated paths.

---

Summary

1. ulimit -c unlimited (and systemd/Docker as needed).
2. gdb ./program core or coredumpctl debug.
3. bt → frame 0 → inspect pointers and logic.
4. Suspect null, UAF, stack, overrun in order.
5. -fsanitize=address when you can reproduce.
6. Automate core handling and keep debug symbols.

Next: CMake link errors (#49-2)

Next article: CMake link errors LNK2019 / undefined reference

Previous article: Custom memory pool (#48-3)

---

Related articles

• Asio deadlock debugging (#49-3)
• DB engine basics (#49-1)
• Query optimization (#49-3)