Previous
Next
C++ Segmentation Fault & Core Dump: GDB/LLDB Debugging

C++ Segmentation Fault & Core Dump: GDB/LLDB Debugging

· 18분 읽기 · 수정 Intermediate Troubleshooting
C++ Segmentation Fault & Core Dump: GDB/LLDB Debugging

이 글의 핵심

Enable core dumps, analyze crashes with GDB or LLDB, and catch use-after-free with AddressSanitizer. Practical workflow for C++ segfault debugging in development and production.

Introduction: When you see “Segmentation fault (core dumped)“

A segmentation fault means the OS terminated your process after invalid memory access: null dereference, dangling pointers, stack overflow, buffer overrun, etc. This article walks through enabling core dumps, loading them in GDB/LLDB, reading backtraces, and using AddressSanitizer (ASan) to catch issues earlier. Topics covered:

  • Core dumps: ulimit, systemd/kernel settings
  • Post-mortem analysis: load core in GDB/LLDB, bt, frame navigation, variables
  • Common causes: null deref, use-after-free, stack overflow
  • ASan for proactive detection
  • Production: automated core collection and analysis See also: GDB/LLDB, AddressSanitizer.

Experience from real projects

On large C++ codebases, book theory rarely matches production crashes. Profiling and code review helped turn vague failures into concrete fixes. The patterns below are meant to shorten your own trial-and-error loop.

Problem scenarios

Scenario 1: Occasional crashes in production

Logs show only Segmentation fault (core dumped) and no core file. Cause: ulimit -c is 0, core_pattern discards cores, or systemd limits cores. Fix: Enable cores → collect with coredumpctl or core_pattern → gdb ./program core and bt.

Scenario 2: Crash inside a third-party library

Cause: Bad pointer arguments, size/type mismatch, use-after-free across the FFI boundary. Fix: bt full, move to caller frames with frame N, inspect arguments; info sharedlibrary for symbols.

Scenario 3: Recursion or huge stack objects

Cause: Default stack size (~8 MB) exceeded. Fix: Repeated frames in bt → recursion; ulimit -s or heap allocation.

Scenario 4: Buffer overrun only on certain inputs

Fix: Re-run under ASan or Valgrind for exact line and size.

Summary

ScenarioHintApproach
Intermittent prod crashNo coreulimit, coredumpctl
Crash in .soTrace callersbt full, inspect args
Stack overflowDeep btulimit -s or heap
Input-dependentBoundsASan / Valgrind

Segfault debugging flow

flowchart TB
    subgraph Detect[Segfault]
        A[Segmentation fault] --> B{Core file?}
    end
    subgraph NoCore[No core]
        B -->|No| C[ulimit -c unlimited]
        C --> D[Check kernel.core_pattern]
        D --> E[Reproduce]
    end
    subgraph WithCore[Core available]
        B -->|Yes| F[gdb ./program core]
        F --> G[bt backtrace]
        G --> H[frame 0]
        H --> I[print variables]
        I --> J{Root cause?}
    end
    subgraph Prevent[Prevention]
        K[ASan build] --> L[Run tests]
        L --> M[Fix on first report]
    end
    E --> F
    J -->|Reproducible| K

Table of contents

  1. Enabling core dumps
  2. Analyzing cores with GDB/LLDB
  3. Complete debugging examples
  4. Common causes
  5. Debugging strategies
  6. Defensive coding
  7. ASan
  8. Production patterns
  9. Common tool errors
  10. FAQ

1. Enabling core dumps

ulimit

  • ulimit -c: if 0, no core file is written.
  • ulimit -c unlimited: in the shell session that runs the binary.
  • For persistence: /etc/security/limits.conf or shell rc files.
ulimit -c
ulimit -c unlimited
echo "ulimit -c unlimited" >> ~/.zshrc

Location and naming

  • Often core or core. in the process CWD.
  • /proc/sys/kernel/core_pattern: e.g. core.%e.%p.%t.
cat /proc/sys/kernel/core_pattern
sysctl kernel.core_pattern
echo "core.%e.%p.%t" | sudo tee /proc/sys/kernel/core_pattern

systemd

  • Set DefaultLimitCORE=infinity (and service LimitCORE= as needed).
  • coredumpctl list, coredumpctl debug or time range.
# /etc/systemd/system/myapp.service
[Service]
DefaultLimitCORE=infinity

2. GDB/LLDB core analysis

Loading a core

Use the same binary build that produced the crash, preferably with -g.

gdb ./your_program core
lldb -c core ./your_program

Backtrace

  • bt: call stack; #0 is the faulting frame.
  • bt full: locals per frame (GDB); bt -f (LLDB).

Frames and variables

  • frame 0, list, print ptr
  • *print ptr may fail if the address is invalid in the dump. | Task | GDB | LLDB | |------|-----|------| | Backtrace | bt | bt | | Locals | bt full | bt -f | | Select frame | frame N | frame select N | | Print | print x | frame variable x | | Disassembly | disas | disassemble |

3. Complete examples

Example 1: Null pointer dereference

// g++ -g -o segfault_demo segfault_demo.cpp && ./segfault_demo
#include <iostream>
int main() {
    int* p = nullptr;
    std::cout << *p << "\n";  // null deref → segfault
    return 0;
}
gdb ./segfault_demo core
(gdb) bt
(gdb) print p
$1 = (int *) 0x0

Example 2: Use-after-free

int* p = new int(42);
delete p;
std::cout << *p << "\n";  // UAF

Rebuild with -fsanitize=address and run; ASan prints heap-use-after-free with free and use sites.

Example 3: Stack overflow (deep recursion)

bt shows the same function repeated many times → reduce recursion or increase stack / use heap.

Example 4: Buffer overflow

Use ASan: stack-buffer-overflow with file and line.

Example 5: Double free

ASan: attempting double-free with allocation and both frees.

Example 6: LLDB on macOS

lldb -c core ./myapp
(lldb) bt
(lldb) bt -f
(lldb) frame select 0

Enable cores: ulimit -c unlimited; cores may appear under ~/Library/Logs/DiagnosticReports/.

4. Common causes

CauseGDB / cluesASan
Null derefprint ptr → 0x0Crash at site
UAFmappings / ASanheap-use-after-free
Stack overflowRepeated frames—
Buffer overrunHard to seestack/heap-buffer-overflow
Double free—attempting double-free

5. Strategies

  1. Ensure cores or ASan repro.
  2. bt → frame 0 → print.
  3. For library crashes, inspect caller frames and arguments.
  4. CI with ASan tests; Valgrind where ASan is awkward.
flowchart LR
    A[Core?] --> B[GDB / print]
    A --> C[ASan run]
    B --> D[Fix]
    C --> D

6. Defensive patterns

  • Check pointers before use; assert in debug.
  • delete p; p = nullptr;
  • std::unique_ptr / std::shared_ptr
  • Prefer strncpy / snprintf / std::string over unchecked C APIs.
  • RAII for files, locks, memory.

7. AddressSanitizer

g++ -g -fsanitize=address -fno-omit-frame-pointer -o myapp myapp.cpp

CMake: target_compile_options / target_link_options with -fsanitize=address.

8. Production patterns

  • systemd: DefaultLimitCORE=infinity, LimitCORE=infinity
  • Docker: ulimit / core pattern volume
  • Scripts: gdb -batch -ex “bt full” -ex quit on program + core
  • coredumpctl: coredumpctl debug, dump -o
  • CI: ASan build + ctest
  • Split debug info: objcopy —only-keep-debug, debuglink

Common tool errors

  1. Core format mismatch: Same arch, same build machine where possible; file core vs file ./app.
  2. No symbols: Rebuild with -g, matching binary.
  3. Cannot access memory: In dumps, some addresses are unmapped—avoid dereferencing in GDB if it fails.
  4. ulimit cannot change: systemd / limits.conf / Docker —ulimit core=-1

FAQ

Q. No core file?
A. Set ulimit -c unlimited, fix core_pattern, systemd limits, and use coredumpctl on systemd. Q. No source in GDB?
A. Debug or RelWithDebInfo, -g, same build as the crashing binary. Q. ASan in production?
A. Typically no—use ASan in test/CI; production uses cores + symbols. Q. When is this guide useful?
A. Whenever you need a repeatable workflow from segfault → core → bt → fix, plus ASan for earlier detection. Q. What to read next?
A. Follow Previous/Next links at the bottom or the C++ series index. Q. Learn more?
A. cppreference, sanitizer docs, GDB/LLDB manuals.

Keywords

Segmentation fault, core dumped, GDB, LLDB, core dump, debugging, use-after-free, null pointer, AddressSanitizer.

Practical tips

  • Reproduce with minimal tests; fix compiler warnings first.
  • Measure before optimizing unrelated paths.

Summary

  1. ulimit -c unlimited (and systemd/Docker as needed).
  2. gdb ./program core or coredumpctl debug.
  3. bt → frame 0 → inspect pointers and logic.
  4. Suspect null, UAF, stack, overrun in order.
  5. -fsanitize=address when you can reproduce.
  6. Automate core handling and keep debug symbols. Next: CMake link errors (#49-2) Next article: CMake link errors LNK2019 / undefined reference Previous article: Custom memory pool (#48-3)

자주 묻는 질문 (FAQ)

Q. 이 내용을 실무에서 언제 쓰나요?

A. Enable core dumps, analyze crashes with GDB or LLDB, and catch use-after-free with AddressSanitizer. Practical workflow … 실무에서는 위 본문의 예제와 선택 가이드를 참고해 적용하면 됩니다.

Q. 선행으로 읽으면 좋은 글은?

A. 각 글 하단의 이전 글 또는 관련 글 링크를 따라가면 순서대로 배울 수 있습니다. C++ 시리즈 목차에서 전체 흐름을 확인할 수 있습니다.

Q. 더 깊이 공부하려면?

A. cppreference와 해당 라이브러리 공식 문서를 참고하세요. 글 말미의 참고 자료 링크도 활용하면 좋습니다.

같이 보면 좋은 글 (내부 링크)

이 주제와 연결되는 다른 글입니다.

이 글에서 다루는 키워드 (관련 검색어)

C++, Segmentation fault, core dump, GDB, LLDB, debugging, AddressSanitizer 등으로 검색하시면 이 글이 도움이 됩니다.