Hardening SSH on Linux Servers: sshd_config, Key Authentication, Agent Forwarding and Jump Hosts

Introduction

SSH (Secure Shell) is more than an encrypted remote shell: SCP/SFTP, port forwarding, SOCKS proxies, and the transport for Git, Ansible and docker context all run over it. That makes the SSH daemon the most exposed and most privileged service on a typical Linux server, and a mistake in its configuration tends to be copied to every machine built from the same image.

This article explains the parts of the protocol that hardening decisions depend on (key exchange, host verification, user authentication, channels), then walks through OpenSSH server configuration that actually takes effect, known_hosts hygiene, agent forwarding versus ProxyJump, and port forwarding controls.

After reading this post

• Explain what KEX, host keys and user keys each prove
• Switch a server to key-only login without locking yourself out
• Verify the effective sshd configuration, not just the file you edited
• Choose ProxyJump over agent forwarding, and restrict forwarding on the server

---

How SSH works

SSH (SSH-2, RFC 4251–4254) is three protocols layered over one TCP connection, by default on port 22:

1. Transport layer: the client and server negotiate algorithms, run a key exchange (KEX) to derive symmetric session keys, and the server proves its identity by signing the exchange with its host key. Everything after this point is encrypted and integrity-protected.
2. User authentication: inside the encrypted channel, the client proves who the user is, by public key, password, or keyboard-interactive (used for OTP prompts).
3. Connection protocol: the authenticated connection is multiplexed into channels: a shell, an exec command, the SFTP subsystem, or forwarded TCP connections.

sequenceDiagram
  participant C as Client
  participant S as SSH Server
  C->>S: TCP connect :22
  C->>S: Version and algorithm negotiation
  C->>S: Key exchange (KEX)
  S->>C: Host key + signature over exchange hash
  C->>C: Verify host key against known_hosts
  C->>S: User authentication (public key signature)
  S->>C: Success, open shell / SFTP / forwarding channels

Three different keys are involved, and confusing them leads to wrong hardening decisions:

Key	Lifetime	What it proves
Session keys (from KEX)	One connection	Nothing about identity; they only encrypt this session.
Host key (/etc/ssh/ssh_host_*_key)	Life of the server	”This is the same server you connected to before.”
User key (~/.ssh/id_ed25519)	Until you rotate it	”This user holds the private key matching an entry in authorized_keys.”

With public key authentication, the private key never leaves the client: it signs data bound to the session, and the server checks the signature against the public key in authorized_keys. That is why a compromised server cannot steal your key from a normal login, and also why agent forwarding (below) is a real change in the threat model.

On algorithms, current OpenSSH defaults are good and hand-tuned KexAlgorithms/Ciphers lists copied from old guides are more likely to weaken a server than improve it. Recent releases default to hybrid post-quantum key exchange (sntrup761x25519-sha512 from 9.0, mlkem768x25519-sha256 from 10.0) and AEAD ciphers such as ChaCha20-Poly1305 and AES-GCM. Keeping OpenSSH updated is the algorithm policy for most teams.

---

Keys and client setup

Generating and installing a key

# Ed25519: short keys, fast, and the default choice for new keys
ssh-keygen -t ed25519 -C "you@laptop" -f ~/.ssh/id_ed25519
# RSA only when a legacy system requires it; 3072 bits or more
ssh-keygen -t rsa -b 4096 -C "you@laptop" -f ~/.ssh/id_rsa
# Hardware-backed key (FIDO2 security key, OpenSSH 8.2+ on both ends)
ssh-keygen -t ed25519-sk -C "you@yubikey"

# Append the public key to ~/.ssh/authorized_keys on the server
ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

Give every private key a passphrase and let ssh-agent hold the decrypted key, so a stolen laptop backup or a leaked dotfiles repository does not hand over server access. Use separate keys for personal access, CI and deploy automation, so revoking one does not break the others, and record where each public key is installed.

-sk keys are worth the setup for administrators: the private key material stays on the hardware token, and every authentication requires a touch, which also defeats malware that tries to use your agent silently.

Restricting keys in authorized_keys

Each line in authorized_keys can carry options that limit what that key may do. This is the right tool for automation keys:

# Backup job: only from the backup host, only this command, nothing else
from="10.0.5.20",command="/usr/local/bin/backup-receive",restrict ssh-ed25519 AAAAC3... backup@job

restrict (OpenSSH 7.2+) disables port, agent and X11 forwarding and PTY allocation in one word; command= forces the command regardless of what the client asks to run; from= limits source addresses. A leaked CI key restricted this way is a nuisance instead of a shell.

Client config

~/.ssh/config turns long command lines into names and keeps jump settings consistent:

Host bastion
  HostName bastion.example.com
  User deploy
  IdentityFile ~/.ssh/id_ed25519
  IdentitiesOnly yes

Host internal-*
  User app
  ProxyJump bastion
  IdentityFile ~/.ssh/id_ed25519
  IdentitiesOnly yes

Host db-tunnel
  HostName app.internal
  ProxyJump bastion
  LocalForward 15432 localhost:5432

ssh internal-api   # connects through the bastion
ssh -N db-tunnel   # local 15432 → Postgres on app.internal, no remote shell

IdentitiesOnly yes makes the client offer only the named key instead of every key loaded in the agent, which avoids “Too many authentication failures” on servers with a low MaxAuthTries.

SCP and SFTP

scp ./build.tar.gz user@host:/var/app/
sftp user@host
# sftp> put local.bin /remote/path/
rsync -avz -e ssh ./dist/ user@host:/var/www/   # resumable, only sends changes

Since OpenSSH 9.0, scp uses the SFTP protocol under the hood instead of the legacy SCP protocol, which had a long history of filename-handling vulnerabilities. For large or repeated transfers, rsync over SSH is usually the better tool.

---

Hardening sshd_config

The settings that matter

A reasonable baseline for an internet-facing server, placed in a drop-in file:

# /etc/ssh/sshd_config.d/00-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey
AllowGroups ssh-users
MaxAuthTries 3
LoginGraceTime 30
X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no

What each line buys you:

• PermitRootLogin no: administrators log in as themselves and use sudo, so logs show who did what. Use prohibit-password instead only if automation genuinely needs root with a key.
• PasswordAuthentication no and KbdInteractiveAuthentication no: both must be off to stop password prompts; keyboard-interactive can present a password prompt through PAM even when PasswordAuthentication is off. (ChallengeResponseAuthentication is the deprecated older name.)
• AuthenticationMethods publickey: makes the requirement explicit. For MFA, publickey,keyboard-interactive requires a key and a PAM-driven OTP prompt.
• AllowGroups: only members of a dedicated group can log in at all, so a service account created by a package with a shell and a weak password is not reachable over SSH.
• AllowTcpForwarding no and AllowAgentForwarding no: most users never need them. Re-enable per group with Match blocks (see the forwarding section) rather than globally.

Make sure the settings actually apply

This is where hardening most often fails silently. For most keywords, sshd uses the first value it reads, not the last. Debian, Ubuntu and many cloud images start the main sshd_config with Include /etc/ssh/sshd_config.d/*.conf, so any drop-in beats lines you add at the bottom of the main file, and drop-ins are read in lexical order.

The case I have run into on cloud VMs: an image ships /etc/ssh/sshd_config.d/50-cloud-init.conf containing PasswordAuthentication yes. Editing the main file to say no changes nothing, sshd -t reports no error, and password login keeps working. Naming your drop-in 00-hardening.conf so it sorts first, and checking the effective configuration, catches this:

sudo sshd -t                                   # syntax check; prints nothing on success
sudo sshd -T | grep -Ei 'passwordauth|permitroot|kbdinteractive|allowtcp'

sshd -T prints the configuration sshd will actually run with, after includes and defaults. To see what applies to a particular user or source address after Match blocks, add -C user=alice,host=laptop,addr=203.0.113.7.

Reload without locking yourself out

Before reloading, keep your current session open and test from a second terminal. Existing sessions survive a reload, so if the new config rejects you, you can still fix it:

sudo sshd -t && sudo systemctl reload ssh    # the unit is "sshd" on RHEL/Fedora/Arch
ssh -o PreferredAuthentications=password user@server   # should now be refused
ssh user@server                                        # key login should still work

On Ubuntu 22.10 and later, sshd is socket-activated by default. Port and ListenAddress changes are applied through ssh.socket, so after changing them run sudo systemctl daemon-reload && sudo systemctl restart ssh.socket; reloading only the service leaves it listening on the old port. Moving SSH off port 22 cuts log noise from scanners, but it is not a security control on its own.

---

known_hosts and host verification

Host key checking is what stops a man-in-the-middle. On first connection the client has nothing to compare against, so it asks you to accept the fingerprint (trust on first use). That prompt only protects you if you compare the fingerprint with one obtained out of band, for example from the server console or your provisioning output:

# On the server (or from your provisioning logs)
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

After that, ~/.ssh/known_hosts pins the key, and a change produces REMOTE HOST IDENTIFICATION HAS CHANGED. Usually the cause is benign (reinstalled server, reused IP address, new image), but “usually” is the point of the warning: verify the new fingerprint, then remove only that entry:

ssh-keygen -R server.example.com

For automation, do not use StrictHostKeyChecking no: it accepts any key, including an attacker’s, and silently defeats host authentication for the job that probably holds the most powerful credentials. Better options, in order of effort:

• Commit the expected host keys to the CI configuration and write them to known_hosts before connecting (for GitHub, use the fingerprints GitHub publishes).
• StrictHostKeyChecking accept-new accepts keys for new hosts but still refuses changed keys. Acceptable for ephemeral build agents talking to a fleet that churns.
• Host certificates for larger fleets: sign host keys with an SSH CA and trust the CA once with a @cert-authority *.example.com ssh-ed25519 AAAA... line in known_hosts, so rebuilt servers do not trigger warnings at all.

---

Agent forwarding and jump hosts

Why agent forwarding is risky

ssh -A (or ForwardAgent yes) exposes your local agent on the remote host through a Unix socket named in SSH_AUTH_SOCK. The key itself is not copied, but anyone who can access that socket, meaning root on that host or anyone who has compromised it, can ask your agent to sign authentication requests for as long as your session is open. In practice, a compromised bastion with forwarded agents is a path to every server those keys open.

If you must forward, add keys with ssh-add -c so each use prompts for confirmation locally, forward only to specific hosts in ~/.ssh/config (never under Host *), and use a separate agent holding only the key needed for that hop.

ProxyJump: the usual replacement

Most agent forwarding exists to hop through a bastion. ProxyJump does that without exposing the agent:

ssh -J [email protected] [email protected]
# multiple hops
ssh -J user@hop1,user@hop2 [email protected]

The client connects to the bastion and asks it to open a TCP connection to the target (the same mechanism as ssh -W). The client then runs a second, end-to-end SSH session to the target through that pipe. The bastion only relays encrypted bytes: it never sees your keys or your session contents, and the target’s host key is still verified against your known_hosts. ProxyCommand remains useful for unusual transports (an HTTP proxy, a cloud provider’s tunnel command), but for plain jump hosts ProxyJump is simpler and harder to misconfigure.

Locking down the bastion itself

A bastion needs TCP forwarding enabled for ProxyJump to work, but it should allow only that. For jump-only users:

Match Group jump-users
    AllowTcpForwarding yes
    PermitOpen 10.0.0.5:22 10.0.0.6:22
    PermitTTY no
    X11Forwarding no
    AllowAgentForwarding no
    ForceCommand /usr/sbin/nologin

PermitOpen limits which destinations may be reached. It matches the host exactly as the client requests it (a name and its IP address are different entries) and does not accept CIDR ranges, so list targets explicitly, or allow *:22 and enforce destinations with firewall rules on the bastion. PermitTTY no and ForceCommand mean these users cannot get a shell on the bastion even with a valid key.

---

Port forwarding and its controls

# Local: your :8080 → target.internal:80, resolved from the SSH server's side
ssh -N -L 8080:target.internal:80 user@bastion
# Remote: the server's :9090 → your local :3000
ssh -N -R 9090:localhost:3000 user@public-host
# Dynamic: a SOCKS5 proxy on your :1080 that exits from the server
ssh -N -D 1080 user@bastion

A common case is a database that listens only on the server’s own loopback interface: ssh -N -L 5432:localhost:5432 user@server works because localhost in the forward target is resolved on the server side. Your client then connects to localhost:5432 locally.

Three details cause most confusion:

• Forwards bind to loopback by default. -L 0.0.0.0:8080:... exposes the tunnel to your whole local network. On the server side, -R listeners stay on the server’s loopback unless the server sets GatewayPorts yes or clientspecified.
• Add -o ExitOnForwardFailure=yes in scripts. Without it, ssh connects successfully even when the port could not be bound, and the script keeps running without a working tunnel.
• -N skips the remote command. Combine it with -f to background the tunnel, or run tunnels under a service manager or autossh for anything long-lived.

From the server’s point of view, forwarding lets any authenticated user reach anything the server can reach, bypassing network firewalls between the user and internal services. That is why the baseline above turns it off and then allows it where needed:

Match Group db-tunnel-users
    AllowTcpForwarding local
    PermitOpen 127.0.0.1:5432

AllowTcpForwarding local permits -L but not -R; PermitListen does the equivalent restriction for remote forwards.

---

Brute force, rate limiting and patching

Once password authentication is off, brute-force attempts cannot succeed, but they still fill logs and consume connection slots. Options:

• OpenSSH’s built-in penalties (PerSourcePenalties, on by default since 9.8) temporarily refuse connections from addresses that repeatedly fail authentication or trigger crashes.
• fail2ban or firewall rate limits (nftables/iptables limit) add longer bans and cover the older OpenSSH versions still common on LTS distributions. They are essential if any account still accepts passwords.
• Restrict the source: when only an office VPN or a bastion needs access, a security group or firewall rule allowing port 22 only from those addresses removes almost all of the noise and much of the attack surface.

Patching matters more than any single setting. CVE-2024-6387 (“regreSSHion”) was a pre-authentication remote code execution bug in OpenSSH’s signal handling, fixed in 9.8p1; a server with perfect key-only configuration was still exposed until patched. Distribution security updates backport such fixes, so check your distribution’s advisory rather than the upstream version number alone.

---

Client-side quality of life

Connection multiplexing

Host *
  ControlMaster auto
  ControlPath ~/.ssh/cm-%C
  ControlPersist 10m

Later connections to the same host reuse the first one’s authenticated TCP connection, so ssh, scp and Ansible runs start almost instantly. %C is a hash of the connection parameters, which avoids socket paths exceeding the Unix socket length limit. The trade-off: while the master is alive, anyone with access to your account can use that socket without authenticating, and a stuck master can make new connections hang (ssh -O exit host closes it).

Keepalives

Host *
  ServerAliveInterval 30
  ServerAliveCountMax 4

Sends an encrypted keepalive every 30 seconds, so NAT gateways and firewalls do not drop idle sessions, and a dead connection is detected after about two minutes instead of hanging.

ssh-agent

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add --apple-use-keychain ~/.ssh/id_ed25519   # macOS: store the passphrase in Keychain
ssh-add -t 8h ~/.ssh/id_ed25519                  # drop the key from the agent after 8 hours

---

Permission denied, host key changes, and timeouts

Symptom	Check
Permission denied (publickey)	Permissions on the server (~ not group-writable, ~/.ssh 700, authorized_keys 600), the right username, and which key the client offers (ssh -v). The server log (journalctl -u ssh or /var/log/auth.log) usually names the reason.
Password login still works after hardening	An earlier drop-in wins; check sshd -T and the files in /etc/ssh/sshd_config.d/.
REMOTE HOST IDENTIFICATION HAS CHANGED	Reinstall or IP reuse is common, but verify the new fingerprint before ssh-keygen -R host.
Connection timeout	Security groups and firewalls, NAT, IPv4 vs IPv6 resolution, a missing ProxyJump, or (Ubuntu 22.10+) a port change not applied to ssh.socket.
Too many authentication failures	The agent offers too many keys before the right one; set IdentitiesOnly yes and an explicit IdentityFile.
Forwarding fails (“administratively prohibited”)	AllowTcpForwarding, PermitOpen/PermitListen and restrict options on the key; for -R on a non-loopback address, GatewayPorts.

---

Checking that the hardening is actually live

SSH hardening is less about exotic settings than about making a few decisions hold: keys only, no root login, a short list of allowed users, forwarding off unless needed, ProxyJump instead of agent forwarding, and real host key verification, including in CI. The part most often missed is verification: sshd -T and a refused password login from a second terminal tell you the configuration is live, while an edited file only tells you what you intended.

References

• man sshd_config, man ssh_config, man ssh-keygen, man sshd (AUTHORIZED_KEYS FILE FORMAT)
• OpenSSH release notes (default algorithm changes, PerSourcePenalties)
• RFC 4251–4254 (SSH architecture, transport, authentication and connection protocols)

---

Frequently Asked Questions (FAQ)

Q. I copied my public key to the server but still get “Permission denied (publickey)”. What is usually wrong?

A. Most often it is file permissions: with OpenSSH’s default StrictModes yes, the server ignores authorized_keys when ~/.ssh or the file itself is writable by group or others, or when the home directory is. Setting ~/.ssh to 700 and authorized_keys to 600 fixes the common case. The other usual suspects are the wrong username, or the client offering a different key than the one you installed - ssh -v shows which keys are tried, and the server’s auth log (journalctl -u ssh or /var/log/auth.log) usually states the exact reason.

---