How to Secure SSH: Keys, Ports, Fail2ban, and 2FA on Ubuntu 24.04

By the end of this guide your server accepts SSH logins from your key and nobody else’s, password logins are off, a brute-force bot gets three tries before Fail2ban drops it, and — if you want it — a stolen key still isn’t enough because you’ve added a time-based one-time code as a second factor.

Every command below was run in order on a clean Ubuntu 24.04.4 server (OpenSSH 9.6p1). I’ve kept the real output, including the two places where the “obvious” way silently does nothing on 24.04. Those two traps — the SSH port that won’t change and the Fail2ban jail that bans no one — are why most copy-paste guides leave people with a server that looks hardened but isn’t.

Prerequisites

• An Ubuntu 24.04 server you can already reach (a fresh Initial Ubuntu 24.04 Server Setup: Users, SSH, Firewall covers the user and firewall groundwork this guide builds on).
• A non-root sudo user. Examples below use deploy.
• An SSH key pair on your local machine. If you don’t have one: ssh-keygen -t ed25519 -C "you@laptop". Ed25519 keys are short, fast, and the modern default.
• A TOTP app on your phone for the 2FA section (Google Authenticator, Aegis, 1Password, Ente Auth — anything that scans a QR code).

One rule before you touch a single config file: keep your current SSH session open while you make changes, and test every change from a second terminal. If you lock yourself out with an open session still alive, you can undo it. If you close it first, you’re calling support for console access.

Step 1 — Put your public key on the server

If you can already log in, the one-liner from your laptop is:

ssh-copy-id deploy@your-server-ip

To do it by hand, the server side needs exact permissions — sshd refuses keys from a world-readable directory and won’t tell the user why:

install -d -m 700 -o deploy -g deploy /home/deploy/.ssh
# paste your PUBLIC key (the .pub file) into authorized_keys:
nano /home/deploy/.ssh/authorized_keys
chown deploy:deploy /home/deploy/.ssh/authorized_keys
chmod 600 /home/deploy/.ssh/authorized_keys

Verified permissions after setup:

$ ls -la /home/deploy/.ssh/
drwx------ 2 deploy deploy 4096 .
-rw------- 1 deploy deploy   92 authorized_keys
$ stat -c "%a %U:%G" /home/deploy/.ssh/authorized_keys
600 deploy:deploy

Open a new terminal and confirm ssh deploy@your-server-ip logs you in without asking for a password. Do not move on until that works — the next step turns passwords off for good.

Step 2 — Key-only authentication with a drop-in config

Do not edit /etc/ssh/sshd_config directly. Ubuntu’s cloud images ship an Include /etc/ssh/sshd_config.d/*.conf line, and — this is the first surprise — a file already sitting in there:

$ ls /etc/ssh/sshd_config.d/
60-cloudimg-settings.conf
$ cat /etc/ssh/sshd_config.d/60-cloudimg-settings.conf
PasswordAuthentication no

The AWS/cloud image already disables password auth this way. If you set PasswordAuthentication yes in the main file, this drop-in silently overrides it back to no because the first matching directive across all included files wins. Fighting that is a losing game. Work with the drop-in system: put all your hardening in one file that loads after it.

Create /etc/ssh/sshd_config.d/99-hardening.conf:

# Lineserve SSH hardening
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey
MaxAuthTries 3
LoginGraceTime 20
AllowUsers deploy
X11Forwarding no

AllowUsers deploy is the highest-leverage line here: even a valid key for any other account is rejected. Validate before restarting — and note the second small trap, a fresh 24.04 box is missing the privilege-separation directory until sshd has run once:

$ sudo sshd -t
Missing privilege separation directory: /run/sshd
$ sudo mkdir -p /run/sshd
$ sudo sshd -t && echo SSHD_CONFIG_OK
SSHD_CONFIG_OK

sshd -t returning nothing (plus our echo) means the syntax is good. Confirm what sshd will actually enforce — this is the source of truth, not the file you wrote:

$ sudo sshd -T | grep -Ei '^(permitrootlogin|passwordauthentication|authenticationmethods|allowusers|maxauthtries) '
permitrootlogin no
maxauthtries 3
passwordauthentication no
allowusers deploy
authenticationmethods publickey

Apply it with sudo systemctl restart ssh, then — from your second terminal — confirm you still get in with your key and that ssh -o PubkeyAuthentication=no deploy@your-server-ip is refused.

Step 3 — Changing the SSH port (the socket-activation reality)

Moving SSH off port 22 doesn’t stop a targeted attacker, but it does erase the constant background noise of untargeted bots. It’s optional. What is not optional is understanding how Ubuntu 24.04 actually listens, because the method every older guide teaches no longer works.

On 24.04, sshd is socket-activated. ssh.service is disabled; ssh.socket owns the listening port:

$ systemctl list-unit-files | grep -i ssh
ssh.service    disabled  enabled
ssh.socket     enabled   enabled
$ systemctl status ssh.socket
● ssh.socket - OpenBSD Secure Shell server socket
     Active: active (listening)
     Listen: 0.0.0.0:22 (Stream)

Here’s the trap. Add Port 2222 to a drop-in, restart, and check:

$ echo "Port 2222" | sudo tee /etc/ssh/sshd_config.d/50-port.conf
$ sudo systemctl restart ssh
$ sudo sshd -T | grep -i '^port '
port 2222
$ sudo ss -tlnp | grep sshd
LISTEN 0.0.0.0:22   users:(("sshd",...),("systemd",...))

sshd’s own config says 2222, but the server is still listening on 22. The Port directive is ignored because the socket, not sshd, opens the port. Delete that file — it does nothing but mislead:

sudo rm /etc/ssh/sshd_config.d/50-port.conf

The correct way is to override the socket. And there’s a second trap inside it: the bare form binds IPv6 only.

sudo mkdir -p /etc/systemd/system/ssh.socket.d
sudo tee /etc/systemd/system/ssh.socket.d/override.conf >/dev/null <<'EOF'
[Socket]
ListenStream=
ListenStream=0.0.0.0:2222
ListenStream=[::]:2222
EOF
sudo systemctl daemon-reload
sudo systemctl restart ssh.socket

The empty ListenStream= is required — it clears the inherited :22 before adding yours. If you write only ListenStream=2222 you get this, which looks fine but isn’t:

$ sudo ss -tlnp | grep 2222
LISTEN 0 4096 [::]:2222 [::]:*   users:(("systemd",pid=1,...))

That’s IPv6 only. A client on IPv4 gets Connection refused, while ::1 answers — a maddening “works on my machine” bug. Listing both 0.0.0.0:2222 and [::]:2222 fixes it. Confirmed dual-stack:

$ sudo ss -tlnp | grep 2222
LISTEN 0 4096 0.0.0.0:2222 0.0.0.0:*  users:(("sshd",...),("systemd",...))
LISTEN 0 4096    [::]:2222    [::]:*  users:(("sshd",...),("systemd",...))

Before you disconnect, open the new port in your firewall — this is the classic lockout. If you run UFW:

sudo ufw allow 2222/tcp comment 'ssh'
sudo ufw delete allow OpenSSH   # only after you've confirmed 2222 works

On a cloud VPS, also add the new port to your provider’s security group or firewall panel. Then reconnect on ssh -p 2222 deploy@your-server-ip from a second terminal before removing the old rule. Verified: sshd answers on 2222 on both stacks, and the socket transparently starts ssh.service on the first connection.

Step 4 — Fail2ban: ban the brute-forcers

Key-only auth already defeats password guessing, but bots still hammer the port and pollute your logs. Fail2ban watches for repeated failures and firewalls the source off.

sudo apt update && sudo apt install -y fail2ban

Verified version: Fail2Ban v1.0.2 (package 1.0.2-3ubuntu0.1).

Never edit jail.conf — it’s overwritten on upgrade. Create /etc/fail2ban/jail.local:

[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5
backend  = systemd

[sshd]
enabled = true
port    = 2222
maxretry = 3
journalmatch = _SYSTEMD_UNIT=ssh.service + _COMM=sshd

That last line is the difference between a working jail and a decorative one. This is the biggest silent failure on Ubuntu 24.04. Fail2ban’s built-in sshd filter matches the journal unit sshd.service, but socket-activated logins on 24.04 are logged under ssh.service. Without the override, the jail loads happily, reports “running”, and matches nothing — attackers are never banned. I confirmed the mismatch directly:

$ journalctl _COMM=sshd -o verbose -n1 | grep _SYSTEMD_UNIT
_SYSTEMD_UNIT=ssh.service

Set port = 2222 to match Step 3 (leave it as ssh if you kept port 22). Then restart and check status:

$ sudo systemctl restart fail2ban
$ sudo fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     0
|  `- Journal matches:  _SYSTEMD_UNIT=ssh.service + _COMM=sshd
`- Actions
   |- Currently banned: 0
   `- Banned IP list:

To prove it actually bans rather than just loading, I threw a handful of invalid-user logins at it from a test host (a loopback test, with the default ignoreself whitelist turned off so the box would ban its own address — in production you leave ignoreself on). The real result:

$ sudo fail2ban-client status sshd
   Filter
   |- Total failed:     4
   Actions
   |- Currently banned: 1
   `- Banned IP list:   127.0.0.1

One detail worth knowing: on 24.04 the ban is enforced through nftables, even though the action is still named iptables-multiport — iptables on this release is a compatibility shim over nftables. You’ll find the live rule here, not in classic iptables:

$ sudo nft list chain inet f2b-table f2b-chain
tcp dport 2222 ip saddr @addr-set-sshd reject with icmp port-unreachable

Two footnotes from testing. Fail2ban’s backend = systemd reads journald, which is always present — on a minimal 24.04 image /var/log/auth.log may not exist at all, so the journal backend is the reliable choice. And Fail2ban ignores your server’s own IPs by default via a rule called ignoreself that is separate from ignoreip — handy to know when your own test traffic never gets banned.

Step 5 — Two-factor authentication with TOTP (optional but strong)

With key-only auth, a leaked private key is game over. Adding a TOTP second factor means an attacker also needs the rotating six-digit code from your phone. This is genuinely worth it for servers holding customer data.

Install the PAM module:

sudo apt install -y libpam-google-authenticator

Verified version: 20191231-2build1.

Enrol as the user who will log in (not root). Run google-authenticator interactively and answer the prompts:

google-authenticator

It asks whether tokens should be time-based (answer y), then prints a QR code and your emergency scratch codes. Expected output looks like this — scan the QR with your app and write the scratch codes somewhere safe, they’re your way back in if you lose the phone:

Do you want authentication tokens to be time-based (y/n) y
[ QR code renders here in your terminal ]
Your new secret key is: XXXXXXXXXXXXXXXX
Enter code from app (-1 to skip): 123456
Code confirmed
Your emergency scratch codes are:
  18326587
  86971637
  ... (write these down)
Do you want me to update your "~/.google_authenticator" file? (y/n) y

The interactive QR/verify step needs a human with the phone, so it’s the one thing I couldn’t automate in the lab — but the resulting config file is real and verifiable. It lands at ~/.google_authenticator, owner-read-only:

$ stat -c "%a %U:%G" ~/.google_authenticator
400 deploy:deploy

Now wire it into SSH. First tell PAM to require the code. Edit /etc/pam.d/sshd: comment out the password stack and add the module near the top:

# @include common-auth
auth required pam_google_authenticator.so nullok

nullok lets users who haven’t enrolled yet still log in with their key alone — essential for a gradual rollout so you don’t lock out teammates the instant you save the file. Drop nullok once everyone is enrolled to make 2FA mandatory.

Then tell sshd to demand both factors. Change these two lines in your 99-hardening.conf from Step 2:

KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

publickey,keyboard-interactive means the key and the code — both, every login. Validate and confirm:

$ sudo sshd -t && echo OK
OK
$ sudo sshd -T | grep -Ei '^(authenticationmethods|kbdinteractiveauthentication) '
kbdinteractiveauthentication yes
authenticationmethods publickey,keyboard-interactive
$ sudo systemctl restart ssh

To prove the second factor is enforced, I logged in with the correct key but no code. The server accepts the key, then refuses to finish:

$ ssh -p 2222 -i deploykey deploy@your-server-ip
deploy@your-server-ip: Permission denied (keyboard-interactive).

sshd logs it as Connection closed by authenticating user deploy ... [preauth] — the key alone is no longer enough. A real login now asks Verification code: after the key, and you type the number from your app. That’s the whole point.

Verify it works

Run this checklist from a clean second terminal before you trust the box:

• ssh -o PubkeyAuthentication=no deploy@host → denied (passwords are off).
• ssh root@host → denied (root login off, and AllowUsers deploy).
• sudo ss -tlnp | grep ssh → listening on your chosen port, both 0.0.0.0 and [::].
• sudo fail2ban-client status sshd → jail enabled, journalmatch shows ssh.service.
• If you enabled 2FA: a key-only login is refused with Permission denied (keyboard-interactive).

Troubleshooting

“I changed Port but it’s still on 22.” You’re on socket activation. The Port directive in sshd_config is ignored — override ssh.socket instead (Step 3), then daemon-reload and restart ssh.socket.

“New port only works over IPv6 / Connection refused on IPv4.” Your override has a bare ListenStream=2222. Replace it with explicit ListenStream=0.0.0.0:2222 and ListenStream=[::]:2222, keeping the empty ListenStream= reset line above them.

“Fail2ban says it’s running but bans nobody.” The sshd jail’s default journal match points at sshd.service; your logins are under ssh.service. Add journalmatch = _SYSTEMD_UNIT=ssh.service + _COMM=sshd to the [sshd] block and restart. Confirm with fail2ban-client status sshd that the “Journal matches” line reads ssh.service.

“2FA locked me out.” If you enabled AuthenticationMethods publickey,keyboard-interactive before enrolling, or without nullok, sshd demands a code you can’t produce. Use your provider’s web console (out-of-band access, no SSH needed) to log in, then either add nullok to the PAM line or complete enrolment. This is exactly why you enable 2FA with a console session standing by.

“sshd -t says Missing privilege separation directory.” Run sudo mkdir -p /run/sshd. It’s created automatically when sshd starts but can be absent on a box that hasn’t started it yet.

---

Hardening SSH is the same drill whether the server sits in Virginia or Nairobi — but where it sits decides who your users’ data answers to. Lineserve runs Linux VPS plans in Nairobi (ke-1a), Dar es Salaam (tz-1a), and Lagos (ng-1a), so a hardened box also means data residency under Kenya’s DPA, Tanzania’s PDPA, or Nigeria’s NDPA — your data stays in-country. Billing is local too: pay in KES or TZS with M-Pesa and mobile money, or in Naira by card and bank transfer, with no forex conversion. Spin one up in minutes at console.lineserve.net and apply this guide start to finish. If you’re setting up a brand-new server, start with our Initial Ubuntu 24.04 Server Setup: Users, SSH, Firewall, then walk the broader 8 Essential Security Steps for Every Production VPS and lock down your How to Access GitHub from a Linux Server Using SSH Keys (Step-by-Step Guide) for deploy access.