When inspecting the system logs of some other issue, I noticed that I was getting lots of SSH attempts in real time. Specifically I have about 10’000 attempts daily!

$ grep Disconnected /var/log/auth.log | wc -l #since March 6
46149

This was quite an eye-opener to me, as I am behind a NAT and normally have 0 logon attempts. From the logs I got a clear hint as to what was going on: the logon attempts were all from localhost!

Mar 10 11:27:38 devbox sshd[11409]: Disconnected from invalid user mn 127.0.0.1 port 45822 [preauth]
Mar 10 11:27:46 devbox sshd[11426]: Disconnected from authenticating user root 127.0.0.1 port 45824 [preauth]

The only thing I could come up with that made sense, was that these connections originated from the reverse SSH tunnel I have established towards my jump host to be able to login to my work computer from home. And indeed, when doing sudo systemctl stop reverse-tunnel all activity stopped immediately! So something about my setup is fishy. Up until now, I had assumed that my setup required the client side to be able to login to the jump host using a certificate valid for that host to be allowed to initiate a connection, but this was clearly wrong!

Setup

The client side .ssh/config

Host jump
    HostName jumphost.somedomain.com
    User ubuntu
    IdentityFile ~/.ssh/jump

# local -> jump -> jump:30001 --> work
# reverse tunnel: work -> jump:22 sets up jump:30001 -> work
Host work
    ProxyJump jump
    User my-user
    IdentityFile ~/.ssh/work
    HostName jumphost.somedomain.com
    Port 30001

The above seems fine, as that just covers the “client side” bit, telling my local computer how to connect.

The reverse tunnel service

This is setup as a systemd service that always is running and is what actually allows inbound connections from the jump host to my work computer. This is the prime suspect.

# The SSH tunnel is configured mostly through options specified in the default ssh config file (such as private key)
# It needs to be copied to /etc/systemd/system/ to be picked up and then run
#    sudo systemctl daemon-reload
#    sudo systemctl enable tunnel
#    sudo systemctl start tunnel
# Original implementation: https://askubuntu.com/a/1316825/165026

[Unit]
Description=Maintain Reverse SSH Tunnel
After=network.target

[Service]
User=my-user
ExecStart=/usr/bin/autossh  -o ServerAliveInterval=30 -o "ServerAliveCountMax 3" -M 44444 -o ExitOnForwardFailure=yes -gnNT -R 30001:localhost:22 jump
RestartSec=15
Restart=always
KillMode=mixed

[Install]
WantedBy=multi-user.target

I suspect this bit to be the culprit: -R 30001:localhost:22 jump