To fully document this ssh mystery, first I give the setup:

We have Party-A, which has a black box firewall appliance.

Party-A has ssh-listener-a, a debian server that has port 22 forwarded to it by the firewall, but otherwise all of it’s internet traffic goes through the firewall. So it has an ssh server that sits on the internet as well as the internal network of Party-A.

Party-A also has server-a which is only on the internal network but has outgoing internet access via the firewall.

We have Party-B, which has a debian server server-b sitting on the internet which is a do-everything server and firewall, with an internet IP address (party-b.com) as well as an internal network address, and does all the normal firewall stuff between the two addresses. It also is a file server, email server, database server, and it has home directories on it as well.

Party-B also has desktop-b, which is pretty much what it sounds like, with all internet traffic going through the firewall server-b and no direct connect capability.

Now the symptom:

I ssh from normal user account luser on desktop-b into a normal user account on ssh-listener-a. This connection happens without password, instead using authorized key of luser on desktop-b. From there, I ssh into root@server-a which requires a password. The root account on server-a has only one file in its .ssh directory, and that is known_hosts. No keys and no authorized_keys.

Once logged in, I then do

# scp data-file nuser@party-b.com:directory/

and the scp transfers the file with no password required!

Um, how is that possible? It’s acting as if the nuser@party-b.com account has an authorized key from the root@server-a account, but root@server-a doesn’t have any keys! Nevermind that there’s no such key in the authorized_keys file for the nuser account on server-b ! The nuser@server-b account has exactly 3 authorized keys, all for hosts on the internal network and none for hosts on the internet.

#WTF