What you need to know about OpenSSH key management


OpenSSH is one of my favourite tools, and one I take for granted because I use it all day, every day. OpenSSH is a Swiss Army-knife of coolness that can be used to provide secure connections to insecure sites in insecure places like free Wi-Fi-offering coffee shops. It can be used to remotely administer systems, provide encrypted file sharing via sshfs, bypass draconian corporate firewall policies (okay, maybe that isn’t the best example of OpenSSH coolness), and a whole lot more.

Before you’re really able to appreciate all that OpenSSH has to offer, you have to learn the basics, and that means key management. So we’re going to look at how to manage and use OpenSSH public/private keypairs.

Generating OpenSSH keys is easy, and doing so allows for passphrase-based keys to be used for login authentication instead of providing your password. This means you have the private key stored locally, and the public key is stored remotely. The two keys together form a cryptographically secure keypair used to perform authentication, without sending a password over the network.

To generate an RSA2 key (default 2048 bits) with a special comment to identify its use and saved to ~/.ssh/server1_rsa and ~/.ssh/server1_rsa.pub, use:

$ ssh-keygen -C "special key for server foo" -t rsa -f ~/.ssh/server1_rsa
Generating public/private rsa key pair.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:
Your identification has been saved in /home/joe/.ssh/server1_rsa.
Your public key has been saved in /home/joe/.ssh/server1_rsa.pub.
The key fingerprint is:
fb:8a:23:82:b9:96:a1:9c:d5:62:58:15:9a:8f:f9:ed special key for server1
The key's randomart image is:

+--[ RSA 2048]----+
|     ..          |
|    o.           |
|   o.            |
|   .+            |
|  oo..  S        |
| o +...  .       |
|oo* .. ..        |
|+=. . o. .       |
|o. . ..E...      |
+-----------------+

Keeping this key to yourself isn’t useful, so it needs to be copied to a remote server where it will be used. You can do this manually by copying it over and then moving it into place, or you can use the ssh-copy-id command:

$ ssh-copy-id -i ~/.ssh/server1_rsa joe@server1.domain.com

Once you provide the account password, the ~/.ssh/server1_rsa’s public key will be copied to the remote server and placed into ~/.ssh/authorized_keys. You should then be able to log in using the key, and its passphrase, from that point forward.

Using the ~/.ssh/config file can really make life easier. With that configuration file, you can easily setup various options for different hosts. If you wanted to have multiple SSH public/private keypairs, and want to use a specific keypair for a specific host, using ~/.ssh/config to define it will save some typing. For instance:

Host server1 server1.domain.com
  Hostname server1.domain.com
  User joe
  IdentityFile ~/.ssh/server1_rsa
Host server2 server2.domain.com
  Hostname server2.domain.com
  User root
  IdentityFile ~/.ssh/server2_rsa

In this example, when you do ssh server1, it will connect to server1.domain.com using the private key in ~/.ssh/server1_rsa, logging in as “joe”. Likewise, when connecting to server2.domain.com, the ~/.ssh/server2_rsa key is used and you will connect as the root user.

If you have changed the remote server’s SSH key (either by installing a new operating system, re-using an old IP address, changing the server keys, whatever), and you have strict key checking enabled (usually a default), you may see a message like this:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!

Warnings like this should be taken seriously. If you don’t know why the key has changed, before making any changes and assuming it’s benign, or even completing the login, find out from the administrator of the box. If you know this is an expected change, then use the ssh-keygen tool to remove all keys that belong to that particular host from the known_hosts file (as there may be more than one entry for the host):

$ ssh-keygen -R server1.domain.com

This is especially useful if you are using hashed hostnames. What are hashed hostnames, you ask? Hashed hostnames are a way to make the known_hosts file not store any identifying information on the host. So in the event of a compromise, an attacker would be able to obtain very little information from the hashed file. If you had an entry like this in your ~/.ssh/known_hosts file:

server4,10.10.10.43 ssh-rsa

AAAAB3NzaC1yc2EAAAABIwAAAIEAtNuBVGgUhMchJoQiDTZ+Nu1jzJOXxG9vo5pVWSbbic4kdAMggWrdhXBU6K3RFIEwxx9MQKR81g6F8shV7us0cc0qnBQxmlAItNRbJI8yA4Ur+2ggFPFteqUEvOhA+I7E8REcPX87urxejWK3W11UqOXyjs7cCjoqdps8fEqBT3c=

This clearly identifies that you have at some point connected to “server4″, which has an IP address of 10.10.10.43. To make this information unidentifiable, you can hash the known_hosts file to make the above look like this instead:

|1|sPWy3K2SFjtGy0jPTGmbOuXb3js=|maUi1uexwObad7fgjp4/TnTvpMI= ssh-rsa

AAAAB3NzaC1yc2EAAAABIwAAAIEAtNuBVGgUhMchJoQiDTZ+Nu1jzJOXxG9vo5pVWSbbic4kdAMggWrdhXBU6K3RFIEwxx9MQKR81g6F8shV7us0cc0qnBQxmlAItNRbJI8yA4Ur+2ggFPFteqUEvOhA+I7E8REcPX87urxejWK3W11UqOXyjs7cCjoqdps8fEqBT3c=

It’s the same host, but now in a format that only ssh and sshd will understand. This is where the ssh-keygen -R command is so necessary, since trying to find the entry for host “foo.domain.com” in a hashed file would be impossible. To hash your known_hosts file, use:

$ ssh-keygen -H

There is so much that can be done with OpenSSH, and this tip mostly dealt with key management. Next, we will look at some quick one-liner commands to help accomplish some basic, and some not-so-basic, tasks with OpenSSH.