Stop Wasting Time: The Only Guide You’ll Ever Need to Setup/Fix SSH on EC2

If you’ve ever SSH’d into an EC2 box, tried to run:

git clone git@github.com:YourOrg/YourRepo.git

and immediately got:

Permission denied (publickey).
fatal: Could not read from remote repository.

you’re in good company. In this post I’ll walk through every step, complete with why each one matters, so that you can:

• Generate or reuse an SSH key on your EC2 instance
• Add it to GitHub (and enable SAML SSO if your org requires it)
• Test and verify your setup
• Clone without ever typing a password again

Why does “Permission denied (publickey)” happen?

GitHub now requires public‐key SSH or a Personal Access Token; password logins over HTTPS are gone. When you ask git clone git@github.com…, SSH checks your keys in ~/.ssh. If it doesn’t find one GitHub recognizes, it balks.

1. Check for an existing SSH key

First, let’s see what’s already there:

ls -al ~/.ssh

You might see files like:

• id_rsa & id_rsa.pub
• id_ed25519 & id_ed25519.pub

If you have one public key (*.pub), you can reuse it; skip to step 3. If not, let’s make one.

2. Generate a new SSH key pair

Run this on your EC2 prompt:

ssh-keygen -t ed25519 -C "your_email@example.com"

• -t ed25519 chooses a modern, secure key type
• -C adds a comment so you know where it came from
• When prompted for file accept the default (~/.ssh/id_ed25519) by pressing Enter
• When prompted for passphrase just press Enter again (no passphrase needed for unattended deployments)

You’ll end up with:

~/.ssh/id_ed25519
~/.ssh/id_ed25519.pub

3. Add your public key to GitHub

1. On EC2, display the key:

cat ~/.ssh/id_ed25519.pub

2. Copy the entire output (starts with ssh-ed25519 AAAA…).

3. In your browser, go to GitHub → Settings → SSH and GPG keys → New SSH key.

• Title: e.g. “EC2 deploy key”
• Key: paste the contents you copied

4. If your organization enforces SAML SSO, GitHub will show an “Enable SSO” button next to your new key.

• Click it, select your org, and authorize.
• Without this, GitHub will still reject the key with a publickey error.

4. Load the key into the SSH agent

To make sure your shell actually uses the key:

eval "$(ssh-agent -s)" # start the agent
ssh-add ~/.ssh/id_ed25519 # add your key

Tip: If you skip this, you may be prompted for your key’s passphrase (if you set one), or SSH might not try that key first.

5. Verify the SSH connection

Now test your link to GitHub:

ssh -T git@github.com

You should see something like:

Hi your_username! You've successfully authenticated, but GitHub does not provide shell access.

If you still get Permission denied, double-check:

• That you added the public key (the .pub file) to GitHub
• That you clicked Enable SSO if your org uses SAML
• That your agent has the key loaded (ssh-add -l lists it)

6. Clone your repository

Everything’s in place, now clone without hassle:

git clone git@github.com:YourOrg/YourRepo.git

No passwords, no PATs, just pure SSH magic.

Troubleshooting tips

Host authenticity prompt
On first SSH to GitHub you’ll see:

The authenticity of host 'github.com (IP)' can't be established.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Type yes and hit Enter. That stores GitHub’s host key in ~/.ssh/known_hosts.

File permissions
SSH demands strict permissions. If Git complains:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 600 ~/.ssh/authorized_keys

Multiple keys
If you have more than one private key, you can specify which to use in ~/.ssh/config:

Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519

Wrapping up

By following these steps you’ve:

• Generated or located a secure SSH key
• Authorized it with GitHub (and your org’s SSO)
• Configured your EC2 to use the key automatically
• Verified and cloned without any more publickey errors

Now every git pull, git push, or git clone on that server will flow over SSH, securely and seamlessly. Enjoy your password-free deployments!