dlx-ansible/docs/JENKINS-SSH-AGENT-TROUBLESH...

6.3 KiB

Jenkins SSH Agent Authentication Troubleshooting

Date: 2026-02-09 Issue: Jenkins cannot authenticate to remote build agent Error: Authentication failed when connecting to remote SSH agent

Problem Description

Jenkins is configured to connect to a remote build agent via SSH but authentication fails:

SSHLauncher{host='45.16.76.42', port=22, credentialsId='dlx-key', ...}
[SSH] Opening SSH connection to 45.16.76.42:22.
[SSH] Authentication failed.

Root Cause

The SSH public key associated with Jenkins's 'dlx-key' credential is not present in the ~/.ssh/authorized_keys file on the remote agent server (45.16.76.42).

Quick Diagnosis

From jenkins server:

# Test network connectivity
ping -c 2 45.16.76.42

# Test SSH connectivity (should fail with "Permission denied (publickey)")
ssh dlxadmin@45.16.76.42

Solution Options

Option 1: Add Jenkins Key to Remote Agent (Quickest)

Step 1 - Get Jenkins's public key from Web UI:

  1. Open Jenkins: http://192.168.200.91:8080
  2. Go to: Manage JenkinsCredentialsSystemGlobal credentials (unrestricted)
  3. Click on the 'dlx-key' credential
  4. Look for the public key display (if available)
  5. Copy the public key

Step 2 - Add to remote agent:

# SSH to the remote agent
ssh dlxadmin@45.16.76.42

# Add the Jenkins public key
echo "ssh-rsa AAAA... jenkins@host" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

# Verify authorized_keys format
cat ~/.ssh/authorized_keys

Step 3 - Test connection from Jenkins server:

# SSH to jenkins server
ssh dlxadmin@192.168.200.91

# Test connection as jenkins user
sudo -u jenkins ssh -o StrictHostKeyChecking=no dlxadmin@45.16.76.42 'echo "Success!"'

Option 2: Create New SSH Key for Jenkins (Most Reliable)

Step 1 - Run the Ansible playbook:

ansible-playbook playbooks/setup-jenkins-agent-ssh.yml -e "agent_host=45.16.76.42"

This will:

  • Create SSH key pair for jenkins user at /var/lib/jenkins/.ssh/id_rsa
  • Display the public key
  • Create helper script to copy key to agent

Step 2 - Copy key to agent (choose one method):

Method A - Automatic (if you have SSH access):

ssh dlxadmin@192.168.200.91
/tmp/copy-jenkins-key-to-agent.sh

Method B - Manual:

# Get public key from jenkins server
ssh dlxadmin@192.168.200.91 'sudo cat /var/lib/jenkins/.ssh/id_rsa.pub'

# Add to agent's authorized_keys
ssh dlxadmin@45.16.76.42
echo "<paste-public-key>" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Step 3 - Update Jenkins credential:

  1. Go to: http://192.168.200.91:8080/manage/credentials/
  2. Click on 'dlx-key' credential (or create new one)
  3. Click Update
  4. Under "Private Key":
    • Select Enter directly
    • Copy content from: /var/lib/jenkins/.ssh/id_rsa on jenkins server
  5. Save

Step 4 - Test Jenkins agent connection:

  1. Go to: http://192.168.200.91:8080/computer/
  2. Find the agent that uses 45.16.76.42
  3. Click Launch agent or Relaunch agent
  4. Check logs for successful connection

Option 3: Use Existing dlxadmin Key

If dlxadmin user already has SSH access to the agent:

Step 1 - Copy dlxadmin's key to jenkins user:

ssh dlxadmin@192.168.200.91

# Copy key to jenkins user
sudo cp ~/.ssh/id_ed25519 /var/lib/jenkins/.ssh/
sudo cp ~/.ssh/id_ed25519.pub /var/lib/jenkins/.ssh/
sudo chown jenkins:jenkins /var/lib/jenkins/.ssh/id_ed25519*
sudo chmod 600 /var/lib/jenkins/.ssh/id_ed25519

Step 2 - Update Jenkins credential with this key

Verification Steps

1. Test SSH Connection from Jenkins Server

# SSH to jenkins server
ssh dlxadmin@192.168.200.91

# Test as jenkins user
sudo -u jenkins ssh -o StrictHostKeyChecking=no dlxadmin@45.16.76.42 'hostname'

Expected output: The hostname of the remote agent

2. Check Agent in Jenkins

# Via Jenkins Web UI
http://192.168.200.91:8080/computer/

# Look for the agent, should show "Connected" or agent should successfully launch

3. Verify authorized_keys on Remote Agent

ssh dlxadmin@45.16.76.42
cat ~/.ssh/authorized_keys | grep jenkins

Expected: Should show one or more Jenkins public keys

Common Issues

Issue: "Host key verification failed"

Solution: Add host to jenkins user's known_hosts:

sudo -u jenkins ssh-keyscan -H 45.16.76.42 >> /var/lib/jenkins/.ssh/known_hosts

Issue: "Permission denied" even with correct key

Causes:

  1. Wrong username (check if it should be 'dlxadmin', 'jenkins', 'ubuntu', etc.)
  2. Wrong permissions on authorized_keys:
    chmod 700 ~/.ssh
    chmod 600 ~/.ssh/authorized_keys
    
  3. SELinux blocking (if applicable):
    restorecon -R ~/.ssh
    

Issue: Jenkins shows "dlx-key" but can't edit/view

Solution: Credential is encrypted. Either:

  • Replace with new credential
  • Use Jenkins CLI to export (requires admin token)

Alternative: Password Authentication

If SSH key auth continues to fail, temporarily enable password auth (NOT RECOMMENDED for production):

# On remote agent
sudo vim /etc/ssh/sshd_config
# Set: PasswordAuthentication yes
sudo systemctl restart sshd

# In Jenkins, update credential to use password instead of key

Files and Locations

  • Jenkins Home: /var/lib/jenkins/
  • Jenkins SSH Keys: /var/lib/jenkins/.ssh/
  • Jenkins Credentials: /var/lib/jenkins/credentials.xml (encrypted)
  • Remote Agent User: dlxadmin
  • Remote Agent SSH Config: /home/dlxadmin/.ssh/authorized_keys
# View Jenkins credential store (encrypted)
sudo cat /var/lib/jenkins/credentials.xml

# Check jenkins user SSH directory
sudo ls -la /var/lib/jenkins/.ssh/

# Test SSH with verbose output
sudo -u jenkins ssh -vvv dlxadmin@45.16.76.42

# View SSH daemon logs on agent
journalctl -u ssh -f

# Check Jenkins logs
sudo tail -f /var/log/jenkins/jenkins.log

Summary Checklist

  • Network connectivity verified (ping works)
  • SSH port 22 is reachable
  • Jenkins user has SSH key pair
  • Jenkins public key is in agent's authorized_keys
  • Permissions correct (700 .ssh, 600 authorized_keys)
  • Jenkins credential 'dlx-key' updated with correct private key
  • Test connection: sudo -u jenkins ssh dlxadmin@AGENT_IP 'hostname'
  • Agent launches successfully in Jenkins Web UI