233 lines
6.3 KiB
Markdown
233 lines
6.3 KiB
Markdown
# Jenkins SSH Agent Authentication Troubleshooting
|
|
|
|
**Date**: 2026-02-09
|
|
**Issue**: Jenkins cannot authenticate to remote build agent
|
|
**Error**: `Authentication failed` when connecting to remote SSH agent
|
|
|
|
## Problem Description
|
|
|
|
Jenkins is configured to connect to a remote build agent via SSH but authentication fails:
|
|
|
|
```
|
|
SSHLauncher{host='45.16.76.42', port=22, credentialsId='dlx-key', ...}
|
|
[SSH] Opening SSH connection to 45.16.76.42:22.
|
|
[SSH] Authentication failed.
|
|
```
|
|
|
|
## Root Cause
|
|
|
|
The SSH public key associated with Jenkins's 'dlx-key' credential is not present in the `~/.ssh/authorized_keys` file on the remote agent server (45.16.76.42).
|
|
|
|
## Quick Diagnosis
|
|
|
|
From jenkins server:
|
|
```bash
|
|
# Test network connectivity
|
|
ping -c 2 45.16.76.42
|
|
|
|
# Test SSH connectivity (should fail with "Permission denied (publickey)")
|
|
ssh dlxadmin@45.16.76.42
|
|
```
|
|
|
|
## Solution Options
|
|
|
|
### Option 1: Add Jenkins Key to Remote Agent (Quickest)
|
|
|
|
**Step 1** - Get Jenkins's public key from Web UI:
|
|
1. Open Jenkins: http://192.168.200.91:8080
|
|
2. Go to: **Manage Jenkins** → **Credentials** → **System** → **Global credentials (unrestricted)**
|
|
3. Click on the **'dlx-key'** credential
|
|
4. Look for the public key display (if available)
|
|
5. Copy the public key
|
|
|
|
**Step 2** - Add to remote agent:
|
|
```bash
|
|
# SSH to the remote agent
|
|
ssh dlxadmin@45.16.76.42
|
|
|
|
# Add the Jenkins public key
|
|
echo "ssh-rsa AAAA... jenkins@host" >> ~/.ssh/authorized_keys
|
|
chmod 600 ~/.ssh/authorized_keys
|
|
|
|
# Verify authorized_keys format
|
|
cat ~/.ssh/authorized_keys
|
|
```
|
|
|
|
**Step 3** - Test connection from Jenkins server:
|
|
```bash
|
|
# SSH to jenkins server
|
|
ssh dlxadmin@192.168.200.91
|
|
|
|
# Test connection as jenkins user
|
|
sudo -u jenkins ssh -o StrictHostKeyChecking=no dlxadmin@45.16.76.42 'echo "Success!"'
|
|
```
|
|
|
|
### Option 2: Create New SSH Key for Jenkins (Most Reliable)
|
|
|
|
**Step 1** - Run the Ansible playbook:
|
|
```bash
|
|
ansible-playbook playbooks/setup-jenkins-agent-ssh.yml -e "agent_host=45.16.76.42"
|
|
```
|
|
|
|
This will:
|
|
- Create SSH key pair for jenkins user at `/var/lib/jenkins/.ssh/id_rsa`
|
|
- Display the public key
|
|
- Create helper script to copy key to agent
|
|
|
|
**Step 2** - Copy key to agent (choose one method):
|
|
|
|
**Method A - Automatic** (if you have SSH access):
|
|
```bash
|
|
ssh dlxadmin@192.168.200.91
|
|
/tmp/copy-jenkins-key-to-agent.sh
|
|
```
|
|
|
|
**Method B - Manual**:
|
|
```bash
|
|
# Get public key from jenkins server
|
|
ssh dlxadmin@192.168.200.91 'sudo cat /var/lib/jenkins/.ssh/id_rsa.pub'
|
|
|
|
# Add to agent's authorized_keys
|
|
ssh dlxadmin@45.16.76.42
|
|
echo "<paste-public-key>" >> ~/.ssh/authorized_keys
|
|
chmod 600 ~/.ssh/authorized_keys
|
|
```
|
|
|
|
**Step 3** - Update Jenkins credential:
|
|
1. Go to: http://192.168.200.91:8080/manage/credentials/
|
|
2. Click on **'dlx-key'** credential (or create new one)
|
|
3. Click **Update**
|
|
4. Under "Private Key":
|
|
- Select **Enter directly**
|
|
- Copy content from: `/var/lib/jenkins/.ssh/id_rsa` on jenkins server
|
|
5. Save
|
|
|
|
**Step 4** - Test Jenkins agent connection:
|
|
1. Go to: http://192.168.200.91:8080/computer/
|
|
2. Find the agent that uses 45.16.76.42
|
|
3. Click **Launch agent** or **Relaunch agent**
|
|
4. Check logs for successful connection
|
|
|
|
### Option 3: Use Existing dlxadmin Key
|
|
|
|
If dlxadmin user already has SSH access to the agent:
|
|
|
|
**Step 1** - Copy dlxadmin's key to jenkins user:
|
|
```bash
|
|
ssh dlxadmin@192.168.200.91
|
|
|
|
# Copy key to jenkins user
|
|
sudo cp ~/.ssh/id_ed25519 /var/lib/jenkins/.ssh/
|
|
sudo cp ~/.ssh/id_ed25519.pub /var/lib/jenkins/.ssh/
|
|
sudo chown jenkins:jenkins /var/lib/jenkins/.ssh/id_ed25519*
|
|
sudo chmod 600 /var/lib/jenkins/.ssh/id_ed25519
|
|
```
|
|
|
|
**Step 2** - Update Jenkins credential with this key
|
|
|
|
## Verification Steps
|
|
|
|
### 1. Test SSH Connection from Jenkins Server
|
|
```bash
|
|
# SSH to jenkins server
|
|
ssh dlxadmin@192.168.200.91
|
|
|
|
# Test as jenkins user
|
|
sudo -u jenkins ssh -o StrictHostKeyChecking=no dlxadmin@45.16.76.42 'hostname'
|
|
```
|
|
|
|
Expected output: The hostname of the remote agent
|
|
|
|
### 2. Check Agent in Jenkins
|
|
```bash
|
|
# Via Jenkins Web UI
|
|
http://192.168.200.91:8080/computer/
|
|
|
|
# Look for the agent, should show "Connected" or agent should successfully launch
|
|
```
|
|
|
|
### 3. Verify authorized_keys on Remote Agent
|
|
```bash
|
|
ssh dlxadmin@45.16.76.42
|
|
cat ~/.ssh/authorized_keys | grep jenkins
|
|
```
|
|
|
|
Expected: Should show one or more Jenkins public keys
|
|
|
|
## Common Issues
|
|
|
|
### Issue: "Host key verification failed"
|
|
**Solution**: Add host to jenkins user's known_hosts:
|
|
```bash
|
|
sudo -u jenkins ssh-keyscan -H 45.16.76.42 >> /var/lib/jenkins/.ssh/known_hosts
|
|
```
|
|
|
|
### Issue: "Permission denied" even with correct key
|
|
**Causes**:
|
|
1. Wrong username (check if it should be 'dlxadmin', 'jenkins', 'ubuntu', etc.)
|
|
2. Wrong permissions on authorized_keys:
|
|
```bash
|
|
chmod 700 ~/.ssh
|
|
chmod 600 ~/.ssh/authorized_keys
|
|
```
|
|
3. SELinux blocking (if applicable):
|
|
```bash
|
|
restorecon -R ~/.ssh
|
|
```
|
|
|
|
### Issue: Jenkins shows "dlx-key" but can't edit/view
|
|
**Solution**: Credential is encrypted. Either:
|
|
- Replace with new credential
|
|
- Use Jenkins CLI to export (requires admin token)
|
|
|
|
## Alternative: Password Authentication
|
|
|
|
If SSH key auth continues to fail, temporarily enable password auth (NOT RECOMMENDED for production):
|
|
|
|
```bash
|
|
# On remote agent
|
|
sudo vim /etc/ssh/sshd_config
|
|
# Set: PasswordAuthentication yes
|
|
sudo systemctl restart sshd
|
|
|
|
# In Jenkins, update credential to use password instead of key
|
|
```
|
|
|
|
## Files and Locations
|
|
|
|
- **Jenkins Home**: `/var/lib/jenkins/`
|
|
- **Jenkins SSH Keys**: `/var/lib/jenkins/.ssh/`
|
|
- **Jenkins Credentials**: `/var/lib/jenkins/credentials.xml` (encrypted)
|
|
- **Remote Agent User**: `dlxadmin`
|
|
- **Remote Agent SSH Config**: `/home/dlxadmin/.ssh/authorized_keys`
|
|
|
|
## Related Commands
|
|
|
|
```bash
|
|
# View Jenkins credential store (encrypted)
|
|
sudo cat /var/lib/jenkins/credentials.xml
|
|
|
|
# Check jenkins user SSH directory
|
|
sudo ls -la /var/lib/jenkins/.ssh/
|
|
|
|
# Test SSH with verbose output
|
|
sudo -u jenkins ssh -vvv dlxadmin@45.16.76.42
|
|
|
|
# View SSH daemon logs on agent
|
|
journalctl -u ssh -f
|
|
|
|
# Check Jenkins logs
|
|
sudo tail -f /var/log/jenkins/jenkins.log
|
|
```
|
|
|
|
## Summary Checklist
|
|
|
|
- [ ] Network connectivity verified (ping works)
|
|
- [ ] SSH port 22 is reachable
|
|
- [ ] Jenkins user has SSH key pair
|
|
- [ ] Jenkins public key is in agent's authorized_keys
|
|
- [ ] Permissions correct (700 .ssh, 600 authorized_keys)
|
|
- [ ] Jenkins credential 'dlx-key' updated with correct private key
|
|
- [ ] Test connection: `sudo -u jenkins ssh dlxadmin@AGENT_IP 'hostname'`
|
|
- [ ] Agent launches successfully in Jenkins Web UI
|