515 lines
16 KiB
Markdown
515 lines
16 KiB
Markdown
# hiveops-remote Implementation Plan
|
|
|
|
## Overview
|
|
|
|
Create **hiveops-remote** at `/source/hiveops-src/hiveops-remote/` with:
|
|
- **hiveops-remote-server** - Java/Spring Boot control plane + module JAR hosting
|
|
- **hiveops-remote-module** - Java module JAR implementing AgentModule SPI
|
|
|
|
The module JAR is downloaded from the server when the remote feature is activated, then loaded dynamically into hiveops-agent.
|
|
|
|
## Requirements
|
|
|
|
- **Screen Capture**: Static on-demand screenshots
|
|
- **File System**: Browse directories + download files
|
|
- **License Tiers**: Pro and Enterprise only
|
|
- **Distribution**: Module JAR downloaded from server on activation
|
|
- **Location**: `/source/hiveops-src/hiveops-remote/`
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ hiveops-remote │
|
|
├─────────────────────────────┬───────────────────────────────────┤
|
|
│ hiveops-remote-server │ hiveops-remote-module │
|
|
│ (Spring Boot) │ (AgentModule JAR) │
|
|
│ │ │
|
|
│ - Control Plane API │ Downloaded & loaded into │
|
|
│ - Command Queue │ hiveops-agent at runtime │
|
|
│ - Module JAR Hosting │ │
|
|
│ - Binary Result Storage │ - Screen Capture │
|
|
│ │ - File Browser │
|
|
│ PostgreSQL │ - File Download │
|
|
│ │ - Result Upload │
|
|
└─────────────────────────────┴───────────────────────────────────┘
|
|
|
|
Flow:
|
|
1. Agent checks for remote module availability
|
|
2. Downloads hiveops-remote-module.jar from server
|
|
3. ModuleLoader loads JAR via SPI
|
|
4. Module starts polling server for commands
|
|
```
|
|
|
|
---
|
|
|
|
## Project Structure (Maven Multi-Module)
|
|
|
|
```
|
|
/source/hiveops-src/hiveops-remote/
|
|
├── pom.xml # Parent POM
|
|
├── hiveops-remote-server/ # Spring Boot server
|
|
│ ├── pom.xml
|
|
│ └── src/main/java/com/hiveops/remote/server/
|
|
│ ├── RemoteServerApplication.java
|
|
│ ├── config/
|
|
│ │ └── SecurityConfig.java
|
|
│ ├── controller/
|
|
│ │ ├── AgentController.java # Agent endpoints (poll, upload)
|
|
│ │ ├── RemoteController.java # User endpoints (commands)
|
|
│ │ └── ModuleController.java # Module JAR download
|
|
│ ├── service/
|
|
│ │ ├── AgentService.java
|
|
│ │ ├── CommandService.java
|
|
│ │ └── BinaryService.java
|
|
│ ├── repository/
|
|
│ │ ├── AgentRepository.java
|
|
│ │ ├── CommandRepository.java
|
|
│ │ └── BinaryChunkRepository.java
|
|
│ ├── entity/
|
|
│ │ ├── RemoteAgent.java
|
|
│ │ ├── RemoteCommand.java
|
|
│ │ └── BinaryChunk.java
|
|
│ └── dto/
|
|
│ ├── AgentRegistrationRequest.java
|
|
│ ├── CommandRequest.java
|
|
│ └── ...
|
|
├── hiveops-remote-module/ # Agent module JAR
|
|
│ ├── pom.xml
|
|
│ └── src/main/java/com/hiveops/remote/module/
|
|
│ ├── RemoteModule.java # AgentModule SPI implementation
|
|
│ ├── RemoteCommandProcessor.java # Command polling & dispatch
|
|
│ ├── RemoteHttpClient.java # Server communication
|
|
│ ├── screen/
|
|
│ │ ├── ScreenCaptureService.java
|
|
│ │ ├── LinuxScreenCapture.java # X11/Wayland capture
|
|
│ │ └── WindowsScreenCapture.java # Robot/GDI capture
|
|
│ ├── filesystem/
|
|
│ │ ├── FileSystemService.java
|
|
│ │ ├── DirectoryLister.java
|
|
│ │ ├── FileDownloader.java
|
|
│ │ └── PathValidator.java # Security
|
|
│ └── dto/
|
|
│ ├── RemoteCommand.java
|
|
│ ├── ScreenshotParams.java
|
|
│ └── FileListParams.java
|
|
├── hiveops-remote-common/ # Shared DTOs between server & module
|
|
│ ├── pom.xml
|
|
│ └── src/main/java/com/hiveops/remote/common/
|
|
│ ├── dto/
|
|
│ │ ├── CommandType.java
|
|
│ │ ├── CommandStatus.java
|
|
│ │ ├── PollResponse.java
|
|
│ │ └── FileEntry.java
|
|
│ └── security/
|
|
│ └── PathSecurityUtils.java
|
|
└── deployments/
|
|
├── docker/
|
|
│ └── Dockerfile
|
|
└── docker-compose.yml
|
|
```
|
|
|
|
---
|
|
|
|
## Module Loading Flow
|
|
|
|
### 1. Check for Remote Module (in hiveops-agent)
|
|
|
|
The existing hiveops-agent can be enhanced to check for downloadable modules:
|
|
|
|
```java
|
|
// In AgentApplication or a new ModuleDownloader
|
|
public void checkRemoteModules() {
|
|
String moduleUrl = config.get("remote.module.url");
|
|
if (moduleUrl != null && remoteEnabled) {
|
|
Path modulePath = downloadModule(moduleUrl, "hiveops-remote-module.jar");
|
|
moduleLoader.loadFromPath(modulePath);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2. Module SPI Registration
|
|
|
|
**File**: `hiveops-remote-module/src/main/resources/META-INF/services/com.hiveops.core.module.AgentModule`
|
|
```
|
|
com.hiveops.remote.module.RemoteModule
|
|
```
|
|
|
|
### 3. RemoteModule Implementation
|
|
|
|
```java
|
|
public class RemoteModule implements AgentModule {
|
|
private RemoteCommandProcessor processor;
|
|
|
|
@Override
|
|
public String getName() {
|
|
return "hiveops-remote";
|
|
}
|
|
|
|
@Override
|
|
public void initialize(ModuleContext context) {
|
|
String serverUrl = context.getConfig("remote.server.url");
|
|
String agentToken = context.getConfig("remote.agent.token");
|
|
|
|
RemoteHttpClient client = new RemoteHttpClient(serverUrl, agentToken);
|
|
processor = new RemoteCommandProcessor(client, context);
|
|
}
|
|
|
|
@Override
|
|
public void start() {
|
|
processor.startPolling();
|
|
}
|
|
|
|
@Override
|
|
public void stop() {
|
|
processor.stopPolling();
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Database Schema (PostgreSQL)
|
|
|
|
```sql
|
|
-- In hiveops-remote-server
|
|
|
|
CREATE TABLE remote_agents (
|
|
id BIGSERIAL PRIMARY KEY,
|
|
agent_id UUID UNIQUE NOT NULL DEFAULT gen_random_uuid(),
|
|
name VARCHAR(255) NOT NULL,
|
|
hostname VARCHAR(255),
|
|
platform VARCHAR(50) NOT NULL,
|
|
agent_version VARCHAR(50),
|
|
module_version VARCHAR(50),
|
|
status VARCHAR(50) NOT NULL DEFAULT 'OFFLINE',
|
|
last_heartbeat_at TIMESTAMPTZ,
|
|
capabilities JSONB,
|
|
license_key VARCHAR(255),
|
|
machine_id VARCHAR(255),
|
|
created_at TIMESTAMPTZ DEFAULT NOW(),
|
|
updated_at TIMESTAMPTZ DEFAULT NOW(),
|
|
UNIQUE(license_key, machine_id)
|
|
);
|
|
|
|
CREATE TABLE remote_commands (
|
|
id BIGSERIAL PRIMARY KEY,
|
|
command_id UUID UNIQUE NOT NULL DEFAULT gen_random_uuid(),
|
|
agent_id BIGINT NOT NULL REFERENCES remote_agents(id) ON DELETE CASCADE,
|
|
type VARCHAR(50) NOT NULL,
|
|
status VARCHAR(50) NOT NULL DEFAULT 'PENDING',
|
|
parameters JSONB,
|
|
result JSONB,
|
|
error_message TEXT,
|
|
created_at TIMESTAMPTZ DEFAULT NOW(),
|
|
started_at TIMESTAMPTZ,
|
|
completed_at TIMESTAMPTZ
|
|
);
|
|
|
|
CREATE TABLE binary_chunks (
|
|
id BIGSERIAL PRIMARY KEY,
|
|
command_id BIGINT NOT NULL REFERENCES remote_commands(id) ON DELETE CASCADE,
|
|
chunk_index INT NOT NULL,
|
|
total_chunks INT NOT NULL,
|
|
data BYTEA NOT NULL,
|
|
checksum VARCHAR(64),
|
|
UNIQUE(command_id, chunk_index)
|
|
);
|
|
|
|
-- Module JAR versions
|
|
CREATE TABLE module_versions (
|
|
id BIGSERIAL PRIMARY KEY,
|
|
version VARCHAR(50) NOT NULL UNIQUE,
|
|
filename VARCHAR(255) NOT NULL,
|
|
checksum VARCHAR(64) NOT NULL,
|
|
size_bytes BIGINT NOT NULL,
|
|
released_at TIMESTAMPTZ DEFAULT NOW(),
|
|
is_latest BOOLEAN DEFAULT FALSE
|
|
);
|
|
```
|
|
|
|
---
|
|
|
|
## API Endpoints
|
|
|
|
### Module Distribution
|
|
|
|
```
|
|
GET /api/v1/modules/latest - Get latest module version info
|
|
GET /api/v1/modules/{version}/download - Download module JAR
|
|
```
|
|
|
|
### Agent-Facing (for hiveops-remote-module)
|
|
|
|
```
|
|
POST /api/v1/agents/register - Register agent with module
|
|
GET /api/v1/agents/poll - Poll for pending commands (long-poll 15s)
|
|
POST /api/v1/agents/heartbeat - Heartbeat update
|
|
POST /api/v1/commands/{id}/status - Update command status
|
|
POST /api/v1/commands/{id}/upload - Upload binary result (chunked)
|
|
```
|
|
|
|
### User-Facing
|
|
|
|
```
|
|
GET /api/v1/agents - List registered agents
|
|
GET /api/v1/agents/{id} - Get agent details
|
|
DELETE /api/v1/agents/{id} - Unregister agent
|
|
|
|
POST /api/v1/commands/screenshot - Request screenshot
|
|
POST /api/v1/commands/file-list - List directory
|
|
POST /api/v1/commands/file-download - Download file
|
|
GET /api/v1/commands/{id} - Get command status
|
|
GET /api/v1/commands/{id}/result - Download binary result
|
|
```
|
|
|
|
---
|
|
|
|
## Screen Capture Implementation
|
|
|
|
### Linux (X11)
|
|
|
|
```java
|
|
public class LinuxScreenCapture {
|
|
public byte[] capture(ScreenshotParams params) throws Exception {
|
|
// Option 1: Use Robot (requires X11 display)
|
|
Robot robot = new Robot();
|
|
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
|
|
BufferedImage image = robot.createScreenCapture(screenRect);
|
|
|
|
// Encode as JPEG
|
|
ByteArrayOutputStream baos = new ByteArrayOutputStream();
|
|
ImageIO.write(image, "jpg", baos);
|
|
return baos.toByteArray();
|
|
}
|
|
}
|
|
```
|
|
|
|
### Windows
|
|
|
|
```java
|
|
public class WindowsScreenCapture {
|
|
public byte[] capture(ScreenshotParams params) throws Exception {
|
|
// Same Robot API works on Windows
|
|
Robot robot = new Robot();
|
|
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
|
|
BufferedImage image = robot.createScreenCapture(screenRect);
|
|
|
|
ByteArrayOutputStream baos = new ByteArrayOutputStream();
|
|
ImageIO.write(image, "jpg", baos);
|
|
return baos.toByteArray();
|
|
}
|
|
}
|
|
```
|
|
|
|
**Note**: Java's `Robot` class works cross-platform for basic screen capture. For headless Linux, may need Xvfb or alternative approach.
|
|
|
|
---
|
|
|
|
## File System Service
|
|
|
|
```java
|
|
public class FileSystemService {
|
|
private final PathValidator pathValidator;
|
|
|
|
public List<FileEntry> listDirectory(String path) {
|
|
pathValidator.validate(path); // Security check
|
|
|
|
Path dir = Paths.get(path);
|
|
return Files.list(dir)
|
|
.map(p -> new FileEntry(
|
|
p.getFileName().toString(),
|
|
Files.isDirectory(p) ? "DIRECTORY" : "FILE",
|
|
Files.size(p),
|
|
Files.getLastModifiedTime(p).toInstant(),
|
|
getPosixPermissions(p)
|
|
))
|
|
.collect(Collectors.toList());
|
|
}
|
|
|
|
public InputStream downloadFile(String path) {
|
|
pathValidator.validate(path);
|
|
pathValidator.checkMaxSize(path);
|
|
return Files.newInputStream(Paths.get(path));
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Security
|
|
|
|
1. **Agent Auth**: License key + machine ID → JWT token (validated against hiveops-mgmt or standalone)
|
|
2. **Path Security**: Configurable allowed paths, block `..` traversal
|
|
3. **File Size Limit**: Configurable max (default 100MB)
|
|
4. **Module Signature**: JAR checksum verification on download
|
|
|
|
---
|
|
|
|
## Maven Configuration
|
|
|
|
**Parent pom.xml**:
|
|
```xml
|
|
<groupId>com.hiveops</groupId>
|
|
<artifactId>hiveops-remote</artifactId>
|
|
<version>1.0.0-SNAPSHOT</version>
|
|
<packaging>pom</packaging>
|
|
|
|
<modules>
|
|
<module>hiveops-remote-common</module>
|
|
<module>hiveops-remote-module</module>
|
|
<module>hiveops-remote-server</module>
|
|
</modules>
|
|
|
|
<properties>
|
|
<java.version>21</java.version>
|
|
<spring-boot.version>3.4.1</spring-boot.version>
|
|
</properties>
|
|
```
|
|
|
|
**hiveops-remote-module pom.xml**:
|
|
```xml
|
|
<dependencies>
|
|
<dependency>
|
|
<groupId>com.hiveops</groupId>
|
|
<artifactId>hiveops-core</artifactId>
|
|
<version>3.0.1-SNAPSHOT</version>
|
|
<scope>provided</scope> <!-- Provided by hiveops-agent -->
|
|
</dependency>
|
|
<dependency>
|
|
<groupId>com.hiveops</groupId>
|
|
<artifactId>hiveops-remote-common</artifactId>
|
|
<version>${project.version}</version>
|
|
</dependency>
|
|
</dependencies>
|
|
|
|
<build>
|
|
<plugins>
|
|
<plugin>
|
|
<groupId>org.apache.maven.plugins</groupId>
|
|
<artifactId>maven-shade-plugin</artifactId>
|
|
<configuration>
|
|
<!-- Bundle only remote-common, exclude hiveops-core -->
|
|
</configuration>
|
|
</plugin>
|
|
</plugins>
|
|
</build>
|
|
```
|
|
|
|
---
|
|
|
|
## Docker Deployment
|
|
|
|
**docker-compose.yml**:
|
|
```yaml
|
|
version: '3.8'
|
|
services:
|
|
hiveops-remote-server:
|
|
build: .
|
|
ports:
|
|
- "8090:8090"
|
|
environment:
|
|
- DB_HOST=postgres
|
|
- DB_NAME=hiveops_remote
|
|
volumes:
|
|
- ./modules:/app/modules # Module JARs served from here
|
|
depends_on:
|
|
- postgres
|
|
|
|
postgres:
|
|
image: postgres:16-alpine
|
|
environment:
|
|
- POSTGRES_DB=hiveops_remote
|
|
- POSTGRES_USER=hiveops
|
|
- POSTGRES_PASSWORD=hiveops
|
|
volumes:
|
|
- pgdata:/var/lib/postgresql/data
|
|
|
|
volumes:
|
|
pgdata:
|
|
```
|
|
|
|
---
|
|
|
|
## Implementation Order
|
|
|
|
### Phase 1: Project Setup
|
|
1. Create Maven multi-module project at `/source/hiveops-src/hiveops-remote/`
|
|
2. Setup parent pom.xml with modules
|
|
3. Create hiveops-remote-common with shared DTOs
|
|
|
|
### Phase 2: Server (hiveops-remote-server)
|
|
4. Create Spring Boot application
|
|
5. Create entities and repositories
|
|
6. Create AgentController (register, poll, upload)
|
|
7. Create RemoteController (user commands)
|
|
8. Create ModuleController (JAR download)
|
|
9. Database migrations
|
|
|
|
### Phase 3: Module (hiveops-remote-module)
|
|
10. Create RemoteModule implementing AgentModule SPI
|
|
11. Create RemoteCommandProcessor (polling loop)
|
|
12. Implement ScreenCaptureService
|
|
13. Implement FileSystemService + PathValidator
|
|
14. SPI registration in META-INF/services
|
|
|
|
### Phase 4: Agent Integration
|
|
15. Add module download capability to hiveops-agent
|
|
16. Test dynamic module loading
|
|
17. Configuration for remote module URL
|
|
|
|
### Phase 5: Docker & Testing
|
|
18. Create Dockerfile
|
|
19. Create docker-compose.yml
|
|
20. End-to-end testing
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
**hiveops-agent hiveops.properties** (to enable remote module):
|
|
```properties
|
|
# Remote module configuration
|
|
remote.enabled=true
|
|
remote.server.url=http://localhost:8090
|
|
remote.module.download.url=http://localhost:8090/api/v1/modules/latest/download
|
|
remote.license.key=${LICENSE_KEY}
|
|
remote.machine.id=${MACHINE_ID}
|
|
```
|
|
|
|
**hiveops-remote-server application.yml**:
|
|
```yaml
|
|
server:
|
|
port: 8090
|
|
|
|
spring:
|
|
datasource:
|
|
url: jdbc:postgresql://${DB_HOST:localhost}:5432/${DB_NAME:hiveops_remote}
|
|
username: ${DB_USER:hiveops}
|
|
password: ${DB_PASSWORD:hiveops}
|
|
|
|
remote:
|
|
poll-timeout-ms: 15000
|
|
command-timeout-ms: 300000
|
|
module-storage-path: /app/modules
|
|
```
|
|
|
|
---
|
|
|
|
## Verification
|
|
|
|
1. **Build**: `mvn clean package` in hiveops-remote
|
|
2. **Start Server**: `docker-compose up -d`
|
|
3. **Test Module Download**:
|
|
```bash
|
|
curl http://localhost:8090/api/v1/modules/latest
|
|
curl -O http://localhost:8090/api/v1/modules/1.0.0/download
|
|
```
|
|
4. **Integration Test**:
|
|
- Start hiveops-agent with remote.enabled=true
|
|
- Verify module downloads and loads
|
|
- Request screenshot via API
|
|
- Verify result returned
|