Building a Parallel SSH Command Executor with Bash and Docker

Stop SSH-ing One by One: Building a Parallel Command Executor in Bash

Alan Varghese’s Multi-Server SSH Executor uses Bash background processes to run commands across server clusters simultaneously. This approach reduces execution time from the sum of all timeouts to only the duration of the slowest single connection.

Why This Matters

While heavy configuration management tools like Ansible are industry standards, engineers often require lightweight, zero-dependency alternatives for immediate, ad-hoc status checks. In high-pressure environments, the overhead of setting up complex automation can be a bottleneck compared to a robust shell script that handles parallel execution and configuration parsing natively.

Key Insights

• Parallelism in Bash is achieved using the ’&’ operator to background tasks, followed by the ‘wait’ command to synchronize execution.
• Configuration parsing requires setting a custom Internal Field Separator (IFS) to handle whitespace and ensure the final line of a file is processed via ’|| [[ -n “$name” ]]’.
• Non-interactive SSH requires ‘BatchMode=yes’ to prevent hanging on password prompts and ‘StrictHostKeyChecking=no’ for dynamic environments like Docker.
• Race conditions in parallel scripts are avoided by writing individual process outputs to unique temporary files using ‘mktemp’ before aggregating results.

Working Examples

Core logic for parallel execution using background processes and PID tracking.

for server in "${servers[@]}"; do ssh $user@$host "$command" > "/tmp/result_$server.txt" & pids+=($!); done; wait

Robust method for reading configuration files with colon delimiters and ensuring the final line is captured.

while IFS=':' read -r name hostname port username || [[ -n "$name" ]]; do # Process server... done < "$config_file"

Docker Compose configuration for simulating a multi-server environment locally.

services: web1: image: rastasheep/ubuntu-sshd:18.04 ports: ["2221:22"] web2: image: rastasheep/ubuntu-sshd:18.04 ports: ["2222:22"]

Practical Applications

• Fleet Health Checks: Executing ‘df -h’ or ‘uptime’ across multiple production nodes simultaneously; Pitfall: Writing to a single shared output file causes interleaved, garbled text.
• Local Cluster Simulation: Using Docker Compose to spin up SSH-enabled Ubuntu containers for script testing; Pitfall: Failing to use ConnectTimeout results in the script hanging indefinitely on unreachable hosts.

References:

• https://dev.to/alanvarghese-dev/stop-ssh-ing-one-by-one-building-a-parallel-command-executor-in-bash-55m1
• https://github.com/alanvarghese-dev/Bash_Scripting/tree/main/ssh_multi_server_executor.git