AWS_EC2_SSH_PowerShell_Onboarding - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki

AWS EC2 SSH Access from Windows PowerShell (GFE Onboarding)

TL;DR

New Government-Furnished Equipment (GFE) users cannot use Git Bash to reach the AWS EC2 development instance via the localhost:2222 tunnel. This document describes the verified, PowerShell-only success path that reproduces the same working SSH connection using only tools present by default on a standard GFE Windows 11 image — no installs, no admin, no Git for Windows.

If you follow the four steps in Onboarding Steps you will end up with a working ssh AWS_EC2_tmcg_ps command from powershell.exe.


Problem Statement

The established workflow uses a Git Bash alias:

ssh AWS_EC2_tmcg   # works reliably in Git Bash / MSYS OpenSSH 10.2p1

New GFE users do not have Git for Windows installed and are not permitted to install it. Their only shells are:

Shell Executable Available on GFE
Windows PowerShell 5.1 powershell.exe Yes (built-in)
PowerShell 7 (pwsh) pwsh.exe No (requires install)
PowerShell ISE powershell_ise.exe Yes (built-in, avoid for interactive ssh)

The only SSH client they have is the Windows-bundled OpenSSH:

C:\Windows\System32\OpenSSH\ssh.exe
OpenSSH_for_Windows_9.5p2, LibreSSL 3.8.2

This client differs from Git Bash's MSYS ssh in subtle ways (see Client Differences) that break the existing .ssh/config alias for some users.

Why This Doc Documents Success, Not Failure

The exact failure reported by affected users could not be reproduced on the reference machine used to author this guide. Rather than block on capturing the failure, we verified a known-good PowerShell path end-to-end so that new users have a deterministic recipe to follow. If a user still fails after completing every step here, they should capture the output of Diagnostic: capture full failure log and share it for triage.


Environment Assumptions (Standard GFE Windows 11)

Assumed present, no admin needed:

  • C:\Windows\System32\OpenSSH\ssh.exe (SSH client)
  • C:\Windows\System32\OpenSSH\ssh-keygen.exe (keypair generator)
  • powershell.exe (Windows PowerShell 5.1)
  • icacls.exe
  • %USERPROFILE% writable (typically C:\Users\<username>)

Assumed already established out of band (not covered here):

  • The localhost:2222 tunnel to the EC2 bastion is running on the user's machine (e.g., via AWS SSM start-session --document-name AWS-StartPortForwardingSession, or an equivalent mechanism). Verify with:

    Test-NetConnection -ComputerName localhost -Port 2222 -InformationLevel Detailed

    You need TcpTestSucceeded : True.

  • The remote EC2 host is running OpenSSH and has a user account matching the User value used below (default in this guide: terry.mcguinness).


Onboarding Steps

All commands are pasted into a powershell.exe window (Start Menu → "Windows PowerShell" — not ISE).

Step 1 — Generate an Ed25519 keypair

The private key is written to %USERPROFILE%\.ssh\id_ed25519_ec2test and locked down to the owner via NTFS ACL. If the file already exists, the script aborts to avoid overwriting.

Save the following as gen-ec2-test-key.ps1 (or paste it directly):

$ErrorActionPreference = 'Stop'

$sshDir  = Join-Path $env:USERPROFILE '.ssh'
$keyFile = Join-Path $sshDir 'id_ed25519_ec2test'

if (-not (Test-Path $sshDir)) {
    New-Item -ItemType Directory -Path $sshDir | Out-Null
    Write-Host "[created] $sshDir"
} else {
    Write-Host "[exists]  $sshDir"
}

if (Test-Path $keyFile) {
    Write-Host "[abort]   $keyFile already exists - remove it or pick another name"
    exit 1
}

Write-Host "---- generating Ed25519 keypair ----"
$comment = "ec2-test-$env:USERNAME"
& 'C:\Windows\System32\OpenSSH\ssh-keygen.exe' `
    -t ed25519 `
    -f $keyFile `
    -N '""' `
    -C $comment

if ($LASTEXITCODE -ne 0) {
    Write-Host "[error]   ssh-keygen exited with code $LASTEXITCODE"
    exit $LASTEXITCODE
}

Write-Host "`n---- locking down NTFS ACL (owner only) ----"
icacls $keyFile /inheritance:r | Out-Null
icacls $keyFile /grant:r "$($env:USERNAME):F" | Out-Null
icacls $keyFile

Write-Host "`n===== PUBLIC KEY (paste into remote ~/.ssh/authorized_keys) ====="
Get-Content "$keyFile.pub"
Write-Host "================================================================="
Write-Host "`nPrivate key: $keyFile"
Write-Host "Public key : $keyFile.pub"

Run it:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\gen-ec2-test-key.ps1

Expected tail of output — the public key line to hand off:

===== PUBLIC KEY (paste into remote ~/.ssh/authorized_keys) =====
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5... ec2-test-<yourusername>
=================================================================

Note on the passphrase: this test key is intentionally passphrase-less. Windows OpenSSH's ssh-agent service is Disabled by default on GFE and cannot be started without admin, so a passphrase-protected key would require typing the passphrase on every connection. The NTFS ACL applied above restricts read access to the owner (plus SYSTEM and Administrators, which Windows OpenSSH tolerates), providing at-rest protection roughly equivalent to a Linux 600-perm key.

Step 2 — Deliver the public key to the remote host

Send the single ssh-ed25519 AAAA... ec2-test-<username> line to whoever administers the remote EC2 account. They will append it to ~/.ssh/authorized_keys for the target Linux user (e.g., terry.mcguinness).

Do not send the private key (the file without the .pub extension) to anyone or paste it anywhere.

Step 3 — Add the SSH client config block

Open (or create) %USERPROFILE%\.ssh\config and append the following block. In PowerShell:

$cfg = Join-Path $env:USERPROFILE '.ssh\config'
$block = @"

Host AWS_EC2_tmcg_ps
    HostName localhost
    Port 2222
    User terry.mcguinness
    IdentityFile ~/.ssh/id_ed25519_ec2test
    IdentitiesOnly yes
    StrictHostKeyChecking no
    UserKnownHostsFile NUL
"@
Add-Content -Path $cfg -Value $block

Field-by-field rationale:

Directive Value Why
HostName localhost Traffic is tunneled to the bastion
Port 2222 Local end of the SSM/SSH tunnel
User terry.mcguinness Linux account on the EC2 host
IdentityFile ~/.ssh/id_ed25519_ec2test The key generated in Step 1
IdentitiesOnly yes Only offer this key — ignore ssh-agent identities and other keys in ~/.ssh
StrictHostKeyChecking no The tunnel endpoint's host key is not persistent; skip prompt
UserKnownHostsFile NUL Windows null device — do not record the volatile host key (Unix equivalent: /dev/null)

Step 4 — Test the connection

Non-interactive test (runs exit on the remote and comes back):

& 'C:\Windows\System32\OpenSSH\ssh.exe' -v AWS_EC2_tmcg_ps exit

Interactive test (drops you into a shell on the EC2 host):

ssh AWS_EC2_tmcg_ps

Expected success markers in the -v output:

OpenSSH_for_Windows_9.5p2, LibreSSL 3.8.2
debug1: Reading configuration data C:\Users\<user>/.ssh/config
debug1: C:\Users\<user>/.ssh/config line NN: Applying options for AWS_EC2_tmcg_ps
debug1: Connecting to localhost [::1] port 2222.
debug1: Connection established.
debug1: identity file C:\Users\<user>/.ssh/id_ed25519_ec2test type 3
...
debug1: Offering public key: ...id_ed25519_ec2test ED25519 SHA256:... explicit
debug1: Server accepts key: ...id_ed25519_ec2test ED25519 SHA256:... explicit
Authenticated to localhost ([::1]:2222) using "publickey".
...
debug1: Exit status 0

If you see Authenticated to localhost ([::1]:2222) using "publickey" and Exit status 0, you are done.


Client Differences (MSYS vs Windows OpenSSH)

Understanding these avoids future confusion when copying ~/.ssh/config snippets between shells.

Behavior Git Bash (MSYS OpenSSH 10.2p1) Windows OpenSSH 9.5p2
Config path shown in logs /c/Users/<u>/.ssh/config (POSIX) C:\Users\<u>/.ssh/config (mixed)
~ expansion %USERPROFILE% (via MSYS) %USERPROFILE% (native)
UserKnownHostsFile /dev/null Fully supported (real POSIX null) Silently tolerated; portable form is NUL
Private key permission check POSIX perms + tolerant NTFS ACL enforced — key rejected if group/other has read access
Reads user config from ~/.ssh/config %USERPROFILE%\.ssh\config (same file, different notation)
System config path /etc/ssh/ssh_config %PROGRAMDATA%\ssh\ssh_config
ssh-agent Started by Git Bash session helpers Windows service, Disabled by default on GFE

The two config directives in the legacy AWS_EC2_tmcg block that are Unix-isms (IdentityFile ~/.ssh/id_rsa and UserKnownHostsFile /dev/null) happen to work on the reference machine but are replaced here with cross-compatible equivalents (IdentitiesOnly yes for determinism, UserKnownHostsFile NUL for portability).


Troubleshooting

Diagnostic: capture full failure log

Run this in the user's PowerShell window and share the resulting file:

& 'C:\Windows\System32\OpenSSH\ssh.exe' -v AWS_EC2_tmcg_ps exit 2>&1 |
    Tee-Object -FilePath "$env:USERPROFILE\ssh-fail.log"

Symptom-to-cause table

Observed output Likely cause Remedy
'ssh' is not recognized as an internal or external command Windows OpenSSH Client feature disabled Ask GFE admin to enable optional feature OpenSSH.Client
Could not open a connection to your authentication agent ssh-agent disabled (expected on GFE) Ignore — the config uses IdentitiesOnly yes and does not need the agent
Permissions for '...id_ed25519_ec2test' are too open. It is required that your private key files are NOT accessible by others. NTFS ACL grants read to non-owners Re-run Step 1's icacls block against the key file
Connection refused on port 2222 Tunnel is not running Restart the SSM / port-forwarding tunnel; verify with Test-NetConnection localhost -Port 2222
no matching host key type found Rare — extreme algorithm mismatch Report; capture full -v log
Repeated Permission denied (publickey) after all steps Public key not (yet) in remote authorized_keys, or in the wrong user's authorized_keys, or file permissions wrong on remote Verify with the remote admin that the exact line from your .pub file is present under the intended Linux user
Hangs with no output ISE was launched instead of powershell.exe — ISE cannot host real terminals Close ISE, launch Windows PowerShell (not ISE)

Manual verification checklist

# 1. Which ssh will be used?
Get-Command ssh | Format-List Source, Version
ssh -V

# 2. Does the config file exist and contain the alias?
Test-Path "$env:USERPROFILE\.ssh\config"
Get-Content "$env:USERPROFILE\.ssh\config" |
    Select-String -Pattern 'AWS_EC2_tmcg_ps' -Context 0,7

# 3. Does the private key exist? What is its ACL?
$key = "$env:USERPROFILE\.ssh\id_ed25519_ec2test"
Test-Path $key
if (Test-Path $key) { icacls $key }

# 4. Is the local tunnel listening?
Test-NetConnection -ComputerName localhost -Port 2222 -InformationLevel Detailed

Appendix A — File Inventory (per user)

Path Purpose Created by
%USERPROFILE%\.ssh\ SSH config directory Step 1 script
%USERPROFILE%\.ssh\config SSH client config with AWS_EC2_tmcg_ps block Step 3
%USERPROFILE%\.ssh\id_ed25519_ec2test Private key (owner-locked ACL) Step 1 script
%USERPROFILE%\.ssh\id_ed25519_ec2test.pub Public key (safe to share) Step 1 script

Appendix B — Alias Compatibility Notes

The legacy AWS_EC2_tmcg alias intentionally coexists with the new AWS_EC2_tmcg_ps alias so Git Bash users can continue using the RSA key while PowerShell-only users adopt the Ed25519 key. Neither interferes with the other. Once all users have migrated, the old block can be retired or updated to use the same portable directives.

Appendix C — Reference Environment

The success path in this document was validated against:

Component Version
Client OS Windows 11 (build 26100.8655)
Client shell Windows PowerShell 5.1.26100.8655 (Desktop)
Client SSH OpenSSH_for_Windows_9.5p2, LibreSSL 3.8.2
Remote SSH OpenSSH_8.7 (RHEL/Amazon Linux)
Tunnel localhost:2222 → EC2 bastion
Key type used Ed25519
⚠️ **GitHub.com Fallback** ⚠️