Why am I receiving errors when I use the browser-based SSH console to access my Lightsail instance?

7 minute read
0

I receive an UPSTREAM_ERROR [515], UPSTREAM_NOT_FOUND [519], or CLIENT_UNAUTHORIZED [769] message when I use the browser-based SSH console to connect to my Amazon Lightsail instance.

Short description

If you receive SSH-related errors when you connect to your Lightsail instance, then you might see the following messages:

  • "Your instance encountered an error and has closed the connection. Try again or contact customer support. UPSTREAM_ERROR [515]"
  • "An error occurred and we were unable to connect or stay connected to your instance. If this instance has just started up, try again in a minute or two. UPSTREAM_NOT_FOUND [519]"
  • "Login failed. If this instance has just started up, try again in a minute or two. CLIENT_UNAUTHORIZED [769]"

The UPSTREAM_ERROR [515] and UPSTREAM_NOT_FOUND [519] errors indicate that you can't reach your Lightsail instance through SSH. The following are common reasons for these errors:

  • There are issues with the instance, such as a boot failure, status check failure, or resource over utilization.
  • An OS-level firewall is blocking SSH port access.
  • The SSH port (22) isn't the default.
  • The SSH service is down.

The CLIENT_UNAUTHORIZED [769] error indicates that there's an SSH authentication issue in your Lightsail instance. The following are common reasons for this error:

  • You misconfigured the /etc/ssh/lightsail_instance_ca.pub Lightsail system key.
  • You upgraded your Ubuntu instance version to 20.04 or later.

Resolution

There's an instance boot failure, instance status check failure, or resource over utilization

View your instance metrics to determine if the instance failed the system status check or the instance status check.

System status check

If the system status check failed, then the instance's underlying hardware is unhealthy. To resolve the issue, stop and then start the instance to migrate the instance to healthy hardware.

Warning: The instance's public IP address changes at every stop and start of the instance. If you don't want the public IP address to change on every stop and start, then attach a static IP address before you stop the instance.

Instance status check

If the instance status check failed, then an operating system-level issue might be causing boot errors. Or, the instance's resources, such as CPU or memory, might be over utilized. For troubleshooting steps, see How can I troubleshoot common issues that cause my Lightsail instance to be unresponsive?

An OS-level firewall is blocking SSH port access

Browser-based SSH access uses the Amazon internal IP address to connect to your Lightsail instance. Some OS-level firewalls and access control files allow SSH access to only one set of IP addresses. In this case, you receive the UPSTREAM_NOT_FOUND [519] or UPSTREAM_ERROR [515] error. The same issue occurs when a firewall is completely blocking SSH access.

Note: Examples of OS-level firewalls include Iptables and UFW. An example of a control file is /etc/hosts.deny that hosts access control in cPanel-based instances.

To resolve this issue, take one of the following actions:

  • If you can use SSH to access the instance from the terminal or PuTTY application, then log in to the instance. Remove the deny rules from firewall and the /etc/hosts.deny file.
  • If you have a cPanel instance, then access the WHM console to remove the firewall rules.
  • If you can't use SSH to connect to the instance from the terminal or PuTTY application, then use a launch script to deactivate the firewall. You can add the launch script only when you're launching an instance.

To use a launch script to deactivate OS-level firewalls, Iptables, and UFW, complete the following steps:

  1. Open the Amazon Lightsail console.

  2. Create a manual snapshot of the instance.

  3. On the Snapshots tab, under Manual snapshots, choose the three dots next to the new snapshot.

  4. Choose Create new instance.

  5. Select the same Availability Zone as the previous instance.

  6. Choose Add launch script, and then add the following script.
    Note: The following example script deactivates the UFW firewall, flushes all the Iptable chains or firewall rules, and renames the /etc/hosts.deny file to deactivate it:

    sudo ufw disable
    sudo iptables -F
    sudo mv /etc/hosts.deny /etc/hosts.deny_backup
    sudo touch /etc/hosts.deny
    sudo systemctl enable sshd
    sudo systemctl restart sshd
  7. Choose a new instance plan, or use the same plan as the previous instance.

  8. Enter a name for the instance, and then choose Create instance.

After the new instance starts to run, wait 10-15 minutes, and then use the browser-based SSH console to try to connect to the instance.

Note: If the previous instance had a static IP address, then use the static IP address on the new instance. On the Networking tab in the Lightsail console, detach the static IP address, and then attach it to the new instance.

The SSH service is down

If the SSH service isn't running or active on the instance, then the SSH connection fails and you receive the UPSTREAM_NOT_FOUND [519] error. To troubleshoot this issue, configure AWS Systems Manager Session Manager for your Lightsail instance. Then, access the instance without the SSH service to resolve the SSH issue.

Basic troubleshooting steps for SSH issues include:

  • Depending on the OS distribution, review the SSH authentication logs in the /var/log/auth.log or /var/log/secure file to identify the errors.
  • Test the SSH configuration file syntax, and then correct any errors:
sudo sshd -t
sudo systemctl restart sshd

You misconfigured the /etc/ssh/lightsail_instance_ca.pub Lightsail system key

Lightsail uses the /etc/ssh/lightsail_instance_ca.pub system key to activate browser-based SSH access. If this file is missing, then SSH authentication fails and you receive the CLIENT_UNAUTHORIZED [769] error. The same error occurs when the Lightsail system key isn't specified in the TrustedUserCAKeys parameter in the /etc/ssh/sshd_config SSH configuration file.

If you can use SSH to connect to the instance through the terminal or PuTTY application, then log in to the instance. Confirm that the /etc/ssh/sshd_config file exists and that it contains the ssh-rsa key.

If the file is missing, then complete the following steps to recreate the file:

  1. Run the following command to confirm that you get the ssh-rsa key in the command's output:

    sudo cat /var/lib/cloud/instance/user-data.txt | grep ^ssh-rsa
  2. If the command returns the ssh-rsa key in the output, then run the following commands to copy it to /etc/ssh/lightsail_instance_ca.pub:

    sudo sh -c "cat /var/lib/cloud/instance/user-data.txt | grep ^ssh-rsa > /etc/ssh/lightsail_instance_ca.pub"
        sudo sh -c "echo >> /etc/ssh/sshd_config" 
        sudo sh -c "echo 'TrustedUserCAKeys /etc/ssh/lightsail_instance_ca.pub' >> /etc/ssh/sshd_config"
        sudo systemctl restart sshd
  3. If you don't get an ssh-rsa key, or you can't use SSH to connect to the instance, then create a snapshot of the instance. When you launch a new instance from the snapshot, the Lightsail system key is automatically added to the server at instance launch.

You upgraded your Ubuntu instance version to 20.04 or later

For instances that run Ubuntu 20.04 or later, you must allow certificate authorities (CAs) to use the ssh-rsa algorithm to sign certificates. Otherwise, authentication fails and you receive the CLIENT_UNAUTHORIZED [769] error. In Ubuntu instance version 20.04 or later, CAs are allowed to use the ssh-rsa algorithm by default . However, if you upgrade your earlier Ubuntu instance version to 20.04, then you must manually allow the CAs.

To manually allow the CAs, complete the following steps:

  1. Use an SSH client, such as the terminal or PuTTY application to connect to the instance.

  2. Open the SSH configuration file (/etc/ssh/sshd_config).

  3. Add CA parameters to the file for your Ubuntu instance version:

    Ubuntu instance version upgraded to 20.04

    Manually enter the CASignatureAlgorithms parameter line into the sshd_config file:

    $ sudo vi /etc/ssh/sshd_config
    $ cat /etc/ssh/sshd_config | egrep "CASignature"
    CASignatureAlgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa

    Ubuntu instance version upgraded to 22.04

    Manually enter the CASignatureAlgorithms and PubkeyAcceptedAlgorithms parameter lines into the sshd_config file:

    $ sudo vi /etc/ssh/sshd_config
    $ cat /etc/ssh/sshd_config | egrep "CASignature|PubkeyAccepted"
    CASignatureAlgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa-cert-v01@openssh.com,ssh-rsa
  4. Run the following command to validate the sshd configuration file content:

    $ sudo sshd -T
  5. Run the following command to restart the sshd service:

    $ sudo systemctl restart sshd
AWS OFFICIAL
AWS OFFICIALUpdated 2 months ago
No comments