When opening Exchange PowerShell(EMS), It is thrown an error The connection to the specified remote host was refused Error.
If any proxy is configured in the Exchange Server may cause for the issue. So run the below commands to get the proxy information and remove the proxy.
To See the Proxy: Netsh Winhttp Show Proxy
To remove the Proxy: Netsh winhttp reset Proxy
Now if you try to open PowerShell, It will open without any issues.