Last I checked, powershell doesn’t support ssh-copyid. There are scripts out there that you can download which mimic the command but I just took the route of copying the ssh key manually so that I can keep it simple for the audience.
Yes, the vscode extension allows you to act as if vscode is installed on the remote machine... because it is. VSCode ssh will install a thin "cli" client on the remote machine. You can install extensions. Work in the scope of a project. One-off edits to any file you like. You can even add *minimal* configuration to have vscode to act as the $EDITOR for any command. The same goes for any of the remote tools in "ms-vscode-remote.vscode-remote-extensionpack" (extension id). WSL: yes. SSH: yes. Tunnels: yes. Devcontainers: yes! You can even nest connections. Want to connect to a devcontainer on a remote machine? You can do that!