ssh -p 21 example@example.com <<EOF
  echo 'printing pwd'
  echo "\\$(pwd)"
  ls -a
  find '*.txt'
EOF

$ is escaped because we do not want it to be expanded by the current shell i.e $(pwd) is to be executed on the remote shell.

Another way:

ssh -p 21 example@example.com <<'EOF'
  echo 'printing pwd'
  echo "$(pwd)"
  ls -a
  find '*.txt'
EOF

Note: The closing EOF should be at the beginning of the line (No whitespaces before). If indentation is required, tabs may be used if you start your heredoc with <<-. See the Indenting here documents and Limit Strings examples for more information.