All the examples in this paragraph print the line

!"#$&'()*;<=>?  @[\\]^`{|}~

A backslash quotes the next character, i.e. the next character is interpreted literally. The one exception is a newline: backslash-newline expands to the empty string.

echo \\!\\"\\#\\$\\&\\'\\(\\)\\*\\;\\<\\=\\>\\?\\ \\ \\@\\[\\\\\\]\\^\\`\\{\\|\\}\\~

All text between single quotes (forward quotes \\', also known as apostrophe) is printed literally. Even backslash stands for itself, and it’s impossible to include a single quote; instead, you can stop the literal string, include a literal single quote with a backslash, and start the literal string again. Thus the 4-character sequence '\\'' effectively allow to include a single quote in a literal string.

echo '!"#$&'\\''()*;<=>?  @[\\]^`{|}~'
#          ^^^^

Dollar-single-quote starts a string literal $'…' like many other programming languages, where backslash quotes the next character.

echo $'!"#$&\\'()*;<=>?  @[\\\\]^`{|}~'
#           ^^            ^^

Double quotes " delimit semi-literal strings where only the characters " \\\\ $ and ``` retain their special meaning. These characters need a backslash before them (note that if backslash is followed by some other character, the backslash remains). Double quotes are mostly useful when including a variable or a command substitution.

echo "!\\"#\\$&'()*;<=>?  @[\\\\]^\\`{|}~"
#      ^^                 ^^  ^^
echo "!\\"#\\$&'()*;<=>?  @[\\]^\\`{|}~"
#      ^^                 ^  ^^      \\[ prints \\[

Interactively, beware that \\! triggers history expansion inside double quotes: "!oops" looks for an older command containing oops; "\\!oops" doesn’t do history expansion but keeps the backslash. This does not happen in scripts.