Test GMAIL from command line

1. SMTP and SMTPS

1.1 MTA vs MSA

At the very beginning, there was no separate MSA (Mail Submission Agent), an MUA (Mail User Agent) can connect to an MTA (Mail Transfer Agent) server just like another MTA client. Such an MTA is called an open relay server. 

As security became more and more important, an SMTP server began to run separate MSA process listening on 587, which is dedicated used for accept connection from MUA. Even (MTA to MTA) and (MUA to MSA) share the same SMTP protocol, the latter usually requires authentication, SMTP AUTH.

1.2 Port 587 vs Port 465

Besides authentication, another big security issue is SMTP protocol is based on plain text without encryption. So SSL/TLS was adapted as a secure layer for SMTP later.

There are two ways to use SSL/TLS for SMTP, one is STARTTLS while the other is implicit TLS. STARTTLS starts with plain text and then explicitly uses TLS while implicit TLS uses TLS at the very beginning of the TCP session.

STARTTLS works on port 587, while implicit TLS on 465. For some reason, implicit TLS was deprecated, but now has been recovered and is preferred to SMARTTLS.  Please refer to rfc8314 .

GMAIL server supports both SMARTTLS and implicit TLS.

$ nmap -Pn smtp.gmail.com

Starting Nmap 7.70 ( https://nmap.org ) at 2020-01-30 22:34 EST

Nmap scan report for smtp.gmail.com (173.194.205.109)

Host is up (0.055s latency).

Other addresses for smtp.gmail.com (not scanned): 2607:f8b0:400d:c02::6c

rDNS record for 173.194.205.109: qm-in-f109.1e100.net

Not shown: 996 filtered ports

PORT    STATE SERVICE

465/tcp open  smtps

587/tcp open  submission

993/tcp open  imaps

995/tcp open  pop3s

As implicit TLS (port 465) is the preferred way, we will use it for the test.

2. Use OpenSSL command to test the GMAIL server 

Traditional telnet doesn't support TLS, so we use "openssl" command tool to test.

2.0 Make sure the tested gmail account is configured as "less secured"

This is required, otherwise, only authorized clients can log in.

2.1 Get base64 encoded credentials

As being said before, MSA requires authentication. So we must provide credentials to connect to the Gmail server.

The credential format is 

"NULusername@gmail.comNULpassword"

The Gmail server requests a base64 encoded string of the above credential. To make this blog more practical, the following is the real account I tested.

# -n to avoid extra newline character, -e to enable \xHH

$ echo -ne '\x00agent.smallstrong@gmail.com\x00z*****9' | base64

A************************************5

2.2 The complete test session

In this test, an email was sent from agent.smallstrong@gmail.com to smstong@126.com.

The following is the real session record with some ouput deleted for clarity.

$ openssl s_client -connect smtp.gmail.com:465 -crlf -ign_eof

CONNECTED(00000004)

....... (more details deleted here)

220 smtp.gmail.com ESMTP m27sm4430762qta.21 - gsmtp

EHLO myclient

250-smtp.gmail.com at your service, [2607:fea8:5760:15f8:ff39:3701:3428:99ad]

250-SIZE 35882577

250-8BITMIME

250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH

250-ENHANCEDSTATUSCODES

250-PIPELINING

250-CHUNKING

250 SMTPUTF8

AUTH PLAIN A*****************************************************5

235 2.7.0 Accepted

MAIL FROM: <agent.smallstrong@gmail.com>

250 2.1.0 OK m27sm4430762qta.21 - gsmtp

RCPT TO: <smstong@126.com>

250 2.1.5 OK m27sm4430762qta.21 - gsmtp

DATA

354  Go ahead m27sm4430762qta.21 - gsmtp

Subject: TEST

HERE IS BODY

.

250 2.0.0 OK  1580443663 m27sm4430762qta.21 - gsmtp

QUIT

221 2.0.0 closing connection m27sm4430762qta.21 - gsmtp

read:errno=0

2.3 verification

sender side.

recipient side.

3 Envelope vs Mail content

3.1 Tips: display raw mail message

Both 126.com and Gmail's WebUI supports displaying raw mail format.

126.com:

Gmail.com:

3.2 "From" and "To" in SMTP vs in Mail Content

To make it more clear, the following script only includes the commands inputted during the test.

 $ openssl s_client -connect smtp.gmail.com:465 -crlf -ign_eof

EHLO myclient

AUTH PLAIN A*****************************************************5

MAIL FROM: <agent.smallstrong@gmail.com>

RCPT TO: <smstong@126.com>

DATA

Subject: TEST

HERE IS BODY

.

QUIT

The two important pieces of information provided are the sender and recipient, which are used by SMTP server to relay the mail. 

The final mail content was:

Bcc: smstong@126.com
Return-Path: <agent.smallstrong@gmail.com>
Received: from myclient ([2607:fea8:5760:15f8:ff39:3701:3428:99ad])
        by smtp.gmail.com with ESMTPSA id m27sm4430762qta.21.2020.01.30.20.05.41
        for <smstong@126.com>
        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
        Thu, 30 Jan 2020 20:07:43 -0800 (PST)
Message-ID: <5e33a80f.1c69fb81.c5ab4.9063@mx.google.com>
Date: Thu, 30 Jan 2020 20:07:43 -0800 (PST)
From: agent.smallstrong@gmail.com
Subject: TEST

HERE IS BODY

In terms of the format of the mail content, rfc5322 is the main specification document. Based on this document, the mail content contains "From:" and "To:" as well. e.g.

 $ openssl s_client -connect smtp.gmail.com:465 -crlf -ign_eof

EHLO myclient

AUTH PLAIN A*****************************************************5

MAIL FROM: <agent.smallstrong@gmail.com>

RCPT TO: <smstong@126.com>

DATA

From: "Myclient" <agent.smallstrong@gmail.com>

To: "best smstong" <smstong@126.com>

Subject: TEST2

HERE IS BODY2

.

QUIT

The final content was:

Return-Path: <agent.smallstrong@gmail.com>
Received: from myclient ([2607:fea8:5760:15f8:ff39:3701:3428:99ad])
        by smtp.gmail.com with ESMTPSA id p19sm4491457qte.81.2020.01.30.20.58.28
        for <smstong@126.com>
        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
        Thu, 30 Jan 2020 20:59:59 -0800 (PST)
Message-ID: <5e33b44f.1c69fb81.9a817.978d@mx.google.com>
Date: Thu, 30 Jan 2020 20:59:59 -0800 (PST)
From: myclient <agent.smallstrong@gmail.com>
To: best smstong <smstong@126.com>
Subject: TEST2

HERE IS BODY2

We can see that both GMAIL and 126.com web clients display information based on the red parts rather than the blue inputs.

What will happen if the envelope and content contain different "From" and "To". Let's have a test.

$ openssl s_client -connect smtp.gmail.com:465 -crlf -ign_eof

EHLO myclient

AUTH PLAIN A*****************************************************5

MAIL FROM: <agent.smallstrong@gmail.com>

RCPT TO: <smstong@126.com>

DATA

From: "sender" <xxx@xxx.com>

To: "recipient" <yyy@yyy.com>

Subject: Test3

BODY

.

QUIT

We found that GMAIL changed the "From:" part in mail content to use the real one, but the "To:" part remained as is.

Let look at the final content.

Bcc: smstong@126.com
Return-Path: <agent.smallstrong@gmail.com>
Received: from myclient ([2607:fea8:5760:15f8:ff39:3701:3428:99ad])
        by smtp.gmail.com with ESMTPSA id u13sm4749368qta.30.2020.01.30.21.05.37
        for <smstong@126.com>
        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
        Thu, 30 Jan 2020 21:07:00 -0800 (PST)
Message-ID: <5e33b5f4.1c69fb81.fb6ab.a8c0@mx.google.com>
Date: Thu, 30 Jan 2020 21:07:00 -0800 (PST)
From: sender <agent.smallstrong@gmail.com>
X-Google-Original-From: "sender" <xxx@xxx.com>
To: recipient <yyy@yyy.com>
Subject: Test3

BODY