Also note that this is working without the DAEMON_OPTIONS described below.

When building your self-signed cert, where it asks for your name or your FQDN, be sure to give it this servers actual FQDN as this is used by the cert verification. It must exactly match the server name, which I'm assuming came from a DNS lookup.

sendmail -d0.10

sendmail -O LogLevel=14 -bs -Am
then enter STARTTLS
and check maillog for more info.
I had gotten it all working on a twin system with a self-signed cert. I ftped the self-signed over to this one and set this up to use the self-signed instead of my expensive NetSol cert. Everything suddenly started working, though obviously checktls shows a self-signed. I've tested the real cert extensively, it has integrity, so I'm confused. Maybe work on it some more tmw, at least I have a clue where to focus now.

• Edit /usr/local/etc/apache/Includes/httpd-ssl.conf file.
• Near top find <VirtualHost _default_:443>
• Replace with <VirtualHost your-domainname:443>
  Do NOT include wildcard template, i.e. "*.your-domainname".

• define(`confDOMAIN_NAME', `your-domainname')
  Do NOT include wildcard template, i.e. "*.your-domainname".
• DAEMON_OPTIONS(`Port=smtp, Family=inet, Addr=your-domainname, Name=MTA, M=bh')
DAEMON_OPTIONS(`Port=smtps, Family=inet, Addr=your-domainname, Name=TLSMTA, M=bh')
  Do NOT include wildcard template, i.e. "*.your-domainname".

Anyways, something interesting that I learned, was that the spammers would favor my secondary MX server. As my spam control mechanism was all built on the primary, this caused me a problem. The secondary would accept the spam from the outside world, then try to relay it through the primary, and it would be rejected - the secondary is now stuck holding this stinker. So I just stopped using a secondary MX.

The reason to explain all this, is that now that I'm trying once again to have a secondary MX, I must finally get it working completely. I actually did have it mostly working for about a year now, but I never bothered with the details like STARTTLS, etc.

The issue here, is that to do this properly, you should have a commercial issued cert for this (the secondary), server. I'm not willing to spend hundreds of dollars each year just for this. The alternative is a self-signed cert. Bear in mind that the primary will use my commercial full-out cert, but the secondary will use a self-signed. Lastly, this will not do an authenticate verify, but the email itself will at least be passed encrypted.

openssl req -new -x509 -keyout brix-cakey.pem -out brix-cacert.pem -days 365
openssl req -nodes -new -x509 -keyout brix-sendmailkey.pem -out brix-sendmailcert.pem -days 365
openssl x509 -noout -text -in brix-sendmail.pem
define(`CERT_DIR', `/root/cert/localcert')
define(`confSERVER_CERT', `CERT_DIR/brix-sendmailcert.pem')
define(`confSERVER_KEY', `CERT_DIR/brix-sendmailkey.pem')
define(`confCLIENT_CERT', `CERT_DIR/brix-sendmailcert.pem')
define(`confCLIENT_KEY', `CERT_DIR/brix-sendmailkey.pem')
define(`confCACERT', `CERT_DIR/brix-cacert.pem')
define(`confCACERT_PATH', `CERT_DIR')
define(`confCRL', `CERT_DIR/sendmail.crl')
And of course test with checktls.com
You should expect "Certificate 1 of 1 in chain: Cert VALIDATION ERROR(S): self signed certificate So email is encrypted but the recipient domain is not verified" as the only error/warning.

Ok, now just to make things more confusing, I'm going to try a "free" cert. I really hope I don't regret this. The biggest issue is the damn things all expire every 90 days. And that's the long ones, comodo is only 30 days - what's the fucking use?

10. What Appears in Logs and Headers? As STARTTLS-enabled mail servers communicate among themselves, it is almost certain (from my experience so far) that purchased certificates will fail to verify. ("verify=FAIL" will appear in the maillog.) If the server acting as a client doesn't offer a certificate, then "verify=NO" will appear in the mail log. (Clients acting as clients--which rarely have certificates at all--will also cause "verify=NO" to appear in the log.)

20191215 - Getting back to TLS issues.

Still seeing log entries like the following:
sendmail[78685]: STARTTLS=client, relay=[127.0.0.1], version=TLSv1.2, verify=FAIL, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256/256
I've found an extremely useful test website: http://www.checktls.com/. Try full debug option for gory details. Note that the last thing shown pretty much matches what you'll see from the following openssl commands.

FYI: The above checktls website makes it clear that the email message is being encrypted, what's happening is that the verification of my domain name is failing.

openssl verify -CAfile AddTrustExternalCARoot.crt -untrusted 20170411-vncert-ov_chain.txt SECURE.VINTNERS.NET.crt

openssl s_client -starttls smtp -connect localhost:25

GOT IT!
Went back to NetSol website and looked carefully at the installation instructions, while reading between the lines. They describe several different filenames than the ones they provided me, which is why this never happened right over a year ago. There was no Apache_Plesk_install.txt file. So I followed their instructions, sort of by copying AddTrustExternalCARoot.crt to ca-bundle.com. I then went into my editor, and concatenated onto it OV_NetworkSolutionsOVServerCA2.crt, and then concatenated onto that, the OV_USERTrustRSACertificationAuthority.crt file. Then changed the .mc file confCACERT to CERT_DIR/ca-bundle.crt, remade the .cf file, and restarted sendmail.

Yay!

20190130 - Still struggling with STARTTLS

    Jan 30 18:07:00 pedicel sendmail[79818]: STARTTLS=client, relay=[127.0.0.1], version=TLSv1.2, verify=FAIL, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256/256

I followed instructions to generate a key from this.
openssl req -newkey rsa:2048 -nodes -keyout secure.vintners.net.key -out secure.vintners.net.csr

I have installed saslv2 from pkg. Check with ps ax | grep sasl -- you'll see several occurances of saslauthd -a -pam.

Check that sendmail was compiled with sasl.
sendmail -d0.4 -bv root
Look for SASLv2 in the "Compiled with:" list.

Version 8.15.2
Compiled with: DNSMAP IPV6_FULL LOG MAP_REGEX MATCHGECOS MILTER
MIME7TO8 MIME8TO7 NAMED_BIND NETINET NETINET6 NETUNIX NEWDB NIS
PICKY_HELO_CHECK PIPELINING SASLv2 SCANF STARTTLS TCPWRAPPERS
USERDB XDEBUG
Canonical name: pedicel.vintners.net
UUCP nodename: pedicel.vintners.net
a.k.a.: [207.229.65.53]
a.k.a.: [192.168.7.1]
a.k.a.: [IPv6:0:0:0:0:0:0:0:1]
a.k.a.: localhost.my.domain
a.k.a.: [IPv6:fe80:0:0:0:0:0:0:1]
a.k.a.: [127.0.0.1]

============ SYSTEM IDENTITY (after readcf) ============
(short domain name) $w = pedicel
(canonical domain name) $j = secure.vintners.net
(subdomain name) $m = vintners.net
(node name) $k = pedicel.vintners.net
========================================================

mike@vintners.net... deliverable: mailer esmtp, host gmail.com., user mikelempriere@gmail.com


Check that sendmail was compiled with STARTTLS, it should be in the same list as sasl above.

Check for proper cert installation, at a command line run:
sendmail -O LogLevel=14 -bs -Am
Then tail -n 50 /var/log/maillog and look for errors.
I'm seeing STARTTLS: CRLFile missing. Internet search says this a benign error that can be ignored, but I'm having problems so will try to fix this. Downloading from:
http://crl.cacert.org/revoke.crl (Note file is much longer than it thinks it is and download takes about 5 times the estimated)

I have this installed in a subdirectory under my root user's home directory. The groups and permissions are set appropriately. See below discussion of 20200326.

[old]
Please note that 'make install' in /etc/mail does not work. Use:
make hostname.cf
then:
diff -cw hostname.cf sendmail.cf to check your changes.
then:
cp hostname.cf sendmail.cf to install it.
lastly:
restart-sendmail (this is my own script.)
Actually, looks like my sendmail-restart doesn't do it either -- do:
service sendmail stop
service sendmail start