gnutls, openssl and compiling mutt

[AV]

Does it make any difference if mutt is compiled with '--with-gnutls'
enabled or with '--with-openssl' enabled.

Mutt (Fed20/21/Rawhide package) shows that '--with-gnutls' is used.

I ask because I have no problem connecting to my ISP Ziggo
with mutt when using Fedora.

With the following in .muttrc:
  set ssl_starttls = yes
  set smtp_url = "smtp://[e-mail]@smtp.ziggo.nl:587
the TLS connection is established successfully and I am asked to 
accept the certificate. Everything works OK from then on.

But this does not work on a MacBook Pro with OsX Yosemite.
I use the Homebrew mutt. This mutt is compiled with '--with-openssl',
linked to 'openssl-1.0.1j'.

Using the same .muttrc file I use in fedora the TLS connection
'aborts' and I get the error message:
"SSL failed, I/O error. Could not negotiate TLS connection".

The openssl-1.0.1j from Homebrew seems to be the same as in 
Fedora Rawhide though I have not checked all the patches.

(Of course mutt in Fedora is compiled with gcc and the Homebrew
 mutt with LLVM Clang. Ziggo seems to be using TLSv1)

AV

[AV]

On Sat, Nov 01, 2014 at 07:27:11PM +0100, Heinz Diehl wrote:
>  On 01.11.2014, Alexander Volovics wrote:  
 
>  > I don't expect it will do any good to copy the
> "general" certificates
>  > to the Mac. 
 
>  Are there any CA-certificates installed on the Mac which are
> available
>  to mutt? If not, it could be the cause of your problem. 
I will stop investigating this problem at the moment.
It has already taken up too much time without any success.

I dicovered that mutt was also compiled linked to the latest
openssl in a version of Antergos Linux I had installed in a
spare partition.
Mutt aborted the TLS connection there too. You would think that if you
include mutt and openssl it would work more or less "out of the box",
but no.

As Antergos is Linux I thought it was easier to investigate things
there, but no.

To begin openssl was implemented quite diferently than in Fedora:
no /etc/pki/ and no /etc/pki/tls and no 
cert.pem -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
and nothing resembling 'tls-ca-bundle.pem'.

All the certificates were just lumped together in /etc/ssl/certs/
including one file 'ca-certificates.crt' also mentioned in the
man of muttrc. No docs to explain anything so again you would think
that the mutt/openssl combo would have been 'primed' to work
together in an obvious way.
Nothing I tried 'suggested' by man mutt, man muttrc, man openssl
worked. Googling didn't give usefull hints. Asking in the 
Antergos forum yielded nothing. So I gave up.

Then OSX. In the HomeBrew openssl directory there were 2 "pem"
files: cert.pem and osx_cert.pem. Trying to use these in some
way or other did not work.
The few actual Apple OSX certificates or whatever are stored 
in a format I do not understand, are not easily accessed and 
I think would have to be 'translated' first to be usefull.
So exit Mac.

And then I still do not quite understand what the TLS
connection is trying to accomplish:
1) verify that smtp.ziggo.nl is actually ziggo and not an 'imposter'
2) encrypt the mail before sending (or a combination of 1 & 2)
3) verify that I am connecting to ziggo and not an imposter.

Anyway thank you very much for your time and help.

AV

[Heinz Diehl]

On 01.11.2014, Alexander Volovics wrote: 

>  And then we might be talking about different things.
>  These might be "general" certificates. When I connected to my
>  ISP with mutt the first time and I had to accept a certificate
>  I had the impression that a "personal" certificate was generated
>  to identify ME to the server in the future. And this certificate
>  was saved by mutt in a file "~/.mutt_certificates". 
Nope. When you connect to your mailserver for the first time and its
certificate can not be validated (e.g. because there's a missing link
to /etc/pki/tls et al.), mutt asks you to either accept it once, for
alltime or to discard it. In case you choose to accept it for all
time, it get's stored in ~/.mutt-certs. From now on, mutt will accept
this certificate because *you* told it to do. This certificate carries
a signature of the ca-cert which has issued it, but mutt was unable to
verify it. Thus asking you to decide.

>  The muttrc manpage also mentions the following config variables:
>  'ssl_ca_certificates_file' & 'ssl_client_cert'. But these also do
>  not work with my Fedora '.mutt_certificates' file. 
ssl_ca_certificates_file is meant to point to something like the
tls-ca-certificates.pem file. But it does only work when compiled with
"--with-gnutls".

>  I don't expect it will do any good to copy the
> "general" certificates
>  to the Mac. 
Are there any CA-certificates installed on the Mac which are available
to mutt? If not, it could be the cause of your problem.
 
>  And then the situation is complicated by using Linux programs
>  in OsX via the Homebrew setup.  
I have noe clue :-)

>  Also, ich stecke tief in die scheisse :)  
Det finnes en løsning til alt. Although I'm German, I've been living
in Norway a long time (und ich glaube nicht, dass du sooo tief in
*der* Sch**** steckst) ;-)

[AV]

On Sat, Nov 01, 2014 at 02:19:27PM +0100, Heinz Diehl wrote:
>  On 01.11.2014, Alexander Volovics wrote:  
 
>  > Is that so. I didn't know that. How are you supposed to get
>  > the certificate then.  
 
>  Check if the "cert.pem" symlink points to something like
> this: 
>  lrwxrwxrwx  1 root root    49 Nov  1 14:11 cert.pem ->
>  /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 
It does, on my Fedora PC.
Nothing like this seems to exist on the Mac under OsX.

And then we might be talking about different things.
These might be "general" certificates. When I connected to my
ISP with mutt the first time and I had to accept a certificate
I had the impression that a "personal" certificate was generated
to identify ME to the server in the future. And this certificate
was saved by mutt in a file "~/.mutt_certificates".

The .muttrc manpage indicates that you can refer to this file in
.muttrc by using "set certificate_file = ~/.mutt_certificates".
So I copied this file to the Mac and specified this in .muttrc.
This does not work, still 'SSL failed. I/O error'!
The muttrc manpage also mentions the following config variables:
'ssl_ca_certificates_file' & 'ssl_client_cert'. But these also do
not work with my Fedora '.mutt_certificates' file.

I don't expect it will do any good to copy the "general" certificates
to the Mac.

And then the situation is complicated by using Linux programs
in OsX via the Homebrew setup. They expressly create a 
/usr/local/Cellar directory to install them and compartementalize
from OsX apps and libraries. Furthermore openssl has to be
isolated evenmore ('kegged' they call it) to not interfere with
OsX's own (ancient) openssl.

Running ~/.openssl directly from the Homebrew directory also
aborts unhelpfully (and using s_client, debug, verify produces
no helpfull info).

Also, ich stecke tief in die scheisse :) 

I contacted Homebrew, hope they can help.

AV

[Heinz Diehl]

On 01.11.2014, Alexander Volovics wrote: 

>  Is that so. I didn't know that. How are you supposed to get
>  the certificate then. Given that all the most used mail progs
>  (thunderbird, outlook, apple mail, evolution, etc) connect you
>  "automatically" the ISP's dont hand out certificates. 
Check if the "cert.pem" symlink points to something like this:

[root@kiera tls]# pwd
/etc/pki/tls

[root@kiera tls]# ls -l
total 16
lrwxrwxrwx  1 root root    49 Nov  1 14:11 cert.pem ->
/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
drwxr-xr-x. 2 root root  4096 Aug  8 15:02 certs
drwxr-xr-x. 2 root root    93 Jun  5 21:38 misc
-rw-r--r--  1 root root 10923 Jun  5 15:07 openssl.cnf
drwxr-xr-x. 2 root root    26 Jun  5 21:18 private

>  And is this a recent change because about a year ago I tried
>  mutt from Homebrew on a Mac and it worked then and I was asked
>  to accept the certificate. 
Yes, you're right, it should. Tried it some minutes ago.
Since I'm neither using a Fedora mutt nor a Fedora openssl, my setup
might not be transferable.

If the certificate is in place, maybe you could invoke openssl
directly, to see what causes the negotiation failure?

[AV]

On Sat, Nov 01, 2014 at 12:39:52PM +0100, Heinz Diehl wrote:

>  On 31.10.2014, Alexander Volovics wrote:  
 
>  > Does it make any difference if mutt is compiled with
> '--with-gnutls'
>  > enabled or with '--with-openssl' enabled. 
 
>  When compiled with "--with-ssl", it uses openssl for TLS,
>  and with "--with-gnutls" it uses the gnutls implementation.
>  (Btw: there is no "--with-openssl" configure option). 
Slip of the tongue :)

>  > Using the same .muttrc file I use in fedora the TLS connection
>  > 'aborts' and I get the error message:
>  > "SSL failed, I/O error. Could not negotiate TLS connection". 
 
>  Is the certificate properly installed in /etc/pki/tls/certs?
>  Openssl looks in that place and aborts if the servers certificate
>  could not be validated. Gnutls offers you to accept it manually,
>  if the host name does not match. 
Is that so. I didn't know that. How are you supposed to get
the certificate then. Given that all the most used mail progs
(thunderbird, outlook, apple mail, evolution, etc) connect you
"automatically" the ISP's dont hand out certificates.

So I was expecting Openssl to work like Gnutls.
And is this a recent change because about a year ago I tried
mutt from Homebrew on a Mac and it worked then and I was asked
to accept the certificate. And if the Homebrew folks link to
openssl now I expect they did the same then (with an earlier
version of openssl).

So everything should work if I copy the certificate from my
Linux pc to the Mac?
I shall try it tonight.

Thanks
AV

[Heinz Diehl]

On 31.10.2014, Alexander Volovics wrote: 

>  Does it make any difference if mutt is compiled with
> '--with-gnutls'
>  enabled or with '--with-openssl' enabled. 
When compiled with "--with-ssl", it uses openssl for TLS,
and with "--with-gnutls" it uses the gnutls implementation.
(Btw: there is no "--with-openssl" configure option).
 
>  Using the same .muttrc file I use in fedora the TLS connection
>  'aborts' and I get the error message:
>  "SSL failed, I/O error. Could not negotiate TLS connection". 
Is the certificate properly installed in /etc/pki/tls/certs?
Openssl looks in that place and aborts if the servers certificate
could not be validated. Gnutls offers you to accept it manually,
if the host name does not match.

(Disclaimer: I'm not using a Fedora mutt).