TLS Server Name Indication Extension and Unrecognized_name

Heavy Pond, Jianfengling National Forest Park, Ledong, Hainan, China

It's getting hot that some TLS/HTTPS server failed with "unrecognized_name". For example, the Adobe AIR 3 Code Signing Certificate Problem, the ADT handshake alert, and the jarsigner issue with timestamp.geotrust.com, etc. This entry will discussion some background of the "unrecognized_name" alert, and the TLS Server Name Indication (SNI) extension.

Background

"Unrecognized_name" is an error alert, define by RFC4366.  In section 4 of RFC4366:

- "unrecognized_name": this alert is sent by servers that receive a server_name extension request, but do not recognize the server name.  This message MAY be fatal.

And in section 3.1 of of RFC4366:

If the server understood the client hello extension but does not recognize the server name, it SHOULD send an "unrecognized_name" alert (which MAY be fatal).

From above sections, we see that "unrecognized_name" is related to "the server name" or "server_name" extension. What is the "server name"? In section 3.1 of RFC4366:

3.1. Server Name Indication 

TLS does not provide a mechanism for a client to tell a server the name of the server it is contacting.  It may be desirable for clients to provide this information to facilitate secure connections to servers that host multiple 'virtual' servers at a single underlying network address.

In order to provide the server name, clients MAY include an extension of type "server_name" in the (extended) client hello.

Let's Look Into a Case

There is a lot of discussion about the timestamp service from timestamp.EXAMPLE.com. The service is hosted as "https://timestamp.EXAMPLE.com/tsa". Let's look at the scenarios of the access:

1. Client want to negotiate a TLS connection with timestamp.EXAMPLE.com. By default, the server name indication extension will be included in the client hello. The server name is "timestamp.EXAMPLE.com".  It is expected that:
• The server does not understand the server name indication extension. No problem, the server will ignore the extension, will continue the negotiation.  No impact on the TLS negotiation/handshaking.
• The server understands the server name indication extension, but does not recognize the server  name. In this case, that's to say, the server does not think that the server name in the request, "timestamp.EXAMPLE.com", is its supported/recognized server name.
• The server is able to recognize the server name in the request. The handshaking will go on.


2.  The server response with a "unexpected_message" fatal alert. As means that the server understand the server name indication extension, but it does not recognize the server name. It looks a little weird in this case in that the server is "timestamp.EXAMPLE.com", but it does not admit that it is "timestamp.EXAMPLE.com".
3. The client have to terminate the negotiation because the server has denied to continue.

 It is strange, is it? The server, "timestamp.EXAMPLE.com", said it is not ""timestamp.EXAMPLE.com".

Know nothing about the details in TSA server, timestamp.EXAMPLE.com, it looks more like a configuration problem in the server side.

Workaround

It would be nice if the timestamp.EXAMPLE.com server would like to correct the behaviors in server side.  But before the correction, many applications may need to work day to day. As a workaround, the client can disable the server name indication in client hello.

Java platform starts to support server name indication extension from JDK 7.  However, considering the compatibility issues, a system property, "jsse.enableSNIExtension", is defined to disable the extension in TLS handshaking:

jsse.enableSNIExtension system property.

Server Name Indication (SNI) is a TLS extension, defined in RFC4366.It enables TLS connections to virtual servers, in which multiple servers for different network names are hosted at a single underlying network address.

Some very old SSL/TLS vendors may not be able handle SSL/TLS extensions. In this case, set this property to false to disable the SNI extension.

OK, let looks at an example in jarsigner in JDK 7.  The 2nd one uses the system property, jsse.enableSNIExtension, to disable the server name indication extension in client.

$ jarsigner -keystore myKeyStore -tsa https://timestamp.EXAMPLE.com/tsa \
      -signedjar signedjar.jar toBeSigned.jar myKeyAliasInKeyStore

jarsigner: unable to sign jar: javax.net.ssl.SSLProtocolException: \
      handshake alert:  unrecognized_name

$ jarsigner -J-Djsse.enableSNIExtension=false \
      -keystore myKeyStore -tsa https://timestamp.EXAMPLE.com/tsa \
      -signedjar signedjar.jar toBeSigned.jar myKeyAliasInKeyStore

Hope it helps!



More blog entries about TLS Server Name Indication (SNI) Extension:
JEP 114: TLS SNI Extension - SunJSSE Behavior Changes
JEP 114: TLS SNI Extension - SunJSSE Behavior Changes (Continue)
JEP 114: TLS SNI Extension - Typical User Cases
JEP 114: TLS SNI Extension - Virtual Servers Dispatcher