In part 1 of this blog post, I wrote about the background of TLS. In this second part I show you how VCD (VMware Cloud Director) and often also the Load Balancer can be configured to enhance security, integrity and page speed.
In this multi-part series I hope you benefit from not only knowing the best setting, but more important, why some settings should be used in favor of others.
Part 1: TLS background and preferred settings can be found by clicking this link.
Getting the “A” grade
A proper secured portal should have priority to anyone, since it is the entrance for customers to the offered services. So, which grade should you strive for. In my opinion you should aim for “A” grade by the SSL Labs test tool. When configuring the basics right, it should not pose much of a problem.
Getting an “A” means the average score in each of the categories below is at or above 80 in a scale from 0 to 100.
- Certificate check: Certificate and chain should be okay.
- Protocol check: All SSL and TLS version prior to 1.2 should be disabled.
- Key exchange check: Strong key exchange ECDHE (PFS and AEAD enabled) should be configured and preferred
- Cipher check: Strong ciphers should be configured and preferred (GCM, CCM)
What about the “A+” rating
To get the highest possible “A+” rating, some additional steps need to be taken:
- Enable TLS 1.3
- Configure Strict Transport Security (HSTS) with long duration
- Configure DNS CAA policy for the domain
Back to Cloud Director
If you have configure certificates with Cloud Director previously, you may know that you have control over the major aspects of the TLS configuration. You can enable / disable specific TLS versions and able to change key exchange protocols and ciphers suits. A good thing is that HSTS is enabled by default and does not need additional configuration.
My advise is to check the TLS configuration after every upgrade. Let’s check the current config:
root@cell [ path]#./cell-management-tool ssl-protocols -l Allowed SSL protocols: TLSv1.2
This is fine by default. The product also support TLS v1 and 1.1, but both should not be enabled for the reasons mentioned earlier. Hopefully TLS 1.3 will be added in a future VCD version.
Now on to the key exchange and cipher suits. You can list suits that are available in the product and change them if needed. The default configuration is pretty good and only has the weak (non PFS) TLS_RSA key exchange method enabled. Probably for compatibility reasons. Unless you need it, disable it.
The list below contains the default enabled key exchange and cipher suites, minus the TLS_RSA mentioned above.
root@cell [ path]#./cell-management-tool ciphers -l Product default ciphers: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
If you want to reconfigure the allowed cipher list, export the full list (-a option) to an output file. Afterwards remove the first line + the ciphers that remain enabled. To generate the list, use the command below.
root@cell [ path]#./cell-management-tool ciphers -a > /home/user/ciphers-disallow.txt
The command below you can disables the weak ciphers. After doing so, restart the VCD service to make the new configuration active.
root@cell [ path]#./cell-management-tool ciphers -d "$(< /home/user/ciphers-disallow.txt)"
What you cannot do in VCD is to prefer some key exchange protocols and cipher suits over others. Normally you would use use this to prefer PFS over non PFS or prefer GCM over CBC. Secondly TLS 1.3 is currently not available in VCD.
When using the configuration mentioned above, having an CAA configured and implemented the whole certificate chain, all should be okay to get the “A” grade.
Using Load Balancers
In most VCD production environments you will have 3 or 4 cells that will be connected to the internet via a load balancer. That being said, the advised TLS configuration in this post should be applied to the LB and not only to the cells. The LB will normally perform the SSL termination and depending on its configuration may connect over HTTPS to the cells.
If performing the SSL termination on the LB, the “A” grade is almost always possible. If you’re aiming for the “A+” grade and use VMware products, aim for the NSX Advanced Load Balancer (former Avi Vantage), which supports TLS 1.3 and HSTS. The NSX-T and NSX-V LB only go up to TLS 1.2 and don’t seem to support HSTS.
For a good write-up how to configure VCD together with NSX-T (Advanced) LB check the links section at the bottom for awesome posts by Hugo Phan and Mohammed Alhussein on this topic.
After a quick search for other popular LB’s like F5 and Fortigate, both seem to support TLS 1.3 and HSTS in the latest software versions.
NSX Load Balancers
If you use one of the NSX based Load Balancers, let me show you how to change the TLS and cipher config to your needs.
NSX-V Load Balancer
By default TLS 1.1 and 1.2 are enabled on the NSX Edge. First disable the 1.1 version by creating an “Application Rule” and add the rule to the “Virtual Server” in the LB properties of the Edge.
Secondly you can change the used ciphers in the used “Application Profile”. Change the parameters of the “Server SSL” config, since the LB uses this config to communicate with your browser. The “Client SSL” config is used when the LB itself communicates over HTTPS to the backend. HTTPS to the backend is used when “Application Profile Type”, “HTTPS End-to-End” is used.
NSX-T Load Balancer
To configure TLS and ciphers in the NSX LB, open the NSX-T Manager UI and go to “Networking” > “Load Balancer” > “Profiles” and select “SSL” in the “Select profile type” pull-down menu.
Use one of the predefined server profiles (browser to LB) or create a custom profile that suit your needs. In the server profile you can change the TLS and cipher config.
NSX Advanced (Avi Vantage) Load Balancer
First I must admit to have never actually worked with the product and only saw a couple of presentations so far. Based on this, the product looks very powerful and has many features not present in NSX-V and NSX-T. Hopefully I will able to get hands-on experience with the product soon. Also good to see the Avi product is available to service providers in the VMware VCPP program and will replace the NSX-T LB eventually.
Back to the LB. Fortunately the Avi documentation site provided the right information how to configure TLS versions and cipher suits. HSTS support is available in the product and since version 18.2.6 Avi also TLS 1.3.
The TLS and cipher suits can be configured using SSL/TLS Profiles. You can create profiles by going to “Templates” > “Security” and open the “SSL/TLS Profile” tab.
Now create a new “SSL/TLS Profile”. In the profiles, all necessary options are there (and more). You can even set the preferred cipher order.
Using “Application Profiles” HSTS can be configured. Create these profiles by going to “Templates” > “Profiles” and open the “Applications” tab.
When creating a new “Application Profile” the HSTS setting can be found on the “Security” tab amongst other important settings like “X-Forwarded-Proto”.
Configuring TLS the right way is not the only aspect of a proper configured Cloud Director portal. Also check out the Cloud Director Security guide for additional security recommendations. Getting an “A” grade or better proves you have the proper TLS settings configured.
Over time security requirements change. For example all the major browsers deprecated TLS 1.0 and 1.1 on January 2020. SSL Labs changes their methods of checking and grading accordingly. So periodic checking really makes sense. For example after a yearly renewal of you certificate or after upgrading your Load Balancer version.
While researching and writing this post, together with testing all of this in my lab, I have learned quite a bit about this topic. The good thing is that the knowledge is applicable to TLS secured applications and websites in general.
Two of the sites I used the most for writing this post is are the SSL Labs SSL / TLS Best Practices and Wikipedia: Transport Layer Security pages. Both provide excellent background information in which the former is the best starting point. So, credits to Nauman Shah.
SSL Labs: SSL / TLS Best Practices
SSL vs. TLS, what’s the difference
Understanding SSL Certificates
Wikipedia: Transport Layer Security
VCD NSX-T (Advanced) Load Balancer posts
Protecting Cloud Director with NSX-T Load Balancer L7 HTTP Policies
Load Balancing and Protecting Cloud Director with Avi Networks
Configure NSX-T 3.x Load Balancer for Cloud Director 10.x cells
Wikipedia: Authenticated encryption (AEAD)
Wikipedia: Forward secrecy (PFS)
Wikipedia: HTTP Strict Transport Security
Wikipedia: Hash-based message authentication code (HMAC)
Wikipedia: Secure Hash Algorithm (SHA)