Exchange Extended Protection and HAProxy

A reader, who wishes to remain anonymous, has kindly sent me his scripts and configuration for Let's Encrypt, HAProxy and Exchange 2019 in conjunction with Extended Protection so that I can publish them here. First of all, I would like to thank you very much for this, because I think this configuration (HAProxy as reverse proxy for Exchange, certificates via Let's Encrypt) should be more common.

For Windows Extended Protection to work, the same certificates must be used on all Exchange servers, load balancers and reverse proxies. It is not enough for the certificates to contain the same DNS names; they must be the same certificate. So if you use a reverse proxy such as HAProxy and automatically request and configure a certificate there via Let's Encrypt, you must also use this certificate for the Exchange servers.

The friendly reader has created several small scripts for this purpose. The first script is a customized configuration for Certbot. Certbot uses it to renew the certificate at 4:00 am. The script or configuration is saved in the file /etc/systemd/system/certbot.timer.d/override.conf:

[Timer]
OnCalendar=
OnCalendar=*-*-* 04:00:00
RandomizedDelaySec=900

The next script under /etc/letsencrypt/renewal-hooks/deploy/restart_services ensures that a script called copy_cert.sh is called, which converts the Let's Encrypt certificate into PKCS12 format and saves it on a share:

#! /bin/bash
for DOMAIN in $RENEWED_DOMAINS; do
case $DOMAIN in
exchange.mycompany.com)
/var/local/ExchangeCert/copy_cert.sh
;;
esac
done

The script /var/local/ExchangeCert/copy_cert.sh converts the certificate into the PKCS12 format (PFX file) and copies it to a share so that it can later be imported on the Exchange servers:

#! /bin/bash
#
# Converts the certificate generated by Let's Encrypt for use with the Exchange server and
# saves it together with a timestamp file on a network share for exchange.
LE_PATH="/etc/letsencrypt/live/exchange.mycompany.de"
MOUNT_TARGET="//fileserver/Documents/ExchangeCert"
MOUNTPOINT="/mnt/exchangecert"
PFX="${MOUNTPOINT}/exchange_cert.pfx"
# Determine the directory of this script
SCRIPT_PATH=$(dirname $(realpath $0))
# Mount network share
test -d ${MOUNTPOINT} || mkdir ${MOUNTPOINT}
mountpoint -q ${MOUNTPOINT} || mount -t cifs ${MOUNT_TARGET} ${MOUNTPOINT} -o "credentials=${SCRIPT_PATH}/smbcredentials"
# Convert PEM to PFX.
# A password is required for the import, which is taken into account here.
openssl pkcs12 -inkey ${LE_PATH}/privkey.pem -in ${LE_PATH}/cert.pem -certfile ${LE_PATH}/chain.pem -export -out ${PFX} -name "$(date +'%Y-%m-%d') Letsencrypt from proxy" -passout pass:dummy
# Save timestamp to file
echo "$(date +'%Y-%m-%d %H:%M:%SZ')" > ${MOUNTPOINT}/timestamp_certcopied.txt

The access data for the share is saved in the file /var/local/ExchangeCert/smbcredentials. This file has the following content:

username=
password=
domain=

A PowerShell script is used on the Exchange servers, which imports the certificate from the share into the Exchange server. This script is executed by a scheduled task every day after 04:00:

# With activation of "Exchange Extended Protection" (as recommended by Microsoft) it is necessary,
# that the same certificate is set up on the Exchange server as on the server that accepts the
# connections from the Internet. Otherwise Outlook would always ask for the password of the
# user's password.
#
# This script checks whether the certificate provided by the proxy (or the generated timestamp file) is
# is more up-to-date than the previous one and then installs/activates it for the Exchange server if necessary.
# must run this script at the times when the proxy can be updated,
# must be started regularly via a task so that after the renewal of the proxy and the renewal of
# at the Exchange server as little time as possible passes (as the users are no longer able to access the Exchange server during this time due to the
# Extended Protection, users can no longer access the Exchange server).
Assign # variables
$Source_Dir = "\\fileserver\Documents\ExchangeCert"
$CertFile = "$Source_Dir\exchange_cert.pfx"
$TimestampFile = "$Source_Dir\timestamp_lastimport.txt"
$LogFile = "$Source_Dir\powershell_log.txt"
$MailDomain = "exchange.mycompany.com"
function WriteLog
{
Param ([string]$LogString)
$Stamp = (Get-Date).toString("yyyy-MM-dd HH:mm:ss")
$LogMessage = "$Stamp $LogString"
Add-content $LogFile -value $LogMessage
# Additional output to console
Write-Output $LogMessage
}
# Variable can be (temporarily) set to true for debugging.
[bool] $ShouldImport = $false
# Check whether the PFX file has a newer date than the timestamp of the last import
if (Test-Path -Path $TimestampFile -PathType Leaf) {
$filePFX = Get-ChildItem $CertFile
$fileTimestamp = Get-ChildItem $TimestampFile
WriteLog "Date certificate: $($filePFX.LastWriteTime.ToString("s"))"
WriteLog "Date Timestamp: $($fileTimestamp.LastWriteTime.ToString("s"))"
if ($filePFX.LastWriteTime -gt $fileTimestamp.LastWriteTime) {
WriteLog "Current certificate is newer than the last import"
$ShouldImport = $true
}
} else {
$ShouldImport = $true
}
# Import
if ($ShouldImport) {
Clear # log file to log only the last execution (and save a log rotate)
Clear-Content -Path $LogFile
# Get Exchange-Snapin
# Source: https://github.com/win-acme/win-acme/blob/master/dist/Scripts/ImportExchange.ps1
WriteLog "Searching for Exchange snapin..."
Get-PSSnapin -Registered `
| Where-Object {
$_.Name -match "Microsoft.Exchange.Management.PowerShell" `
-and (
$_.Name -match "Admin" -or
$_.Name -match "E2010" -or
$_.Name -match "SnapIn"
)
} `
| Add-PSSnapin -ErrorAction SilentlyContinue -PassThru `
| WriteLog
# Test if the Cmdlet is there now
$Command = Get-Command "Enable-ExchangeCertificate" -errorAction SilentlyContinue
if ($Command -eq $null)
{
WriteLog "Exchange Management Tools for Powershell not installed"
return
}
WriteLog "Import..."
try
{
# The certificate must be password-protected for the import. When exporting on the proxy and taken into account here accordingly.
# Unfortunately, the import function does not return an object, so that a direct assignment to "$ImportCert" (see below) is not possible.
Import-ExchangeCertificate -FileData ([Byte[]]$(Get-Content -Path $CertFile -Encoding byte -ReadCount 0)) -Password (ConvertTo-SecureString -String 'dummy' -AsPlainText -Force) -ErrorAction Stop | Out-Null
WriteLog "Certificate has been imported"
}
catch
{
WriteLog "Error during import-ExchangeCertificate"
WriteLog "Exception: $_"
throw
}
# Assign certificate to Exchange/IIS
try
{
WriteLog "Updating Exchange services..."
# Determine the latest certificate
$ImportCert = Get-ExchangeCertificate | where {$_.Subject -match $MailDomain} | Sort-Object NotBefore | Select -Last 1
WriteLog "ImportCert: $ImportCert"
WriteLog "Enable-ExchangeCertificate"
Enable-ExchangeCertificate -Services IIS -Thumbprint $ImportCert.Thumbprint -Force -ErrorAction Stop
WriteLog "Certificate assigned to Exchange"
}
catch
{
WriteLog "Error with Enable-ExchangeCertificate"
WriteLog "Exception: $_"
throw
}
# Adjust timestamp
WriteLog "Update timestamp..."
Get-Date -Format "u" | Out-File -FilePath $TimestampFile
# Clean up
try
{
WriteLog "Delete expired certificates"
# Select all expired certificates.
# The certificate that has just been renewed is normally valid for another month and therefore remains valid for the time being.
Get-ExchangeCertificate | where {$_.Subject -match $MailDomain -and $_.NotAfter -lt $today} | Remove-Item
}
catch
{
WriteLog "Error deleting the old certificates"
WriteLog "Exception: $_"
throw
}
WriteLog "Exit"
}

I have not recreated the script or configuration myself. However, the configuration and the scripts look conclusive to me. Last but not least, the friendly reader also sent me his HAProxy configuration:

global
# Log HTTP requests, but later write only those relating to tarpit to haproxy-tarpit.log via rsyslog.
# The remaining log entries are still written normally in haproxy.log.
log /dev/log local0 info
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# generated 2021-12-22, Mozilla Guideline v5.6, HAProxy 2.2.9, OpenSSL 1.1.1k, intermediate configuration
# https://ssl-config.mozilla.org/#server=haproxy&version=2.2.9&config=intermediate&openssl=1.1.1k&guideline=5.6
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
ssl-default-bind-options prefer-client-ciphers no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
defaults
log global
mode http
option httplog
option dontlognull
timeout http-request 5s
timeout connect 5s
# Avoid connection resets due to higher timeouts, see
# https://discourse.haproxy.org/t/high-number-of-connection-resets-during-transfers-exchange-2013/1158/6
# Client/server timeouts were previously set to 50 seconds. At least no problems were noticed.
timeout client 1000s
timeout server 1000s
listen stats
bind *:9000
mode http
stats enable
stats uri /
stats auth admin:ENTER PASSWORD HERE
frontend fe
# SSL: Specifying a directory loads all certificates available in it.
# HAProxy uses the appropriate certificate depending on the requested domain.
bind :80
bind :85 ssl crt /etc/haproxy/ssl/
bind :443 ssl crt /etc/haproxy/ssl/
# Pass through Let's Encrypt for Proxmox Mail Gateway.
# Is processed there.
acl domain_is_mailgateway hdr_dom(host) -i mail.example.de
# Let's Encrypt for everything else
acl is_certbot path_beg /.well-known/acme-challenge/
# Redirect from http to https
# 301 for "Moved Permanently"
http-request redirect scheme https code 301 unless { ssl_fc } or is_certbot or domain_is_mailgateway
# Depending on the condition or domain, redirect to the corresponding target server
use_backend ProxmoxMailGateway if domain_is_mailgateway
use_backend certbot if is_certbot
# Configuration for Exchange
# Uses "exchange.example.de" and "autodiscover.example.de"
no option httpclose
acl eas path_beg -i /Microsoft-Server-ActiveSync
acl autodiscover path_beg -i /autodiscover
acl ews path_beg -i /ews
acl mapi url_beg /mapi
acl oab path_beg -i /oab
acl owa path_beg -i /owa
acl rpc path_beg -i /rpc/rpcproxy.dll
acl ecp path_beg -i /ecp
# Not sure if the favicon.ico is only requested when OWA is called, but since it is otherwise
# ends up in the tarpit, it is simply passed on to OWA.
acl owa path_beg -i /favicon.ico
use_backend ExchangeActiveSync if eas
use_backend ExchangeAutoDiscover if autodiscover
use_backend ExchangeECP if ecp
use_backend ExchangeEWS if ews
use_backend ExchangeMAPI if mapi
use_backend ExchangeOAB if oab
use_backend ExchangeOWA if owa
use_backend OutlookAnywhere if rpc
# The rest (attacks, scans, etc.) ends up in the tarpit.
default_backend tarpit
backend certbot
server certbot 127.0.0.1:9080
backend ProxmoxMailGateway
server MEINPMGSERVER 192.168.120.5
# Exchange backends
# Source:
# https://www.bayreuth.tk/home/linux-und-bsd/haproxy-als-loadbalancer-und-ssl-offloader-fuer-microsoft-exchange-cas-server.html
# Since we only use one Exchange server, everything with sticky table removed.
backend ExchangeActiveSync
option httpchk HEAD /Microsoft-Server-ActiveSync
http-check expect status 401
server MEINEXCHANGESERVER 192.168.120.3:443 ssl verify none
backend ExchangeOWA
option http-server-close
option redispatch
option httpchk GET /owa
http-check expect status 301
server MEINEXCHANGESERVER 192.168.120.3:443 ssl verify none
# Requests to ECP are not forwarded for security reasons, as this is used for administration.
# also blocks things like the recognition of absences, but this is not important for us.
backend ExchangeECP
http-request deny if TRUE
backend ExchangeEWS
option httpchk GET /ews
http-check expect status 401
server MEINEXCHANGESERVER 192.168.120.3:443 ssl verify none
backend ExchangeAutoDiscover
server MEINEXCHANGESERVER 192.168.120.3:443 ssl verify none
backend ExchangeOAB
server MEINEXCHANGESERVER 192.168.120.3:443 ssl verify none
backend ExchangeMAPI
server MEINEXCHANGESERVER 192.168.120.3:443 ssl verify none
backend OutlookAnywhere
option redispatch
server MEINEXCHANGESERVER 192.168.120.3:443 ssl verify none
# Tarpit: Thwart attackers
# https://www.haproxy.com/de/blog/use-haproxy-response-policies-to-stop-threats/
# Would otherwise land on "fe/" and get 503 errors, but would not be delayed.
# Offers no protection, but ties up resources on the attacker's side (also for us).
backend tarpit
timeout tarpit 10s
http-request tarpit deny_status 403 if TRUE

This scenario can therefore be replicated quite easily. The scripts still need to be adapted a little, for example access data, paths and IPs, but this should not be a problem. I will try out these scripts in a quiet minute.

Exchange Extended Protection and HAProxy

17 thoughts on “Exchange Extended Protection und HAProxy”

  1. It works almost fine, only ExchangeMAPI is not working for me. It constantly ask for user and password when connecting via Outlook through /mapi. OWA is working fine, activesync is working fine.

    Reply
  2. Hi, habe mich eingelesen und getestet…nur kommt ein Problem, das Zertifikat wird importiert, ist aber nur in der MMC-Konsole sichtbar und nicht im Admin-Center und auch nicht per Get-ExchangeCertificate zu sehen. Hat da jemand eine Idee ?

    Reply
  3. Leider ist die Anleitung auf meiner OpenSense FW mit HaProxy und LetsEncrypt Plugin nicht umsetzbar.
    Weder die genannten Pfade noch die Dateien existieren, auch eine Volltextsuche brachte keine Licht ins dunkel.
    Hat das schon jemand auf OpenSense/pfSense umgesetzt?

    Reply
    • Falls das Thema noch aktuell ist.
      Ich habe aktuell die gleiche Situation.
      Hier meine Lösung:
      Im opnSense unter Services->ACME Client->Automations einen Eintrag erstellen und „Upload certificate via SFTP“ auswählen.
      (auf dem Exchange entsprechend den openssl-server aktivieren, einen Nutzer einrichten, Ordner anlegen und openSSL entsprechend konfigurieren

      Im Gegensatz zu der hier vorgestellten Lösung würde ich das Importieren der Zertifikats nicht über die Uhrzeit regeln sondern einen Filesystem-Watcher installieren, der bei Änderung des Zertifikats den Import triggert.

      Reply
  4. Interessante Lösung, danke fürs Teilen (und an den Autor ).

    Beim Überfliegen fiel mir allerdings eine Kleinigkeit auf: in Zeile 40 des Powershell-Scripts sollte $ShouldImport doch eher auf false stehen, oder nicht? So wie es aktuell eingestellt ist wird (wenn ich den Code richtig lese) der Timestamp-Check quasi ausgehebelt und das Zertifikat jedes Mal ausgetauscht wenn das Script läuft.

    Reply
    • Nein, der Abschnitt wird ausgeführt, wenn die Marker-Datei noch nicht existiert, also bei der allerersten Ausführung. Da hätte der Kommentar besser formuliert sein, bzw. vor der folgenden if-Abfrage stehen können, um es deutlicher zu machen.

      Reply
  5. hallo

    ist es ok dass in if , beide werte $ShouldImport auf True ??

    if ($filePFX.LastWriteTime -gt $fileTimestamp.LastWriteTime) {
    WriteLog „Aktuelles Zertifikat ist neuer als der letzte Import“
    $ShouldImport = $true
    }
    } else {
    $ShouldImport = $true
    }

    Reply
  6. Genial, das Du und in diesem Fall ein anonymer Kollege uns teilhaben lässt an seiner Arbeit und KnowHow.

    1000*Danke dafür.

    Reply
  7. Sehr schön,
    und danke auch von mir an den Leser!
    ich mach diese Schritte seit einer Weile jeden Monat manuell, weil ich nicht die Zeit finde das zu scripten.
    ich werde es morgen mal bei mir einbauen.
    Ich frage mich allerdings, wieso HAProxy als reverse Proxy benutzt wird und nicht zum Beispiel nginx. da kann man doch gleich Pfade sperren oder nur auf bestimmte Clients freischalten. Zum beispiel owa nur für bestimmte Smartphones mit bestimmten User Logins….

    Reply
    • Hallo Christop
      Ich nutze den HAproxy schon länger als PlugIn auf der pfSense

      >da kann man doch gleich Pfade sperren oder
      Ich denke, HAproxy kann auch mir Pfaden umgehen – oder täusche ich mich da ?

      > nur auf bestimmte Clients freischalten. Zum beispiel owa nur für bestimmte Smartphones mit bestimmten User Logins….
      WIe muss ich mir das vorstelle ? Wie muss ich mir das vorstellen ?
      Danke und Grüße, Rolf

      Reply
    • Dachte mir, eine Proxy-Software wäre besser als ein Webserver für einen Proxy ;-)

      Hatte mit nginx bisher nicht viel zu tun und die Anleitungen, auf denen das basiert, haben halt HAProxy verwendet. Also eigentlich kein konkreter Grund, HAProxy zu verwenden, bzw. bisher keine Notwendigkeit für nginx gehabt.

      Falls Du das mit nginx umsetzen solltest, würde mich Deine Konfig aber auch interessieren.

      Reply
      • Viel Spaß mit Nginx und Ex2019 CU17+

        Ab dann muss entweder der Exchange so auf forced basic-auth umgestellt werden, oder die commercial license von nginx.

        Reply
        • wegen kerberos, welches der gratis nginx nicht kann, oder? ntlm geht auch nicht? geht kerberos mit haproxy?

Leave a Comment