Microsoft Exchange Server Certificates

When Exchange Server certificates expire, it’s the responsibility of the System Administrator to update those certs. Here’s a sequence of execution to ensure server up time.

Quick PowerShell Commands:

Part 1: Self-Signed Certs

# Option A:
# Renew expired certificates in-place
$expiredCertThumbprints=(Get-ExchangeCertificate | where {$_.Status -eq "Valid" -and $_.IsSelfSigned -eq $true -and $_.NotAfter -lt $(get-date)} | Select Thumbprint).Thumbprint
$expiredCertThumbprints|Get-ExchangeCertificate -Thumbprint $_ | New-ExchangeCertificate -PrivateKeyExportable $true

# Option B:
# Check for self-signed certs - empty output means no expiring certs
$daysThreshold=30
Get-ExchangeCertificate | where {$_.Status -eq "Valid" -and $_.IsSelfSigned -eq $true -and $_.NotAfter -lt $(get-date).addDays($daysThreshold)} | `
	Format-List FriendlyName,Subject,CertificateDomains,Thumbprint,NotBefore,NotAfter

# Create a new cert for each expired cert and assign newly generated certs thumbprints into a variable
$exchangeServerName=$env:computername
$domain="$env:userdnsdomain"
$fqdn="$exchangeServerName.$domain"
$expiredCertNames=(Get-ExchangeCertificate | where {$_.Status -eq "Valid" -and $_.IsSelfSigned -eq $true -and $_.NotAfter -lt $(get-date).addDays($daysThreshold)} | Select FriendlyName).FriendlyName
$thumbprints=($expiredCertNames|%{New-ExchangeCertificate -FriendlyName $_ -SubjectName CN=$domain -DomainName $domain,$fqdn -PrivateKeyExportable $true}|Select Thumbprint).Thumpprint

# Add new certs into Exchange
$date=get-date
$thumbprints|%{Set-AuthConfig -NewCertificateThumbprint $_ -NewCertificateEffectiveDate $date}

# Publish new cert
Set-AuthConfig -PublishCertificate

# Remove old cert
Set-AuthConfig -ClearPreviousCertificate

Part 2: Public Certs

$certFile="\\FILESHERVER01\Software\Certificates\kimconnect_cert.pfx"
$plaintextPassword='$!JFQ$j0(jeVE$%@%$JADJAO%a94 jab j0w3jijrarski5034j-i-b-54-353j5-j@!'
$friendlyName=$env:USERDNSDOMAIN -replace ".$($env:USERDOMAIN)"
if($friendlyName){	
	Import-ExchangeCertificate -FileName $certFile `
	-Password (ConvertTo-SecureString -String $plaintextPassword -AsPlainText -Force) `
	-FriendlyName $friendlyName -PrivateKeyExportable $true -Services SMTP,IIS,UM,UMCallRouter,POP,IMAP
	}	

# Check Exchange Service Health
Test-ServiceHealth	

Current Instructions (Applicable to Updating Mailbox roles)

Obtain Public Cert

Get a public cert from a certificate authority such as Commodo, Godaddy, etc.

Optional: Create Updated Self Signed Cert

# Create New Cert
$domain="EXCH01"
$fqdn="$domain`.intra.domain.net"
$friendlyName="Exchange Certificate"
New-ExchangeCertificate -FriendlyName $friendlyName -SubjectName CN=$domain -DomainName $domain,$fqdn -PrivateKeyExportable $true

Add Updated Cert into Certificates Store of Local Machine and Remove Old Cert

Using Exchange PowerShell

$friendlyName="kimconnect.com"
Import-ExchangeCertificate -FileName "\\FILESHERVER01\Software\Certificates\kimconnect_cert.pfx" -Password (ConvertTo-SecureString -String '$!JFQ$j0(jeVE@!' -AsPlainText -Force) -FriendlyName $friendlyName -PrivateKeyExportable $true -Services SMTP,IIS,UM,UMCallRouter,POP,IMAP

Warning

This certificate with thumbprint XXXXXXXXXX and subject '*.kimconnect.com' cannot used for POP SSL/TLS connections because the subject is not a Fully Qualified Domain Name (FQDN). Use command Set-POPSettings to set X509CertificateName to the FQDN of the service.

This certificate with thumbprint XXXXXXXXXX and subject '*.kimconnect.com' cannot used for IMAP SSL/TLS connections because the subject is not a Fully Qualified Domain Name (FQDN). Use command Set-IMAPSettings to set X509CertificateName to the FQDN of the service.

# Set POP & IMAP with fqdn using Exchange PS
Set-POPSettings -X509CertificateName mail.kimconnect.com
Set-IMAPSettings -X509CertificateName mail.kimconnect.com

# Restart affected services using System PowerShell (not Exchange PS)
Stop-Service MSExchangePop3
Start-Service MSExchangePop3
Stop-Service MSExchangeImap4
Start-Service MSExchangeImap4

Using GUI

Run: certlm.msc

1.  Add New Cert
   – Personal > right-click Certificates > All Tasks > Import > navigate to the location of the New Cert > OK > OK
   – Right-click New Cert
2. Back up Old Cert
   Right-click old cert > All tasks > Export > Next > select “Yes, export the private key” > Next > put a check mark next to “Export all extended properties” > Next > put a check mark next to “Password” > input password > Next > input file name (e.g. C:\certs\$certname_expiring_date.pfx) > Next > OK
3. Document private key permissions of old cert
   Right-click old cert > All tasks > Manage Private Keys
   
   Add Network Service with READ permissions
   
4. Remove old cert – WARNING: this will render URL access offline until new cert is rebound within IIS!
   Right-click old cert > delete > OK

Bind New Cert to IIS on Exchange Server

Run: inetmgr.msc > $serverName > Sites > Default Web Site > Bindings > Set these
– https 443 * = new Public Cert
– https 443 127.0.0.1 = Local Server Self Signed Cert (e.g. EXCH01.intranet.company.com)

Restart services

# Restart IIS
iisreset

# Restart Exchange Transport
Stop-Service MSExchangeTransport
Start-Service MSExchangeTransport

# Restart MSExchangeMailboxAssistants
Stop-Service MSExchangeMailboxAssistants
Start-Service MSExchangeMailboxAssistants

# Optional: restart Exchange Information Store
Stop-Service MSExchangeIS
Start-Service MSExchangeIS

Validate Exchange Functionality

# Check Exchange Service Health
Test-ServiceHealth

# Validate access to Exchange Control Panel
$fqdn="EXCH01.intranet.company.com"
Function getDefaultBrowser {
	#Get the default Browser path
	New-PSDrive -Name HKCR -PSProvider registry -Root Hkey_Classes_Root | Out-Null
	$browserPath = ((Get-ItemProperty ‘HKCR:\http\shell\open\command’).'(default)’).Split(‘”‘)[1]
	return $browserPath
}
getDefaultBrowser;
start "$fqdn"

Troubleshooting commands:

# Check for server roles (pay attention to Edge Transport roles as that role will require special Cert updating sequence)
Get-ExchangeServer | select name, serverrole, edition, admindisplayversion, isClientAccessServer | fl

# Check for self-signed certs
Get-ExchangeCertificate | where {$_.Status -eq "Valid" -and $_.IsSelfSigned -eq $true} | Format-List FriendlyName,Subject,CertificateDomains,Thumbprint,NotBefore,NotAfter

Raw Notes from Prior

1. Run inetmgr.exe (Internet Information Services Manager)

• Default Web Site:
  – https * 443 => fqdn public certificate (e.g. Issued To = *.kimconnect.com; Issued By = “DigiCert SHA2 High Assurance Server CA”)
  – https 127.0.0.1 443 => fqdn public certificate
• Exchange Back End:
  – https * 444 => private server certificate (e.g. Issued To = exch01.intranet.kimconnect.com, Issued By = KimConnect SHA2 CA)

2. Run services.msc to restart these services

• IIS Admin Service (IISReset /noforce)
  Microsoft Account Signin Assistance
  Microsoft Exchange Information Store
  Microsoft Exchange Mailbox Assistants
  Microsoft Exchange Forms-based Authentication Service (may not be available on some instances)
• Start an EMS > C:\Program Files\Microsoft\Exchange Server\V15\Bin\UpdateCas.ps1
• Run inetmgr.exe > Reset the OWA virtual directory

3. Check Exchange Service Health to validate that there are no ServicesNotRunning items

[PS] C:\Windows\system32>Test-ServiceHealth

Role                    : Mailbox Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeDelivery, MSExchangeIS,
                          MSExchangeMailboxAssistants, MSExchangeRepl, MSExchangeRPC, MSExchangeServiceHost,
                          MSExchangeSubmission, MSExchangeThrottling, MSExchangeTransportLogSearch, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Client Access Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeIMAP4, MSExchangeMailboxReplication,
                          MSExchangeRPC, MSExchangeServiceHost, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Unified Messaging Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeServiceHost, MSExchangeUM, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Hub Transport Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeEdgeSync, MSExchangeServiceHost,
                          MSExchangeTransport, MSExchangeTransportLogSearch, W3Svc, WinRM}
ServicesNotRunning      : {}

Check the OWA VD settings:

Get-OwaVirtualDirectory | FL Identity,*auth*,*url*
Get-EcpVirtualDirectory | FL Identity,*auth*,*url*

4. Validate access to Exchange Admin Center

– Authenticate to https://{fqdn}/ecp
– If HTTP error 503 or 500 occurs, try https://{fqdn}/ecp/?ExchClientVer=15

5. Additional troubleshooting items

Change the authentication method of the “owa” virtual directory to Windows authentication

set-Owavirtualdirectory -identity "E15MBX\owa (Exchange Back End)" -WindowsAuthentication $True -Basicauthentication $false -Formsauthentication $false
IISReset /noforce

Raw Notes:

Error:
Failed to connect to the Edge Transport server ADAM instance with exception The supplied credential is invalid.. This could be caused by a failure to resolve the Edge Transport server name EXCH-EDGE.intra.net in DNS, a failure trying to connect to port 50636 on EXCH-EDGE.intra.net, network connectivity issues, an invalid certificate, or an expired subscription. Verify your network and server configuration.

Process to Resolve:

Preliminary steps to rule out easy to fix problems:

# Verify connectivity from Hub to Edge
ran check-netconnection function to verify connectivity between exch-hub to EXCH-EDGE port 50636 with success = $true

# Check to see whether there are any error messages in the queue:
Get-queue

Restart some services on Hub and Edge servers
1. Restart the following services on MBX Server
Microsoft Exchange EdgeSync
Microsoft Exchange Transport 
2. Restart the following services on Edge Server
Microsoft Exchange ADAM
Microsoft Exchange Credential service 
Microsoft exchange Transport

# Confirm if the certificate meets the FQDN of Edge Server if it has been enabled for SMTP service
get-exchangecertificate | FL

Intermediate Level steps to address connector issues:

Mail flow:
Outlook client <==> Hub Exchange <==> Edge Exchange <==> Barracuda (smart host) <==> Internet <==> Destination email systems

Generalization:
- Hub uses EdgeSync to connect to the edge server via ADAM credentials and those are periodically changed by the "Edge Credential Service"
- Only the Client Access Role server requires public certs. The rest of the other roles does not require such.
- Connectors between Edge and Hub servers require SSL, and those can be private certs.
- If the Edge server cert is updated, New-EdgeSubscription command needs to be ran to generate a newEdgeSubcription.xml file
- The newEdgeSubscription.xml needs to also be ran on the Hub server to import new Edge connector information
- Make sure the credential service is up and running on the edge. 
- Call start-edgesynchronization is required to synchronize between Edge and Hub is a new subscription has been created
- Send connectors and Receive connectors can be automatically generated

[PS] C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Microsoft Exchange Server 2010>Start-EdgeSynchronization

RunspaceId : f9a541a8-51db-4b87-a392-b727eeae6c42
Result : CouldNotConnect
Type : Recipients
Name : EXCH-EDGE
FailureDetails : The supplied credential is invalid.
StartUTC : 8/9/2019 6:45:46 PM
EndUTC : 8/9/2019 6:45:46 PM
Added : 0
Deleted : 0
Updated : 0
Scanned : 0
TargetScanned : 0

RunspaceId : f9a541a8-51db-4b87-a392-b727eeae6c42
Result : CouldNotConnect
Type : Configuration
Name : EXCH-EDGE
FailureDetails : The supplied credential is invalid.
StartUTC : 8/9/2019 6:45:46 PM
EndUTC : 8/9/2019 6:45:46 PM
Added : 0
Deleted : 0
Updated : 0
Scanned : 0
TargetScanned : 0

[PS] C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Microsoft Exchange Server 2010>Get-EdgeSubscription

Name Site Domain
---- ---- ------
EXCH-EDGE intra.net/Co... intra.net

==============================================================================

Recreate Edge Subscription:

On Hub server

# Generate new private Exchange certificate
$domain="exch-hub"
$fqdn="exch-hub.intra.net"
New-ExchangeCertficate -DomainName $domain, $fqdn -PrivateKeyExportable $true -KeySize 2048

# Check certs
get-ExchangeCertificate

# Get more details about cert
# $newcert = get-ExchangeCertificate | ? { $_.certdate -like "blah blah"} | select name
$newcert="#######"
get-exchangecertificate $number | fl

# set iis to bind to new cert
# perform iisreset 
# backup old cert and remove it

# New-SendConnector -Custom -Name Baracudda -AddressSpaces * -smarthost 10.10.11.1 -ForceHELO $true -SmartHostAuthMechanism None -Source $edgeServer

# Remove Edge Subscription
Get-EdgeSubscription | Remove-EdgeSubscription

On Edge

# Clean up old certs
lmcert.msc > remove Microsoft Exchange ADAM from Personal Certs folder

# Remove Edge Subscription
Get-EdgeSubscription | Remove-EdgeSubscription

# Generate new subscription file
New-EdgeSubscription -Filename c:\newEdgeSubscription.xml
Re-start the Microsoft Exchange ADAM

On Hub server
# New-EdgeSubscription -FileData ([byte[]]$(Get-Content -Path "\\EXCH-EDGE\c$\newEdgeSubscription.xml" -Encoding Byte -ReadCount 0)) #Experimental command
New-EdgeSubscription -Filename c:\newEdgeSubscription.xml 
Start-EdgeSynchronization
Test-EdgeSynchronization

[PS] C:\Windows\system32>New-EdgeSubscription -Filename c:\newEdgeSubscription.xml

Confirm
If you create an Edge Subscription, this Edge Transport server will be managed via EdgeSync replication. As a result,
any of the following objects that were created manually will be deleted: accepted domains, message classifications,
remote domains, and Send connectors. After creating the Edge Subscription, you must manage these objects from inside
the organization and allow EdgeSync to update the Edge Transport server. Also, the InternalSMTPServers list of the
TransportConfig object will be overwritten during the synchronization process.
EdgeSync requires that this Edge Transport server is able to resolve the FQDN of the Hub Transport servers in the
Active Directory site to which the Edge Transport server is being subscribed, and those Hub Transport servers be able
to resolve the FQDN of this Edge Transport server. You should complete the Edge Subscription inside the organization in
the next "1440" minutes before the bootstrap account expires.
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"): y


New-EdgeSubscription : Microsoft Exchange couldn't create or update the Edge Subscription account on the Edge Transport
server for the following reason: The LDAP server is unavailable.. Stack is at System.DirectoryServices.Protocols.LdapConnection.Connect()
at system.DirectoryServices.Protocols.LdapConnection.BindHelper(NetworkCredential newCredential, Boolean needSetCredential)
at Microsoft.Exchange.MessageSecurity.EdgeSync.AdamUserManagement.CreateOrUpdateADAMPrincipal(String user, String password, Boolean bootStrapAccount, TimeSpan expiry)
at Microsoft.Exchange.Management.SystemConfigurationTasks.NewEdgeSubscription.InitiateSubscriptionOnEdge()
At line:1 char:21
+ New-EdgeSubscription <<<< -Filename c:\newEdgeSubscription.xml
+ CategoryInfo : InvalidOperation: (:) [New-EdgeSubscription], InvalidOperationException
+ FullyQualifiedErrorId : 780DB3C3,Microsoft.Exchange.Management.SystemConfigurationTasks.NewEdgeSubscription

# Check status of Exchange ADAM Services
Get-Service *ADAM* | ft Di*,St*

# Check Exchange certificates
[PS] C:\Windows\system32>Get-ExchangeCertificate | fl

AccessRules : {System.Security.AccessControl.CryptoKeyAccessRule, System.Security.AccessControl.CryptoKeyAccessR
ule}
CertificateDomains : {ab0ee702-f37f-4dff-bfb2-66698a441d9a}
HasPrivateKey : True
IsSelfSigned : False
Issuer : CN=280b6975-b30a-4f5b-b2c3-7864e37f1c05
NotAfter : 8/9/2119 1:36:53 PM
NotBefore : 8/9/2019 12:36:53 PM
PublicKeySize : 2048
RootCAType : Unknown
SerialNumber : 73AC7DDB217BA7AF44847CC68A8B9CC9
Services : None
Status : Invalid
Subject : CN=ab0ee702-f37f-4dff-bfb2-66698a441d9a
Thumbprint : CFD78D7F9DFAA0BD537B3755C24089CE3ED0EC55

AccessRules :
CertificateDomains : {EXCH-EDGE, EXCH-EDGE.intra.net}
HasPrivateKey : True
IsSelfSigned : True
Issuer : CN=EXCH-EDGE
NotAfter : 10/11/2017 11:09:54 PM
NotBefore : 10/11/2012 11:09:54 PM
PublicKeySize : 2048
RootCAType : Registry
SerialNumber : 5DC03A0D09D1C594468C11CE9EC919D4
Services : SMTP
Status : DateInvalid
Subject : CN=EXCH-EDGE
Thumbprint : 4157434692710986BAC026FD2DFE32D4352DE9B3

AccessRules :
CertificateDomains : {intra.net, www.intra.net, exch-cas.intra.net, apollo.inglewood.kimconnect.com, autodisc
over.intra.net, autodiscover.inglewood.kimconnect.com, pop.inglewood.kimconnect.com, imap.inglewood.kimconnect.com, inglewood.kimconnect.com, legacy.intra.net, legacy.inglewood.kimconnect.com}
HasPrivateKey : True
IsSelfSigned : False
Issuer : SERIALNUMBER=07969287, CN=Go Daddy Secure Certification Authority, OU=http://certificates.godaddy.
com/repository, O="GoDaddy.com, Inc.", L=Scottsdale, S=Arizona, C=US
NotAfter : 5/16/2016 11:18:35 AM
NotBefore : 5/16/2011 11:18:35 AM
PublicKeySize : 2048
RootCAType : ThirdParty
SerialNumber : 2B94032E16C980
Services : SMTP
Status : DateInvalid
Subject : CN=intra.net, OU=Domain Control Validated, O=intra.net
Thumbprint : A05FBA0E72AD3D3E666973C9AFDE378535E24393

=============================================================================================

# Create New Cert
$domain="EXCH-EDGE"
$fqdn="exch-hub.intra.net"
$friendlyName="Exchange Certificate"
New-ExchangeCertificate -FriendlyName $friendlyName -SubjectName CN=$domain -DomainName $domain,$fqdn -PrivateKeyExportable $true #Optional:-Services SMTP -KeySize 2048

# Check for self-signed certs
Get-ExchangeCertificate | where {$_.Status -eq "Valid" -and $_.IsSelfSigned -eq $true} | Format-List FriendlyName,Subject,CertificateDomains,Thumbprint,NotBefore,NotAfter

# Restart Exchange Transport
Stop-Service MSExchangeTransport
Start-Service MSExchangeTransport

# Create new Subscription on Edge servers:
New-EdgeSubscription -Filename c:\newEdgeSubscription.xml

# Import subscription on Hub server
New-EdgeSubscription -Filename c:\newEdgeSubscription.xml

# On Hub, trigger New Edge Susbcription via Exchange Management Console GUI
$site='intra.net/Configuration/Sites/DistrictOffice'
New-EdgeSubscription -FileData '<Binary Data>' -Site $site -CreateInternetSendConnector $true -CreateInboundSendConnector $true

# Trigger sync
start-edgesynchronization -forcefullsync

# Restart Exchange Transport
Stop-Service MSExchangeTransport
Start-Service MSExchangeTransport

# Check mail queue
Get-Queue

# Check logs, navigate to:
%exchangeinstallpath%\TransportRoles\Logs\ProtocolLog\SmtpReceive

# Create new connector to point to the smart host (Barracuda spam filter). Make sure that the Source of Send Connector is Edge Server (not Hub Server)
# Disable the automatically generated connector that does not use the smart host

# Example of mail flow issue when the smart host does not accept connections from the Hub server. Resolution was to change the connector Source to the Edge transport

[PS] C:\Windows\system32>Get-Queue

Identity DeliveryType Status MessageCount NextHopDomain
-------- ------------ ------ ------------ -------------
exch-hub\1639048 MapiDelivery Active 17 school-mailboxdb3
exch-hub\1639053 SmartHost... Retry 5675 [10.10.1.11]
exch-hub\1639058 MapiDelivery Active 10 do-mailboxdb
exch-hub\1639059 MapiDelivery Active 12 school-mailboxdb4
exch-hub\1639060 MapiDelivery Active 14 school-mailboxdb2
exch-hub\Submission Undefined Ready 103 Submission
exch-hub\Shadow\1591071 ShadowRed... Ready 62 EXCH-EDGE.intra.net
exch-hub\Shadow\1639036 ShadowRed... Ready 166 EXCH-EDGE.intra.net

[PS] C:\Windows\system32>Get-Queue -Identity exch-hub\1639053 | fl #where 1639053 is Identity of the smart host

RunspaceId : b2e3dae0-ecb1-4508-b307-31da04271141
DeliveryType : SmartHostConnectorDelivery
NextHopDomain : [10.10.1.11]
TlsDomain :
NextHopConnector : 77215356-bf27-49bc-bd41-4603375ac561
Status : Retry
MessageCount : 5656
LastError : 451 4.4.0 Primary target IP address responded with: "421 4.4.2 Connection dropped due to SocketE
rror." Attempted failover to alternate host, but that did not succeed. Either there are no alter
nate hosts, or delivery failed to all alternate hosts.
LastRetryTime : 8/9/2019 5:45:39 PM
NextRetryTime : 8/9/2019 5:50:39 PM
DeferredMessageCount : 0
QueueIdentity : exch-hub\1639053
Identity : exch-hub\1639053
IsValid : True