Categories
exchange network server windows

Uninstall Exchange 2010 on crippled 2008 R2 DC (SBS2011)

FASTTRACK ARTICLE

Exchange 2010 was installed on a domain controller, it actually was a Small Business Server 2011. Something happened to the AD database, backup not good. Server would only boot in AD restore mode.

In AD restore mode, we could login using domain credentials because the other DC (backup) was providing logon and authentication. We could even start a whole bunch of services and also the Exchange services. We seized the Roles on our other DC so that users would be able to logon without issues.In the next days we prepared to move the mailboxes away, in our case to Office365 in the cloud.
After that happened we followed the steps to remove the last Exchange server including disconnecting all mailboxes from AD users, removing public folders and such. On the final step, to uninstall Exchange it would not continue stating that the server was pending a reboot.
“A reboot from a previous installation is pending. Please restart the system and then rerun Setup.”
Restarting did not help of course, because we could only boot in to directory services restore mode.
So I had to find a way to fix the AD.

1) Checking the AD database (ntds.dit) from restore mode:
Open a CMD command prompt (As Administrator);
execute command ntdsutil
at the prompt type: activate instance ntds
at the prompt type: files

A] If you get an error about corruption ->
Move all the .log files in C:\windows\ntds\ to another directory (desktop perhaps);
Open a new CMD command prompt (As Administrator);
execute command: ESENTUTL.EXE /p C:\windows\ntds\ntds.dit
execute command: ESENTUTL.EXE /g C:\windows\ntds\ntds.dit
Try the files command again in the open ntdsutil command prompt.

B] If you get an error about being in the recovered state ->
Open a new CMD command prompt (As Administrator);
execute command: ESENTUTL.EXE /g C:\windows\ntds\ntds.dit
The integrity check will show all is normal, otherwise see step A.
Reboot the computer and set the date in the BIOS a couple of months or from before the backup-date if you tried a AD restore. Or set it a year back or so if you are unsure. Try to boot in normal mode. Normally it should boot up, but could take a while, change the date/time back to the correct values when it’s booted up.

2) Replication and authentication to other domain controllers
If you have other domain controllers to replicate to, then you might probably need to change BURFLAGGS for non-authorative restore (to fix NTFRS corruption on SYSVOL) and first reset the machine account password for the secure channel to the other domain controllers. See this post: http://ares.gobien.be:8080/2013/07/sync-issues-krb_ap_err_modified-0x80090322-target-principal-name-incorrect/

3) Now you can either try to fix everything further or go ahead and uninstall Exchange. Try the uninstall the normal way.
If you get errors about the sate of the Active Directory, try it like this:
Open a new CMD command prompt (As Administrator);
execute command: cd %programfiles%\Microsoft\Exchange Server\v14\bin
execute command: setup.com /m=uninstall /dc:otherdc.domain.local

Make sure the server is still in the “Exchange Servers” security group.
Make sure there are no entries in the hosts file for your DC’s. Because it can also trigger the following error:
Setup encountered a problem while validating the state of Active Directory: ‘server.domain.local’ isn’t a fully qualified domain name (FQDN). Please provide a valid FQDN. For example: ‘SERVER’.

Happy uninstalling!

Log excerpt:

[12/31/2014 08:27:56.0273] [1] Active Directory session settings for 'Get-ExchangeServer' are: View Entire Forest: 'True', Configuration Domain Controller: 'SRV-APP1.contoso.com', Preferred Global Catalog: 'SRV-APP1.contoso.com', Preferred Domain Controllers: '{ SRV-APP1.contoso.com }'
[12/31/2014 08:27:56.0273] [1] Beginning processing Get-ExchangeServer -Identity:'SBS2011'
[12/31/2014 08:27:56.0273] [1] Searching objects "SBS2011" of type "Server" under the root "$null".
[12/31/2014 08:27:56.0273] [1] Previous operation run on domain controller 'SRV-APP1.contoso.com'.
[12/31/2014 08:27:56.0273] [1] Previous operation run on domain controller 'SRV-APP1.contoso.com'.
[12/31/2014 08:27:56.0273] [1] Preparing to output objects. The maximum size of the result set is "unlimited".
[12/31/2014 08:27:56.0273] [1] Ending processing Get-ExchangeServer
[12/31/2014 08:27:56.0491] [1] [REQUIRED] There is a pending reboot from a previous installation of a Windows Server 2008 role or feature. Please restart the system and rerun Setup.
[12/31/2014 08:27:56.0523] [1] Ending processing test-setuphealth
[12/31/2014 08:27:56.0538] [0] **************

[12/31/2014 08:28:01.0312] [1] Ending processing Get-ExchangeServer
[12/31/2014 08:28:01.0702] [1] [REQUIRED] There is a pending reboot from a previous installation of a Windows Server 2008 role or feature. Please restart the system and rerun Setup.
[12/31/2014 08:28:01.0702] [1] Ending processing test-setuphealth
[12/31/2014 08:34:16.0514] [0] End of Setup

[12/31/2014 10:17:29.0782] [1] Ending processing Get-ExchangeServer
[12/31/2014 10:17:30.0047] [1] [REQUIRED] Unable to read data from the Metabase. Ensure that Microsoft Internet Information Services is installed.
[12/31/2014 10:17:30.0047] [1] [REQUIRED] Setup encountered a problem while validating the state of Active Directory: Active Directory operation failed on SBS2011.contoso.com. The supplied credential for 'CONTOSO\Administrator' is invalid.

[REQUIRED] Setup encountered a problem while validating the state of Active Directory: Active Directory operation failed on SBS2011.contoso.com. The supplied credential for 'CONTOSO\Administrator' is invalid.

[12/31/2014 10:53:05.0881] [1] Searching objects "SBS2011" of type "Server" under the root "$null".
[12/31/2014 10:53:05.0897] [1] Previous operation run on domain controller 'SRV-APP1.contoso.com'.
[12/31/2014 10:53:05.0897] [1] Previous operation run on domain controller 'SRV-APP1.contoso.com'.
[12/31/2014 10:53:05.0897] [1] Preparing to output objects. The maximum size of the result set is "unlimited".
[12/31/2014 10:53:05.0912] [1] Ending processing Get-ExchangeServer
[12/31/2014 10:53:06.0287] [1] [REQUIRED] Setup encountered a problem while validating the state of Active Directory: 'SBS2011.contoso.com' isn't a fully qualified domain name (FQDN). Please provide a valid FQDN. For example: 'SBS2011'.
[12/31/2014 10:53:06.0318] [1] Ending processing test-setuphealth

[12/31/2014 10:54:32.0491] [1] Previous operation run on domain controller 'SRV-APP1.contoso.com'.
[12/31/2014 10:54:32.0491] [1] Previous operation run on domain controller 'SRV-APP1.contoso.com'.
[12/31/2014 10:54:32.0491] [1] Preparing to output objects. The maximum size of the result set is "unlimited".
[12/31/2014 10:54:32.0491] [1] Ending processing Get-ExchangeServer
[12/31/2014 10:54:33.0043] [1] [REQUIRED] Active Directory does not exist or cannot be contacted.
[12/31/2014 10:54:33.0043] [1] [REQUIRED] Setup encountered a problem while validating the state of Active Directory: 'SBS2011.contoso.com' isn't a fully qualified domain name (FQDN). Please provide a valid FQDN. For example: 'SBS2011'.
[12/31/2014 10:54:33.0043] [1] Ending processing test-setuphealth

[12/31/2014 10:56:00.0320] [1] Previous operation run on domain controller 'SRV-APP1.contoso.com'.
[12/31/2014 10:56:00.0320] [1] Preparing to output objects. The maximum size of the result set is "unlimited".
[12/31/2014 10:56:00.0320] [1] Ending processing get-EdgeSubscription
[12/31/2014 10:56:00.0574] [1] [REQUIRED] Setup encountered a problem while validating the state of Active Directory: 'SBS2011.contoso.com' isn't a fully qualified domain name (FQDN). Please provide a valid FQDN. For example: 'SBS2011'.
[12/31/2014 10:56:00.0670] [1] Ending processing test-se

[12/31/2014 12:31:07.0542] [1] Previous operation run on domain controller 'SVR-DC1.contoso.com'.
[12/31/2014 12:31:07.0542] [1] Previous operation run on domain controller 'SVR-DC1.contoso.com'.
[12/31/2014 12:31:07.0542] [1] Preparing to output objects. The maximum size of the result set is "unlimited".
[12/31/2014 12:31:07.0542] [1] Ending processing Get-ExchangeServer
[12/31/2014 12:31:07.0791] [1] [REQUIRED] Setup encountered a problem while validating the state of Active Directory: The user-specified domain controller SRV-APP1 does not exist.

Categories
blog server windows

Windows 7 profile SID wrong mstsc can’t login

This is a very strange problem I came across on a windows 7 Embedded thin client. I don’t quite understand what went wrong but I’ll give you a detailed description.

CASE:
The user has a thin client with Windows 7 Embedded that’s been entered in to the Active Directory domain. On the public desktop of the thin client there is a RDP file to connect to a Remote Desktop Server (a.k.a. Terminal Server). The user logs on to the thin client using their AD credentials. The user was able to log on to the server using the RDP file without problems until today.

SYMPTOMS:
– User can’t log on to the Remote Desktop Server, the error received is:

The connection was denied because the user account is not authorized for remote login

TROUBLESHOOTING:
Normally this just means that the user is not a member of the “Remote Desktop Users” local group on the server.
– I verified the user was a member of the correct groups to log on to the server.
– I then tried to log on the server with the same credentials from a different workstation. This worked without a problem. Which led me to conclude at the server-side everything was OK.
– On the troublesome workstation (thin client with WIN 7 E in my case) I launched remote desktop with the “Run As Administrator” option and supplied credentials for an admin account. I tried to connect to the Remote Desktop server using the credentials of the troublesome user account. This worked without a problem.
– I tried again without the run as, and it failed again with the same error.

This led me to my conclusion that something was very wrong with the user profile on the workstation for this domain user.

SOLUTION:
I decided to delete the user profile on the local workstation since nothing is stored in it (they don’t work locally). However when I opened Explorer and went to see in “C:\Users” I saw 2 identical folders with the same name (the username of the troublesome user). It seems there were 2 identical profile folders. I didn’t think it was possible for 2 folders to have the same name.
I deleted both folders!
I then opened REGEDIT and went to HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\WINDOWS NT\CURRENTVERSION\PROFILELIST
I saw multiple user SID’s and checked them all. To my surprise there were 2 different user SID’s that both had a value c:\users\problem.username underneath it. So 2 different user SID’s for the same username. I thought that was impossible. I deleted both registry keys.
After deleting the profiles and the keys I logged back in with the user and profile was recreated and the remote desktop worked perfectly.

So it seems that the remote desktop client was sending the wrong SID to the server and that was the reason for the unauthorized error message.

Categories
blog howto server windows

VMware image customization in progress

Recently I came across an issue with a virtual machine that was deployed from a template. After the newly created virtual machine was booted up it seemed that the generalization did not fully complete. The password was not changed and some other minor things were not as it should have been after full customization runs.

We then saw that the VM kept saying “VMware image customization in progress” on every reboot right before windows boots. If you already have configured and/or are using the VM and don’t want to start over, here’s what you can do.

Open regedit and find the key:
HKLM\SOFTWARE\VMWare,Inc.\Guest Cutomization\
or on a 64-bit system:
HKLM\SOFTWARE\Wow6432Node\VMWare,Inc.\Guest Cutomization\
Change the value for “CustomizationInProgress” to “0”.

Then navigate to the following key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager
Change the value for “BootExecute” to the default of “autocheck autochk *”, you should just delete the “sysprepDecryptor.exe” part and the enters after it. Be careful or your system won’t boot.

You should probably check your SID and see if it’s unique, to make sure the customization did indeed change this. Altough duplicate SID’s should be not a real problem (according to Microsoft), don’t do it on your servers!
http://technet.microsoft.com/en-us/sysinternals/bb897417.aspx

PS: In my case the customization failed because sysprep had already been run 3 times (template was cloned from previous deployed machine) and apparently you can only do it 3 times unless you add the special option not to rearm the licensing state (which vmware doesn’t add when customizing).

PS2: If you manually did sysprep but got stuck because with error

A fatal error occurred while trying to sysprep the machine

because of the rearm failure, try this: http://itfunk.wordpress.com/2013/01/09/resetting-rearm-count-in-windows-7/
Put the following in a bat file called c:\delwpa.bat

reg load HKLM\MY_SYSTEM “%~dp0Windows\System32\config\system”
reg delete HKLM\MY_SYSTEM\WPA /f
reg unload HKLM\MY_SYSTEM

reboot and using F8 choose repair computer, choose keyboard, type password and choose command prompt. Find your correct drive could be C or D or higher (depends on reserved partitions), and execute the BAT file. Rearm state will be wiped, after reboot you will see not genuine but you can rearm again 3 times now.

Categories
blog howto network server windows

Backup domain controller sync issues KRB_AP_ERR_MODIFIED 0x80090322 target principal name incorrect

My case:
1x Windows 2008 Small Business Server (named: SBS2008)
1x Windows 2008 R2 standard on off-site location (named: TS2008) BACKUP DOMAIN CONTROLLER & GC
Connection between the 2 servers was lost for nearly 3 months.
Replication would only work from SBS2008 to TS2008 but not from TS2008 to SBS2008.
I couldn’t view the shares on \\SBS2008 from the console on TS2008, i received the error “The target principal name is incorrect”. On SBS2008 I could view the shares on TS2008.
In the eventlog there were errors:

The Knowledge Consistency Checker (KCC) was unable to form a complete spanning tree network topology. As a result, the following list of sites cannot be reached from the local site.

Sites:
CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=domein,DC=local

The Knowledge Consistency Checker (KCC) has detected problems with the following directory partition.

Directory partition:
DC=domain,DC=local

There is insufficient site connectivity information for the KCC to create a spanning tree replication topology. Or, one or more directory servers with this directory partition are unable to replicate the directory partition information. This is probably due to inaccessible directory servers.

All directory servers in the following site that can replicate the directory partition over this transport are currently unavailable.

Site:
CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=domain,DC=local
Directory partition:
DC=domain,DC=local
Transport:

CN=IP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=domain,DC=local

The File Replication Service is having trouble enabling replication from SBS2008 to TS2008 for c:\windows\sysvol\domain using the DNS name SBS2008.domein.local. FRS will keep retrying.
Following are some of the reasons you would see this warning.

The session setup from the computer SBS2008 failed to authenticate. The name(s) of the account(s) referenced in the security database is SBS2008$. The following error occurred:
Access is denied.

The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server sbs2008$. The target name used was E3514xx-xxxxxxxxxxxxxxx/yyyyyyyyyyyyyyy/domain.local@domain.local. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Please …

The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server sbs2008$. The target name used was DNS/sbs2008.domain.local. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Please …

Demote and the promote for TS2008 would not work without forcing and doing a lot of NTDS cleanup on the PDC. So this was my last resort.

1st reboot -> did not work.
Doing a lot of searching and looking up the SPN for SBS2008 on both DC’s did not show any differences.

Luckily I found an older KB from microsoft on how to reset the kerberos secure channel between two DC’s.
http://support.microsoft.com/kb/288167

I had to disable the KDC (Kerberos Key Distribution Center) service on TS2008 and then reboot.
Immediately after reboot I noticed I could browse the shares on the SBS2008 without error.
This is because TS2008 was no longer supplied Kerberos tickets itself but requesting them from SBS2008.
Now I opened an elevated command prompt and forced a sync of all replica partitions and triggered the KCC checker.
repadmin /syncall /ade
repadmin /kcc

To check the replication backlog queue use:
repadmin /queue

After replication was succesful I put the KDC service back to automatic and started it. Problem solved.

If you can’t get replication working yet, you’ll need these extra steps.
klist /purge
netdom resetpwd /server:sbs2008 /userd:domain\Administrator /passwordd:* (the * will make it prompt for password).

Also you might need to check your DNS settings and put the IP adres of SBS2008 as primary DNS IP on the NIC of TS2008.

Other helpful information:
http://support.microsoft.com/kb/2090913