Exchange Online has a limit of 10,000 folders within a mailbox. If you try and migrate a mailbox with more than this number of folders then it will fail – and that would be expected. But what happens if you have a mailbox with less than this number of folders and it still fails for this same reason? This is the problem, with resolution, I outline below.
I was moving some mailboxes to Exchange Online when I came across the following error in the migration batch results:
Data migrated: 18.18 MB (19,060,890 bytes)
Migration rate: 0 B (0 bytes)
Error: MigrationMRSPermanentException: Error: Could not create folder 2288. –> MapiExceptionFolderHierarchyChildrenCountQuotaExceeded: Unable to create folder. (hr=0x80004005, ec=1253) Diagnostic context: Lid: 55847 EMSMDBPOOL.EcPoolSessionDoRpc called [length=204] Lid: 43559 EMSMDBPOOL.EcPoolSessionDoRpc returned [ec=0x0][length=468][latency=1] Lid: 52176 ClientVersion: 15.20.1730.17 Lid: 50032 ServerVersion: 15.20.1730.6019 Lid: 35180 Lid: 23226 — ROP Parse Start — Lid: 27962 ROP: ropCreateFolder [28] Lid: 17082 ROP Error: 0x4E5 Lid: 25953 Lid: 21921 StoreEc: 0x4E5 Lid: 27962 ROP: ropExtendedError [250] Lid: 1494 —- Remote Context Beg —- Lid: 38698 Lid: 29818 dwParam: 0x0 Msg: f28f1e21-62aa-4999-977f-ce310efea309-61f0997f-74d5-4421-9050-64f8272e5ac2[9]-28A06 Lid: 29920 dwParam: 0xB Lid: 29828 qdwParam: 0x2711 Lid: 29832 qdwParam: 0x2710 Lid: 45884 StoreEc: 0x4E5 Lid: 29876 StoreEc: 0x4E5 Lid: 30344 StoreEc: 0x4E5 Lid: 54080 StoreEc: 0x4E5 Lid: 56384 StoreEc: 0x4E5 Lid: 38201 StoreEc: 0x4E5 Lid: 35904 Lid: 45434 Guid: f12f3e45-67aa-89012-345f-ce678efea901 Lid: 10786 dwParam: 0x0 Msg: 15.20.1730.017:VI1PR0502MB2975:145a3769-3902-4e6b-9fe4-6db564e4eb92 Lid: 1750 —- Remote Context End —- Lid: 31418 — ROP Parse Done — Lid: 22417 Lid: 30609 StoreEc: 0x4E5 Lid: 29073 Lid: 20369 StoreEc: 0x4E5 Lid: 64464 Lid: 64624 StoreEc: 0x4E5
In the above I have highlighted some of the errors I was seeing – with the “could not create folder” message, the first indicator is that I have too many folders to migrate or I have a corrupt mailbox. Running Get-MoveRequestStatistics and including a full report (with -IncludeReport) shows in part the below. This was run to get more info on the move request. This was run from Exchange Online:
26/03/2019 17:10:09 [VI1PR0502MB3855] ‘MigrationService (on behalf of ‘Brian.Reid@domain.co.uk’)’ created move request.
26/03/2019 17:10:15 [DB8PR05MB6025] The Microsoft Exchange Mailbox Replication service ‘DB8PR05MB6025.eurprd05.prod.outlook.com’ (15.20.1730.17 ServerCaps:01FFFFFF, ProxyCaps:07FFFFC7FD6DFDBF5FFFFFCB07EFFF, MailboxCaps:, legacyCaps:01FFFFFF) is examining the request.
26/03/2019 17:10:15 [DB8PR05MB6025] Content from the Shard mailbox (Mailbox Guid: f12f3e45-67aa-89012-345f-ce678efea901, Database: cc980daf-4402-4645-b26c-2a83760b161c) will be merged into the target mailbox.
26/03/2019 17:10:15 [DB8PR05MB6025] Connected to target mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’, database ‘EURPR05DG090-db014’, Mailbox server ‘DB8PR05MB6025.eurprd05.prod.outlook.com’ Version 15.20 (Build 1730.0).
26/03/2019 17:10:20 [DB8PR05MB6025] Connected to source mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’, database ‘DB’, Mailbox server ‘onprem.server.domain.com’ Version 15.0 (Build 847.0), proxy server ‘onprem.server.domain.com’ 15.0.847.40 ServerCaps:, ProxyCaps:, MailboxCaps:, legacyCaps:1FFFCB07FFFF.
26/03/2019 17:10:21 [DB8PR05MB6025] Request processing started.
26/03/2019 17:10:21 [DB8PR05MB6025] Source mailbox information:
Regular Items: 8443, 905.4 MB (949,422,345 bytes)
Regular Deleted Items: 1149, 189.9 MB (199,115,692 bytes)
FAI Items: 4651, 11.72 MB (12,285,701 bytes)
FAI Deleted Items: 9, 19.26 KB (19,721 bytes)
26/03/2019 17:10:21 [DB8PR05MB6025] Cleared sync state for request 2c065e32-3bd5-4524-9aac-03880fa8e961 due to ‘CleanupOrphanedMailbox’.
26/03/2019 17:10:21 [DB8PR05MB6025] Mailbox signature will not be preserved for mailbox ‘tenant.onmicrosoft.com\f12f3e45-67aa-89012-345f-ce678efea901 (Primary)’. Outlook clients will need to restart to access the moved mailbox.
26/03/2019 17:11:20 [DB8PR05MB6025] Stage: CreatingFolderHierarchy. Percent complete: 10.
26/03/2019 17:12:38 [DB8PR05MB6025] Initializing folder hierarchy from mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’: 29048 folders total.
26/03/2019 17:21:21 [DB8PR05MB6025] Folder creation progress: 1102 folders created in mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’.
26/03/2019 17:31:22 [DB8PR05MB6025] Folder creation progress: 2730 folders created in mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’.
26/03/2019 17:41:22 [DB8PR05MB6025] Folder creation progress: 4535 folders created in mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’.
26/03/2019 17:51:23 [DB8PR05MB6025] Folder creation progress: 6257 folders created in mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’.
26/03/2019 18:01:23 [DB8PR05MB6025] Folder creation progress: 7919 folders created in mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’.
26/03/2019 18:11:23 [DB8PR05MB6025] Folder creation progress: 9570 folders created in mailbox ‘tenant.onmicrosoft.com\2c065e32-3bd5-4524-9aac-03880fa8e961 (Primary)’.
26/03/2019 18:14:15 [DB8PR05MB6025] Fatal error StoragePermanentException has occurred
The move request logs show an increasing folder count, and when this exceeds 10,000 a storage error occurs.
So the next thing to do is to check what I have on-premises. I have generally two options to try and fix a mailbox I am moving to Exchange Online. One is to move the mailbox elsewhere on-premises (on the basis that I discard errors on-premises and then move a cleaner mailbox to the cloud) or run repairs on the mailbox. Note that running repairs on-premises is part of the move to the cloud anyway as Exchange Server does this as part of the move.
But this revealed nothing! The move request logs on-premises showed the same – there was over 10,000 folders (indeed some of my mailboxes had over 20,000 folders) and this was enumerated in the move request logs. A New-MailboxRepairRequest did nothing either. But interestingly, Get-MailboxFolderStatistics | Measure showed only 200 folders! Each of my failing mailboxes had between 150 and 263 folders – nothing like the +10,000 that the move request was finding!
So I opened the mailbox in Outlook having granted myself permissions to it – again nothing.
A version of the above cmdlet though indicated something more useful: Get-MailboxFolderStatistics -FolderScope NonIpmRoot | Measure returned a count in excess of 10000.
So I opened MFCMapi and had a look at the folders. Now MFCMapi shows everything in the mailbox, and not just items under the “top of the information store” folder. I went about expanding each subfolder I could find and I came across a subfolder that everytime i expanded it, MFCMapi would hang. I would close and restart MFCMapi and the same thing!
I had found my suspect folder – its a iPhone device that had created the +10,000 folders. Now that I had a good candidate for my issue, the fix was easy. I listed the active-sync devices using Get-MobileDevice -Mailbox “Richard Redmond” | FL Identity and then removed the suspect device using Remove-ActiveSyncDevice “domain.co.uk/OU/Richard Redmond/ExchangeActiveSyncDevices/iPhone§A9BCDE7FG57HIJ81KL1M08NOPQ” -Confirm:$false where the device identity was returned in the Get-MobileDevice cmdlet run just before.
This Remove-ActiveSyncDevice (or Remove-MobileDevice) cleans up this mailbox and deletes the partnership with the device.
Once this was done, I moved the mailbox again and it was ~200 folders and not much higher when including the NonIPMRoot folders and the mailbox moved to Exchange Online without further issue.
Where I tested the move to another Exchange Server database rather than Exchange Online, I found that looking in the move request report (I had prestaged the move and then removed the corrupt mobile device), the move report showed information like the following and all I had done was removed one mobile device from the mailbox!
26/03/2019 17:41:22 [servername] Folder hierarchy changes reported in source ‘Primary (a8c13a2f-535b-d996-908e-ff84b1484a7)’: 200 changed folders, 24080 deleted folders.
From the users perspective, if the phone is an active device and is syncing email, then removing the phone causes it to create a new partnership. If the server allows any device then this is seamless to the user. If the server requires authorization to add a new device, then the user will be told this and service desk/admin will need to approve the device again. So if Allow/Block/Quarantine (ABQ) is not enabled on the server, one wonders if deleting all active sync partnerships before migrating any mailbox is an idea worth considering – there could be mailboxes I have moved that are <10,000 folders but not far from that number and therefore storing up issues for the future!
Leave a Reply