Summary

Remove-DuplicateItems.ps1 is a PowerShell script that uses EWS (requires the EWS Managed API) to identify and remove duplicate items within a mailbox. It can be targeted at specific folders or the whole mailbox.

The following examples are all against Office 365 (Exchange Online), which require OAuth to be configured (basic authentication cannot be used). You can assign the OAuth parameters to local PowerShell variables as follows (assuming application permissions with impersonation):

$mailbox = "[email protected]"

$clientId = "application (client) id"

$tenantId = "tenant id"

$secretKey = "secret key"

Example 1

.\Remove-DuplicateItems.ps1 -Mailbox $mailbox -Office365 -RecurseFolders -OAuth -OAuthClientId $clientId -OAuthTenantId $tenantId -OAuthSecretKey $secretKey

The above will process the whole (Office 365 hosted) mailbox and delete (to Deleted Items folder) any duplicates it finds of items. Duplicates are only searched for in the same folder as the original item (any found in different folders will not be matched or removed).

Example 2

.\Remove-DuplicateItems.ps1 -Mailbox $mailbox -Office365 -RecurseFolders -MatchEntireMailbox  -OAuth -OAuthClientId $clientId -OAuthTenantId $tenantId -OAuthSecretKey $secretKey

The above will process the whole (Office 365 hosted) mailbox and delete (to Deleted Items folder) any duplicates it finds of items. Duplicates will be matched across the entire mailbox.

Example 3

.\Remove-DuplicateItems.ps1 -Mailbox $mailbox -Office365 -OAuth -OAuthClientId $clientId -OAuthTenantId $tenantId -OAuthSecretKey $secretKey -RecurseFolders -MatchEntireMailbox -DuplicatesTargetFolder "\Duplicates"

The above will process the whole (Office 365 hosted) mailbox and move any duplicates it finds of items to the specified folder (in this case, a folder called Duplicates that is found directly underneath Top of Information Store). Duplicates will be matched across the entire mailbox.

Parameters

-Mailbox: Specifies the source mailbox (from which items will be moved/copied).

-Archive: When specified, the archive mailbox will be accessed (instead of the main mailbox).

-PublicFolders: If this switch is present, folder path is required and the path points to a public folder.

-FolderPath: Folder to search - if omitted, the mailbox message root folder is assumed.

-DuplicatesTargetFolder: "Folder to which any duplicates will be moved. If not specified, duplicate items are soft deleted (will go to Deleted Items folder).

-RecurseFolders: When specified, subfolders will also be processed.

-MatchEntireMailbox: When specified, duplicates will be matched anywhere within the mailbox (instead of just within the current folder).

-HardDelete: When speciifed, duplicate items will be hard deleted (normally they are moved to Deleted Items).

-Credential: Credentials used to authenticate with EWS (provided as PSCredential).

-OAuth: when specified, will use OAuth to access the mailbox (required for MFA enabled accounts) - this requires the ADAL dlls to be available.

-OAuthClientId: The application Id as registered in Azure AD. If not specified, a global registration will be used that supports delegated access only (and will need consent to be able to access mailboxes).

-OAuthTenantId: The tenant Id in which the application is registered. If missing, application is assumed to be multi-tenant and the common log-in URL will be used.

-OAuthRedirectUri: The redirect Uri of the Azure registered application (defaults to http://localhost/code).

-OAuthSecretKey: Secret key to be used when obtaining access token. If this is specified, then application permissions are requested and no user log-on will be required.

-OAuthCertificate: The OAuth certificate to be used when obtaining access token. Application permissions are requested in this scenario. You can obtain a certificate from your own certificate store using the thumbprint: Get-Item Cert:\CurrentUser\My\50B510B4AE120D9B0EE3F059B6DD494469CD6D3B.

-Impersonate: If set, ApplicationImpersonation is used to access the mailbox(es).

-EwsUrl: EWS Url (if omitted, and -Office365 not specified, then autodiscover is used).

-Office365: If set, requests are directed to Office 365 endpoint (overrides -EwsUrl).

-ForceTLS12: If specified, only TLS 1.2 connections will be negotiated.

-EWSManagedApiPath: Path to managed API (if omitted, a search of standard paths is performed).

-IgnoreSSLCertificate: If set, invalid SSL certificates will be ignored and the connection made regardless. Use with care, required for self-signed certificates.

-AllowInsecureRedirection: If set, insecure redirection will be allowed during AutoDiscover.

-LogFile: Logs script activity to the specified file.

-TraceFile: Write all EWS traffic (requests/responses/headers) to the specified file.

-WhatIf: If set, no changes will be made to the target mailbox (but actions that would be taken will be logged).