Update: The Julia White tweetchat has been postponed to January 19. This article has been updated to reflect this change.

Today Microsoft Canada is celebrating Azure with various activities throughout the day. Fear not, you do not need to be Canadian to take part in today’s festivities. You do not need to wear flannel and say “eh”. However, I would highly recommend it. Joking aside, anyone can participate in the activities today that are online.

On January 19th between 11:00 am and 12:00 pm eastern time, Julia White (Microsoft Cloud Platform Product Marketing) will be on Twitter answering questions about Azure. Simply follow Microsoft in Business Canada to pose your questions or share your comments with Julia.

All day, Microsoft employees will be tweeting about Azure using the hashtag… wait for it…. #Azure. A wealth of information will be shared using this hashtag not only on Twitter but all social media platforms.

Happy Azure day!

Phishing messages are continuing to evolve and look ever more convincing. It’s scary to see just how legitimate some of these messages can look. Last week I was working with an organization that received a phishing message that looked incredibly legitimate. What stood out for me the most, was that this message included fake Safety tips.

I have blurred the sending email address above but will tell you that the sending domain was comprised of random characters. The sending domain is controlled by the attacker and so SPF checks all pass. The display name, however, is Microsoft, and this is where the attacker is hoping to fool the recipient.

This attack was specifically targeted at the organization I worked with. Clicking the link in the message brought you to a sign-on page which included the organization’s logo, and resembled a typical ADSF sign on page. There were a few hints that the site was malicious, one of them being that any text entered in the password field showed in the clear. Submitting your credentials through this site would give the attackers immediate access to your account.

White this attack was blocked quickly, the messages did reach end users at this organization and a hand full of them had their accounts compromised by entering their credentials on the malicious site. This organization uses Advanced Threat Protection, ATP, and was able to quickly look up how many users clicked the malicious link, and who they were.

Safety Tips

As of today, the only safety tip that we actually insert directly into a message is the Fraud warning.

Our other information tips will only appear in OWA and will appear outside of the message (above the subject). In the above phishing message, while you and I would know that a Trusted Sender banner won’t appear legitimately directly in a message, most end users will have no idea.

As end users are the last line of defense, user education becomes critical. While no amount of education will stop all users from clicking links in messages like the above, it can certainly decrease the number. With attacks ever evolving, the education piece needs to evolve with it and end users need to continually be reminded to be careful. Accomplishing this is no easy task, and so I will leave this up to you, the reader.

For more information on safety tips, see Email Safety Tips in Office 365.

Advanced Threat Protection Safe Links

If you have Advanced Threat Protection in Exchange Online Protection and have configured a Safe Links policy that does not have the option, Do not track user clicks enabled, then you can track which users have clicked what links. To do so, run a URL Trace in the Exchange Online portal.

For this attack, we ran a URL trace scoped to the malicious URL and quickly found all users that clicked the link. Those users were immediately contacted to begin a remediation process. The results provided by the URL Trace will look something like this.

Straight up URL blocking isn’t very efficient with how easily an attacker can change the URL placed in a message. Protection technology is ever evolving, and yesterday we announced two new ATP features, URL Detonation, and Dynamic Delivery. You can find the announcement over on our Office Blog, Evolving Office 365 Advanced Threat Protection with URL Detonation and Dynamic Delivery.

Cheers,

Andrew

Other resources

• Enable or disable safety tips in Office 365
• Seven things to know about Safety Tips

Today we announced that the connector changes that were planned for Exchange Online have now been pushed back from February 1st 2017 to July 5th 2017. These changes impact Exchange Online inbound connectors and require organizations with certain configurations to make changes and updates.

The original blog post and KB article have both been updated to reflect the new timelines. Both articles also have updated content in them, so please visit them for full details on this planned change.

For those that were required to make a change but still had not, breath a sigh of relief, you have more time.

Cheers,

Andrew

I recently worked with an organization that had an Exchange Online inbound connector which accepted mail from their on-premises Exchange environment. This connector was scoped by IP, and the organization wanted to change it to be scoped by certificate instead due to an upcoming change in Exchange Online. For more details on this change see Important notice for Office 365 email customers who have configured connectors.

Mail flow for this organization was as follows.

For Exchange Online to be a relay in this scenario, there needs to be an inbound connector setup in Exchange Online that is of type “on-premises.” To identify mail from the on-premises server, the Exchange Online inbound connector can either use the IP of the on-premises server, or the certificate that will be presented from on-premises. For some environments, after the upcoming change in July, the Exchange Online inbound connector will need to be scoped by certificate. Please see the above link to find out if you need to convert the scope on your Exchange Online inbound connector from IP based to certificate based.

For the organization that I was working with, they had an existing Exchange Online inbound connector which was scoped by IP, and the goal was to change it to be scoped by certificate. To reduce the chance of mail disruption, they created a second Exchange Online inbound connector, which was scoped to identify mail from on-premises by the certificate that was presented.
Because of propagation time in Office 365, we waited for an hour after the second connector was created before disabling the original IP based inbound connector. Shortly after disabling the IP based inbound connector, the on-premises server started receiving the following error from Exchange Online.

550 5.7.64 TenantAttribution; Relay Access Denied

For some reason, mail sent from on-premises to Exchange Online was not being attributed to the Exchange Online inbound connector which was certificate scoped. As such, Exchange Online was rejecting the mail because the service is not an open relay. In my humble experience, the three most common causes of this are as follows.

• The certificate being presented by the on-premises servers had expired
• The certificate being presented by the on-premises servers was not issued by a trusted CA
• The on-premises servers are not presenting a certificate

For these cases, I always first obtain a network capture, as 99% of the time, these will show the exact problem. My go to network capture program is Wireshark. I have a previous blog post that includes handy Wireshark filters that I use in my troubleshooting which is aptly named Useful Wireshark Filters for Mail Flow Troubleshooting.

The trace provided by the organization consisted of 2187 rows, a lot of which was not SMTP traffic. The trace was taken while the IP based connector in Exchange Online was enabled, and so it didn’t contain the “Relay Access Denied” error, and so I cannot search for that. The trace also contains many SMTP conversations, and I’d like to see just one.
I start by filtering my results to only show traffic that was sent over port 25 using the following filter.

Tcp.port == 25

Next I search for the string “mail.protection.outlook.com,” and I look for an SMTP code of 220. This is the name that Exchange Online response with when a connection is first made to it.

As there are many SMTP conversations between Exchange Online and the on-premises servers, I pick my first result.

To view only traffic in this particular conversation, right click, select Follow, and then TCP Stream.

I am now left with 83 lines, which is pretty good considering we started with 2187. The rows that I’m interested in are the ones that show the certificate exchanges between Exchange Online and the on-premises server.

The first highlighted line is Exchange Online presenting its certificate to the on-premises server. Digging into this packet I can see the Common Name, or CN, of the certificate being presented.

If I want to dig further, I can expand the above line and view all sorts of interesting details about the Exchange Online certificate.

Next I look at the second highlighted yellow line which is where the on-premises server sends its certificate to Exchange Online.

Interestingly, there is no certificate being presented by the on-premises server, which explains why the mail is not being attributed to the Exchange Online inbound connector which is certificate scoped.

Our investigation quickly turned to the on-premises server where it was discovered that the organization’s certificate was actually not associated with their Exchange send connector. Once the on-premises certificate was properly associated with the Exchange send connector we tested again. This time we had wonderful success, and there was much rejoicing! The certificate was properly being presented by Exchange on-premises, which allowed for the mail to be properly attributed to the organization’s Exchange Online inbound connector.

Words of wisdom

When in doubt, grab a network capture.

Cheers,

Andrew

If a user account has been compromised and used to send massive amounts of spam, Exchange Online will block the account from sending (if enabled, a notification email can be sent to administrators to alert them when this happens). Once the account password has been reset, the block can be lifted by an administrator from the Action Center, located in the Protection section of the Exchange Online portal.

We often see organizations that would like to give help desk individuals rights to the Action Center so that they can unblock a banned sender. The stipulation being that the help desk individuals won’t have rights to change anything else in the portal, and only have rights to unblock a banned sender. Out of the box this is not possible, as the built-in admin roles that grant access to unblocking users also grant access to other parts of the Exchange Online portal.

Have no fear, we can create a custom RBAC (Role Based Access Control) role which will ONLY grant access to the Action Center. To do this, we are going to create a custom RBAC role through PowerShell which will only grant access to the cmdlets Remove-BlockedSenderAddress & Get-BlockedSenderAddress, which will, in turn, allow delisting through the portal as these are the cmdlets that are run in the background.

Typically the process for creating a custom RBAC role begins with copying an existing one. From there, we will remove cmdlets (or add) until we have the rights that we are looking for. With that in mind, let’s first see which built in roles grant access to the Remove-BlockedSenderAddress cmdlet.

PS > Get-ManagementRoleEntry *\remove-blockedsenderaddress

Running this returns the following.

We have two built-in roles that we can start with. For absolutely no reason in particular, I’m going to start with the “Transport Hygiene” role. First, create a copy of the “Transport Hygiene” management role. I’m going to call my copy “Blocked Sender.”

New-ManagementRole -Parent "Transport Hygiene" -Name "Blocked Sender"

This new management role will contain all the cmdlets that the Transport Hygiene management role contains. Let’s take a look.

Get-ManagementRoleEntry "Blocked Sender\*"

If a user has been assigned this new role, they will be able to run all of these cmdlets. This includes being able to run them from PowerShell, but also gives rights to anything in the portal that runs these cmdlets in the background. We want to remove all the cmdlets except the two the deal with unblocking banned senders. To do this, I’m going to run the following.

$(Get-ManagementRoleEntry "Blocked Sender\*") | where {$_.name -notlike "*blockedsender*"} | foreach {$id =$_.identity + "\" + $_.name;Remove-ManagementRoleEntry $id -confirm:$false}

Once this completes, I’ll re-run the following to see which cmdlets are left in my new management role.

Get-ManagementRoleEntry "Blocked Sender\*"

Excellent, we now have two cmdlets left in our new management role, both of which are required to unblock a banned sender. We can now assign this new Management Role to an admin role through the Portal.

Help desk personal with this role will now be able to unblock banned senders from the Action Center in the Exchange Online portal, and will not have access to any change anything else. If you followed along with me, give yourself a pat on the back!

In my line of work, I am constantly requesting message samples from organizations so that I can analyze the headers. Whether an end user has received a message that they believe should have been marked as spam, or they receive a message that was marked as spam that should not have been, step one of the troubleshooting starts with asking for the original message. Forwarding a sample message does not work as the original headers are destroyed.

Less known by most people is that forwarding a sample as an attachment is also problematic.

When a message is forwarded in this way the Outlook desktop application will compress the attachment to reduce the sending size. The problem with this is that headers in the original message will be stripped, and quite often the EOP headers that we are looking for will be gone.

To guarantee that the original message along with its’ headers are forwarded intact, the message first needs to be saved to your desktop, then compressed (I recommend adding the message to a .zip archive), and then sending the compressed file as an attachment. The Outlook Desktop client will not modify a message in a zip file, and this will ensure the complete message with all headers intact will arrive at the destination.

Let’s look at an example. I have a sample message that I have sent to myself twice. The first time the message was attached using the “Forward as Attachment” button in the Outlook desktop client. The second time it was first added to a .zip file, and than that compressed file was attached to the email. Here are the resulting headers from each send.

The header on the left is from the message that was sent in a .zip file. The header on the right is from the message that was attached directly to a sent message. The text highlighted in red on the left is absent in the header on the right. It looks like Outlook removed half of the headers from the original message!

Headers play such a crucial role in troubleshooting messages and it is imperative that they remain intact when sending them to other people. Moral of this story, always add a message to an archive first before sending it to me.

The recent WannaCrypt Ransomware has been all over the news (you can see this BBC article on the threat) in the last couple of days. Our Information Protection product teams are aware of the ongoing attack and literally working around the clock. EOP and ATP already have definitions in place to detect this malware, but we are watching for new and impacting variants.

This ransomware is attacking Windows machines that aren’t patched by March 2017 MS17-010. The following guidelines for Exchange Online customers aren’t new but are worth the review in light of this recent threat.

• EOP Best practices: includes how to create rules for extension blocking
• Use rules to block executable content
• Report any undetected malware to Microsoft
• Review information about Zero Hour Auto Purge (ZAP) (enabled by default for cloud mailboxes)
• Configure the anti-malware policy
• Utilize Advanced Threat Protection for safe links and safe attachments scanning
• More context and best practices around blocking executables from our Anti-Spam Product Manager, Terry Zink
• Educate end users around phishing

On Friday our security team published a guidance article which is also a very valuable read. We also had a webcast which contains excellent information and can be listened to again here. The deck that was used in this webcast can be downloaded below.

Or you can click here to download the deck if the above hyperlink doesn’t work.

Stay safe out there!

When you have a hybrid environment setup with Exchange Online, you’ll notice a new Accepted Domain in the Exchange Online portal.

<domain>.mail.onmicrosoft.com

This domain is used by Exchange on-premises to route mail to a mailbox that has been migrated from Exchange on-premises to Exchange Online. After a mailbox is migrated from Exchange on-premises to Exchange Online, the remaining on-premises object will have its targetAddress Active Directory attribute populated. Typically the new targetAddress attribute will look something like user@domain.mail.onmicrosoft.com. When an on-premises mailbox sends mail to this user, the targetAddress is used to route the mail to the Exchange Online mailbox.

A quick way to view an objects Active Directory targetAddress attribute is through the Active Directory Users and Computers panel. In AD Users and Computers, ensure that Advanced Features has been enabled under the View menu.

Then right click an object, select properties, and you’ll find the Attribute Editor tab present. This tab won’t be present if you haven’t enabled Advanced Features.

This is only one way to view this attribute. You can also use ADSIEdit or Active Directory Administrative Center.

I recently worked with a customer that found the targetAddress attribute in Active Directory was incorrect for a few mailboxes that had been migrated to Exchange Online. We fixed those few mailboxes, but we wanted to be proactive by looking to see if any other on-premises objects had incorrect targetAddress attributes.

The following PowerShell can be run against on-premises Active Directory and will display any AD user that does not have a targetAddress that ends in mail.onmicrosoft.com.

Get-aduser -filter {targetAddress -notlike “*.mail.onmicrosoft.com”} -properties * | Select-Object Name,targetAddress

This query will not return objects that do not have a targetAddress attribute set. It will only return objects that have a targetAddress that does not end in mail.onmicrosoft.com. For this particular customer that I was working with, there ended up being a few more that were incorrect. We weren’t able to determine why they were wrong, but we were able to correct them. High five!!

In light of the recent malware news, a couple of my colleagues put together a list of Exchange Online resources. This list is by no means definitive or complete, it is just a place to start when thinking about configuration options in Exchange Online Protection.

• EOP Best practices: includes how to create rules for extension blocking
• Use rules to block executable content
• Report any undetected malware to Microsoft
• Review information about Zero Hour Auto Purge (ZAP) (enabled by default for cloud mailboxes)
• Configure the anti-malware policy
• Utilize Advanced Threat Protection for safe links and safe attachments scanning
• More context and best practices around blocking executables from our Anti-Spam Product Manager, Terry Zink
• Educate end users around phishing

Stay safe out there!

For those of you that are Exchange Online Protection veterans, it may be second nature to always head to the Exchange Online portal whenever you need to make any changes to that service. You may not even think of making those changes in the Security and Compliance Center. I’d like to throw out this quick reminder to make sure you are visiting the Security and Compliance Center. If you haven’t been for a while, then it’s time to get out of that comfort zone and go for a visit! It’s easy, just click https://protection.office.com.

There are some new Exchange Online Protection features that you may not be aware of which are only available in the Security and Compliance Center. As an example, us veterans know that you can set an action like “junk folder” for spam, and a different action like “quarantine” for high confidence spam.

But did you know that in the Security and Compliance Center you can also set actions for Phishing and Bulk messages?

For those that want to have phishing mail treated differently than spam and high confidence spam, you can!!

In the Security and Compliance Center there is also a front end for spoof intelligence management (requires the purchase Office 365 Enterprise E5 or Advanced Threat Protection licenses), as well as a check box for enabling or disabling Outlook Web tips. If you were only using the Exchange Online portal, you would have to use PowerShell for management of these features.

If you haven’t checked out the Security and Compliance Center for a while, I would highly recommend a visit to it at https://protection.office.com. Features related to Exchange Online Protection can be found under the Threat Management menu.

When creating a transport rule, please…. PLEASE, do not disable auditing. Your rule auditing setting should not look like this.

Unless of course, you have a security mandate about not auditing transport rules, then please continue on and disable auditing on transport rules. But for those that do not have a security mandate, please do not turn this off!

Why should I not turn it off?

I’m glad you I asked! With auditing disabled (unchecked), a transport rule will not appear in the standard message trace. That’s right, if this rule triggers on a message, and you run a standard message trace on the message, you won’t be able to see what rule triggered. This can make troubleshooting transport rules extremely difficult.

An interesting case

I recently worked with an organization that was not receiving any messages from one of their partners. The sender was not receiving an NDR, and the recipient did not receive the message. When running a message trace, we could see that a transport rule in the recipients’ tenant deleted the message, but the name of the transport rule wasn’t present in the message trace (we later discovered the reason why… auditing had been disabled on this transport rule). All we saw in the message trace was this.

'[{LED=550 5.2.1 Message deleted by the transport rules agent};{MSG=};{FQDN=};{IP=};{LRT=}]'

If auditing had not been disabled, we also would have seen the name of the transport rule that triggered in the message trace.

The organization wanted to know what transport rule was deleting the message. The tricky part is that there were over 100 transport rules and a lot of them had the action of “delete message.” Going through these rules one by one to try and find the rule with criteria that matched the sending message would be extremely time consuming. If only there was another way.

Wait… there is another way!

Another way!

We pulled an extended message trace for the message and loaded it into Excel. In the custom data column, you can view the transport rules that are evaluated against the message. If you see S:TRA=ETRP, this means that a transport rule was evaluated, but did not trigger. I grabbed the following from an Extended Message Trace that we ran on the message.

In looking at all the rules that were evaluated, I counted about 50. Keep in mind that this organization has well over 100 rules. Since only 50 were evaluated, either a rule triggered with the option of “stop processing more rules,” or a rule triggered with the action of delete. When a rule triggers that has an action of delete, we stop processing subsequent rules.

In looking at the above, we can see the GUID of the last transport rule to trigger. To figure out what this rule is, we can run the following PowerShell.

Get-TransportRule -Identity <GUID>

This returned the name of a rule, which suspiciously contained the text “Phishing Rule” in the name. In looking at the details of this rule we found it had an action of “Delete Message”, and criteria which matched the message that was deleted. Also…. Auditing was disabled on the transport rule, which is why it never showed up in the original message trace that was run.

Now that we have the transport rule, we could modify the criteria to prevent the false positive detection that is caused.

Help me find my rules that are set to Do Not Audit

If you would like to quickly see what rules you have that are currently set to Do Not Audit, you can run the following PowerShell.

Get-TransportRule | Where-Object {$_.SetAuditSeverity -like "DoNotAudit"} | fl -property Name, SetAuditSeverity

Wrap up

There are many scenarios that may force you to disable auditing on some transport rules. But if you don’t’ have this justification, run the above PowerShell to verify that you have not accidentally disabled Auditing on any of your transport rules.
For more information, see my previous article on Auditing Transport Rules.

Cheers!

Microsoft Canada recently published a book on Office 365 titled Expert Office 365 – Notes from the Field. Each chapter in the book is about a technology in Office 365, and is written by an expert that works with that technology every day. The authors, myself included, all work for Microsoft Canada, and are Premier Field Engineers & Support Escalation Engineers. We collaborate with organizations every day and help them with their Office 365 products. We have written this book from our experiences in working with Office 365 every day. This book literally contains notes from the field.

The book is published by APress and can be found at http://www.apress.com/us/book/9781484229903. To celebrate the book launch, Microsoft donated $3,000 USD to the Make a Wish foundation. Going forward, all profits from the sales of this book will also go to the Make a Wish foundation.

For those in the email space, Rhoderick Milne, who you may be familiar with from his Technet blog 250 Hello, has a chapter written about Exchange hybrid mode. My chapter is about email protection, and my colleague Bruce Wilson has a chapter about mail flow in Exchange Online. The complete chapter list is as follows.

1. Records Management in SharePoint Online
2. Skype for Business Online
3. Introducing and Preparing for Exchange Hybrid
4. SharePoint Hybrid Team Sites
5. Hybrid Search
6. SharePoint Recovery
7. Azure Rights Management for SharePoint
8. Introduction to the SharePoint Framework
9. Understanding and Troubleshooting Protection in Office 365
10.  Understanding and Troubleshooting Office 365 Mail Flow
11.  Final Thoughts

Writing a chapter in this book was incredibly fun, and I couldn’t be prouder of the Canadian group that came together to make this book possible!

My lack of updates around these parts can be attributed to the craziness of work over the last few months. This afternoon I have some time and am typing this out as quickly as I can before someone notices and gives me something else to work on. Let’s begin.

I’ve recently seen a very big rise in display name spoofing. With technologies in place like DMARC, DKIM, and SPF, attackers are finding it harder and harder to spoof sending domains. Instead, they have reverted to something quite simple, changing their sending display name to be that of an executive in the targeted organization.

For example, an attacker will register a free email account and use any email address. Sometimes the addresses contain the name of the executive that they are trying to spoof. The attacker would then set their display name to match your CEO or some other executive, and then send phishing messages into your organization. The hope is that the recipient won’t look at the sending address, and instead just look at the sending display name. Some recipients may even assume that the sending email is the personal email of the executive and believe it to be real.

The other problem with an attacker using a consumer email account, or even their own domain, is that all checks like DMARC, DKIM, and SPF will all pass (as long as the records are set up correctly) as there is no domain name spoofing happening.

To combat this, I have had customers implement a transport rule that identifies messages that contain the names of key executives in the From field, and which originate from outside of the tenant. The transport rule would look something like that.

Under exceptions, you would add the personal addresses that the executives may use to send mail to the company to ensure those messages aren’t caught by this rule.

The rule is simple and straightforward, but I’ve had customers report having much success with it capturing phishing attempts.

Cheers!

I recently worked with an organization where one of their users had a full Recoverable Items folder. This user had a hold on their mailbox, but also had the Exchange Online Unlimited Archive, so why had the Recoverable Items folder filled up and why were these items not being moved to the online archive? Let’s dig in.

Problem

A user started having problems when trying to accept meeting invites that had been sent to them. They raised the problem with their IT staff who quickly discovered that the users’ Recoverable Items folder was full. This user had previously been put on hold, which gave them a quota of 100 GB for their recoverable items folder (default is 30GB), and this space was maxed. In our investigation, we found that the Versions folder in the Recoverable Items folder was taking all the space in the Recoverable Items folder. This explains why the user was having problems accepting meeting invites, because meeting versions are moved to the Versions folder in the Recoverable Items folder, which was completely full.

Troubleshooting

As any good IT folk would do first, I wanted to verify the problem with my own eyes, so I had the organization run the following PowerShell.

Get-mailboxfolderstatistics -Identity <User Identity> -IncludeAnalysis

This cmdlet will return statistics for all folders in the mailbox, including the folders under the Recoverable Items folder. In looking at the results for the Versions Folder we found the following.

Name                       : Versions
FolderPath                 : /Versions
FolderType                 : RecoverableItemsVersions
ItemsInFolderAndSubfolders : 252
FolderAndSubfolderSize     : 100.6 GB (108,063,890,884 bytes)
TopSubject                 : <Meeting Subject redacted>
TopSubjectSize             : 100.6 GB (108,063,890,884 bytes)
TopSubjectCount            : 252
TopSubjectClass            : IPM.Appointment
TopSubjectPath             : \Top of Information Store\Calendar

We see that there are 252 items in this folder. We also see that the TopSubjectCount is 252, and TopSubjectClass is IPM.Appointment. This indicates that a single item, with 252 multiple versions, is responsible for all the space being used in this folder.

In the past, I have seen the odd case where a corrupt calendar meeting will continually replicate in the versions folder, and this is my current suspicion here. There are 252 versions of a single meeting that are taking 100.6 GB of space. If we do some simple math and assume that each version is the same size, we find out that each version of this meeting is approximately 409 MB in size.

Remember that this user also has an Exchange Online Unlimited Archive. So even if we have a corrupt meeting that keeps replicating, why is it not being moved to the online archive?

From the above information, because the only item we saw here was this single meeting, I was inclined to believe that other items in the Recoverable Items Folder were properly being moved to the user's Online Archive. It was only the large versions of this corrupt meeting that were not. My guess is that the size of these items, each being 409 MB in size, was the reason they were not moving to the Online Archive. Either the size stopped the Managed Folder Assistant (MFA) from moving the items, or the MFA was timing out because the items were so large. I won’t go into details about the MFA in this blog, but all you need to know is that this is the process that purges expired items in a mailbox and/or moves items to the archive, both behaviours are based on retention policies.

To verify that the Managed Folder Assistant was properly running against this mailbox we ran the following.

$ElcProperties = Export-MailboxDiagnosticLogs <user identity> -ExtendedProperties
$XMLProperties = [xml]($ElcProperties.MailboxLog)
$XMLProperties.Properties.MailboxTable.Property | where {$_.Name -like "ELCLastSuccessTimestamp"}

We found that the ELCLastSuccessTimestamp showed a last run time within the last 24 hours. This tells us that MFA is running against the mailbox. Next, I wanted to verify the retention policy was configured  correctly. In particular, we looked for a retention policy tag that had a type of RecoverableItems folder. We found there was one in place.

Name: Recoverable Items 14 days move to archive
Type: RecoverableItems
MessageCLassDisplayName: All Mailbox Content
RetentionAction: MoveToArchive
AgeLimitForRetention: 14.00

The above tag is part of the applied policy. This tells me that the items in the Recoverable Items folder should have moved as long as they were at least 14 days old. We used MFCMAPI (details later on in this post) to view the items in the Versions folder, and sure enough, they were all much older than 14 days. Some were almost a year old.

A wrench here is that this mailbox was on hold. If the mailbox was not on Hold, we would have been able to ask the user to delete and re-create the corrupt meeting, and then disable Single Item Recover and use MFCMAPI to manually delete the items in the Versions folder. Because the mailbox is on Hold, we have a couple of other steps to perform for this cleanup.

Solution

With this mailbox being on Hold, we want to be careful with our actions so that we minimize the chance that mail gets purged and lost when we perform the cleanup. If you want to be the most careful, remove access to the mailbox from the user before performing the following steps.

1. Have the user delete all occurrences of the meeting invite from their calendar. We know the subject of the meeting from the above Get-MailboxFolderStatistics pull. Once deleted they can re-create it.
2. Grant yourself access to the mailbox. This usually takes around 60 minutes for the permissions to replicate. Remove the user's access to the mailbox if you want to be extra careful.
3. Disable MFA from running on the user's mailbox by running Set-Mailbox -Identity <UPN of user> -ElcProcessingDisabled $true. We need this to be disabled because when we remove the mailbox hold, we don’t want the MFA going through the Recoverable Items Folder and purging any items in there. As this command will take some time to propagate, wait at least a few hours before moving on.
4. Disable single item recover on the mailbox by running Set-Mailbox -Identity <UPN of user> -SingleItemRecoveryEnabled $true. If we don’t do this, then deleting items won’t allow the item to actually be completely purged.
5. Remove the hold on the mailbox.

After the above has been completed, use the following steps to perform the cleanup with MFCMAPI.

Create an Outlook Profile for the user's mailbox on your machine, and then use MFCMAPI to connect to the mailbox.

• Start MFCMAPI
• Click Session and then click Logon.
• Select the new mail profile and then click OK.
• Right-click the users’ mailbox you have full access to and then click Open Store.
• Expand the user and then expand Recoverable Items.
• Double click /Versions to open the folder to view the contents.
• Select one or more messages, right-click, and then choose the appropriate action you wish to complete.

Now that the items are deleted, we need to reverse the changes we made above.

1. Re-enable the hold.
2. Re-enable Single Item Recovery (Set-Mailbox -Identity <UPN of user> -SingleItemRecoveryEnabled $true).
3. Re-enable MFA (Set-Mailbox -Identity <UPN of user> -ElcProcessingDisabled $false).

To verify the cleanup worked, we ran the following.

Get-mailboxfolderstatistics -Identity <User Identity> -IncludeAnalysis

The results showed us that the versions folder contained 551 KB of data, down from 100.6GB. Mission accomplished. Pat yourself on the back and move on to the next fire.

Cheers!

References

Clean up or delete items from the Recoverable Items folder

Increase the Recoverable Items quota or mailboxes on hold

Overview of unlimited archiving in Office 365

It has been quiet around these parts as of late. Much too quiet. To put that to an end, I have a handy tip today that could help in your spam troubleshooting adventures. For various reasons, organizations sometimes need to create multiple spam filters in Exchange Online. When troubleshooting a false positive (an inbound message that was marked as spam incorrectly), it can be tricky to figure out which spam filter was used to process the message. Well, that changes today with this tip.

How spam filters are evaluated

Let’s first look at how spam filters are evaluated in Exchange Online. Even when multiple spam filters have been created, every inbound message is only evaluated by a single spam filter. When a message is received, Exchange Online will first look at the spam filter that has a priority of 0. If the “applied to” criteria in that filter matches the message, then that spam filter, and only that spam filter will be applied to the message. If the “applied to” criteria in that first spam filter do not match the message, then Exchange Online will look at the next spam filter. Once a match is found, then only that spam filter will be used. If no match is found, then the Default spam filter will be used.

SpamDiagnosticMetadata

So you have a false positive and need to figure out which of the spam filters in EOP caused the message to be junked. Maybe the message was junked because of the sender of the message being on the block list in one of your spam filters, or maybe it was junked because of an Advanced Spam Filter (if this is the reason, you’ll see an x-CustomSpam header) option that you now want to disable in the spam filter that processed the message. How do we figure out which spam filter processed the message? Grab the message header and look for the following header.

SpamDiagnosticMetadata

This header will contain the name of the spam filter that was used to evaluate this message. As an example, I have the following spam filters configured in my Exchange Online tenant.

I sent a spam message to my own tenant and by looking at the headers I can see that my spam filter named “Who throws a shoe” was used for this message.

SpamDiagnosticMetadata: Who+throws+a+shoe

Any other awesome headers?

Another handy header is X-MS-Exchange-AntiPhishPolicyId. If a message was marked as spam by your Phish policy, then this header will be present and contain the name of your Phish policy which processed the message.

Important stuff

From what I’ve seen, the above headers are only present in messages that have been marked as spam, and the component which was responsible for the spam determination is what will be present in the message header (ie. If the ATP Phish policy marked the message as phishing, then you’ll see X-MS-Exchange-AntiPhishPolicyId in the message header).

The above headers are subject to change at any point and may even be removed. But as of today, you can use them to make your troubleshooting adventures a little bit easier.

Cheers!

ZAP, also known as Zero-hour Auto Purge, is a protection feature in Exchange Online that can move spam, phish, or malware messages from users’ inbox to their junk folder. This feature works in the background and can often be forgotten about. If a message is delivered to a mailbox, and then at a later time it is determined to be junk, ZAP can reactively move the message to the users' junk folder.  For more information about ZAP, check out Zero-hour auto purge – protection against spam and malware.

I recently worked with an organization that was seeing some messages from a particular sender appearing in their recipients' inbox, but others were appearing in their recipients' junk mail folder. The strange part here was that the headers of the messages that landed in the junk folder showed that EOP had marked the messages as not spam (SFV:NSPM, SCL:1). If you suspect that a device might be the culprit, then pulling mailbox audit logs is the best place to start. But if you suspect that ZAP may be at play, then take a look at the message headers of a message in the junk folder. When looking at the headers for a message in the junk folder, look at the very bottom. If ZAP moved the message to the junk folder you’ll see a header like the following.

X-Microsoft-Antispam-ZAP-Message-Info: <a long string of characters>

If this header is present, then ZAP was the reason for the message residing in the junk folder.

What if I don’t want ZAP to move messages from my inbox to my junk folder?

Well, there are two options here. If you don’t want ZAP to do anything, then it can be turned off on your content filter. If you only want ZAP to be disabled on a subset of your users, you can create a new content filter which is scoped to only those users, and then disable ZAP on that content filter. The PowerShell to turn ZAP on and off is very simple.

PS C:\> Set-HostedContentFilterPolicy Default -ZapEnabled $false
PS C:\> Get-HostedContentFilterPolicy Default | fl *zap*
ZapEnabled : False

Another option is to whitelist the message. This can be done either with a transport rule, or by using the Allow lists in the spam filter. If a message arrives at your inbox with an SCL of -1 from EOP, then ZAP should not be able to touch it.

Above I mentioned mailbox audit logs. These logs contain actions that have been performed by either the owner, delegate, or admin. ZAP moves will not appear in the mailbox audit logs because this move is performed by Transport. But as mentioned above, just look at the headers for a quick way of seeing if ZAP was at play.

That’s your quick December tip from me. If I don’t post again before the holidays, I hope you have a wonderful holiday season!!

Links

• Zero-hour auto purge – protection against spam and malware
• Find and investigate malicious email that was delivered (OFfice 365 Threat Intelligence). This is like manually "zapping" messages.
• Exchange Online message headers

Consider the following mail flow.

On-premises environment --> Your Exchange Online tenant --> External Recipient

With the above mail flow, you may find yourself in a situation where you need to validate that the outbound message was properly attributed to your Exchange Online tenant. I recently worked with an organization that controlled two Exchange Online tenants and found that their mail was not relaying out of the tenant they expected.

You could simply run a message trace in your tenant as a trace will only show you results for messages that have passed through your tenant. However, you can also use the headers of the message as they look to the recipient. When looking at the Receive headers of a message, you’ll typically see a server name of <server name>.mail.protection.outlook.com for messages that have been sent to Exchange Online. This will indicate when a message entered an Exchange Online tenant.

Received: from mail-yw1-f45.google.com (209.85.161.45) by
QB1CAN01FT005.mail.protection.outlook.com (10.152.120.70) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id
15.20.1471.13 via Frontend Transport; Wed, 2 Jan 2019 17:40:34 +0000

But what tenant in Exchange Online actually received this message? When a message is inbound to Exchange Online, it will be stamped with the following header once it has been attributed to a tenant.

X-EOPTenantAttributedMessage

The value of this header will be a GUID which represents a unique Office 365 tenant. To view the GUID of your own Office 365 tenant, connect via PowerShell to your Office 365 tenant and run the following.

Get-MsolCompanyInformation | Select-Object InitialDomain,ObjectID

If your tenant GUID matches the value in this header, then the message was attributed to your tenant. For a message that has left an Office 365 tenant and arrived at an external recipient, you may also see the following header.

X-MS-Exchange-CrossTenant-id

This will often represent the GUID of the tenant which sent the message, but won’t always, depending on how the mail routed through Exchange Online and if there were hops to third party devices as a message moved from one tenant to another.

The easiest way to tell if a message has gone through your tenant by using the headers is to grab your tenant GUID using the above PowerShell and then searching the headers for that GUID.

Cheers.

This is a quick tip for something that I'm asked quite often. The scenario is that you have created an inbound connector in Exchange Online and want to verify that mail which should be coming in through that connector really is; as opposed to mail coming in through a different custom connector or even the default hidden inbound connector.

The quickest way I’ve found to do this is through PowerShell. First, find a message that was received by Exchange Online. Copy the MessageID from the message and then plug it into the following PowerShell which you’ll need to run against your tenant.

Get-MessageTrace -MessageId "<MessageID>" | Get-MessageTraceDetail -Event Receive | fl

After running the above you should see something like this.

Copying out the Data field, we can see the name of the inbound connector in Exchange Online in which this message was attributed to. Just look for “S:InboundConnectorData.”

<root><MEP Name="ConnectorId" String="CY4PR03MB2759\Default CY4PR03MB2759"/><MEP Name="ClientIP" String="2a01:111…<MEP Name="CustomData" Blob="S:ProxyHop1=TO1CAN01FT009.mail.protection.outlook.com(10.152.122.134);S:ProxyHop2=BN3PR03CA0051.outlook.office365.com(2a01:111:e400:7a4d::11);'S:InboundConnectorData=Name=For blog post;ConnectorType=Partner;TenantId=…</root>

I can quickly see that this particular message came into my tenant through my connector named “For blog post.” If you do not see “S:InboundConnectorData,” then the message came in through the default hidden inbound connector.

Cheers!

If you follow @MSDNService on Twitter you may have seen that MSDN & TechNet blogs will be archived into read-only blogs soon.

That day for my blog, EOP Field Notes, is fast approaching and I wanted to do one final post. I created this blog back in 2014 when I was helping customers on-board to Exchange Online Protection. I have since moved to a Support Engineer role for Exchange Online and continue to enjoy writing and sharing content. For a number of reasons, I have not had as much time over the last couple of years to publish as many articles as I would like. Although my time for this blog had decreased, I loved being able to share what I found to be interesting about Exchange Online and Exchange Online Protection.

I thought about setting up a new blog on my own, outside of Microsoft, but as of now I do not plan on doing that. I have loved having this blog and interacting with all of you through comments and the occasional ping on Twitter. Thank you for being here and I hope you have taken away some knowledge from my articles. I do not know where this blog will be moved to once it's archived, but I'm sure it won't be too hard to find.

While I don't post much on Twitter, I do use it every day and can be found at @AndrewStobart. Feel free to ping me there if you have a question about EOP, or want to talk video games or photography.

Cheers all!