Ongoing Outlook Sync issues with ESET

[ShaneDT 13]

Thanks Craig, was going to go looking for a PowerShell command today so I can compare the size of these folders with the total size of a users mailbox. Help to identify those with the biggest problem.

I've found in testing yesterday, if you have Outlook on two computers with ESET installed with Outlook open at the same time, every single email received gets duplicated to Conflicts, some times twice. Makes sense as ESET is scanning and tagging these emails twice. This was the case for my customer in the thread I started. She has an office PC that she normally leaves on with Outlook open at the office... But even with Outlook only open on one computer I'm still seeing maybe 10-20% of all emails getting duplicated.

Re ESET, they have known about this problem for years, there are settings in policy that are supposed to remove this tagging that have been there for years, they just don't work!

This will be a problem for EVERY user that has Outlook and the ESET Outlook plug-in. Most just don't know it as they never see these Sync Issues folders as they're hidden in the default Mail view in Outlook. It's only if you switch to Folder List view that you see them.

ESET just needs to fix it. 

How did you go with the xml file Marcos provided? Did this work for you in the end?

I assume this is imported in EES on the actual client? For a managed client (policy assigned by ESET Protect Server) I'm just worried it may screw up settings pushed down by the server. 

If this xml policy works I don't understand why they can't just add it to security polices that are applied from the server!?

[ShaneDT 13]

PS: Probably better to use Get-Mailbox -ResultSize unlimited as this will then include Shared Mailboxes.

[Marcos 5,267]

I assume that configuring Outlook email protection to not scan received email on one of the machines that concurrently download email on two or more devices could reduce the number of sync conflicts. Email would still be scanned on read.

[ShaneDT 13]

No Marcos that would not be an acceptable solution.

If that computer is being used and the other isn't then emails aren't being scanned as they are received.

[CraigC 1]

9 hours ago, ShaneDT said:

 

How did you go with the xml file Marcos provided? Did this work for you in the end?

 

It doesn't work. I was advised to open a case locally here in UK with version 8.1, I supplied some debug info, and ESET HQ confirmed its broken, with no timescale to fix. I would suggest you do same, and refer to this thread and the many others.

TBH, I don't think this issue is an easy problem to fix, if it was we wouldn't still be chatting here about the issue that has been around for a long time. The issue is just hidden for majority of users, and is only noticed when looking at mailbox size. The issue is not just about mailbox size, but performance / overhead of the AV software on the device, again this is hidden with fast devices with SSD drives, but the issue is still there.

I do hope ESET are having discussions with Microsoft on how to resolve.

@Marcos There are a few use cases ESET need to consider when testing with Office365 mailboxes:

1) Multiple devices sharing same mailbox. This often happens in companies, users have their own mailbox and a shared mailbox, eg 'customerserivice@', 'finance@' etc. Also Manager / PA / Secretary type

2) Multiple devices with same mailbox, eg laptop and PC

3) Brand new devices with ESET installed, that will sync a whole mailbox.

 

 

 

[ShaneDT 13]

1 hour ago, CraigC said:

TBH, I don't think this issue is an easy problem to fix, if it was we wouldn't still be chatting here about the issue that has been around for a long time. The issue is just hidden for majority of users, and is only noticed when looking at mailbox size. The issue is not just about mailbox size, but performance / overhead of the AV software on the device, again this is hidden with fast devices with SSD drives, but the issue is still there.

Obviously not understanding technically how it works, I don't understand why not tagging the metadata of an email that is scanned would be so difficult? OK so ESET no longer knows which emails have been scanned and which haven't. Is that really a problem. All this really needs to do is scan emails as they are received in the Inbox. Maybe add an option to do a full mailbox scan on a schedule, but even then it shouldn't need to tag the emails.

Yes tagging the email would be ideal yada yada but if it causes such a major issue (I've been running Powershell on my customers tenants all day, most are around 10-30% capacity in Conflicts, some are up to 80%!) that has been a problem for so long, why persist with it?! 

Just simplify it down to just scan new emails as they are received in the mailbox, no tagging, and optionally add an option to schedule a full scan. Leave the option to manually scan folders but again without tagging.

Re your point 3 above, that used to be a major problem when first installing ESET on a customers computer. First thing it would do is scan the entire mailbox already downloaded in Outlook, which would trigger Outlook to resync the whole mailbox (EDIT: Actually from memory it was only the Inbox)! From memory disabling some of the settings in the security policy did fix this at the time. That's where I remember I had these settings set to Never.

Edited August 12, 2021 by ShaneDT

[volkswagner 0]

I landed here when searching "esetmessageflag causing sync issues outlook".

I'm running NOD32 AV 14.2.24.0. This has been an issue for a long time (without me knowing it).

I only just found out after a user reached 98% of 50Gig capacity for their mailbox. There are

over 30,000 messages in the conflicts folder (over 14Gigs).

So there's still no fix, huh?

This is an insane amount of work if someone had to sift through these messages to

see which of the conflicts should be kept. Do we just blindly delete everything in this folder?

This is not a free product.

ESET is left with two choices, fix it or lose customers. 

[Marcos 5,267]

It depends on the conflicting property. Most conflicts can be avoided either by setting up email protection in a way that emails are not modified (e.g. that the subject is not modified or that no signature is appended to scanned email). However, there's one property that may cause conflicts and that cannot be currently avoided. You will be able to avoid the conflicts as of v15 by enabling this setting in the email protection setup:

[volkswagner 0]

5 hours ago, Marcos said:

It depends on the conflicting property. Most conflicts can be avoided either by setting up email protection in a way that emails are not modified (e.g. that the subject is not modified or that no signature is appended to scanned email). However, there's one property that may cause conflicts and that cannot be currently avoided. You will be able to avoid the conflicts as of v15 by enabling this setting in the email protection setup:

Version 15 has the solution, what do we do now? Do we need to turn off email scanning completely?

When will version 15 be released?

 

Virus scanners should not be modifying email messages, period!

Eset needs another way to keep track of scanned/not yet scanned messages.

Maintain your own DB if necessary.

[itman 1,746]

6 hours ago, Marcos said:

However, there's one property that may cause conflicts and that cannot be currently avoided. You will be able to avoid the conflicts as of v15 by enabling this setting in the email protection setup:

This setting exists on my current EIS ver. 14.2.24 installation. Are you stating that the setting currently doesn't work?

[Marcos 5,267]

1 hour ago, itman said:

This setting exists on my current EIS ver. 14.2.24 installation. Are you stating that the setting currently doesn't work?

Yes, it was stated in another topic regarding sync issues. Will be fixed in v15.

[Marcos 5,267]

1 hour ago, volkswagner said:

Virus scanners should not be modifying email messages, period!

Eset needs another way to keep track of scanned/not yet scanned messages.

An AV must modify email - it must detect and remove malware which is a kind of modification. We also modify the subject and append tag messages to infected email, however, this can be customized.

As I wrote, with v15 which is going to be released in a few weeks you will be able to get rid of syn issues regardless of the conflicting property.

[CraigC 1]

Hello @Marcos, I originally started this thread specifically regarding v8.1, the business product that is more commonly used with Outlook and Exchange. You refer to v15, the home version? Will this ongoing sync issue also be fixed in the business version?  When will this be release? I did open a case with ESET as you originally suggested, but I have not heard anything back from ESET with regard to this fix or release date?

The ticket case ref in the ESET system is #CASE_00129486.

Regards,

Craig

 

Edited September 11, 2021 by CraigC

[Marcos 5,267]

A new service release of Endpoint v8.1 is going to be released with the next few days. I'll need to check with devs if the "Disable checking upon inbox content change" setting is already fixed there.

[volkswagner 0]

9 hours ago, Marcos said:

An AV must modify email - it must detect and remove malware which is a kind of modification. We also modify the subject and append tag messages to infected email, however, this can be customized.

As I wrote, with v15 which is going to be released in a few weeks you will be able to get rid of syn issues regardless of the conflicting property.

AV agent should have permission to modify and only modify when needed. It should not modify EVERY SINGLE email sent or received, plus every calendar event, etc.

Period!

If I have 30-60k emails and have never sent nor received any malware, why does ESET need to modify all 60K items?

[ShaneDT 13]

On 9/12/2021 at 1:08 AM, Marcos said:

A new service release of Endpoint v8.1 is going to be released with the next few days. I'll need to check with devs if the "Disable checking upon inbox content change" setting is already fixed there.

Marcos did you check with the 'devs' whether this setting was being fixed in the business version and what date this release was please?

[Marcos 5,267]

53 minutes ago, ShaneDT said:

Marcos did you check with the 'devs' whether this setting was being fixed in the business version and what date this release was please?

The fix won't be included in the upcoming Endpoint 8.1 service release yet, however, we will be able to provide you with a fixed dll. Also we should be able to provide you with newer installers that you could install via a software install task to upgrade existing clients. However, this version will not undergo QA tests and we'll be able to test it just briefly before giving it out to particular users. A drawback of using an unofficial version is that it cannot be upgraded via the so-called uPCU in the future and you will need to use a software install task. The fix will be included in Endpoint v9 available later this year as well as in possible futrther service release of Endpoint v8.1 that may be released after v9 too.

[ShaneDT 13]

9 hours ago, Marcos said:

The fix won't be included in the upcoming Endpoint 8.1 service release yet, however, we will be able to provide you with a fixed dll. Also we should be able to provide you with newer installers that you could install via a software install task to upgrade existing clients. However, this version will not undergo QA tests and we'll be able to test it just briefly before giving it out to particular users. A drawback of using an unofficial version is that it cannot be upgraded via the so-called uPCU in the future and you will need to use a software install task. The fix will be included in Endpoint v9 available later this year as well as in possible futrther service release of Endpoint v8.1 that may be released after v9 too.

That is disappointing. This problem has been known for so such a long time, and a fix has been promised (and not working) also for a long time. I'm not interested in deploying an 'only briefly tested' version to hundreds of customers computers. Will the dll fix be able to be deployed by ESET Protect? Will this affect future 'uPCU' updates?

Whatever happened to automatic program updates by the way? This feature has been in policy for a long time and also has never worked. Wasn't this supposed to be fixed in version 8.0?

[Marcos 5,267]

6 hours ago, ShaneDT said:

This problem has been known for so such a long time,

Not really. The bug with "Disable checking upon inbox content change" was discovered and confirmed by developers on July 23, 2021.

6 hours ago, ShaneDT said:

Whatever happened to automatic program updates by the way? This feature has been in policy for a long time and also has never worked. Wasn't this supposed to be fixed in version 8.0?

We have already rolled out a uPCU to the latest v8.0.2039 for users with older Endpoint v8.0. As for uPCU to v8.1, a new service build is going to be released within a few days. After 30-60 days it will be released as uPCU.

You could avoid sync issues by reading email only on one device. Accessing it from more devices with ESET installed and Outlook email protection enabled at a time increases the chances of sync issues.

[ShaneDT 13]

16 minutes ago, Marcos said:

Not really. The bug with "Disable checking upon inbox content change" was discovered and confirmed by developers on July 23, 2021.

We have already rolled out a uPCU to the latest v8.0.2039 for users with older Endpoint v8.0. As for uPCU to v8.1, a new service build is going to be released within a few days. After 30-60 days it will be released as uPCU.

You could avoid sync issues by reading email only on one device. Accessing it from more devices with ESET installed and Outlook email protection enabled at a time increases the chances of sync issues.

I'm pretty sure this problem with Outlook has been reported well before July 23 this year...

So the automatic updates are only for minor updates? I thought this was for minor and major updates post 8.0? So computers currently on 8.0 would receive 8.1 automatically? This doesn't seem to be happening. (Yes I did notice a minor update on some 8.0 computers that I hadn't yet manually deployed the 8.1 update to).

Edit: Also having ESET and Outlook on multiple devices (which is very very common) doesn't increase the chances of duplicates in the Conflicts folder, it GUARANTEES it! Every single email gets duplicated. As has already been reported by me and several others.

Edited September 15, 2021 by ShaneDT

[Marcos 5,267]

49 minutes ago, ShaneDT said:

I'm pretty sure this problem with Outlook has been reported well before July 23 this year..

I've checked it in the internal bug ticketing system and the bug was created on that date immediately after developers confirming it.

Quote

So the automatic updates are only for minor updates? I thought this was for minor and major updates post 8.0? So computers currently on 8.0 would receive 8.1 automatically? This doesn't seem to be happening. (Yes I did notice a minor update on some 8.0 computers that I hadn't yet manually deployed the 8.1 update to).

Not in the case of business products. We plan to release an uPCU update to v8.1 30-60 days after the release of the upcoming service release of Endpoint 8.1 for all v8.0.x and 8.1.x users.

[CraigC 1]

@Marcos maybe some confusion here. There has been an ongoing issue with ESET and Outlook Sync issues for many years. There may have been some fixes, that rely on importing config files, as the options are not visible on GUI, but the issue has not been addressed as a proper solution. It seems this fix is broken, and has been reported as a specific BUG in a specific version of ESET that I reported to ESET UK in July.

If we look at the Windows Server ESET product, it automatically detects the environment its being installed on and will apply exclusions etc.

I wonder if the same should be applied to the ESET client on Windows, if it detects Outlook and Exchange, any 'fixes' or configs should be automatically applied as default?

I know this issue and specific bug are maybe not seen as high priority, as its only affecting performance, excessive IO/CPU cycles, disk space. On modern PCs this is masked with fast CPU, SSD drives, and massive Office 365 mailboxes, and most people are unaware of issue, so is not widely reported.

I also don't understand the logic of prioritising a fix in a Home version before the Business version? In my experience Home users do not tend to use Outlook and Exchange.

1 hour ago, Marcos said:

You could avoid sync issues by reading email only on one device. Accessing it from more devices with ESET installed and Outlook email protection enabled at a time increases the chances of sync issues.

You can not avoid Sync issues even using one device with ESET/Outlook/Exchange, you will still get sync issues. ESET must be seeing this is QA/Testing?

 

 

[rpremuz 6]

I'd just like to add  that we are also seeing this issue with ESET Endpoint Antivirus 8.1.2037.2 on MS Windows 10 Pro. machines and Outlook for Microsoft 365 clients (ver. 2109 Build 16.0.14430.20292 64-bit) connecting to Exchange Online.

-- rpr.

[Lockbits 10]

On 9/14/2021 at 11:26 AM, Marcos said:

The fix won't be included in the upcoming Endpoint 8.1 service release yet, however, we will be able to provide you with a fixed dll. Also we should be able to provide you with newer installers that you could install via a software install task to upgrade existing clients. However, this version will not undergo QA tests and we'll be able to test it just briefly before giving it out to particular users. A drawback of using an unofficial version is that it cannot be upgraded via the so-called uPCU in the future and you will need to use a software install task. The fix will be included in Endpoint v9 available later this year as well as in possible futrther service release of Endpoint v8.1 that may be released after v9 too.

Hi Marcos, can you share with us this fix? We've a customer with sync issues and even disabling Outlook integration doesn't work. Outlook is stuck at updating mailboxes.

Thanks.

[Marcos 5,267]

4 hours ago, Lockbits said:

Hi Marcos, can you share with us this fix? We've a customer with sync issues and even disabling Outlook integration doesn't work. Outlook is stuck at updating mailboxes.

Thanks.

You can try Endpoint v9 beta and enable the setting "Disable checking upon inbox content change".