ESET protection fluctuating?

[foxtigerjungle 1]

Hello,

In some tests, ESET scores "only" 98%. Why is that?
Is the competition more accurate/better?

Are the default settings not enough?

https://www.av-comparatives.org/comparison/

https://www.av-comparatives.org/vendors/eset/

 

Or am I misunderstanding something?

 

Greetings

 

[itman 1,746]

9 hours ago, foxtigerjungle said:

Are the default settings not enough?

For the average home user, Eset default settings are adequate.

9 hours ago, foxtigerjungle said:

Or am I misunderstanding something?

AV lab tests are approximations on how a given AV product will perform using malware samples collected by the lab at any given time. Eset historically has performed better on tests by other AV labs than AV-C. AV-C makes it a point in their tests to always include a few samples in the "grayware" category such as POC's, joke malware, PUA's, etc. that Eset might not detect.

Edited Sunday at 04:18 PM by itman

[foxtigerjungle 1]

24 minutes ago, itman said:

AV-C makes it a point in their tests to always include a few samples in the "grayware" category such as POC's, joke malware, PUA's, etc. that Eset might not detect.

Is this then a weakness of ESET?
Does it mean that ESET's protection is weak if it doesn't detect something like this or detects it incorrectly?

[czesetfan 29]

To the best of my knowledge, PUA default is not enabled. It's up to the user's choice during installation.
This can have a significant impact on detection. 

[itman 1,746]

1 hour ago, foxtigerjungle said:

Does it mean that ESET's protection is weak if it doesn't detect something like this or detects it incorrectly?

No. In most cases, it's an issue with the AV lab testing. Case in point;

Quote

Update 22 August 2024 – what has changed?

On 19 August 2024, two days after the missed deadline for providing feedback on the technical data from the Advanced In-The-Wild Malware test, we received evidence from an ESET representative that a malware sample had been detected which, according to our algorithms, was initially missed by ESET Smart Security software on 14 July 2024 when the sample was tested at that time.

As a tester at AMTSO (Anti-Malware Testing Standard Organisation), we would like to bring this to the public’s attention. After reviewing the evidence, we believe that ESET’s request is legitimate. In the interest of full transparency, we also include the following technical data and logs:

SHA256 sample checksum:

5fb58f1f2e8e61c2a67fb5a8a14b9a5e676fc52a8e7e864a9e1bdb17d1dcf1b2

Exact start date of sample test:

2024-07-14 05:10:57.463

Detailed test logs in ZIP format (password: infected):

[ download ]

After downloading and extracting the archive, the following logs will be available, among others:

• a. output.txt file as a real-time console record of the actions performed by the AVLab testing system
• b. malware sample with the original link where it was downloaded from
• c. XML Sysmon report
• d. /ProgramData/ESET/ESET Security/Logs/warnlog.dat <- the following file contains information that the sample was transferred to the Eset cloud for analysis during the test. This is sufficient evidence of sample detection:

The Eset software stores logs in encrypted form. In order to view them, you will need to download the Eset Log Collector software and preview the specified file.

We have complied with the ESET’s request to include this evidence for a sample that was initially incorrectly marked as Fail. As a result, the score for the Eset software has been corrected to 100% blocking of all malware samples used in the Advanced In-The-Wild Malware Test in the July 2024 edition.

 

https://avlab.pl/en/how-is-security-software-protect-against-new-threats-wild-sample-used-in-results-for-july-2024/

Edited Sunday at 05:58 PM by itman

[foxtigerjungle 1]

@czesetfan

thank you

 

@itman

thank you for the information.

Was the result then also adjusted?

ESET's reputation has always been very good in the past. Over time, we've heard that it's not so good anymore.

 

 

A little off topic:
Does ESET take suggestions for improvements here on the forum seriously?
Are suggestions for improvements also implemented?

I reported a few there.

Edited Sunday at 06:08 PM by foxtigerjungle

[itman 1,746]

1 minute ago, foxtigerjungle said:

Was the result then also adjusted?

Yes. Again, from the article except I posted above;

Quote

As a result, the score for the Eset software has been corrected to 100% blocking of all malware samples used in the Advanced In-The-Wild Malware Test in the July 2024 edition.

I will also bring up the point as to malware samples used in AV lab testing and if they are really indeed "in-the-wild" samples? As noted in the AVLabs posted article excerpt, they performed their testing on;

Quote

on 14 July 2024 when the sample was tested at that time.

However, this sample was posted to VirusTotal on;

Again, AV lab tests are beneficial but don't rely on them as an absolute factor in determining overall AV product malware detection effectiveness.

[foxtigerjungle 1]

Thank you @itman

 

I didn't know avlab.pl yet.

But the site seems to be a bit broken.

Edited Sunday at 07:11 PM by foxtigerjungle

[itman 1,746]

I forgot to mention this.

The overall function of AV labs is product certification. The AMTSO standard for product certification for "in-the-wild" malware testing is 98% or better detection rate.

Some AV labs go beyond the AMTSO certification detection standard for their certification requirements. One such AV lab is MRG-Effitas: https://www.mrg-effitas.com/wp-content/uploads/2024/08/MRG_Effitas_360_Q2_2024_Final-2.pdf which only tests commercial AV versions. MRG certifies by malware category. For example, the ransomware category requires no missed real or simulated ransomware detection's to be certified against ransomware. 

Edited Sunday at 07:48 PM by itman

[josh_bdn 0]

My personal experience with Eset also got worse during this year. I was not pleased with the protection and because of that I used another product for the past few months. Yesterday I tried Eset again and tested it with a few samples. I tested with just 5 files (mostly infostealers) and it didnt detect 2 of them. Both were successful at stealing the information and one installed a Coin Miner that Eset wasnt able to remove.

Also I sent an undetected sample to Eset a month ago and I still have no response and it is still undetected. In the past it normaly took under 24 hours for them to add a detection.

[Marcos 5,257]

Please cla

28 minutes ago, josh_bdn said:

My personal experience with Eset also got worse during this year. I was not pleased with the protection and because of that I used another product for the past few months. Yesterday I tried Eset again and tested it with a few samples. I tested with just 5 files (mostly infostealers) and it didnt detect 2 of them. Both were successful at stealing the information and one installed a Coin Miner that Eset wasnt able to remove.

Please clarify what submissions do you mean. I see only 2 sent from your forum email address this year. The last one was sent less than an hour ago and besides an executable, which we are now analyzing, contains also 2 benign files which are definitely not subject to detection:

The other submission with the subject "Stealer" has been detected as OSX/PSW.Agent.BN trojan since June 18, 2024.

[josh_bdn 0]

Thanks for the reply.

Yes these are the two submissions I mean.

About the first one: The exe didnt run without the other files on my end so I just kept them in the zip file just in case.

The second one: When I last checked on my mac Eset didnt detect the file. Maybe this is an issue on my end I will check later.

Then I said that there were two samples missed when I tested yesterday. The other one was a large files (800 MB). Eset has a detection for it but still lets it execute. I reduced the file size and uploaded it to VT. 

9ff1ca0678c81ef0cd6bef34c76fb73ed5bcd571dc3c3de356422f1859072720

Even for this file: When I manually scan the file it blocks it. When I download it and execute it it isnt blocked. Eset detects RedLine stealer in memory after execution and then continues to detect a CoinMiner every few seconds but isnt able to remove it.

 

[Marcos 5,257]

27 minutes ago, josh_bdn said:

About the first one: The exe didnt run without the other files on my end so I just kept them in the zip file just in case.

A detection was added for the executable - JS/TrojanDownloader.Agent.ABMP trojan, about 1,5 hour after the file was submitted.

29 minutes ago, josh_bdn said:

The second one: When I last checked on my mac Eset didnt detect the file. Maybe this is an issue on my end I will check later.

Since it's a dmg file, it could be that you didn't have archive scanning enabled. The detection was added on June 18. Please check your scanner settings and re-scan the file.

34 minutes ago, josh_bdn said:

The other one was a large files (800 MB). Eset has a detection for it but still lets it execute. I reduced the file size and uploaded it to VT. 

9ff1ca0678c81ef0cd6bef34c76fb73ed5bcd571dc3c3de356422f1859072720

Even for this file: When I manually scan the file it blocks it. When I download it and execute it it isnt blocked. Eset detects RedLine stealer in memory after execution and then continues to detect a CoinMiner every few seconds but isnt able to remove it.

After I ran the NSIS installer, a detection "Advanced memory scanner;file;Operating memory » RegAsm.exe(7772);a variant of MSIL/Spy.RedLine.A trojan;contained infected files" was triggered.  Then I I ran a memory scan and no threat was detected in memory and the process RegAsm.exe was not running either. Nevertheless, we are checking the 784MB Obsidium-protected file WeAura.exe if it's subject to detection and add one if necessary. We'll also block the entire NSIS too, just to prevent somebody from running it although the threat is detected and neutralized after execution.

[itman 1,746]

1 hour ago, josh_bdn said:

The other one was a large files (800 MB). Eset has a detection for it but still lets it execute. I reduced the file size and uploaded it to VT. 

9ff1ca0678c81ef0cd6bef34c76fb73ed5bcd571dc3c3de356422f1859072720

Even for this file: When I manually scan the file it blocks it. When I download it and execute it it isnt blocked. Eset detects RedLine stealer in memory after execution and then continues to detect a CoinMiner every few seconds but isnt able to remove it.

According to the VT analysis, it's deploying and using a vulnerable driver, WinRing0.sys, that only three vendors are flagging as such. My suspicion is this driver is what is deploying the coin miner at system startup time.

[josh_bdn 0]

3 minutes ago, Marcos said:

After I ran the NSIS installer, a detection "Advanced memory scanner;file;Operating memory » RegAsm.exe(7772);a variant of MSIL/Spy.RedLine.A trojan;contained infected files" was triggered.  Then I I ran a memory scan and no threat was detected in memory and the process RegAsm.exe was not running either. Nevertheless, we are checking the 784MB Obsidium-protected file WeAura.exe if it's subject to detection and add one if necessary. We'll also block the entire NSIS too, just to prevent somebody from running it although the threat is detected and neutralized after execution.

On my system I constantly get the detection Win64/CoinMiner.IZ 

Every few seconds a file gets dropped C:\ProgramData\fmtjrnlncwpn\diltklqafxsg.exe 
Eset deletes the file and then it gets created again. Cmd.exe is constantly running with about 800 MB of RAM use. Still happens even after full scan and reboot.
But I think this is the same file as WeAura.exe 

[Marcos 5,257]

It'd be good to get ELC logs from the machine. WeAura.exe will be detected as Win32/Packed.Obsidium.LG trojan.

[josh_bdn 0]

11 minutes ago, Marcos said:

It'd be good to get ESET Log Collector logs from the machine. WeAura.exe will be detected as Win32/Packed.Obsidium.LG trojan.

I sent the logs to you via DM

[itman 1,746]

22 minutes ago, itman said:

According to the VT analysis, it's deploying and using a vulnerable driver, WinRing0.sys, that only three vendors are flagging as such.

Below is the vulnerable driver being used. It's ancient; circa 2008, and appears not be on Microsoft vulnerable driver list. Or as I suspect, it's a device driver.

[josh_bdn 0]

Okay so I ran the sample again and the dropped file was quickly detected as Win32/Packed.Obsidium.LG
So as far as I can tell Eset now blocks the file before it does any damage. Thanks for the quick implementation of the detection.

The only thing that bothers me is: Why didnt Eset block the sample automatically before?
I mean it adds exceptions to Defender to evade detection, it breaks and deletes Windows Update.
Arent that actions that should be prevented by HIPS/Behavioral Detection?

 

[itman 1,746]

As far as Eset detection  of WinRing0x64.sys, this appears to be a variant not seen by Eset previously. It has detected another variant of it previously: https://forum.eset.com/topic/32116-evga-component-driver-winring0x64sys-detected-after-five-years/ .

[Marcos 5,257]

55 minutes ago, josh_bdn said:

The only thing that bothers me is: Why didnt Eset block the sample automatically before?
I mean it adds exceptions to Defender to evade detection, it breaks and deletes Windows Update.
Arent that actions that should be prevented by HIPS/Behavioral Detection?

I didn't encounter any issues so I can't tell. As I wrote, the malware was detected after execution by Advanced memory scanner in memory and the process was likely killed as the subsequent memory scan didn't report any active threat.

Please make sure that the CloudCar test file is detected upon download as "Suspicious" to ensure that the LiveGrid reputation system works alright. There was a communication error logged in the logs and LiveGrid information for files was missing.

[foxtigerjungle 1]

5 hours ago, Marcos said:

Since it's a dmg file, it could be that you didn't have archive scanning enabled. The detection was added on June 18. Please check your scanner settings and re-scan the file.

Where can i find this Option?

Are there any other Settings to turn on, to get a better/safer Protection?

[itman 1,746]

42 minutes ago, foxtigerjungle said:

Where can i find this Option?

@Marcos was referring to archive settings for Eset Mac OS product.

[foxtigerjungle 1]

So under Windows there is no option for scanning Archives?

[itman 1,746]

20 minutes ago, foxtigerjungle said:

So under Windows there is no option for scanning Archives?

I don't know what context you are referring to for archive scanning.

Eset real-time protection by default scans archives once they are extracted on the local device.

For default manual off-line Eset Smart scan, archives are not scanned by default. Enabling archive scanning in off-line scans can result in significantly longer scan times and system overhead. If you chose to still do so, refer to the below screen shot and enable archive scanning;

 

 

Edited Monday at 07:57 PM by itman