AV-Comparatives Real-World Protection Test February-June 2018

[Sammo 8]

Eset didn't fare so well. 😟

https://www.av-comparatives.org/comparison/?usertype=consumer&chart_chart=chart2&chart_year=2018&chart_month=Feb-June&chart_sort=1&chart_zoom=3

[TomFace 539]

2 hours ago, Sammo said:

Eset didn't fare so well. 😟

https://www.av-comparatives.org/comparison/?usertype=consumer&chart_chart=chart2&chart_year=2018&chart_month=Feb-June&chart_sort=1&chart_zoom=3

It must have taken a bit of digging to find a test from a year ago.

Test like this are not worth their weight in salt.

So what is your purpose and point in posting this?

Regards,

Tom

Edited June 18, 2019 by TomFace

[itman 1,667]

I assume the reference is to this year's most recent A-V C Realtime test where Eset scored 98.4%; approximately the same as it has previously scored recently in this test series.

If one has concerns about Eset, refer to this more comprehensive test series where over 10,000 malware samples are used: https://www.av-comparatives.org/tests/malware-protection-test-march-2019/ . Eset scored 99.86% for malware protection.

Again, this is only one AV Lab's test; and test series for that lab. Refer to all the AV lab tests that Eset participates in and you will observe that Eset is a top scorer overall.

Edited June 18, 2019 by itman

[Sammo 8]

2 hours ago, TomFace said:

It must have taken a bit of digging to find a test from a year ago.

Test like this are not worth their weight in salt.

So what is your purpose and point in posting this?

Regards,

Tom

I hope you are right about this not being worth it's salt. Also, to see the most recent test you need to change the date on that chart  to 2019 Feb to May. Eset only scored 98.4% next to dead last. Can't really be right.

[itman 1,667]

A-V C is "very creative" when it comes to finding samples for its Realtime test series. It's not uncommon for it to slip in a few samples that are geographically restricted to one country and/or region within with an "in-the-wild" dispersion of < 10. The odds of encountering one these samples in close to zero.

Edited June 18, 2019 by itman

[novice 20]

12 hours ago, itman said:

The odds of encountering one these samples in close to zero.

The odds of encountering a sample cannot justify the acceptance of ESET low performance; when even Microsoft scores better , the expectation would be that somebody from ESET would step in and offer an official statement.

[itman 1,667]

11 hours ago, novice said:

when even Microsoft scores better

Here we go again. Windows Defender had a whopping 74 false positives in this test. Refer to the below screen shot that clearly shows that WD "block-at-first-sight" was set to aggressive setting level; basically blocking execution of any process without established reputation. Whereas this might be acceptable to advanced security level professionals, it certainly isn't so for the average user; especially for corp. users.

 

-EDIT- Also 55 of the WD 74 false positives were user dependent block/allow action. It is a no-no to have the user decide if a process is malicious or not:

Ref.: https://www.av-comparatives.org/tests/real-world-protection-test-february-may-2019/

Finally and most important, note the following. A-V C does not factor false positive scoring into its protection scores for its realtime tests as is done for its more comprehensive malware protection test series. Using the above false positive scoring criteria of 50% of user decisions are wrong, WD would have scored 27/752 or 96.4% placing it at the bottom of the protection scoring heap.

Edited June 19, 2019 by itman

[TomFace 539]

I have always been very leery of comparison tests.

Sorry if I came across as a bit snarky.

Regards,
 

Tom

[itman 1,667]

1 hour ago, BeanSlappers said:

Here is the new one if anyone is interested:

 

https://www.av-comparatives.org/comparison/?usertype=consumer&chart_chart=chart2&chart_year=2018&chart_month=Feb-June&chart_sort=1&chart_zoom=3

Again, this is for 2018. I posted a link above for the current 2019 test.

[Sammo 8]

3 minutes ago, BeanSlappers said:

 

All links are at 2018, I think we have to manually change it to 2019 when we get to the site.

https://www.av-comparatives.org/comparison/?usertype=consumer&chart_chart=chart2&chart_year=2019&chart_month=Feb-May&chart_sort=1&chart_zoom=3

[Sammo 8]

8 minutes ago, BeanSlappers said:

Why did it work for you and not for me?  That is crazy.

I just changed the year and months in the link to make it work. 😊

[novice 20]

1 hour ago, Sammo said:

https://www.av-comparatives.org/comparison/?usertype=consumer&chart_chart=chart2&chart_year=2019&chart_month=Feb-May&chart_sort=1&chart_zoom=3

So ESET is not even mentioned here....

[novice 20]

10 hours ago, BeanSlappers said:

Yes it is.

Sorry, I did not see it at first.

But 98.4%????    .

We have Avira (free) , Kasersky (free) at 100%, Bitdfender (free) at 99.9%, Microsoft (free) at 99.6% .

 

[novice 20]

4 minutes ago, BeanSlappers said:

because there are new malware every day

Most anti viruses, if not all , have these days sophisticated mechanisms to deal with unknown malwares : behavior blockers, heuristics, HIPS, generic signatures....

To expect to get a sample first and add detection after is impossible these days ; I remember one of ESET officials saying " this sample was seen only 10 times by ESET , in the whole word, that's why we did not detect it"

I was shocked by such statement.

[itman 1,667]

11 hours ago, novice said:

To expect to get a sample first and add detection after is impossible these days ; I remember one of ESET officials saying " this sample was seen only 10 times by ESET , in the whole word, that's why we did not detect it"

I was shocked by such statement.

This again shows your obvious disconnect with the "real malware world." Not the simulated one put forth in AV lab testing.

Someone recently sent me a malware 0-day sample that only recently had been detected by 6 AV vendors at Virus Total. Half of those vendors specialize in malware detection circulated in the country where the malware had been discovered. The remaining detection vendors specialize in malware detection in the specific region.

BTW - this malware specifically targeted Windows Defender and bypassed it. So if other AV solutions did not detect it, is that a missed detection since it was not a threat to them?

[novice 20]

59 minutes ago, itman said:

This again shows your obvious disconnect with the "real malware world." Not the simulated one put forth in AV lab testing.

BTW - this malware specifically targeted Windows Defender and bypassed it. So if other AV solutions did not detect it, is that a missed detection since it was not a threat to them?

I was referring to this:

The official explanation: " It's a Chinese ransomware written in Python with Chinese instructions. It's been seen on less than 10 machines in total. "

While the OP complained : 

" another of real-life experience with ransomware bypassing ESET protection layers. It is still "at large" even for now with ver15819 definition and has 3 days of reputation history... Other vendors have successfully blocked the encryption through their behavioral detection layer "

 

[itman 1,667]

1 hour ago, BeanSlappers said:

So does eset do it for one region or all area's of the world? 

Microsoft a while back got a lot of free press on how Windows Defender ATP was able to detect a a zero day malware. What Microsoft didn't publicly disclose at the time but did so later via a blog detailed analysis of the incident is the following. At least 6 WD ATP installations were infected by the malware prior to Azure AI cloud server analysis returned a positive identification of malware status. BTW - those infected installations were all located in a specific region within Russia.

Bottom line - there is no such thing as 100% 0-day protection. If there was, that concern would in short order be the only security solution used and all other AV vendors would cease to exist.

[itman 1,667]

6 minutes ago, BeanSlappers said:

Did you miss the question?  I didn't ask about microsoft, I didn't specifically ask about 0 day either.

Eset and other AV vendors get data from malware feeds and honeypots world-wide. The problem is that there are certain geographic areas such as China for example, where access to such data is restricted, filtered, or otherwise difficult to obtain in  a timely fashion. Of course, malware dispersion and frequency is a major factor in detection by the aforementioned. If only a few samples exist in the wild, their targets are restricted to a specific area or business concern, etc., the likelihood of quick detection by existing monitoring methods are quite low. 

[novice 20]

1 minute ago, itman said:

If only a few samples exist in the wild, their targets are restricted to a specific area or business concern, etc., the likelihood of quick detection by existing monitoring methods are quite low.

Still I did not get it: if ESET encountered 10 times a certain malware which otherwise was detected by a significant number of vendors, why did not add a rule or something to have that particular malware detected?

Why was necessary for an user to pinpoint the problem and to persuade ESET to implement a detection????

[itman 1,667]

13 hours ago, novice said:

Still I did not get it: if ESET encountered 10 times a certain malware which otherwise was detected by a significant number of vendors, why did not add a rule or something to have that particular malware detected?

I guess you do still do not understand my previous reply on this occurrence. An "in-the-wild" occurance of 10 statistically equates to a near zero probability of capture, analysis, and mitigation using existing capture methods. The Eset forum response as to "10 times" was in regards to the "in-the-wild" instance of the malware; not how many times an Eset product detected it.

The OP's complaint at the time was that three days had elapsed since his posting about his detection and still no specific signature for it had been issued by Eset. I can't recollect if the OP actually official submitted the malware via Eset in-product method to do so. I just recently did so for a malware sample Eset wasn't detecting that also originated geographically from this region with a low "in-the-wild" count. Eset promptly responded with detection capability in a few hours; the exact elapsed time I don't know since I wasn't specifically monitoring for that.

Edited June 21, 2019 by itman

[camelia 6]

What is Windows Defender? 🤣🤣

Came

[TomFace 539]

38 minutes ago, camelia said:

What is Windows Defender? 🤣🤣

Came

In Windows 7 (which is what I run) it acts as an antispyware program.

In Windows 10, I "think" it tries to act as an A/V scanner. See https://forum.eset.com/topic/19330-another-av-to-complement-eset/?do=findComment&comment=94318  

A Win 10 user can add their input.

Regards,

Tom

 

[novice 20]

7 hours ago, itman said:

The Eset forum response as to "10 times" was in regards to the "in-the-wild" instance of the malware; not how many times an Eset product detected it.

I do not think so. Marco's answer was very clear :" It's been seen on less than 10 machines in total "  which suggests that "10 machines with ESET"

Would be impossible for ESET to know that my machine (with Kaspersky let's say) encountered that specific malware.

Regardless how are you trying to sugarcoat it, the fact remains: for a while now ESET is subpar compared with other players on the market. Strange thing, all these players which performed better than ESET , have a free version to offer (Avast!, Bitdefender, Avira, Kaspersky, Microsoft)

 

Edited June 21, 2019 by novice

[itman 1,667]

4 minutes ago, novice said:

I do not think so. Morco's answer was very clear :" It's been seen on less than 10 machines in total "  which suggests that "10 machines with ESET"

@Marcos, care to clarify the above comment you made?

[TomFace 539]

4 hours ago, novice said:

I do not think so. Marco's answer was very clear :" It's been seen on less than 10 machines in total "  which suggests that "10 machines with ESET"

Would be impossible for ESET to know that my machine (with Kaspersky let's say) encountered that specific malware.

Regardless how are you trying to sugarcoat it, the fact remains: for a while now ESET is subpar compared with other players on the market. Strange thing, all these players which performed better than ESET , have a free version to offer (Avast!, Bitdefender, Avira, Kaspersky, Microsoft)

 

I do not see any link to the quote that "novice" is claiming that Marcos posted in this Forum.

Could you share it "novice"?

Regards,

Tom

Edited June 22, 2019 by TomFace