Jump to content
Sammo

AV-Comparatives Real-World Protection Test February-June 2018

Recommended Posts

Posted (edited)
2 hours ago, Sammo said:

It must have taken a bit of digging to find a test from a year ago:lol:.

Test like this are not worth their weight in salt.

So what is your purpose and point in posting this?

Regards,

Tom

Edited by TomFace

Share this post


Link to post
Share on other sites
Posted (edited)

I assume the reference is to this year's most recent A-V C Realtime test where Eset scored 98.4%; approximately the same as it has previously scored recently in this test series.

If one has concerns about Eset, refer to this more comprehensive test series where over 10,000 malware samples are used: https://www.av-comparatives.org/tests/malware-protection-test-march-2019/ . Eset scored 99.86% for malware protection.

Again, this is only one AV Lab's test; and test series for that lab. Refer to all the AV lab tests that Eset participates in and you will observe that Eset is a top scorer overall.

Edited by itman

Share this post


Link to post
Share on other sites
2 hours ago, TomFace said:

It must have taken a bit of digging to find a test from a year ago:lol:.

Test like this are not worth their weight in salt.

So what is your purpose and point in posting this?

Regards,

Tom

I hope you are right about this not being worth it's salt. Also, to see the most recent test you need to change the date on that chart  to 2019 Feb to May. Eset only scored 98.4% next to dead last. Can't really be right.

Share this post


Link to post
Share on other sites
Posted (edited)

A-V C is "very creative" when it comes to finding samples for its Realtime test series. It's not uncommon for it to slip in a few samples that are geographically restricted to one country and/or region within with an "in-the-wild" dispersion of < 10. The odds of encountering one these samples in close to zero.

Edited by itman

Share this post


Link to post
Share on other sites
12 hours ago, itman said:

The odds of encountering one these samples in close to zero.

The odds of encountering a sample cannot justify the acceptance of ESET low performance; when even Microsoft scores better , the expectation would be that somebody from ESET would step in and offer an official statement.

Share this post


Link to post
Share on other sites
Posted (edited)
11 hours ago, novice said:

when even Microsoft scores better

Here we go again. Windows Defender had a whopping 74 false positives in this test. Refer to the below screen shot that clearly shows that WD "block-at-first-sight" was set to aggressive setting level; basically blocking execution of any process without established reputation. Whereas this might be acceptable to advanced security level professionals, it certainly isn't so for the average user; especially for corp. users.

98025-1ce44734e65cff0d219e30c4b352002b.jpg.f7b49c7bcc3013a24b033483b162b40b.jpg 

-EDIT- Also 55 of the WD 74 false positives were user dependent block/allow action. It is a no-no to have the user decide if a process is malicious or not:

wd-fp.png.75931fa86463df66462e1af49fc03904.png

Ref.: https://www.av-comparatives.org/tests/real-world-protection-test-february-may-2019/

Finally and most important, note the following. A-V C does not factor false positive scoring into its protection scores for its realtime tests as is done for its more comprehensive malware protection test series. Using the above false positive scoring criteria of 50% of user decisions are wrong, WD would have scored 27/752 or 96.4% placing it at the bottom of the protection scoring heap.

Edited by itman

Share this post


Link to post
Share on other sites

I have always been very leery of comparison tests.

Sorry if I came across as a bit snarky.

Regards,
 

Tom

Share this post


Link to post
Share on other sites

 

4 minutes ago, itman said:

Again, this is for 2018. I posted a link above for the current 2019 test.

All links are at 2018, I think we have to manually change it to 2019 when we get to the site.

Share this post


Link to post
Share on other sites
8 minutes ago, BeanSlappers said:

Why did it work for you and not for me?  That is crazy.

I just changed the year and months in the link to make it work. 😊

Share this post


Link to post
Share on other sites
14 minutes ago, novice said:

So ESET is not even mentioned here....

Yes it is.

Share this post


Link to post
Share on other sites
10 hours ago, BeanSlappers said:

Yes it is.

Sorry, I did not see it at first.

But 98.4%????    .

We have Avira (free) , Kasersky (free) at 100%, Bitdfender (free) at 99.9%, Microsoft (free) at 99.6% .

 

Share this post


Link to post
Share on other sites
Posted (edited)

It's all good, you made me triple check to lol.

No AV is 100%, that is near impossible because there are new malware every day. 

Edited by BeanSlappers

Share this post


Link to post
Share on other sites
4 minutes ago, BeanSlappers said:

because there are new malware every day

Most anti viruses, if not all , have these days sophisticated mechanisms to deal with unknown malwares : behavior blockers, heuristics, HIPS, generic signatures....

To expect to get a sample first and add detection after is impossible these days ; I remember one of ESET officials saying " this sample was seen only 10 times by ESET , in the whole word, that's why we did not detect it"

I was shocked by such statement.

Share this post


Link to post
Share on other sites
11 hours ago, novice said:

To expect to get a sample first and add detection after is impossible these days ; I remember one of ESET officials saying " this sample was seen only 10 times by ESET , in the whole word, that's why we did not detect it"

I was shocked by such statement.

This again shows your obvious disconnect with the "real malware world." Not the simulated one put forth in AV lab testing.

Someone recently sent me a malware 0-day sample that only recently had been detected by 6 AV vendors at Virus Total. Half of those vendors specialize in malware detection circulated in the country where the malware had been discovered. The remaining detection vendors specialize in malware detection in the specific region.

BTW - this malware specifically targeted Windows Defender and bypassed it. So if other AV solutions did not detect it, is that a missed detection since it was not a threat to them?

Share this post


Link to post
Share on other sites
59 minutes ago, itman said:

This again shows your obvious disconnect with the "real malware world." Not the simulated one put forth in AV lab testing.

BTW - this malware specifically targeted Windows Defender and bypassed it. So if other AV solutions did not detect it, is that a missed detection since it was not a threat to them?

I was referring to this:

The official explanation: " It's a Chinese ransomware written in Python with Chinese instructions. It's been seen on less than 10 machines in total. "

While the OP complained

" another of real-life experience with ransomware bypassing ESET protection layers. It is still "at large" even for now with ver15819 definition and has 3 days of reputation history... Other vendors have successfully blocked the encryption through their behavioral detection layer "

 

Share this post


Link to post
Share on other sites
1 hour ago, itman said:

This again shows your obvious disconnect with the "real malware world." Not the simulated one put forth in AV lab testing.

Someone recently sent me a malware 0-day sample that only recently had been detected by 6 AV vendors at Virus Total. Half of those vendors specialize in malware detection circulated in the country where the malware had been discovered. The remaining detection vendors specialize in malware detection in the specific region.

BTW - this malware specifically targeted Windows Defender and bypassed it. So if other AV solutions did not detect it, is that a missed detection since it was not a threat to them?

Very true.  So does eset do it for one region or all area's of the world?

Share this post


Link to post
Share on other sites
1 hour ago, BeanSlappers said:

So does eset do it for one region or all area's of the world? 

Microsoft a while back got a lot of free press on how Windows Defender ATP was able to detect a a zero day malware. What Microsoft didn't publicly disclose at the time but did so later via a blog detailed analysis of the incident is the following. At least 6 WD ATP installations were infected by the malware prior to Azure AI cloud server analysis returned a positive identification of malware status. BTW - those infected installations were all located in a specific region within Russia.

Bottom line - there is no such thing as 100% 0-day protection. If there was, that concern would in short order be the only security solution used and all other AV vendors would cease to exist.

Share this post


Link to post
Share on other sites
Posted (edited)
20 minutes ago, itman said:

Microsoft a while back got a lot of free press on how Windows Defender ATP was able to detect a a zero day malware. What Microsoft didn't publicly disclose at the time but did so later via a blog detailed analysis of the incident is the following. At least 6 WD ATP installations were infected by the malware prior to Azure AI cloud server analysis returned a positive identification of malware status. BTW - those infected installations were all located in a specific region within Russia.

Bottom line - there is no such thing as 100% 0-day protection. If there was, that concern would in short order be the only security solution used and all other AV vendors would cease to exist.

Did you miss the question?  I didn't ask about microsoft, I didn't specifically ask about 0 day either.

Edited by BeanSlappers

Share this post


Link to post
Share on other sites
6 minutes ago, BeanSlappers said:

Did you miss the question?  I didn't ask about microsoft, I didn't specifically ask about 0 day either.

Eset and other AV vendors get data from malware feeds and honeypots world-wide. The problem is that there are certain geographic areas such as China for example, where access to such data is restricted, filtered, or otherwise difficult to obtain in  a timely fashion. Of course, malware dispersion and frequency is a major factor in detection by the aforementioned. If only a few samples exist in the wild, their targets are restricted to a specific area or business concern, etc., the likelihood of quick detection by existing monitoring methods are quite low. 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...