AV-Comparatives: Real-World Protection Test - November 2017

[TomD 0]

Another poor result from Eset... fourth from bottom.

hxxp://chart.av-comparatives.org/chart1.php?chart=chart2&year=2017&month=11&sort=1&zoom=4

Edited December 15, 2017 by TomD

[peteyt 393]

14 minutes ago, TomD said:

Another poor result from Eset... fourth from bottom.

hxxp://chart.av-comparatives.org/chart1.php?chart=chart2&year=2017&month=11&sort=1&zoom=4

As mentioned by multiple people in multiple posts, these tests are never 100 percent. If you google for test results you will find multiple tests with different results. One may have one doing well, one not so well etc. it all depends on multiple factors. Microsoft for example had a bug discovered recently that if someone created a virus that utilised the bug, just the scanning of it would have caused a memory corruption and given access to the system. This isn't mentioned in the results yet I'd class that as a serious issue and it isn't the only issue Microsoft has had lately. This is why it is best to take these tests with a grain of salt. 

[SCR 195]

12 minutes ago, peteyt said:

As mentioned by multiple people in multiple posts, these tests are never 100 percent. If you google for test results you will find multiple tests with different results. One may have one doing well, one not so well etc. it all depends on multiple factors. Microsoft for example had a bug discovered recently that if someone created a virus that utilised the bug, just the scanning of it would have caused a memory corruption and given access to the system. This isn't mentioned in the results yet I'd class that as a serious issue and it isn't the only issue Microsoft has had lately. This is why it is best to take these tests with a grain of salt. 

I think you're giving these types of test results to much credit at a grain of salt. 

[Marcos 5,074]

If those are poor results, then how come I virtually never hear from users that they got infected with ESET installed. Of course, there are users who do get infected but they usually have RDP enabled and a user has a weak password which can be easily bruteforced, allowing an attacker to remote in and disable or uninstall antivirus. Another type of users who get infected are those with an older version that lacks some protection features or have ESET misconfigured (e.g. the whole c: drive is excluded, some protection features are disabled, etc.). But I virtually never come across cases when users who practice secure computing and have the latest version of ESET installed get infected.

[TomD 0]

27 minutes ago, Marcos said:

If those are poor results, then how come I virtually never hear from users that they got infected with ESET installed. Of course, there are users who do get infected but they usually have RDP enabled and a user has a weak password which can be easily bruteforced, allowing an attacker to remote in and disable or uninstall antivirus. Another type of users who get infected are those with an older version that lacks some protection features or have ESET misconfigured (e.g. the whole c: drive is excluded, some protection features are disabled, etc.). But I virtually never come across cases when users who practice secure computing and have the latest version of ESET installed get infected.

You do raise a valid point there, however, It's not a one off result from AV Comparatives. How come the other AV vendors consistently get a good score on AV-Comparatives compared to ESET?

[novice 20]

37 minutes ago, Marcos said:

then how come I virtually never hear from users that they got infected with ESET installed

Based on your logic, I can say the same thing about MSE: I have it on 1 PC since several years and no infection. (and MSE DID SCORE 100%, cumulated!!!!!)

It's been more than 1 year since ESET performs poorly in AV Comparatives test, despite of being more and more sophisticated.

1-st rule of engineering: keep it simple.

Edited December 15, 2017 by John Alex

[peteyt 393]

14 minutes ago, John Alex said:

Based on your logic, I can say the same thing about MSE: I have it on 1 PC since several years and no infection. (and MSE DID SCORE 100%, cumulated!!!!!)

It's been more than 1 year since ESET performs poorly in AV Comparatives test, despite of being more and more sophisticated.

1-st rule of engineering: keep it simple.

Again do any of the results talk about this https://www.computerworld.com/article/3240936/microsoft-windows/microsoft-quietly-repairs-windows-defender-security-hole-cve-2017-11937.html - it has just been fixed so will have existed at the time of the test. My point is no security suite is perfect - you can say WD is great but i can show you evidence of the contrary 

[itman 1,659]

50 minutes ago, John Alex said:

and MSE DID SCORE 100%, cumulated!!!!!)

The recent AV-C test was performed on 64-bit Win 10, I believe the latest 1709 release, using Windows Defender + WD SmartScreen which is the latest terminology for the SmartScreen version built into Win 10. WD scored the highest false positive rate; all of which required user interaction and decision making into whether to allow or block the process. What this recent test does show is Microsoft's much hyped AI cloud scanning is clearly deficient since no user interaction should be needed if it was properly doing its job.

Edited December 15, 2017 by itman

[novice 20]

55 minutes ago, itman said:

What this recent test does show is Microsoft's much hyped AI cloud scanning is clearly deficient

So, you are saying that ESET cloud scanning is efficient???? ESET had 0.3% "user dependent" and 0.8% "compromised = total 1.1%

MSE had 1.1% "user dependent" and 0% compromised.

[SCR 195]

My personal real world test of Eset products has been running for many years now. I've never been infected with anything. I've had a few warnings over the years but never an infection nor even a false positive. So my findings based on my real world experience is that Eset has been 100% effective in my real world test. That's all that matters to me.

However, I do practice secure computing methods along with Eset's protection.

My Real World testing of Eset products continues.

[novice 20]

38 minutes ago, SCR said:

That's all that matters to me

"I have been driving Ford cars for almost 30 years now; I never been injured so far, so the airbags of my car offer 100% protection. That's all that matters to me."

Eh, how does it sound?????

 

[TomFace 539]

49 minutes ago, SCR said:

My personal real world test of Eset products has been running for many years now. I've never been infected with anything. I've had a few warnings over the years but never an infection nor even a false positive. So my findings based on my real world experience is that Eset has been 100% effective in my real world test. That's all that matters to me.

However, I do practice secure computing methods along with Eset's protection.

My Real World testing of Eset products continues.

Amen SCR. Just like the A/V tests....a dog will never catch his tail. It just goes on and on and on and on.... More fodder for the trolls.

Edited December 16, 2017 by TomFace

[novice 20]

42 minutes ago, TomFace said:

Amen SCR. Just like the A/V tests....a dog will never catch his tail. It just goes on and on and on and on.... More fodder for the trolls.

Yeah, if it doesn't fit, ignore it!

[galaxy 11]

The tests are not really helpful

[cyberhash 188]

6 hours ago, John Alex said:

Yeah, if it doesn't fit, ignore it!

I don't believe anyone ignores it especially the vendors of the products tested, but the chart makes something look worse than it actually is. A fraction of a single percent is what separates all the vendors. There is not really a bad score there ..........

[peteyt 393]

10 hours ago, John Alex said:

So, you are saying that ESET cloud scanning is efficient???? ESET had 0.3% "user dependent" and 0.8% "compromised = total 1.1%

MSE had 1.1% "user dependent" and 0% compromised.

It seems like these tests your ignoring details. 0 percent compromised is okay but when the actual product is compromised it makes you wonder. This is my point, these tests never show the true full picture. If you went by this test you would think WD was perfect yet had a massive bug and lots of false positives. If you want to use a compromised program be my guest but i will stick to eset. Never been infected but im also a safe user 

[novice 20]

31 minutes ago, peteyt said:

when the actual product is compromised it makes you wonder

You missed the point here : Microsoft quietly repairs Windows Defender security hole CVE-2017-11937 .

Software programs are inherent to vulnerabilities, the important thing is to have them patched or repaired in a timely matter.

In rest, whatever has been discussed so for , is still valid; ESET finished again on the 4-th place from bottom in AV Comparative, for the 11-th consecutive month, behind MSE.

 

[cyberhash 188]

12 minutes ago, John Alex said:

You missed the point here : Microsoft quietly repairs Windows Defender security hole CVE-2017-11937 .

Software programs are inherent to vulnerabilities, the important thing is to have them patched or repaired in a timely matter.

In rest, whatever has been discussed so for , is still valid; ESET finished again on the 4-th place from bottom in AV Comparative, for the 11-th consecutive month, behind MSE.

 

The full report for the same period only awarded MSE a rating of advanced , where ESET achieved advanced+ ......

https://www.av-comparatives.org/wp-content/uploads/2017/12/avc_prot_2017b_en.pdf

[novice 20]

13 minutes ago, cyberhash said:

The full report for the same period only awarded MSE a rating of advanced , where ESET achieved advanced+ ......

https://www.av-comparatives.org/wp-content/uploads/2017/12/avc_prot_2017b_en.pdf

* downgraded by one rank due to the score of wrongly blocked sites/files (FPs); see page 13

Expert users who do not care about wrongly blocked files/websites (false alarms) or user-dependent

detections, are free to rely on the protection rates on page 9 instead of our awards ranking which takes

those in consideration.

 

On page 9, ESET is the 7-th from the bottom, well behind  MSE

[TomD 0]

We can all agree that Eset rarely has false positives. The problem is the detection which are worrying.

[SCR 195]

12 hours ago, John Alex said:

"I have been driving Ford cars for almost 30 years now; I never been injured so far, so the airbags of my car offer 100% protection. That's all that matters to me."

Eh, how does it sound?????

 

I don't drive Fords, never owned one. So with no real world experience I'm not qualified to respond to your question.

[novice 20]

10 minutes ago, SCR said:

I don't drive Fords, never owned one. So with no real world experience I'm not qualified to respond to your question.

You didn't get the irony, did you?

[SCR 195]

1 minute ago, John Alex said:

You didn't get the irony, did you?

I wasn't looking for "irony." This and the post I responded to does not seem to be on topic, which is not about Fords, airbags or irony.

[itman 1,659]

14 hours ago, John Alex said:

So, you are saying that ESET cloud scanning is efficient???? ESET had 0.3% "user dependent" and 0.8% "compromised = total 1.1%

MSE had 1.1% "user dependent" and 0% compromised.

AV-C actually runs to tests in this Real World test series. The first test is a malware test and the second is a false positive test. Different samples are used in each test. In regards to the false positive test, the results are aggregated on page 11 for the test period; in this case July through November, 2017. WD wrongly blocked 10 samples with user interaction required for 38 samples. This yielded a wrongly blocked score of 29; the second highest score of all products tested.

[novice 20]

1 hour ago, itman said:

AV-C actually runs to tests in this Real World test series. The first test is a malware test and the second is a false positive test. Different samples are used in each test. In regards to the false positive test, the results are aggregated on page 11 for the test period; in this case July through November, 2017. WD wrongly blocked 10 samples with user interaction required for 38 samples. This yielded a wrongly blocked score of 29; the second highest score of all products tested.

MSE/WD was just an example of free AV getting better detection than a paid one.

There are at least 9 others performing better than ESET, with 1-3 FP and some of them free (Avast!, Avira, Bitdefender, Panda)

This is not the right approach in trying to justify ESET's failure, over and over again.

I would expect ESET developers to contact AV Comparatives, ask for undetected sample, analyze and provide a documented answer.

"I have been using ESET for X years, and never got infected" is not an answer.

Edited December 16, 2017 by John Alex