MSE performing better than ESET @ AVTest

[novice 20]

ESET detection rate: 98.1% (February)

MSE detection rate:100% (February)

[Marcos 5,071]

1, Do you also believe that MS would protect you better in real life than ESET?

2, Do you believe there are security solutions that detect 100% of malware?

3, Do you believe that 1,9% difference in detection in a test is that big?

[TJP 143]

Whom to believe...AV-C shows a different viewpoint for 2017: https://chart.av-comparatives.org/chart1.php?chart=chart2&year=2016&month=7&sort=1&zoom=2

Eset 99.2% with zero false positives or 99.5% with user interaction

MS 96.6% with 9 false positives

Let's look at the last part of 2016: https://chart.av-comparatives.org/chart1.php?chart=chart2&year=2016&month=7&sort=1&zoom=2

Eset 99.5% with zero false positives

MS 97% with 23 false positives!!

Look here at the lastest gaffe: https://www.theregister.co.uk/2017/04/03/ms_defender_bluber_false_alarm/

Performance test (i.e. system resource usage): https://chart.av-comparatives.org/chart1.php?chart=chart2&year=2016&month=7&sort=1&zoom=2

Eset - equal 1st with Avira with 0.3 impact score

MS - dead last with 22.7 impact score.

There is a reason why MSE is free and Eset isn't

Edited April 5, 2017 by TJP

[novice 20]

3 hours ago, Marcos said:

Do you also believe that MS would protect you better in real life than ESET?

I used ESET for few years; even though ESET is quite sophisticated, yet all alerts were signature based; never got an alert from HIPS or something else . MSE is good enough on detection based on signatures.

 

3 hours ago, Marcos said:

Do you believe there are security solutions that detect 100% of malware?

Is not "what I believe" is what AVTest said. 

3 hours ago, Marcos said:

Do you believe that 1,9% difference in detection in a test is that big?

No, will not make a big difference , so even though MSE would be at 98% is still exceptionally good.

 

Now, if you pair up MSE with MBAM Pro v3 , you will get a layered approach in PC security with better results than just using ESET.

Edited April 5, 2017 by MSE

[Marcos 5,071]

8 minutes ago, MSE said:

I used ESET for few years; even though ESET is quite sophisticated, yet all alerts were signature based; never got an alert from HIPS or something else . MSE is good enough on detection based on signatures.

I hardly remember an old-fashioned signature detection by ESET. Maybe with eicar and some other few files where a traditional signature suffices. ESET have used sophisticated DNA definitions for years which are based on code emulation by advanced heuristics, so not traditional signatures. I'd bet that most detections you've seen were thanks to various ESET's technologies. For more info, read https://www.eset.com/int/about/technology/.

[novice 20]

31 minutes ago, Marcos said:

I hardly remember an old-fashioned signature detection by ESET. Maybe with eicar and some other few files where a traditional signature suffices. ESET have used sophisticated DNA definitions for years which are based on code emulation by advanced heuristics, so not traditional signatures. I'd bet that most detections you've seen were thanks to various ESET's technologies. For more info, read https://www.eset.com/int/about/technology/.

Yet, on Virus Radar , I can see Update 15205 :Total: 57 (1 Android, 3 HTML, 3 JS, 11 MSIL, 39 Win32)

Android/Spy.Chrysaor.C
HTML/Phishing.BankOfAmerica.A
HTML/Phishing.NatWestBank.A
HTML/Phishing.Webmail.C
JS/TrojanDownloader.Nemucod.CRG
JS/TrojanDownloader.Nemucod.CRH
JS/TrojanDownloader.Nemucod.CRJ
MSIL/Agent.APN
MSIL/Agent.RTO
MSIL/Injector.RVY
MSIL/Injector.RVZ
MSIL/Injector.RWA
MSIL/Injector.RWB
MSIL/Kryptik.ITR
MSIL/PSW.Agent.QFA
MSIL/Spy.Agent.AIN

......................................................

 

 

[Marcos 5,071]

That doesn't mean those are traditional signatures. They are (with the exception of script signatures) mainly smart DNA detections based on application's behavior which means that one detection may theoretically cover an unlimited number of threat variants.

[itman 1,659]

I would be a bit suspicious of recent AV-Test results. They "crossed the line" in their recent testing of Cylance, a Next Gen AI/machine learning solution, by using 100% "synthetic" malware samples.

Additionally Window Defender is also receiving these vastly improved detection scores in likewise recent AV-Test results.

I always recommend that the best way to use AV lab test results is to average the results of all lab tests and use that as your baseline for evaluation purposes. Additionally, make it a point to read the methodology details of each test. The less details given, the more suspicious one should be of the results. The same applies to the more non-standard and/or unorthodox the individual tests employed. Finally and most importantly is the type and volume of samples malware used plus the source and method they were obtained.

-EDIT- Also in regards to these recent MSE and Windows Defender vastly improved 0-day detection scores, I am more than skeptical on the results, Neither product employs behavior analysis but uses the same reputational and heuristic analysis Eset employs. This leads me to believe that the malware samples used in the tests were "cherry picked" to be detectable by both products generic signatures versus those used by other AV vendor products. 

Edited April 5, 2017 by itman

[novice 20]

On ‎4‎/‎5‎/‎2017 at 9:12 AM, itman said:

This leads me to believe that the malware samples used in the tests were "cherry picked" to be detectable by both products generic signatures versus those used by other AV vendor products. 

I highly doubt that AV Test would use different sets of malware to test different security software.

From what I know , the set of malwares is the same, they will re-image the test PC and run it again for each and every entity tested.

[itman 1,659]

The AV-Comparatives lab tests mentioned previously rated Windows Defender one of the lowest rated products.  

Another lab, Virus Bulletin, rates Eset second highest in Proactive protection as noted in the below recent test graphic. The difference between Proactive and Reactive scores in the chart is that Proactive testing disables the network connection which results in the product being tested without any reputation analysis capability. This is an important test in that malware will often tramper with or disable network access. Eset scored second highest in Proactive detection among products tested.

Of interest is Microsoft does not participate in Virus Bulletin tests. I believe the following related to their test procedures might be the reason:

VB100 certification

The VB100 award is a certification of products which meet the basic standards required to be recognized as legitimate and properly functioning anti-malware solutions.

To display a VB100 logo, a product must:

•Prove it can detect 100% of malware samples listed as 'In the Wild' by the WildList Organization
•Generate no false positives when scanning an extensive test set of clean samples
•All this must be done with default, out-of-the-box settings in the VB lab environment.

Of note is a product will not be certified unless all 0-day malware are detected. Also notable, it is stated where the source of VB malware samples are selected from.

 

Edited April 6, 2017 by itman

[novice 20]

10 hours ago, itman said:

Of interest is Microsoft does not participate in Virus Bulletin tests. I believe the following related to their test procedures might be the reason

MSE is a free software ; I do not see a reason for MSE not to participate in any tests or to fake any tests.

It is free.

On the other hand, you can see "reactive detection rate" between 90%and 95% for most of the VB candidates, which is not impressive.

Proactive , less than 75%.

Edited April 7, 2017 by MSE

[itman 1,659]

14 hours ago, MSE said:

On the other hand, you can see "reactive detection rate" between 90%and 95% for most of the VB candidates, which is not impressive.

It's a "realistic" detection rate. No AV vendor including Microsoft will guarantee a 100% detection rate. If that were possible, the product would the only one used and all others would cease to exist.

14 hours ago, MSE said:

Proactive , less than 75%.

It is a vivid example of how important cloud based reputation analysis is in malware detection today. Without cloud based analysis, these percentages would be the norm versus the 95%+ detection rates with cloud analysis detection.

Since you seem to think MSE and WD are adequate protection, also check out the AV labs test results for both against ransomware. Both are deficient in that area.

[novice 20]

13 hours ago, itman said:

It is a vivid example of how important cloud based reputation analysis is in malware detection today

see this:

https://blogs.technet.microsoft.com/mmpc/2014/09/22/microsoft-cloud-protection/

Microsoft is using cloud protection to help keep our customers safe. In fact, nearly any detection made by Microsoft security products could be the result of cloud protection. Software developers often ask us how this cloud protection works and how they can improve our cloud’s impression of their software.......

[TomFace 539]

I have two questions for MSE...do you work for Microsoft or are you receiving any form of compensation from them?

Edited April 10, 2017 by TomFace

[novice 20]

2 hours ago, TomFace said:

I have two questions for MSE...do you work for Microsoft or are you receiving any form of compensation from them?

Oh yeah... I am vice-president  at MSE and I receive millions of dollars just to post on ESET forum.....

Why do you have to find all kind of explanations and cannot accept the simply fact that , indeed, MSE improved its detection rate?

  

 

[Marcos 5,071]

Well, a cloud blacklisting system may be simple based on hashes of files or advanced. ESET leverages mainly a blacklist of hashes calculated from the results of emulation provided by advanced heuristics, so-called DNA hashes. For that reason, one DNA hash can block a lot of (even not yet existing) malware.

[itman 1,659]

5 hours ago, MSE said:

Why do you have to find all kind of explanations and cannot accept the simply fact that , indeed, MSE improved its detection rate?

Because I haven't seen any other supporting AV lab test results to support the recent AV-Test results. The Feb., 2017 AV-Comparatives test contradicts the AV-Test result.

Other AV Labs should be posting 1st quarter 2017 test results soon. When those are released, I will post same.

 

[novice 20]

10 hours ago, Marcos said:

ESET leverages mainly a blacklist of hashes calculated from the results of emulation provided by advanced heuristics

In spite of all this sophistication, I never had an ESET detection other that signature based, always with clear name of item detected.

And this for more than 5 years; nothing like HIPS detection, behavior detection, etc.

[Marcos 5,071]

1 hour ago, MSE said:

In spite of all this sophistication, I never had an ESET detection other that signature based, always with clear name of item detected.

You've never seen Win32/Injector, Win32/Kryptik or Win32/GenKryptik detections? Even those with particular malware names are often smart DNA detections based on emulation by advanced heuristics which can be detected by AMS in cooperation with HIPS upon execution.

[novice 20]

 Win32/Injector is detected by MSE even though doesn't have such sophisticated mechanisms as ESET:

 https://www.microsoft.com/security/portal/threat/encyclopedia/entry.aspx?Name=VirTool%3AWin32%2FInjector

 

[Marcos 5,071]

I would kindly ask everyone to refrain from personal attacks. It really doesn't matter if someone works for Microsoft or another company; everybody has the right to join discussion as long as he or her stays objective. Therefore some inappropriate comments have been removed.

[itman 1,659]

I said I would post back when another AV Lab comparative test was published. A-V Comparatives just did so for their March real world test: https://www.av-comparatives.org/wp-content/uploads/2017/04/avc_factsheet2017_03.pdf

On Win 7, MSE did score marginally higher than Eset.  

[novice 20]

2 hours ago, itman said:

On Win 7, MSE did score marginally higher than Eset.  

More exactly, ESET  99.1% and MSE 99.7%

Not quite "marginal".

[itman 1,659]

1 hour ago, MSE said:

Not quite "marginal".

.6%

[novice 20]

9 hours ago, itman said:

.6%

The last 1% is the most difficult to achieve.