ESET How to treat it ，SKD Labs think EIS ranking “ second to last”

[yanchenyu 0]

https://www.skdlabs.com/2021/news_0208/669.html

Is it because of localization? It has always been thought that eset is the top ranking, but the result is regretful. Is the third-party evaluation conclusion true?

 

Attachment pdf, the original report！

20210208060154284.pdf

[Nightowl 202]

PDF is only accessible to ESET Staff but I doubt they will download it.

[Marcos 5,074]

Never heard of that testing organization. Test methodology is not known and we didn't get missed samples for verification from them either which is something that prestigious test organizations do. During verification we often point out a lot of bad samples (clean/grey/non-functional,...) that are not subject to detection and had to be removed from the test set. This was not the case of this "test". Also the fact that all prestigious testing organizations rank ESET high and just one "test" ranks it very low tells something. Another very suspicious thing about this is something with a Chinese name on places 1-4.

[itman 1,659]

Background data on SDK Labs here: http://www.skdlabs.com/html/english/ . Of note is they are an AMTSO member which I verified on the AMTSO web site.

Also of interest is SDK Labs state that Eset in a test participant.

My best guess to Eset's poor performance is they are located in Peoples Republic of China. As such, their malware samples might reflect attacks prevalent within China. There have been past multiple discussions on that regard to Eset's detections of those.

What I believe is the issue is malware identification data for in-China distributed malware is limited. Appears the major AV vendors there like Qihoo do not always upload their samples to main malware harvesting databases. As such, Eset is "hamstringed" in malware detection there; relying primarily in what Eset installed products can upload for further analysis. Since Eset is not a "major player" in the PRC AV market, the number of malware samples it can harvest is limited.

Edited February 9, 2021 by itman

[Marcos 5,074]

They list ESET among their customers but I've got a confirmation that this is untrue and we are not a customer of theirs.

Also a colleague in charge of communication with testing organizations confirmed that we neither received missed samples for verification nor the testing methodology is known which is something that trustworthy testers must disclose and share.

[itman 1,659]

2 hours ago, Marcos said:

They list ESET among their customers but I've got a confirmation that this is untrue and we are not a customer of theirs.

Also a colleague in charge of communication with testing organizations confirmed that we neither received missed samples for verification nor the testing methodology is known which is something that trustworthy testers must disclose and share.

Not surprised by these findings. Eset should file a complaint with AMTSO.

What I am surprised about is they are a Microsoft MVI certification source: https://docs.microsoft.com/en-us/windows/security/threat-protection/intelligence/virus-initiative-criteria

Edited February 9, 2021 by itman

[yanchenyu 0]

1. I think you may understand that it may be somewhat inaccurate. "SKD Labs" is the evaluation and certification organization of "West Coast Labs" in China. Strictly speaking, it is localized, but it is not a Chinese company.
2. The second and fourth are "McAfee AVERT" and "Kaspersky" respectively, which are just Chinese names, not that they are also products of Chinese companies. 360 antivirus, also ranked 9th.
3、they are a Microsoft MVI certification source: https://docs.microsoft.com/en-us/windows/security/threat-protection/intelligence/virus-initiative-criteria
4. Therefore, I think the key point is to communicate with them by email objectively and fairly, analyze the specific reasons, rather than doubt whether they are Chinese companies. Because "McAfee AVERT" and "Kaspersky" are not localized, nor are they Chinese companies.

[yanchenyu 0]

I translated some of the names：

[itman 1,659]

13 hours ago, yanchenyu said:

1. I think you may understand that it may be somewhat inaccurate. "SKD Labs" is the evaluation and certification organization of "West Coast Labs" in China. Strictly speaking, it is localized, but it is not a Chinese company.

Appears this has something to do with West Coast Labs affiliation with NEWSKY SECURITY LLC in 2015. NewSky concentrates on IoT security. It also is the developer of checkmarkcertified.com.

[itman 1,659]

20 hours ago, yanchenyu said:

4. Therefore, I think the key point is to communicate with them by email objectively and fairly, analyze the specific reasons, rather than doubt whether they are Chinese companies. Because "McAfee AVERT" and "Kaspersky" are not localized, nor are they Chinese companies.

McAfee Avert, aka Stringer, is not a real-time AV solution but rather a malware stand-alone cleaning tool:

Quote

McAfee Stinger is a standalone utility used to detect and remove specific viruses. It’s not a substitute for full antivirus protection, but a specialized tool to assist administrators and users when dealing with infected system. Details on new or enhanced signatures added with each Stinger build are listed in the Readme details.

https://www.mcafee.com/enterprise/en-us/downloads/free-tools/stinger.html

This raises some serious questions as to what this comparative test is about.

-EDIT- Likewise, Kaspersky has a free stand-alone scanner named Virus Removal Tool; i.e. KVRT, that is also quite effective in removing pre-existing entrenched resident malware: https://support.kaspersky.com/us/8527

Norton has a stand-alone scanner and cleaner named Power Eraser: https://support.norton.com/sp/en/us/home/current/solutions/kb20100824120155EN .

Etc, etc..

If the goal of this comparative was to test various AV available products against pre-existing entrenched resident malware, SDK Labs should have used Eset's free Online scanner or its bootable SysRescue product. Also, Eset has stand-alone tools for specific entrenched malware types here: https://support.eset.com/en/kb2372-stand-alone-malware-removal-tools

Edited February 10, 2021 by itman

[itman 1,659]

I finally converted the entire blog article into English.

SKD Labs refers to the McAfee product tested as "McAfee Comprehensive Security Protection Suite." Since no product of that name exists, I assume they are referring to McAfee Total Security.

Ditto for Kaspersky and Norton. Appears they are referring to their Internet Security versions.

[yanchenyu 0]

@itman The description is correct. 

Some problems in my translation。
They are all EIS similar products of eset! The comparison is all anti-virus software, otherwise the evaluation is meaningless.

 

Ranked fourth ：卡巴斯基反病毒软件  = Kaspersky Anti-Virus

Ranked second ：迈克菲安全保护套装  =  McAfee Total Protection

ESET = EIS

Avast Free Antivirus 

ESET I nternet Security

 

 

 

 

 

 

 

 

 

Edited February 21, 2021 by yanchenyu

[Marcos 5,074]

There is nothing to discuss until they take AV testing seriously, disclose the methodology and provide vendors with missed samples for verification. That said,we'll draw this topic to a close.