av-comparatives rating

[New_Style_xd 69]

Guys passing by to inform you once again ESET does not do well in the test:

Real-World Protection Test February-May 2022

https://www.av-comparatives.org/tests/real-world-protection-test-february-may-2022/

Business Security Test March-April 2022 – Factsheet

https://www.av-comparatives.org/tests/business-security-test-march-april-2022-factsheet/

Once again we are behind other products much weaker than ESET.

[itman 1,659]

First in regards to the  Business Security Test March-April 2022, we are talking about the Malware Protection component of this test; not the Real-World Protection Test component that Eset performed well on. So let's review the methodology employed in Malware Protection testing:

Quote

The Malware Protection Test assesses a security program’s ability to protect a system against infection by malicious files before, during or after execution. The methodology used for each product tested is as follows. Prior to execution, all the test samples are subjected to on-access scans (if this feature is available) by the security program (e.g. while copying the files over the network). Any samples that have not been detected by the on-access scanner are then executed on the test system, with Internet/cloud access available, to allow e.g. behavioural detection features to come into play. If a product does not prevent or reverse all the changes made by a particular malware sample within a given time period, that test case is considered to be a miss. For this test, 1,007 recent malware samples were used.

It appears that initially the malware samples are being tested w/o Internet access available. This means that LiveGuard Advanced cloud scanning would not have been deployed. It should be noted that LiveGuard cloud scanning is only performed upon file creation time as I understand it; not at file execution time with the exception being files contained on external media.

Edited June 15, 2022 by itman

[itman 1,659]

In regards to the Real-World Protection Test February-May 2022 test, Eset ranked third from last place; tied with two other vendor products. One of those products was Microsoft Defender. This does again bring up the question if EIS is worth the cost.

 

Edited June 15, 2022 by itman

[New_Style_xd 69]

22 minutes ago, itman said:

In regards to the Real-World Protection Test February-May 2022 test, Eset ranked third from last place; tied with two other vendor products. One of those products was Microsoft Defender. This does again bring up the question if EIS is worth the cost.

 

I don't think it's worth the cost.
because it has free products doing the same role as ESET.

[Marcos 5,074]

40 minutes ago, New_Style_xd said:

I don't think it's worth the cost.
because it has free products doing the same role as ESET.

You are free to make a choice. What is the reason then that you're using ESET? Generally it is recommended to test various AVs and use the one that works best you.

[MartinPe 10]

I wonder if the Eset Security Premium would have fared better with Liveguard.

[itman 1,659]

I will state this about Microsoft Defender. It has two fundamental flaws.

The first is hackers ability to find ways around its self-protection despite Microsoft's best efforts otherwise.

The second is its exploit protection is dismal. MRG Effitas that performs exploit testing for its real-time test series, consistently shows MD "at the bottom of the heap" on exploit protection scores. This means if you're an average home user that does not keep their OS and app software updated as soon as one is available, you run the risk of being exploited. I assume there are large number of home users that fall into this category.

Finally, MD with default settings is not adequate. It needs to be "hardened" by applying all available ASR mitigations plus tweeting block-at-first-sight cloud scanning settings. Whereas this can be done easily via Group Policy in Win Pro+ versions, Win Home users will have to deploy registry modifications to do the same. This again is beyond the capabilities of most Windows Home version users.

Edited June 15, 2022 by itman

[New_Style_xd 69]

1 hour ago, Marcos said:

You are free to make a choice. What is the reason then that you're using ESET? Generally it is recommended to test various AVs and use the one that works best you.

I'll list what I like about the eset.
1- I like the product because it doesn't harm the performance of my system.
2- The way to update the new version of the antivirus I think is very good, it doesn't force you to install it automatically.
3- I like the eset icon on the taskbar with animation.
4- I like the details and information about when and which modules were updated.

I'll list what I don't like about the eset.
1- In the product tests, the eset is below several others, even the free ones. I'm talking 10 years to today's date, it didn't stay at the top for long at first.
2- Dark mode has not been set.
3- It takes a long time to release updates and improvements to customers.
4- The product's interface is the same as version 9, it hasn't had a significant change for over 10 years.
5- Feedback from users asking for antivirus modifications and improvements has not been answered for several years.
6- The password vault could be free or already included in the internet security version as a program to be installed on the machine as kaspersky does with its password vault.
7- Could have a VPN, knowing that you always say that an AV has to be just an AV.
8- Does not have System Watcher module.
9- Program updater.
10- Have a research center here in BRAZIL, because there are several Malware that only works in Brazil as an example Malware that creates fake payment slips.

[AnthonyQ 48]

I don't know if ESET is constantly collecting and analyzing undetected samples from VirusTotal. But I do notice that many competitors like Kaspersky, McAfee and Symantec do so. This practice can improve the detection rate.

After I sent a fresh malware sample to VirusTotal, it would soon appear on Kaspersky's OpenTip with a sandbox analysis report, but at the same time, the LiveGrid reputation is still unavailable (blank).

Edited June 16, 2022 by AnthonyQ

[TheStill 29]

18 hours ago, New_Style_xd said:

Guys passing by to inform you once again ESET does not do well in the test

I must be looking at different results then as to me, the real-world results show ESET with no false positives. This puts it at the top of that category. The difference between ESET and the others in the protection category is 1% which in reality is nothing at all especially when we are talking about between 99% and 100%. It gets much worse though when you look at the number of false positives for the companies at the top of the protection list. Most of them are all at the bottom for really high false positives. 

Security isn't just about blocking every single thing that a user may interact with. If a user is constantly being annoyed by false positives then they are going to start turning the anti-virus off occasionally so they can do what they want to do. The same with if they are trying to use their computer and they can't because they see the anti-virus hogging all of the resources. These things all need to be carefully balanced in order to be the most effective. 

[peteyt 393]

6 hours ago, TheStill said:

I must be looking at different results then as to me, the real-world results show ESET with no false positives. This puts it at the top of that category. The difference between ESET and the others in the protection category is 1% which in reality is nothing at all especially when we are talking about between 99% and 100%. It gets much worse though when you look at the number of false positives for the companies at the top of the protection list. Most of them are all at the bottom for really high false positives. 

Security isn't just about blocking every single thing that a user may interact with. If a user is constantly being annoyed by false positives then they are going to start turning the anti-virus off occasionally so they can do what they want to do. The same with if they are trying to use their computer and they can't because they see the anti-virus hogging all of the resources. These things all need to be carefully balanced in order to be the most effective. 

Yeah it is interesting when you look at some of the apparent top ones, based on blocking.

NortonLifeLock blocked 100 percent of stuff but had 50 false positives. In relation Eset blocked 99 percent of stuff but had no false positives.

I've always said I'd like eset to add new features that competitors have but at the end of the day there's always a risk if you make an AV too aggressive. A false positive might seem like nothing but what if it was a system file or linked to a key application that a business relied on

[itman 1,659]

Since we are the subject of A-V Comparatives, one test worth reviewing is the Advanced Threat Protection test for Consumer AV products: https://www.av-comparatives.org/tests/advanced-threat-protection-test-2021-consumer/ .

In this test, Eset and Kaspersky had identical scores; each missing two samples.

The important point to note in this test is BitDefender's score which was poor. Now BitDefender traditionally scores high in the AV lab's Real-time tests. The point here being that all AV lab tests available must be analyzed in accessing an AV product overall effectiveness.

Also, Microsoft was not listed in this test. This means they either declined to be tested, or Microsoft Defender's score was poor enough to have the results omitted in the public published report.

Edited June 16, 2022 by itman

[New_Style_xd 69]

The bad thing is that this test is already 1 year ago, nowadays other products have implemented new techniques in their products and new technology.
eset livegrand example was implemented a short time ago.
The kaspersky sandbox has been around for years. without counting other resources that exist.

Edited June 17, 2022 by New_Style_xd

[rotaru 10]

8 hours ago, itman said:

This means they either declined to be tested, or Microsoft Defender's

So, that means nothing, in fact.

 

When ESET decided not to be tested in AV Test, is OK???

[AnthonyQ 48]

17 hours ago, peteyt said:

Yeah it is interesting when you look at some of the apparent top ones, based on blocking.

NortonLifeLock blocked 100 percent of stuff but had 50 false positives. In relation Eset blocked 99 percent of stuff but had no false positives.

I've always said I'd like eset to add new features that competitors have but at the end of the day there's always a risk if you make an AV too aggressive. A false positive might seem like nothing but what if it was a system file or linked to a key application that a business relied on

The FP rate is considered by AV-Comparative to classify tested products. A high enough FP rate will cause a product with a 100% block rate to be classified as "Tested". As such, Norton only got "Standard" award, even if it blocks all threats.

However, let's look at products awarded "Adcanced+": They have very high protection rates yet very low FP rates. 

[itman 1,659]

5 hours ago, rotaru said:

When ESET decided not to be tested in AV Test, is OK???

I believe you are referring to the special ransomware and  data stealer test performed here: https://www.av-test.org/en/news/29-protection-solutions-against-data-stealers-and-ransomware-under-windows-10/ .

And I agree with you that Eset should have participated in the commercial product test and had its results published publically. Note: Eset no longer participates in consumer product testing at AV-Test. The primary reason being that the AV labs rarely perform testing exclusively in this category. I never did "buy into" the reason exclusively was the test cost.

Edited June 17, 2022 by itman

[itman 1,659]

As far as Microsoft Defender goes, reference is had to how it performed when tested by AVlab in Poland: https://forum.eset.com/topic/31961-where-is-eset-in-the-av-test-test/?do=findComment&comment=149193 .

Full report here: https://avlab.pl/en/results-january-2022/

Note that AVlab is not an AMTSO member and does not have to adhere to its recommended guidelines: https://www.amtso.org/wp-content/uploads/2019/12/AMTSO-Testing-Protocol-Standard-for-the-Testing-of-Anti-Malware-Solutions-v1.3.pdf in regards to malware testing. Those guidelines were developed in agreement with AV vendors who are AMTSO members. This means that AVlab can be more "creative" in its testing of malware.

Eset test results here: https://avlab.pl/en/recent-results/

Edited June 17, 2022 by itman

[itman 1,659]

Another source to consider when evaluating anti-virus software are trusted third party web sites.

One such site is PC Magazine that has been reviewing AV software for as long as I can remember. The plus in PC Magazine reviews is it does its own malware testing and also factors in AV lab results in the final determination of a AV product's effectiveness against against malware. Also, by performing it's own ad hoc testing, PC Magazine is not constrained by the AMTSO testing standard that applies to AV labs. On this regard, Rubenking knows what he is doing. He's been performing this type of testing for years.

So what does PC Magazine think of Eset consumer product effectiveness against malware?

Quote

Mixed Malware Protection Scores

I’m always happy to have results reported by the independent labs, but not every product makes it into those reports. Even when results are available, I still run hands-on malware protection testing, to see the product's defenses in action.

When I opened the folder containing my current collection of malware samples, NOD32's real-time protection gave them the once-over. However, it only eliminated 32% of them at this point. That’s uncommonly low—most products score in the 80s or better. Adaware Antivirus Free impressively eliminated 90% of this same sample collection on sight, though it came up short in other areas.

Notably, NOD32 recognized less than half of the ransomware samples on sight. Of a dozen other products whose real-time protection wipes out known threats on sight, eight eliminated all the ransomware samples on sight and four eliminated all but one.

Continuing the test, I launched the remaining samples. Clearly the antivirus applies a tougher standard to programs that are about to launch. It prevented quite a few samples from launching at all. That included all the remaining ransomware samples, most of which it identified by name. It did flag some samples as PUAs, and I chose to delete all of those. In other cases, it caught a malware component during the installation process.

NOD32 detected 89% of the samples one way or another. However, the fact that it let several samples install executable files brought its overall score down to a dismal 7.9 points, even worse than the 8.3 points it scored in my previous review. Tested with this same sample set, Malwarebytes managed 100% detection and a perfect 10 points. McAfee came close, with 100% detection and 9.9 points. And Webroot SecureAnywhere AntiVirus detected 99% and scored 9.8.

NOD32’s score in this test is the lowest of any product tested with the current sample set. That result doesn’t line up at all with its many excellent lab test scores. When my results don’t jibe with the labs, I give the labs more weight.

It takes me quite a while to collect and analyze a new set of malware samples, so those necessarily stay the same for months. To check a product's protection against the latest in-the-wild threats, I start with a feed of malware-hosting URLs detected in the last few days by researchers at MRG-Effitas. I launch each URL in turn and note whether the antivirus prevents access to the URL, eliminates the malware payload, or utterly fails to detect any threat.

Ransomware Protection

According to ESET, NOD32’s ransomware protection has been beefed up in this latest edition. It falls under HIPS in settings, meaning I could test it by turning off ordinary real-time protection and leaving HIPS turned on. I did just that and then tested a dozen real-world ransomware samples. The results weren’t pretty.

One of the samples didn’t try any chicanery; without ransomware behavior, the ransomware detector naturally didn’t react. Four file encrypting samples proceeded to do their dirty deeds without a peep from NOD32, as did one whole disk encrypting sample. Four more got caught after launch by NOD32’s scan for active malware in memory.

That leaves exactly two detected by ransomware protection. This detection took the form of a warning about a program trying to modify files in a suspicious way. It didn’t mention ransomware. Denying the activity saved the day in one case. The other managed to encrypt over 4,000 files before NOD32 took it down.

As with ransomware protection layers in other antivirus products, NOD32’s isn’t intended as the first line of defense, or even the second. With all cylinders firing, NOD32 eliminated almost half the samples on sight and wiped out the rest when they tried to launch. But this test suggests the ransomware-specific protection layer could use another round of enhancements.

https://www.pcmag.com/reviews/eset-nod32-antivirus

Unfortunately, the current review by PC Magazine parallels its past like reviews of Eset in regards to software protection capability. The bottom line is if you're in the "take AV lab reviews with a gain of salt" camp, you would run away screaming from Eset if PC Magazine reviews were your only evaluation category. However, I have seen enough other third party reviews like this to seriously question AV lab test reliability.

Edited June 17, 2022 by itman

[New_Style_xd 69]

44 minutes ago, itman said:

Outra fonte a considerar ao avaliar o software antivírus são sites confiáveis de terceiros.

Um desses sites é a PC Magazine, que revê o software AV há tanto tempo quanto me lembro. A vantagem nas análises da PC Magazine é que ele faz seus próprios testes de malware e também fatores no laboratório AV resultam na determinação final da eficácia de um produto AV contra malware. Além disso, ao realizar seus próprios testes ad hoc, a PC Magazine não é restringida pelo padrão de teste AMTSO que se aplica aos laboratórios AV. A esse respeito, Rubenking sabe o que está fazendo. Ele realiza esse tipo de teste há anos.

Então, o que a PC Magazine pensa da eficácia do produto de consumo Eset contra malware?

https://www.pcmag.com/reviews/eset-nod32-antivirus

Infelizmente, a revisão atual da PC Magazine é paralela ao seu passado, como as análises da Eset em relação à capacidade de proteção de software. O ponto principal é que, se você estiver no campo "faça revisões do laboratório AV com um ganho de sal", você fugiria gritando com Eset se as críticas da PC Magazine fossem sua única categoria de avaliação. No entanto, já vi outras análises de terceiros como essa para questionar seriamente a confiabilidade dos testes de laboratório AV.

I really enjoyed your research, on the subject it only makes it more certain that ESET is not good against ransomware.
Unfortunately, ransomware and ESET do not have a good effectiveness over ransomware, unfortunately, nowadays.

[itman 1,659]

Here's another factor when evaluating security software. It's the "put your money where your mouth is" criteria.

How many security vendors provide full reimbursement for expenses related to a security breach? As far as I am aware of, only one - Cloudstrike. It will reimburse expenses up to $1,000,0000 based on the number of license seats that exist for an installation: https://www.crowdstrike.com/products/managed-services/falcon-complete/warranty-faq/

Edited June 17, 2022 by itman

[itman 1,659]

23 hours ago, New_Style_xd said:

Unfortunately, ransomware and ESET do not have a good effectiveness over ransomware, unfortunately, nowadays.

I would qualify this by stating it is not good at detecting and preventing 0-day ransomware.

Yesterday morning I actually found a 0-day ransomware sample and it was a "doozy." 0-day context here is no one at VT detected it upon initial submission. The verdict is out on VT listed vendor behavior detection since those features for many vendors are not deployed on their VT versions.

Within a couple of hours, Eset along with two other vendors had a sig. for the bugger. So I consider that a good performance by Eset. Of course if this was a targeted attack and you were the initial target, your installation would be "dead meat."

By a "doozy" of a ransomware attack, I mean this bugger was designed to evade cloud sandbox analysis. The initial Joe's Cloud Sandbox scan came back clean. Joe must have detected something suspicious; like all sandbox files encrypted, and next time ran it under "fuzzy" command line criteria. Then, the bugger was detected.

Bottom line x 10, back up your files to off-line storage media!

Edited June 18, 2022 by itman

[New_Style_xd 69]

8 minutes ago, itman said:

I would qualify this by stating it is not good at detecting and preventing 0-day ransomware.

Yesterday morning I actually found a 0-day ransomware sample and it was a "doozy." 0-day context here is no one at VT detected it upon initial submission. The verdict is out on VT listed vendor behavior detection since those features for many vendors are not deployed on their VT versions.

Within a couple of hours, Eset along with two other vendors had a sig. for the bugger. So I consider that a good performance by Eset. Of course if this was a targeted attack and you were the initial target, your installation would be "dead meat."

By a "doozy" of a ransomware attack, I mean this bugger was designed to evade cloud sandbox analysis. The initial Joe's Cloud Sandbox scan came back clean. Joe must have detected something suspicious and this time scanned it under "fuzzy" command line criteria and then detected the bugger.

Bottom line x 10, back up your files to off-line storage media!

Based on what we are commenting here on the forum, we know that ESET is not good against ransomware and has a lot to improve.
I wanted to know your opinion or even sites that can tell us, if ESET for phishing protection is really good or not.
If possible have a link showing that it has a good detection against phishing.

[itman 1,659]

15 minutes ago, New_Style_xd said:

can tell us, if ESET for phishing protection is really good or not.
If possible have a link showing that it has a good detection against phishing.

Refer to the PC Magazine article link I posted above - 'Phishing Protection Tests' section. Eset scores in the "middle of the pack" of AV products tested.

In an A-V Comparatives Anti-Phishing test commissioned by Avast, Eset scored about the same: https://www.av-comparatives.org/wp-content/uploads/2022/04/avast_phishing_04-2022.pdf

Edited June 17, 2022 by itman

[peteyt 393]

One thing I will add is I've seen mods ask people why they use eset when they question the product.

Personally I find this a bit of a bad practise. I know the mods on here aren't actually responsible directly for product development and not every feature people ask for can be implemented, but if quite a few users are asking for something then maybe there's a reason.

Surely any AV vendor is going to want to make their product as good as possible. This is also why I find it strange that some features aren't considered because they could be bypassed but anything could be bypassed. By this way of thinking a virus could bypass an AV so why have an AV.

Just to add I do hope no one eset wise takes this personally. I have no plans of stopping using eset and they have been good to me, I just want it to be the best it can be 

 

 

 

 

[itman 1,659]

BTW, A-V Comparatives runs a phishing certification test once a year. Eset did well on this test scoring 92%: https://www.av-comparatives.org/news/anti-phishing-certification-test-2021/