ESET Internet Security 11 Review Results

[Bernaro34 0]

Should I worry about this bad result?

 

[Marcos 5,074]

Bad result? I don't see any bad results there but there's definitely an issue with the methodology.

As for the urls, it appears to be there were no malicious ones that were not blocked by ESET but those not blocked was mainly fresh phishing from today which was not blocked at that time by any AV.

As for the on-demand scan "test", I "like" tests where one puts all mess (benign files, apps with Chinese gui, PUAs, etc.) into a folder, then scans the files and presents undetected files as misses AVs that detect such files have FPs and should be penalized for that but in these "tests" they get good points for detecting FPs.

A credible tester should know what he or she has in the test set, should be able to distinguish malware from PUAs, greyware and other benign applications and remove such from the test set. Including non-malicious files usually substantially skews the final results.

Also note that on-demand scans do not reflect the real-world scenario. In real world, malware is usually downloaded by malicious scripts on compromised websites or spread by spammed email. Running just an on-demand scan cannot test other protection layers that might have prevented the malware from being downloaded and executed.

[cyberhash 188]

Don't know why a user installed bitcoin miner would actually be classed as malicious. Likewise i doubt that anyone would be likely to install anything that was not in their native language or one they could not understand.

Some of the things that did install might even be legitimate and some people might actually use them, but since it's in Chinese i guess we will never know. Like @Marcos also said, PUAs are not typically malicious by nature and penalising against them distorts the results.

Additionally there was nothing else to compare it against , it was a single set of tests on a single product. Would any other suite have caught any more or less ?? We will never know

After watching the video, i don't see any bad results either

[illumination 5]

I would not take this tester above seriously in any way shape or form. He is concerned with YouTube traffic and not testing correctly. First I should mention, he likes to claim the use of zero days, anyone with any experience testing will know instantly, that these samples are far from this. They are collected from Virussign in sample packs, file extension renamed, and normally he renames the files themselves. He does not vet the samples for working/broken or legitimacy nor does he vet the samples for age and detection. He does not take the time to learn the product, how it functions and works as designed. As Marcos pointed out and is quite correct, his methodology is beyond flawed. 

 

[Malware Blocker 0]

1 hour ago, illumination said:

I would not take this tester above seriously in any way shape or form. He is concerned with YouTube traffic and not testing correctly. First I should mention, he likes to claim the use of zero days, anyone with any experience testing will know instantly, that these samples are far from this. They are collected from Virussign in sample packs, file extension renamed, and normally he renames the files themselves. He does not vet the samples for working/broken or legitimacy nor does he vet the samples for age and detection. He does not take the time to learn the product, how it functions and works as designed. As Marcos pointed out and is quite correct, his methodology is beyond flawed. 

 

Several things:

1) I am not concerned with Youtube traffic & I spend my time creating these videos to inform other people.

2) Yes my methodology is flawed because I don't have the time nor resources to have fully realistic testing methodology.

3) I don't use malware from Virussign so please don't spread the word that I do...I use samples from several sources including Hybrid Analysis, Malshare, VirusShare, etc.

4) Next thing, no I don't rename files themselves, I download the samples individually or as ZIP files created by others & then change the file extensions usually from .bin to .exe.

5) On top of all the other time spent creating a video you expect me to execute every sample? I don't have the time to do that, I am not unemployed & I do have other activities going on in my life.

6) If I am downloading the samples individually I do upload them using the VirusTotal uploader to VirusTotal & check the first submission date for each sample. If I am using samples in packs provided by other people then I just have to trust that what they say is accurate as once again I don't have the time to check each .exe file.

7) That's correct I don't take the time to "learn the product" because again I don't have that time. A key thing to note is that normal users don't necessarily take the time to learn the product either - lots of users install the product & leave it alone because they just want protection. That's why I do the tests on default settings.

I agree with you that my methodology is flawed, if you think you can do better then feel free to join the Youtube community & start uploading tests yourself. I don't have the resources nor time to "test correctly" as in test with tremendous accuracy & I don't understand how you can expect someone like myself to have that much free time to spend.

Finally, as a message to the moderator reading this - none of what I have written above is intended to be offensive or insulting towards anyone on this thread, I am simply trying to correct this person's comment because he appears to have just plucked random information out of thin air (eg. that I use VirusSign for samples).

[Malware Blocker 0]

On 10/28/2017 at 10:21 PM, Marcos said:

Bad result? I don't see any bad results there but there's definitely an issue with the methodology.

As for the urls, it appears to be there were no malicious ones that were not blocked by ESET but those not blocked was mainly fresh phishing from today which was not blocked at that time by any AV.

As for the on-demand scan "test", I "like" tests where one puts all mess (benign files, apps with Chinese gui, PUAs, etc.) into a folder, then scans the files and presents undetected files as misses AVs that detect such files have FPs and should be penalized for that but in these "tests" they get good points for detecting FPs.

A credible tester should know what he or she has in the test set, should be able to distinguish malware from PUAs, greyware and other benign applications and remove such from the test set. Including non-malicious files usually substantially skews the final results.

Also note that on-demand scans do not reflect the real-world scenario. In real world, malware is usually downloaded by malicious scripts on compromised websites or spread by spammed email. Running just an on-demand scan cannot test other protection layers that might have prevented the malware from being downloaded and executed.

Hi Marcos, yes the phishing links used were new & most likely undetected by most vendors (but in other tests using the same phishing link source I've seen products like Kaspersky do much better & I collect the links about 5 minutes before recording).

PUAs/PUPs aren't exactly false positives in every case as I'm sure you know, but if the samples are being detecting by other vendors then why not detect it unless you have a different policy on PUAs/PUPs & hacktools, cracks, etc.

PUAs/PUPs aren't malicious always, but the point is that they get detected because they cause annoyance to the user - eg. popups (adware), running at startup. I see no reason to remove these sorts of files from the test set when regular users are possibly going to run into these types of files which they should be protected against.

I agree that malware can be downloaded via scripts, email, etc. But the reality is that users do download malware from all sorts of places - you can look for game hacks & end up downloading a trojan. You could be looking for a crack & get a keylogger or stealer. The second opinion scans are there to detect things the product has missed - if something is detected in HitmanPro for example then it should be detected by the product in most cases because other vendors like Kaspersky have detected that file.

I hope you can agree & please do not take this as an insult in anyway - I am just giving my stance on the situation.

[Malware Blocker 0]

I wouldn't say it was a bad result. It detected a lot of files in the scan & the leftover files were PUAs/PUPs & crack/hack tools - which some will say shouldn't be detected or included. But a lot of cracks/hacks are detected & then if it's reported as a false positive the detection won't be removed because it's a program that can be used illegally (some vendors will change the detection name to riskware though). In terms of URLs I shouldn't worry because those were really new phishing links.

[itman 1,659]

15 hours ago, Malware Blocker said:

PUAs/PUPs aren't malicious always, but the point is that they get detected because they cause annoyance to the user - eg. popups (adware), running at startup. I see no reason to remove these sorts of files from the test set when regular users are possibly going to run into these types of files which they should be protected against.

When you installed IS ver. 11, it will ask you if you want PUA protection. As I recollect, that option is set on by default in ver. 11 whereas in other versions it was not. I would also verify that the Antivirus scanner options shown  below are enabled prior to testing:

Also for a PUP/PUA alert to be generated upon download from the Internet, ThreatSense cleaning level in Web Access Protection section must be set to "normal" cleaning level as shown in the below screen shot. If it is set to "strict," no alert will be generated and the download will be automatically deleted leading one to believe that the PUP/PUA was not detected.

Edited November 1, 2017 by itman

[persian-boy 22]

1-The detection of the potentially unsafe programs is disabled by default and he needs to enable it.
2- kmspico is not a virus!
3-The way he tests the Avs are wrong!he needs to say the results may get changes if you touch the settings! and stop giving wrong advice to ppl
4- Chinese products are not a virus (i used a lot of them and still use)!most of them are potentially unwanted programS and maybe Adware! ESET isn't adware cleaner!Consider that Chinese use Eset and if Eset blocks their software(because you don't like it) they may get mad.
5-Before you decide to click smth you can simply right-click the file and check the reputation! the live grid is there to help you and will show you the safety level of the file!
6-The malware blocker didn't use the hips stuff and I'm sure he doesn't even know  what is Hips!
There is different between Avira free which has no settingS with Eset internet security! 

Edited November 1, 2017 by persian-boy

[illumination 5]

18 hours ago, Malware Blocker said:

Several things:

1) I am not concerned with Youtube traffic & I spend my time creating these videos to inform other people.

2) Yes my methodology is flawed because I don't have the time nor resources to have fully realistic testing methodology.

3) I don't use malware from Virussign so please don't spread the word that I do...I use samples from several sources including Hybrid Analysis, Malshare, VirusShare, etc.

4) Next thing, no I don't rename files themselves, I download the samples individually or as ZIP files created by others & then change the file extensions usually from .bin to .exe.

5) On top of all the other time spent creating a video you expect me to execute every sample? I don't have the time to do that, I am not unemployed & I do have other activities going on in my life.

6) If I am downloading the samples individually I do upload them using the VirusTotal uploader to VirusTotal & check the first submission date for each sample. If I am using samples in packs provided by other people then I just have to trust that what they say is accurate as once again I don't have the time to check each .exe file.

7) That's correct I don't take the time to "learn the product" because again I don't have that time. A key thing to note is that normal users don't necessarily take the time to learn the product either - lots of users install the product & leave it alone because they just want protection. That's why I do the tests on default settings.

I agree with you that my methodology is flawed, if you think you can do better then feel free to join the Youtube community & start uploading tests yourself. I don't have the resources nor time to "test correctly" as in test with tremendous accuracy & I don't understand how you can expect someone like myself to have that much free time to spend.

Finally, as a message to the moderator reading this - none of what I have written above is intended to be offensive or insulting towards anyone on this thread, I am simply trying to correct this person's comment because he appears to have just plucked random information out of thin air (eg. that I use VirusSign for samples).

1) You used to be a member of a security forum, that you were spamming and asking how you could gain many followers quickly on youtube. You left said forum because you were stopped from advertising. You connected with other youtube members and were discussing revenue from youtube. 

2) I do test security products and have for a very long time, I use multiple resources to do so, and recognize those packs from Virussign you have been using, as I have used some myself, and from doing so, I know they are not fresh samples, just a wider variety of, which is why I use them sometimes myself personally, you may be lucky and find 8 to 9 fresher samples in one of those packs. I do not upload to youtube as my testing is for personal use only. 

3) Please explain if you do not have the time to vet samples correctly and or learn products correctly or take time to refine your methodolgy, why you even test in the first place if it is not for youtube traffic. As all your tests do, being performed this way, is grossly misinform users. 

4) Before you spam this thread with 5 more consecutive posts, please take the time to re-read these first 3 points, and let them sink in a little. Eset is a great product, and why I have joined here in this forum after a couple years use and testing of it. I am here to support them. 

[peteyt 393]

19 hours ago, Malware Blocker said:

Several things:

1) I am not concerned with Youtube traffic & I spend my time creating these videos to inform other people.

2) Yes my methodology is flawed because I don't have the time nor resources to have fully realistic testing methodology.

3) I don't use malware from Virussign so please don't spread the word that I do...I use samples from several sources including Hybrid Analysis, Malshare, VirusShare, etc.

4) Next thing, no I don't rename files themselves, I download the samples individually or as ZIP files created by others & then change the file extensions usually from .bin to .exe.

5) On top of all the other time spent creating a video you expect me to execute every sample? I don't have the time to do that, I am not unemployed & I do have other activities going on in my life.

6) If I am downloading the samples individually I do upload them using the VirusTotal uploader to VirusTotal & check the first submission date for each sample. If I am using samples in packs provided by other people then I just have to trust that what they say is accurate as once again I don't have the time to check each .exe file.

7) That's correct I don't take the time to "learn the product" because again I don't have that time. A key thing to note is that normal users don't necessarily take the time to learn the product either - lots of users install the product & leave it alone because they just want protection. That's why I do the tests on default settings.

I agree with you that my methodology is flawed, if you think you can do better then feel free to join the Youtube community & start uploading tests yourself. I don't have the resources nor time to "test correctly" as in test with tremendous accuracy & I don't understand how you can expect someone like myself to have that much free time to spend.

Finally, as a message to the moderator reading this - none of what I have written above is intended to be offensive or insulting towards anyone on this thread, I am simply trying to correct this person's comment because he appears to have just plucked random information out of thin air (eg. that I use VirusSign for samples).

I should add that this user has said in his video that he does like Eset and while some commentators have stated it was poor he has in a way defended eset. I just commented on the video earlier mentioning the fact that actually the youtube tests are generally all flawed. I could make a video that made a specific security suite look great or one that made them look bad.  

[itman 1,659]

This discussion is also a great example to only rely on vetted AV Lab test results. They test with default product settings. Many use the AMTSO malware database for their samples ensuring a standardized and verified source. They include with the test results or reference the methodology used. Most AV Labs do not use VM's but stand alone test rigs. Etc., etc.. 

Edited November 1, 2017 by itman

[illumination 5]

13 minutes ago, itman said:

This discussion is also a great example to only rely on vetted AV Lab test results. They test with default product settings. Many use the AMTSO malware database for their samples ensuring a standardized and verified source. They include with the test results or reference the methodology used. Most AV Labs do not use VM's but stand alone test rigs. Etc., etc.. 

Exactly, and even these professional testing centers have disclaimers to take their results with a grain of salt as they may or may not be exactly accurate. 

 

Real world testing that includes the "mark of the web" ect is definitely a more accurate painting of the whole picture. 

When testing for example, I have 3 email accounts, one for personal, one for product licensing and forums, and one strictly for spam collecting for testing. What happens when you open that email that has one link titled "Website" and an invitation to click it while running Eset, once clicked, Eset jumps into action and terminates the connection stopping that Trojan from ruining your day. 

What happens when you leave the products realtime active and go to these malware sample sites to download the samples, do they even make it onto the desktop, probably not... 

 

These youtube tests do nothing but misinform users, leaving them doubting their security they just paid for. They can actually endanger average users with misinformation. 

[itman 1,659]

My favorite AV Lab is SE Labs in the U.K.. I state this because they go to lengths in their comparative tests to not only show the results but also the methods those results are based upon.

For example in their latest consumer security product test for July/Aug/Sept 2017 which can be downloaded here: https://selabs.uk/en/reports/consumers , I am posting the extract of the scoring methodology used in determining protection effectiveness. Of note is that evaluation is not a simple "pass or fall" result employed by amateur security testers. Rather a number of factors need to be evaluated in determining a product's overall effectiveness against malware. BTW - Kaspersky edged out "by a hair" Eset for first place:
 

Quote

2. PROTECTION RATINGS

The results below indicate how effectively the products dealt with threats. Points are earned for detecting the threat and for either blocking or neutralising it.

• Detected (+1)
If the product detects the threat with any degree of useful information, we award it one point.

• Blocked (+2)
Threats that are disallowed from even starting their malicious activities are blocked. Blocking products
score two points.

• Neutralised (+1)
Products that kill all running malicious processes ‘neutralise’ the threat and win one point.

• Complete remediation (+1)
If, in addition to neutralising a threat, the product removes all significant traces of the attack, it gains an additional one point.

• Compromised (-5)
If the threat compromises the system, the product loses five points. This loss may be reduced to four
points if it manages to detect the threat (see Detected, above), as this at least alerts the user, who may now
take steps to secure the system.

Rating calculations

We calculate the protection ratings using the following formula:

Protection rating = (1 x number of Detected) + (2 x number of Blocked) + (1 x number of Neutralised) +
(1 x number of Complete remediation) + (-5 x number of Compromised)

The ‘Complete remediation’ number relates to cases of neutralisation in which all significant traces of the attack were removed from the target. Such traces should not exist if the threat was ‘Blocked’ and so Blocked results imply Complete remediation.

These ratings are based on our opinion of how important these different outcomes are. You may have a different
view on how seriously you treat a ‘Compromise’ or ‘Neutralisation without complete remediation’. If you
want to create your own rating system, you can use the raw data from 4. Protection Details on page 11 to roll
your own set of personalised ratings.

 

Edited November 1, 2017 by itman

[illumination 5]

2 hours ago, itman said:

My favorite AV Lab is SE Labs in the U.K.. I state this because they go to lengths in their comparative tests to not only show the results but also the methods those results are based upon.

For example in their latest consumer security product test for July/Aug/Sept 2017 which can be downloaded here: https://selabs.uk/en/reports/consumers , I am posting the extract of the scoring methodology used in determining protection effectiveness. Of note is that evaluation is not a simple "pass or fall" result employed by amateur security testers. Rather a number of factors need to be evaluated in determining a product's overall effectiveness against malware. BTW - Kaspersky edged out "by a hair" Eset for first place:
 

 

This is one I have not looked into, but have book marked it and will look into it later tonight, thank you for sharing it. 

[peteyt 393]

15 hours ago, itman said:

My favorite AV Lab is SE Labs in the U.K.. I state this because they go to lengths in their comparative tests to not only show the results but also the methods those results are based upon.

For example in their latest consumer security product test for July/Aug/Sept 2017 which can be downloaded here: https://selabs.uk/en/reports/consumers , I am posting the extract of the scoring methodology used in determining protection effectiveness. Of note is that evaluation is not a simple "pass or fall" result employed by amateur security testers. Rather a number of factors need to be evaluated in determining a product's overall effectiveness against malware. BTW - Kaspersky edged out "by a hair" Eset for first place:
 

 

Interesting criteria. Do they add any points for puas and remove any for false positives?

[itman 1,659]

They like most AV Labs penalize for FP's.

Overall, AV Labs don't use non-malicious PUA/PUP samples. Strictly speaking they are not malware. Some labs might include a separate test for them but results are not factored in for certification status. 

[Malware Blocker 0]

On 11/1/2017 at 3:34 PM, illumination said:

1) You used to be a member of a security forum, that you were spamming and asking how you could gain many followers quickly on youtube. You left said forum because you were stopped from advertising. You connected with other youtube members and were discussing revenue from youtube. 

2) I do test security products and have for a very long time, I use multiple resources to do so, and recognize those packs from Virussign you have been using, as I have used some myself, and from doing so, I know they are not fresh samples, just a wider variety of, which is why I use them sometimes myself personally, you may be lucky and find 8 to 9 fresher samples in one of those packs. I do not upload to youtube as my testing is for personal use only. 

3) Please explain if you do not have the time to vet samples correctly and or learn products correctly or take time to refine your methodolgy, why you even test in the first place if it is not for youtube traffic. As all your tests do, being performed this way, is grossly misinform users. 

4) Before you spam this thread with 5 more consecutive posts, please take the time to re-read these first 3 points, and let them sink in a little. Eset is a great product, and why I have joined here in this forum after a couple years use and testing of it. I am here to support them. 

I have read all of your points & here is my reply:

1) I was a member of MalwareTips & then a staff member tried to insult me via DM for asking him a question so I decided to leave. I was not banned, I was warned several times for posting videos in the wrong place on their forum, but there's a section there where you can advertise your videos (it's allowed on their forum). The last point here is completely false when did I discuss revenue with another Youtuber?

2) That's great, but I swear on my life that none of those samples are from Virussign - if you don't believe me then fine, but I am telling the truth & you clearly are ignoring my statements because you dislike me. I haven't used any Virussign samples since starting this channel in 2016 - most of them are from Hybrid Analysis & Malshare for example.

3) To be honest I really have no reason to continue doing Youtube when it all brings is unfair criticism from people like yourself - you are ignoring all my replies for some reason & not believing that anything I am saying is true. They don't misinform users anymore than AVTest or AVComparatives who shows AV products getting 100% in tests - which they are not capable of getting in the real world.

4) I also like ESET & I don't understand where you got the idea that I dislike the product? It's in my top 5 Paid AVs list!

[Malware Blocker 0]

On 11/1/2017 at 2:34 PM, persian-boy said:

1-The detection of the potentially unsafe programs is disabled by default and he needs to enable it.
2- kmspico is not a virus!
3-The way he tests the Avs are wrong!he needs to say the results may get changes if you touch the settings! and stop giving wrong advice to ppl
4- Chinese products are not a virus (i used a lot of them and still use)!most of them are potentially unwanted programS and maybe Adware! ESET isn't adware cleaner!Consider that Chinese use Eset and if Eset blocks their software(because you don't like it) they may get mad.
5-Before you decide to click smth you can simply right-click the file and check the reputation! the live grid is there to help you and will show you the safety level of the file!
6-The malware blocker didn't use the hips stuff and I'm sure he doesn't even know  what is Hips!
There is different between Avira free which has no settingS with Eset internet security! 

1) I believe I enabled detection of PUPs in the installer.

2) kmspico is riskware - it's an activation tool for Windows that can be used illegally - lots of AV vendors detect it as unsafe.

3) I never said not to change the settings?  I test of default settings because normal users with limited computer knowledge are unlikely to tweak any settings. Please do not bring false information into this conversation.

4) Some Chinese & foreign products can be considered PUPs or adware - they can bring pop ups & unwanted extras. ESET is a product designed to prevent harm coming to the computer - this includes adware because it brings annoyance to the user & can slow the system down, etc.

6) I do know what Hips is & Avira Free does have settings:

 

[Malware Blocker 0]

On 11/1/2017 at 7:13 PM, illumination said:

Exactly, and even these professional testing centers have disclaimers to take their results with a grain of salt as they may or may not be exactly accurate. 

 

Real world testing that includes the "mark of the web" ect is definitely a more accurate painting of the whole picture. 

When testing for example, I have 3 email accounts, one for personal, one for product licensing and forums, and one strictly for spam collecting for testing. What happens when you open that email that has one link titled "Website" and an invitation to click it while running Eset, once clicked, Eset jumps into action and terminates the connection stopping that Trojan from ruining your day. 

What happens when you leave the products realtime active and go to these malware sample sites to download the samples, do they even make it onto the desktop, probably not... 

 

These youtube tests do nothing but misinform users, leaving them doubting their security they just paid for. They can actually endanger average users with misinformation. 

Final point before I will leave because I'm clearly not making any progress. All tests of AV products are flawed - it doesn't matter if it's done unprofessionally or professionally. Obviously professional tests can provide more accurate results (although 100% scores are unrealistic) & the tests performed are of greater quality - this is why companies like AVTest & AVComparatives charge AV vendors money to have their products tested. The truth is that nothing can emulate a real environment because they are all different - you need to take every test of these types of products with a grain of salt as you like to say. 

To be honest I thought it was obvious that you needed to take tests like these with a grain of salt, but perhaps to a lot of people it isn't for some reason.

[itman 1,659]

Yes, AV Labs tests are approximations. That is why they are called "tests." No test can duplicate all the variables involved in the actual malware attack. This is simply because those variables don't exist in a test environment. For example, an infected Word document where the user unwittingly enables macros which starts the whole chain of malware infection events.

The main thing the AV Labs do is to follow established AMTSO test guidelines since most are AMTSO members. Well, most of the time that is. The recent NSS Labs, AV-Test, etc. shenanigans in regards to using "simulated" malware to facilitate Next Gen/AI solutions was another matter altogether.

Edited November 2, 2017 by itman

[illumination 5]

2 hours ago, Malware Blocker said:

I have read all of your points & here is my reply:

1) I was a member of MalwareTips & then a staff member tried to insult me via DM for asking him a question so I decided to leave. I was not banned, I was warned several times for posting videos in the wrong place on their forum, but there's a section there where you can advertise your videos (it's allowed on their forum). The last point here is completely false when did I discuss revenue with another Youtuber?

2) That's great, but I swear on my life that none of those samples are from Virussign - if you don't believe me then fine, but I am telling the truth & you clearly are ignoring my statements because you dislike me. I haven't used any Virussign samples since starting this channel in 2016 - most of them are from Hybrid Analysis & Malshare for example.

3) To be honest I really have no reason to continue doing Youtube when it all brings is unfair criticism from people like yourself - you are ignoring all my replies for some reason & not believing that anything I am saying is true. They don't misinform users anymore than AVTest or AVComparatives who shows AV products getting 100% in tests - which they are not capable of getting in the real world.

4) I also like ESET & I don't understand where you got the idea that I dislike the product? It's in my top 5 Paid AVs list!

1) I did not say you were banned, I said you left after being told you could not advertise there any more. Do you deny spamming the forum with profile statuses and post asking how to quickly build your youtube channel with followers? Do you deny asking the other youtubers how to get built up quickly, do you deny discussing with a staff member possibilities of making money from the channel that you learned of from another youtuber. Do keep in mind, I was a staff member there when all this took place. 

2) Both Malshare and Hybrid analysis only provide single samples not sample packs of 300 or 400 or 1000. Are you telling me you take the time to individually download each sample and build those massive packs that way. I should mention Virussign comes in pre-packed sample packs. 

3)So you believe pouring salt on an ax wound is better then trying to heal the issue? It is ok to misinform others because others are doing it? 

4) I never once stated you did not like Eset, nor did I state you tried to make it look bad or any other statement you may try to use to justify, re-read the above messages again. 

I am no longer a member of that forum myself, because I stood up to many that misinform users, it is wrong period. It is not a matter of liking you or not liking you, as I do not personally know you, but I know what your videos represent, and that, I do not like. 

 

Now before this thread becomes a book of back and forth banter, if you wish to speak to me some more on this subject, you are welcome to personal message me. 

Edited November 2, 2017 by illumination

[itman 1,659]

One finally comment I am going to make.

When testing with samples from malware packs, you are in essence testing the malware payload. Delivery of malware in testing is a critical factor. When an AV Lab such as A-V Comparatives performs its periodic realtime tests, it is using actual URLs where malware is present. It considers a detection to be anything that prevents the malware from executing on the test device. This means that if access to the URL is blocked, the dropper download is blocked,  or if the malware dropper execution used to deliver the malware payload is blocked, the AV solution passed the test. In other words, preventing the malware from being delivered to the target PC is actually more important than actually detecting any malicious activities from it.

[illumination 5]

20 minutes ago, itman said:

One finally comment I am going to make.

When testing with samples from malware packs, you are in essence testing the malware payload. Delivery of malware in testing is a critical factor. When an AV Lab such as A-V Comparatives performs its periodic realtime tests, it is using actual URLs where malware is present. It considers a detection to be anything that prevents the malware from executing on the test device. This means that if access to the URL is blocked, the dropper download is blocked,  or if the malware dropper execution used to deliver the malware payload is blocked, the AV solution passed the test. In other words, preventing the malware from being delivered to the target PC is actually more important than actually detecting any malicious activities from it.

This is why I mentioned having an actual real email account to test emails from, and or leaving realtime enabled while downloading samples from various sites, as these methods are how malware are realistically introduced to the system and of course test products how they are actually designed to function. While I'm not a professional tester by any means myself, methods can be used to simulate realistic scenarios. Samples executed from the desktop, still have their place, at least as far as removal media is concerned. Testing statically is pointless with old samples. Using older, wider variety of samples to test Dynamically how ever is not, as then all modules have their chance to shine. Tests can be useful to gather a glimpse of the products abilities, but they certainly need samples vetted and scenarios adjusted to be more realistic. 

[Marcos 5,074]

I don't mind amateur "tests", even if they don't reflect real-world scenario but at least those guys should remove anything that has gui as it's unlikely to be actual malware, especially if it's in Chinese, Russian, etc. and one cannot verify the purpose of such app. The fact that a particular AV detects it at VT does not make it malicious; it can be a perfectly legitimate application detected only due to the packer used. Also the "testers" should be able to provide at least hashes of tested files to AV vendors for verification and be open to correct the verdict if a particular vendor confirms that some of the files are not malicious. Last but the least there should be a notice that the test does not show how the security product protects users in real world due to various infection vectors and protection layers being in place.