ESET AV for Business versus Ransonware

[TTATITI 0]

I saw a related post in the Home product forum and while that one is closed I want to express my concerns about the effectiveness of the ESET ransomware protection.  Here goes!

I used the KnowBe4 RanSim vulnerability test against Vipre AV  and it scored a perfect 10 on on the protection. Stopped everything. 

https://www.knowbe4.com/ransomware-simulator

I ran the same test on ESET AV for Business and it failed all 10 tests.  I was told that ESET support believes the simulation isn't "fair" since the RanSim application both creates the temporary test folders and files and then performs the simulated crypto attack on them.  AVG, Symantec and even MS Windows Defender stopped at least some of 10 attack methods.  

Maybe there are some settings in ESET I'm missing.

Also, what is a "fair" test to demonstrate ESET's effectiveness compared to other products?   If I want clients to change AV protection I need to show them that ESET is better than their current solution.  A strong KnowBe4 test result would have made this a very simple choice.  

Thanks,

 Troy Taylor

[Marcos 5,074]

This is just a simulator of a specific behavior. It doesn't tell how well a particular AV protects from ransomware. That said, AV that fails the "tests" may protect you way better from ransomware file encryption than most of AVs that pass them. We don't detect innocuous applications as part of the detection process is also checking its code in memory for resemblance with actual malware to prevent FPs and this application (simulator) is indeed innocuous.

By the way, I reckon that in order to pass the tests it should be enough to create a HIPS rule that would ask for an action if a write operation on "my documents" folder is attempted.

As long as you use the latest version (ie. Endpoint v6 in business environment) and have all features enabled, the chance of getting files encrypted by malware should be pretty low. I don't tell none because there's no security solution in the world that would provide 100% protection from all threats without excessive number of false positives.

[TTATITI 0]

Marcos,

 

 This is not an innocuous application.  KnowBe4 has put together a suite of tools to help companies improve their processes and environments and RanSim uses the same techniques the real threats do.  In the early days of AV we had the EICAR test to help prove that the protection would work.  I need that same validation to show why a customer should change to ESET.   If you haven't already I suggest trying the RanSim test for yourself.  Just please don't do like I did and run it against a production PC!  It triggered our proprietary Crypto protection which shut-down all the server files shares.  I've posted my most recent results when testing against Vipre AV.

Regards,

  Troy

 

[Marcos 5,074]

3 hours ago, TTATITI said:

 This is not an innocuous application.

If it's not innocuous that you're saying it's not a simulator but actual malware that encrypts files without user's consent.

Quote

KnowBe4 has put together a suite of tools to help companies improve their processes and environments and RanSim uses the same techniques the real threats do.  

Nope, RanSim does not use the same techniques as most of ransowmare does. If it would, AVs that pass the tests would 100% protect users from encryption but that's not the case in the real-world scenario. Quite the contrary, despite failing these tests ESET provides excellent protection when it comes to Filecoders and file encryption. Also malware writers can use RanSim to find out what techniques of encryption to avoid in order to get around those AVs behavioral detections. It may give you a false sense of security that using a software that passed the test will 100% protect you from malicious file encryption.

Don't blindly trust some fancy graphs without knowing what is behind and how things differ in real world.

[TTATITI 0]

 

RanSim behaves exactly like a real threat.  If a new, unknown program appears and starts encrypting and acting on large amounts of local and network files my AV should at a minimum provide a warning.

I don’t want to be combative.  I need data in order to make a strong recommendation that a client should move to a different solution and it is not the graph but the science behind it that makes RanSim an effective evaluation too.

https://knowbe4.zendesk.com/hc/en-us/articles/229040167-RanSim

Troy

[Marcos 5,074]

We tested RanSim and it didn't encrypt users' data. Otherwise it'd not be a simulator but actual trojan that would be detected by ESET. RanSim does not tell how well a particular AV would protect you from file encryption by Filecoder ransomware.

I recommend reading Itman's post https://forum.eset.com/topic/10792-ransomware-simulators-a-detailed-analysis/.

[Pankaj 2]

Hi,

I have read the various ESET knowledge base articles regarding preventing data encryption by Filecoders and have made various HIPS rules to that effect. And instead of blocking I made it "to ask" to make it more practical for Home use. This is where I was genuinely impressed with Internet Security 10. The power that you get with custom HIPS rules is just unbelievable.

I agree with Marcos on this one. RanSim is a legitimate program and runs the encryption inside a simulation, therefore it should not be detected as a threat by any AV. If an AV catches it as a threat then that AV is just doing something specific to that program just to pass that test. Any AV can create test specific rules/settings in their software to pass that test but that would not mean that they would pass all kinds of that test. For example an AV that can pass the AMTSO phishing test will not necessarily block all phishing attempts. Even worse that AV may only pass just that AMTSO test but could fail badly in real world phishing cases.

All in all a it is important that the behaviour patterns of all known Ransomware is monitored and caught by an AV and not passing a test which may or may not a real world scenario.

[TTATITI 0]

Why is RanSim treated as a legitimate program and who makes that determination? ESET? What value is there to doing that?

 I'm frustrated by a reoccurring theme of, "You can't use a test to test because we know it is a test." when I ask why ESET totally fails to detect a threat that even Windows Defender stops at least part of.  

  Do you have supporting data that shows where ESET outperforms the competition when they are in the same ransomware scenario?

Troy