IDS use over 70% of CPU & too many attacks detected

[kamiran.asia 5]

Hi dear ESET Admins,

In These 2-3 days we have a problem in many VPS that FS V7.3 or 8.0 are installed.

Over 70-90 % of Cpu use by Ekrn, When IDS and Botnet Protection is disable there is no problem (Ekrn cpu usage will be less that 1%).

our support team disable all firewall policy , Block All inbound UDP and all TCP inbount in Windows Firewall (and Limit RDP with IP Whiltelist in Windows FireWall).

Still we see many Attacks in IDS log and many Blocked IP !

Ekrn dump and EpfwLog.pcapng are uploaded here : https://we.tl/t-raMXXS0y2n

What is the cause of this attack while all tcp and udp port are closed by Windows firewall !?

 

 

Edited July 14, 2021 by kamiran.asia
Edit title

[Marcos 5,074]

Looks like the server is being heavily attacked. Please provide logs collected with ESET Log Collector.

[kamiran.asia 5]

48 minutes ago, Marcos said:

Looks like the server is being heavily attacked. Please provide logs collected with ESET Log Collector.

Thank you dear  @Marcos for rapid reply as always, The Man Number1 of ESET Forum Administrators 😍

ESET Log Collector : https://we.tl/t-gEPyQZyBeK

[Nightowl 202]

You don't have a port when you click Details in Network Protection Troubleshoot?

[kamiran.asia 5]

1 minute ago, Nightowl said:

You don't have a port when you click Details in Network Protection Troubleshoot?

No , As you can see in screenshots Destination Port and any other info is N/A !

All inbount TCP and UDP are block by windows firewall but still ESET IDS is involved with attacks.

[Nightowl 202]

Put your Windows Firewall to log dropped attempts and see what is being dropped since you denied all INCOMING it must be logged there

2 minutes ago, kamiran.asia said:

No , As you can see in screenshots Destination Port and any other info is N/A !

All inbount TCP and UDP are block by windows firewall but still ESET IDS is involved with attacks.

 

[kamiran.asia 5]

3 minutes ago, Nightowl said:

Put your Windows Firewall to log dropped attempts and see what is being dropped since you denied all INCOMING it must be logged there

 

We are confused that If ports are dropped , Why ESET IDS will involved ?! ESET Network driver are working before Windows firewall ?

We will check Windows firewall dropped log.

[Nightowl 202]

Could be your Windows Firewall is misconfigured and not blocking properly , could be there is a malware inside the server that is making way for the communication for all these attacks

Try to see TCPView https://docs.microsoft.com/en-us/sysinternals/downloads/tcpview

As per the photo you are blocking only TCP 445

Edited July 14, 2021 by Nightowl

[kamiran.asia 5]

Windows Firewall Dropped Log is attached.  also Uploaded to https://we.tl/t-rU7u763VGL

2021-07-14 10:19:31 DROP UDP 51.255.115.138 239.255.255.250 57942 1900 201 - - - - - - - RECEIVE
2021-07-14 10:19:31 DROP UDP 152.228.149.234 239.255.255.250 50664 1900 202 - - - - - - - RECEIVE
2021-07-14 10:19:32 DROP UDP 51.255.115.138 239.255.255.250 57942 1900 201 - - - - - - - RECEIVE
2021-07-14 10:19:32 DROP UDP 152.228.149.234 239.255.255.250 50664 1900 202 - - - - - - - RECEIVE
2021-07-14 10:19:33 DROP UDP 51.255.115.138 239.255.255.250 57942 1900 201 - - - - - - - RECEIVE
2021-07-14 10:19:33 DROP UDP 152.228.149.234 239.255.255.250 50664 1900 202 - - - - - - - RECEIVE
2021-07-14 10:19:34 DROP UDP 51.255.115.138 239.255.255.250 57942 1900 201 - - - - - - - RECEIVE
2021-07-14 10:19:55 DROP UDP 51.255.115.140 239.255.255.250 51999 1900 201 - - - - - - - RECEIVE
2021-07-14 10:19:56 DROP UDP 51.255.115.140 239.255.255.250 51999 1900 201 - - - - - - - RECEIVE
2021-07-14 10:19:57 DROP UDP 51.255.115.140 239.255.255.250 51999 1900 201 - - - - - - - RECEIVE
2021-07-14 10:19:58 DROP UDP 51.255.115.140 239.255.255.250 51999 1900 201 - - - - - - - RECEIVE
2021-07-14 10:20:07 DROP UDP 54.38.229.21 239.255.255.250 55629 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:08 DROP UDP 54.38.229.21 239.255.255.250 55629 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:09 DROP UDP 54.38.229.21 239.255.255.250 55629 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:10 DROP UDP 54.38.229.21 239.255.255.250 55629 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:12 DROP UDP 51.255.115.139 239.255.255.250 60076 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:13 DROP UDP 51.255.115.139 239.255.255.250 60076 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:14 DROP UDP 51.255.115.139 239.255.255.250 60076 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:14 DROP UDP 152.228.149.239 239.255.255.250 52484 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:15 DROP UDP 51.255.115.139 239.255.255.250 60076 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:15 DROP UDP 152.228.149.239 239.255.255.250 52484 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:16 DROP UDP 152.228.149.239 239.255.255.250 52484 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:17 DROP UDP 152.228.149.239 239.255.255.250 52484 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:25 DROP UDP 152.228.149.237 152.228.149.255 138 138 229 - - - - - - - RECEIVE
2021-07-14 10:20:40 DROP UDP 152.228.149.244 239.255.255.250 60322 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:41 DROP UDP 152.228.149.244 239.255.255.250 60322 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:42 DROP UDP 152.228.149.244 239.255.255.250 60322 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:43 DROP UDP 152.228.149.244 239.255.255.250 60322 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:44 DROP UDP 152.228.149.252 239.255.255.250 61878 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:45 DROP UDP 152.228.149.252 239.255.255.250 61878 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:46 DROP UDP 51.255.115.141 239.255.255.250 64900 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:46 DROP UDP 152.228.149.242 239.255.255.250 55633 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:46 DROP UDP 152.228.149.252 239.255.255.250 61878 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:47 DROP UDP 51.255.115.141 239.255.255.250 64900 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:47 DROP UDP 152.228.149.242 239.255.255.250 55633 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:47 DROP UDP 152.228.149.252 239.255.255.250 61878 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:48 DROP UDP 51.255.115.141 239.255.255.250 64900 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:48 DROP UDP 152.228.149.242 239.255.255.250 55633 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:49 DROP UDP 51.255.115.141 239.255.255.250 64900 1900 202 - - - - - - - RECEIVE
2021-07-14 10:20:49 DROP UDP 152.228.149.242 239.255.255.250 55633 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:05 DROP UDP 152.228.149.250 239.255.255.250 53798 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:06 DROP UDP 152.228.149.250 239.255.255.250 53798 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:07 DROP UDP 152.228.149.250 239.255.255.250 53798 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:08 DROP UDP 152.228.149.250 239.255.255.250 53798 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:24 DROP UDP 54.38.229.19 239.255.255.250 60066 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:24 DROP TCP 134.209.122.227 152.228.149.230 52399 80 40 S 4236672370 0 65535 - - - RECEIVE
2021-07-14 10:21:25 DROP UDP 54.38.229.19 239.255.255.250 60066 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:26 DROP UDP 54.38.229.19 239.255.255.250 60066 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:27 DROP UDP 54.38.229.19 239.255.255.250 60066 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:27 DROP UDP 152.228.149.226 239.255.255.250 52031 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:27 DROP UDP 152.228.149.231 239.255.255.250 50707 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:28 DROP UDP 152.228.149.226 239.255.255.250 52031 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:28 DROP UDP 152.228.149.231 239.255.255.250 50707 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:29 DROP UDP 152.228.149.226 239.255.255.250 52031 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:29 DROP UDP 152.228.149.231 239.255.255.250 50707 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:30 DROP UDP 152.228.149.234 239.255.255.250 50665 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:30 DROP UDP 152.228.149.226 239.255.255.250 52031 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:30 DROP UDP 152.228.149.231 239.255.255.250 50707 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:31 DROP UDP 51.255.115.138 239.255.255.250 57943 1900 201 - - - - - - - RECEIVE
2021-07-14 10:21:31 DROP UDP 152.228.149.234 239.255.255.250 50665 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:32 DROP UDP 51.255.115.138 239.255.255.250 57943 1900 201 - - - - - - - RECEIVE
2021-07-14 10:21:32 DROP UDP 152.228.149.234 239.255.255.250 50665 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:33 DROP UDP 51.255.115.138 239.255.255.250 57943 1900 201 - - - - - - - RECEIVE
2021-07-14 10:21:33 DROP UDP 152.228.149.234 239.255.255.250 50665 1900 202 - - - - - - - RECEIVE
2021-07-14 10:21:34 DROP UDP 51.255.115.138 239.255.255.250 57943 1900 201 - - - - - - - RECEIVE
2021-07-14 10:21:55 DROP UDP 51.255.115.140 239.255.255.250 53107 1900 201 - - - - - - - RECEIVE
2021-07-14 10:21:56 DROP UDP 51.255.115.140 239.255.255.250 53107 1900 201 - - - - - - - RECEIVE
2021-07-14 10:21:57 DROP UDP 51.255.115.140 239.255.255.250 53107 1900 201 - - - - - - - RECEIVE
2021-07-14 10:21:58 DROP UDP 51.255.115.140 239.255.255.250 53107 1900 201 - - - - - - - RECEIVE
2021-07-14 10:22:07 DROP UDP 54.38.229.21 239.255.255.250 55630 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:08 DROP UDP 54.38.229.21 239.255.255.250 55630 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:09 DROP UDP 54.38.229.21 239.255.255.250 55630 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:10 DROP UDP 54.38.229.21 239.255.255.250 55630 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:12 DROP UDP 51.255.115.139 239.255.255.250 56678 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:13 DROP UDP 51.255.115.139 239.255.255.250 56678 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:14 DROP UDP 51.255.115.139 239.255.255.250 56678 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:14 DROP UDP 152.228.149.239 239.255.255.250 52485 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:15 DROP UDP 51.255.115.139 239.255.255.250 56678 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:15 DROP UDP 152.228.149.239 239.255.255.250 52485 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:16 DROP UDP 152.228.149.239 239.255.255.250 52485 1900 202 - - - - - - - RECEIVE
2021-07-14 10:22:17 DROP UDP 152.228.149.239 239.255.255.250 52485 1900 202 - - - - - - - RECEIVE

Find many UDP Dropped logs. All from OVH SAS , As this server is a vps in OVH.

But these udp ports are blocked. We can not find why ESET IDS is involved with CPU usage of 70% yet.

 

pfirewall.log

[Marcos 5,074]

Please enable advanced OS logging in the advanced setup under Tools -> Diagnostics when you notice the cpu hog. After 2-3 minutes disable logging. Make sure that there are no big dumps in C:\ProgramData\ESET\ESET Security\Diagnostics (we don't need them) and collect fresh logs with ELC.

As to whether Windows firewall evaluates rules prior to after ESET, we don't know. It should be that ESET scans the communication first.

[kamiran.asia 5]

47 minutes ago, Marcos said:

Please enable advanced OS logging in the advanced setup under Tools -> Diagnostics when you notice the cpu hog. After 2-3 minutes disable logging. Make sure that there are no big dumps in C:\ProgramData\ESET\ESET Security\Diagnostics (we don't need them) and collect fresh logs with ESET Log Collector.

As to whether Windows firewall evaluates rules prior to after ESET, we don't know. It should be that ESET scans the communication first.

Here is your requested log dear marcos : https://we.tl/t-MRdRdaMqvF

[itman 1,659]

2 hours ago, kamiran.asia said:

Windows Firewall Dropped Log is attached

What is being blocked is incoming SSDP; i.e. uPnP traffic. Only local subnet inbound traffic to port 1900 is allowed by both the Win and Eset firewall.

Appears to me your gateway/router has been hacked. It's firewall should have never been allowing this outbound traffic from it. It should only be forwarding local subnet traffic to port 1900.

Edited July 14, 2021 by itman

[kamiran.asia 5]

4 minutes ago, itman said:

What is being blocked is incoming SSDP; i.e. uPnP traffic. Only local subnet inbound traffic to port 1900 is allowed by both the Win and Eset firewall.

Appears to me your gateway/router has been hacked. It's firewall should have never been allowing this outbound traffic from it. It should only be forwarding local subnet traffic.

Hi dear @itman This Server is our customer's VPS in OVH DataCenter. and we have not any access to gateway/router.

We know that s.th is wrong here that ESET IDS is involved. We are working on it and waiting for @Marcos to check the ESET Log Collector.

[Nightowl 202]

Just now, kamiran.asia said:

Hi dear @itman This Server is our customer's VPS in OVH DataCenter. and we have not any access to gateway/router.

We know that s.th is wrong here that ESET IDS is involved. We are working on it and waiting for @Marcos to check the ESET Log Collector.

Maybe OVH can provide some kind of a firewall to eliminate those attacks , or by applying another firewall as virtual machine for example like Fortinet or Sophos etc...

[itman 1,659]

8 minutes ago, kamiran.asia said:

This Server is our customer's VPS in OVH DataCenter. and we have not any access to gateway/router.

Your currently playing "malware Russian roulette" having that server connected to you network. It should have be disconnected when this activity started.

[Nightowl 202]

36 minutes ago, itman said:

Your currently playing "malware Russian roulette" having that server connected to you network. It should have be disconnected when this activity started.

That's how VPS are offered , they normally won't give you a Firewall protection , or will give you a simple IPTABLES to configure , to have more control and a better firewall , you will need to have a seperate machine that would be a firewall whether it's open source one or something like sophos firewall or fortinet firewall

 

EDIT :  I mis-understood your post , you are right about disconnecting the server once there is an attack , but maybe that could make him lose connection to it? , but connectivity to work places yes should be cut.(isolate the server)

Edited July 14, 2021 by Nightowl

[itman 1,659]

I also don't see a problem here pertaining to Eset IDS protection.

I have always assumed it will filter incoming network traffic prior to passing it to the Eset firewall for monitoring.

The solution here as I see it is what @Nightowlposted. A gateway w/firewall needs to be installed to drop this rogue incoming traffic prior to entry to the local network.

Also Sophos offers a free UTM firewall server OS here: https://www.sophos.com/en-us/products/free-tools/sophos-utm-home-edition.aspx . Since it is designed for home users, it only supports 50 IP addresses.

Edited July 14, 2021 by itman

[itman 1,659]

Here's an article I believe is applicable to you: https://www.spamtitan.com/web-filtering/network-segmentation-best-practices/ .

The gist of the article is all external network server interfaces are set up in DMZ's. The DMZ's along with Internet traffic are all filtered through a boarder edge firewall appliance.

Of note:

Quote

Risks of an Unsegmented Network

A real world example of an unsegmented network and resulting attack is the massive Target data breach of 2013.  Reportedly, the Target breach had its origin in a phishing email opened by an employee at a small HVAC company that did business with Target. The malware lurked in the HVAC network for two months before moving on to attack the Target network.

Once inside they were able to move laterally through Target’s internal network, eventually installing malware on point-of-sale (POS) terminals throughout the stores. In the wake of the attack, Target implemented network segmentation to prevent the lateral movement that allows the attackers move with the system in this breach.

 

Edited July 14, 2021 by itman

[kamiran.asia 5]

Dear friends.

Thank you all for you useful information. 🤩

Our customer just rent a vps in OVH ( Exactly a Cloud server at a VPS ) , he have no access to virtualization firewall or ...  , Their support said " These udp attacks are general and normal at many servers !! " They advice him  to block such these traffics by Windows Firewall. ( As we do )

right now we are not sure that IDS high usage of cpu is related to these udp packets.

Right now we block all inbound UDP and TCP port by windows firewall and we must disable IDS and botnet Permanently ( Because they can not work with server due to cpu usage over 70%)

We are waiting for dear @Marcos that if he find any thing in advanced OS logging that can help : https://we.tl/t-MRdRdaMqvF

 

 

[Nightowl 202]

1 hour ago, kamiran.asia said:

Dear friends.

Thank you all for you useful information. 🤩

Our customer just rent a vps in OVH ( Exactly a Cloud server at a VPS ) , he have no access to virtualization firewall or ...  , Their support said " These udp attacks are general and normal at many servers !! " They advice him  to block such these traffics by Windows Firewall. ( As we do )

right now we are not sure that IDS high usage of cpu is related to these udp packets.

Right now we block all inbound UDP and TCP port by windows firewall and we must disable IDS and botnet Permanently ( Because they can not work with server due to cpu usage over 70%)

We are waiting for dear @Marcos that if he find any thing in advanced OS logging that can help : https://we.tl/t-MRdRdaMqvF

 

 

Mostly Cloud Servers/services , will start to get attacked as the first moment they are deployed online , doesn't matter which company you rent from them , seems that lot of hackers have scripts on their IPs ready all the time

I would still try to get something from OVH for a Firewall plus the Windows Firewall solution, it's not safe at all to stay under attack , even if it was ESET usage very low under these attacks , still the server is constantly being attacked and one day they might be able to exploit or penetrate the server , then it will go bad.

I read that you can enable this : https://docs.ovh.com/gb/en/dedicated/firewall-network/

Edited July 15, 2021 by Nightowl

[Marcos 5,074]

Not many exploitation attempts are occurring on the server, it's about 15 per hour which is not a lot.

The case has been reported to developers and is pending for analysis.

[Nightowl 202]

@kamiran.asia, Check in your Firewall rules if you have this enabled

File and Printer Sharing (SMB-In)- TCP 445

  Deny this rule or filter it by IP

And does this server have HTTP opened? And they also try to come by HTTPS.

About HTTP/HTTPS , if it's a server that serves websites , then you cannot filter them unfortunately if I am not mistaken.

If Windows Firewall is properly configured then ESET should stop showing you alerts about attacks , because the Firewall would render them useless(Ports blocked).

Edited July 15, 2021 by Nightowl

[kamiran.asia 5]

8 minutes ago, Nightowl said:

@kamiran.asia, Check in your Firewall rules if you have this enabled

File and Printer Sharing (SMB-In)- TCP 445

And does this server have HTTP opened? And they also try to come by HTTPS.

The exploitation attempts that dear @Marcos mentioned was occurred before  we configure Windows Firewall to block all inbound TCP and UDP. So we have no open port right now. Just a limited secure RDP on special IPs. ekrn still use high cpu in this situation.

 

18 minutes ago, Marcos said:

The case has been reported to developers and is pending for analysis.

Yes we think that s.th go wrong and IDS must not involved like this in such attacks. We temporary Disable IDS so Server work probably and waiting for analysis report and any updates. while there is no open inbound port , there is no worries to temporary disable IDs.

[Marcos 5,074]

Just to make sure, did the performance issue occur even before you created Windows firewall rules to block the malicious communication?

[kamiran.asia 5]

3 minutes ago, Marcos said:

Just to make sure, did the performance issue occur even before you created Windows firewall rules to block the malicious communication?

Nothing changed , We still saw these attacks while no ports was open and still performance issue occur.

seems that ESET Firewall driver work before windows firewall and still analyze inbound packets !