Endpoint Security can't connect to Push Notification Service

[etosolini 1]

Hello, I'm having this same issue after updating some clients to version 9.0.

The diagnostics log "iris.epns.0.log" shows this:

Quote

02.12.2021 12:42:39.421 [1576:14712] INFO Logging turned on
02.12.2021 12:45:57.664 [1576:6032] DEBUG [EPNS] <worker> Connection state changed: UNAVAILABLE => CONNECTION_CLEANUP
02.12.2021 12:45:57.664 [1576:6032] DEBUG [EPNS] <worker> Running connection cleanup; last error: 0
02.12.2021 12:45:57.664 [1576:6032] DEBUG [EPNS] <worker> Connection state changed: CONNECTION_CLEANUP => UNAVAILABLE
02.12.2021 12:51:04.397 [1576:6032] DEBUG [EPNS] <worker> Connection state changed: UNAVAILABLE => CONNECTION_CLEANUP
02.12.2021 12:51:04.397 [1576:6032] DEBUG [EPNS] <worker> Running connection cleanup; last error: 0

After reading this and other posts I doubled checked the proxy config on the Agent and the Endpoint and it not set. Attached are some screenshots showing the config for reference.

There isn't any proxy or firewall at the network level, all outgoing connections are allowed for this testing.

If you need any more information I'm open to provide it.

 

Thanks in advance.

[Marcos 5,135]

If you don't use a proxy server, do you have the following addresses and ports allowed on a firewall?

ESET Push Notification Service

Hostname
epns.eset.com
h1-epnsbroker01.eset.com
h1-epnsbroker02.eset.com
h1-epnsbroker03.eset.com
h1-epnsbroker04.eset.com
h1-epnsbroker05.eset.com
h1-epnsbroker06.eset.com
h1-epnsbroker07.eset.com
h3-epnsbroker01.eset.com
h3-epnsbroker02.eset.com
h3-epnsbroker03.eset.com
h3-epnsbroker04.eset.com
h3-epnsbroker05.eset.com
h3-epnsbroker06.eset.com
h3-epnsbroker07.eset.com
h5-epnsbroker01.eset.com
h5-epnsbroker02.eset.com
h5-epnsbroker03.eset.com
h5-epnsbroker04.eset.com
h5-epnsbroker05.eset.com
h5-epnsbroker06.eset.com
h5-epnsbroker07.eset.com

IP address
91.228.165.144
91.228.165.145
91.228.165.146
91.228.165.147
91.228.165.148
91.228.165.159
91.228.165.160
91.228.167.171
91.228.167.172
91.228.167.187
91.228.167.188
91.228.167.192
91.228.167.193
91.228.167.194
38.90.226.51
38.90.226.52
38.90.226.62
38.90.226.63
38.90.226.64
38.90.226.65
38.90.226.66

Port
8883, 443

[etosolini 1]

Hi @Marcos, there is no restrictions on outgoing traffic.

DNS query for epns.eset.com resolves to: 38.90.226.65
Telnet to epns.eset.com connects from the server and the clients as shown on the attachments.

[ivc52 0]

The same problem, and this warning cannot be disabled

[Marcos 5,135]

@kapela86, developers suggest to the following:

1, Provide the ouput from these commands:

• systemctl status httpd
• ps aux | grep httpd
• all proxy configuration (located in /etc/httpd/) - you've already provided that

2, Enable debug logs in /etc/httpd/conf/httpd.conf:

• set config option: LogLevel debug
• restart httpd service by running: systemctl restart httpd
• start tcpdump of traffic on proxy server by running: tcpdump -s 2000 -w wireshark_out.pcapng
• reproduce the issue - restart the client computer
• gather logs from /var/log/httpd/

Other questions:

The log showed connection to 8883 port being enabled so going through proxy is not required.
Could you turn off the global use of proxy in Endpoint and use it only for explicit services, such as update?

[etosolini 1]

@Marcos is there something to test or try?

Upgraded the same clients to 9.0.2032.2 and the problem persists.

As mentioned there isn't firewall, proxy or any other block at dns level.

[Marcos 5,135]

Those who use Apache HTTP proxy on Linux, could you try installing it on a Windows machine and configure Endpoint to connect through it? One of the users stated that it worked and the problem was only with Linux so we'd like to confirm or deny this as a pattern.

[Kamilos 3]

12 hours ago, Marcos said:

Those who use Apache HTTP proxy on Linux, could you try installing it on a Windows machine and configure Endpoint to connect through it? One of the users stated that it worked and the problem was only with Linux so we'd like to confirm or deny this as a pattern.

mayby in linux (CentOs) we should install Mosquitto MQTT Messaging Broker

sudo yum -y install epel-release
sudo yum -y install mosquitto
sudo systemctl start mosquitto
sudo systemctl enable mosquitto

and allow connections to port 8883

sudo firewall-cmd --permanent --add-port=8883/tcp
sudo firewall-cmd --reload 

[kapela86 10]

On 12/4/2021 at 11:19 AM, Marcos said:

@kapela86, developers suggest to the following:

 

1, Provide the ouput from these commands:

• systemctl status httpd
• ps aux | grep httpd
• all proxy configuration (located in /etc/httpd/) - you've already provided that

2, Enable debug logs in /etc/httpd/conf/httpd.conf:

• set config option: LogLevel debug
• restart httpd service by running: systemctl restart httpd
• start tcpdump of traffic on proxy server by running: tcpdump -s 2000 -w wireshark_out.pcapng
• reproduce the issue - restart the client computer
• gather logs from /var/log/httpd/

Other questions:

The log showed connection to 8883 port being enabled so going through proxy is not required.
Could you turn off the global use of proxy in Endpoint and use it only for explicit services, such as update?

I did 1 and 2, but that "Other" I don't want to, get your devs to create test environment in lab and see for themselves.

eset.zip

[MaFa 0]

I have the same issue here.

10.206.1.161 - - [06/Dec/2021:14:04:56 +0100] "CONNECT epns.eset.com:443 HTTP/1.1" 200 - "-" "-"
10.204.10.156 - - [06/Dec/2021:14:05:00 +0100] "CONNECT epns.eset.com:443 HTTP/1.1" 200 - "-" "-"
10.204.8.138 - - [06/Dec/2021:14:05:00 +0100] "CONNECT epns.eset.com:443 HTTP/1.1" 200 - "-" "-"
10.204.8.21 - - [06/Dec/2021:14:05:01 +0100] "CONNECT epns.eset.com:443 HTTP/1.1" 200 - "-" "-"
10.204.70.153 - - [06/Dec/2021:14:05:02 +0100] "CONNECT epns.eset.com:443 HTTP/1.1" 200 - "-" "-"
10.204.81.158 - - [06/Dec/2021:14:05:37 +0100] "CONNECT epns.eset.com:8883 HTTP/1.1" 403 223 "-" "-"

 

[BradAtkins 4]

Sorry if this is redundant.  But to confirm that the problem happens with us.

• We have Endpoint Security installed on Windows 10. 
• Using ESET Protect Cloud.
• We have no proxy and all outbound connections are allowed on all ports.

Interestingly - ESET Protect Cloud shows "Everything is OK".

 

 

[Gregecslo 8]

+1 here

Same error, firewall is opened, verified with pcap logs and graylog.

We do use linux proxy, Centos appliance.

[Gregecslo 8]

/etc/httpd/conf.d/proxy.conf

Change:

AllowCONNECT 443 2222

TO

AllowCONNECT 443 2222 8883

Restart httpd service

?

[Gregecslo 8]

EDIT:
Seems to return 503 in log.

Others mentioned that even without proxy they get same message.

Is this expected response from server?

curl -v https://epns.eset.com:8883

* About to connect() to epns.eset.com port 8883 (#0)
*   Trying 38.90.226.64...
* Connected to epns.eset.com (38.90.226.64) port 8883 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
*       subject: CN=epns.eset.com
*       start date: Apr 30 00:00:00 2020 GMT
*       expire date: May 23 12:00:00 2022 GMT
*       common name: epns.eset.com
*       issuer: CN=Thawte RSA CA 2018,OU=www.digicert.com,O=DigiCert Inc,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: epns.eset.com:8883
> Accept: */*
>
* Empty reply from server
* Connection #0 to host epns.eset.com left intact
curl: (52) Empty reply from server

Maybe there are server issues?

[Marcos 5,135]

The problem seems to be with an older version of Apache proxy which closes connections after 30s. The issue is still being investigated.

[JonnyDepp 0]

The Same Problem here after Update from Eset Endpoint Antivirus Version 8.1 to 9.

 

[kapela86 10]

https://support.eset.com/en/news8179-eset-protect-hotfix-version-90102-has-been-released

And how do we do it on VA?

yum update

doesn't find anything. And installed version is 2.4.6.

Or is this unrelated to this bug?

Edited December 7, 2021 by kapela86

[JonnyDepp 0]

We use the Linux VA appliance, and the error also occurs with us.

The error seems to be OS/Platform independent.

Edited December 8, 2021 by JonnyDepp

[Gregecslo 8]

Well easy solution for Windows

Virtual appliance will be tricky to fix...

[etosolini 1]

@Kamilos did this worked for you?

On 12/6/2021 at 5:22 AM, Kamilos said:

mayby in linux (CentOs) we should install Mosquitto MQTT Messaging Broker

sudo yum -y install epel-release
sudo yum -y install mosquitto
sudo systemctl start mosquitto
sudo systemctl enable mosquitto

and allow connections to port 8883

sudo firewall-cmd --permanent --add-port=8883/tcp
sudo firewall-cmd --reload 

In the case that it worked, did you changed something else?

Thanks.

[Kamilos 3]

2 hours ago, etosolini said:

@Kamilos did this worked for you?

In the case that it worked, did you changed something else?

Thanks.

 

I did not try, I sent it only as a proposal for those who have a test environment.

Edited December 8, 2021 by Kamilos

[alur 1]

we have the same error, before that there was version 8.0, there were no problems, we updated to 9.0 and problems.

 

[Vitaly2021 1]

On 12/1/2021 at 4:05 PM, Marcos said:

Please upload also httpd.conf from the Apache http proxy.

I have the same issue after upgrading from Endpoint Antivirus 8.1.2037.2 to 9.0.2032.2
I have sent collected data to ESET but still there is no answer from them.

It's seems new version 9 brings new bugs...
Thanks for this post, Yesterday I thought the problem is mine only.

[swb371 0]

Not sure if this will help anyone else, but this worked for us with the Linux VA

We logged into Webmin on the appliance, and clicked on update modules. Then rebooted the appliance. After this, the error went away on workstations after they restarted. 

[Marcos 5,135]

We have found an issue in the configuration of Apache http proxy for Linux. Windows version is not affected. We'll provide more information and fix instructions soon.