Jump to content

The ESET LiveGrid servers cannot be reached


steingat
 Share

Recommended Posts

We get this error on a number of our endpoints who are working from home.  There are all personal internet connections the we do not control.

Is there a way to route this connection though the ESMC server?  This never use to be a problem in older versions of eset until they changed the way that this attempt to reach the server.

Link to comment
Share on other sites

  • Administrators

For a start I'd enable advanced direct cloud logging as well as change the logging severity to Diagnostic. After reproducing the issue (e.g. after several days), disable logging and provide logs collected with ESET Log Collector. Also provide times when the machine was connected to a different network.

Link to comment
Share on other sites

Its not just one machine, its several, is there a policy option that can be enabled to re-route this connection through the ESMC Server? 

Its possible that there are a few machines that change wifi networks or connect/disconnect from the VPN as well.  

If a policy based option is not available, then I would be able to gather the requested logs from some of the worst offenders, however, like i said, we do not control the network connections of most of these endpoints so attempting to reach out to the ISP or modifying router settings is not a realistic option in most cases.

Link to comment
Share on other sites

  • Administrators

Endpoint connects either directly or through an http proxy server, if configured. If there are changes in network, we detect them and use the current DNS server for address resolution.

If a machine sometimes connects through a proxy and sometimes directly, you may need to enable this setting in the proxy setup:

image.png

Link to comment
Share on other sites

  • 2 weeks later...

I think there is something more going on here with the LiveGrid issue that many seem to report.  We had a large number of clients (1000+) running EEA 7.4.2041 for about 1 year without any issue.  I upgraded everything to 8.1.2037.2 and most seem to be reporting that it is unable to connect to LiveGrid.  I checked each of the domains listed on the KB to ensure they are reachable and that correct DNS resolutions are happening and everything was good.  Going off of the knowledge shared on the forum of the communication change with 8.1, I went ahead and upgraded one of the clients with the issue to v9.  This fixed the issue.   Assuming v9 uses the same communication mechanism as v8.1, I lean toward there being something wrong with v8.1 rather than a local network issue.  I have clients running v8.1 with the issue and without the issue that are running the exact same image, behind the same firewalls, configured exactly the same.  

Link to comment
Share on other sites

Ya, What I found is the following:

V9 Seems to help with this issue

Some of the endpoints I was able to fix by changing their Wifi DNS to Google public DNS

Some of the endpoints connect/disconnect from a VPN on a regular basis, These endpoint are still problematic with this error and suspect that the network switching to be the issue.

 

As for logs, I will be enabling logging on a few of the endpoints to narrow this down.

 

Link to comment
Share on other sites

14 minutes ago, kingoftheworld said:

I have clients running v8.1 with the issue and without the issue that are running the exact same image, behind the same firewalls, configured exactly the same.  

My own opinion is the only way to get to the bottom of this is by using tracert.

The ideal test scenerio is on an installation using v8.1 where some connect OK and some do not. "My gut is telling me" Internet backbone relay server/s are the culprit.

Edited by itman
Link to comment
Share on other sites

2 minutes ago, itman said:

My own opinion is the only way to get to the bottom of this is by using tracert.

The ideal test scenerio is on an installation using v8.1 where some connect OK and some do not. "My gut is telling me" Internet backbone relay server/s are the culprit.

That is a good possibility.  Given that most (or all) of the IPs go to Europe which is a long way for at least me in the US. 

Link to comment
Share on other sites

4 minutes ago, Marcos said:

Couldn't it be that the troublesome machines are connected both via wi-fi and wire at the same time?

I have ruled this out as a possibility and verified that the computers are only on one internet connection (wifi), enabled google public DNS on the wifi connection, and disabled IPv6 on the Wifi network controller.  At this point, the most problematic machines are ones that connect/disconnect from our VPN while on wifi.  

 

The vast majority of our systems that connect to this VPN do not have any issues, its just a small handful of PCs

Link to comment
Share on other sites

20 hours ago, itman said:

My own opinion is the only way to get to the bottom of this is by using tracert.

The ideal test scenerio is on an installation using v8.1 where some connect OK and some do not. "My gut is telling me" Internet backbone relay server/s are the culprit.

Further testing says your theory may be correct.  On some machines (working machines) the routes to the LiveGrid servers leave the US, go through Denmark and onto ESET.   On the ones that aren't working, the route appears to leave the US, head through Russia and some other locations but don't appear to make it to ESET.  Curious if anyone else is seeing similar. 

Link to comment
Share on other sites

2 hours ago, kingoftheworld said:

Further testing says your theory may be correct. 

Yesterday I did a tracert on a domain Eset uses for its mPico updates, etc., h1-epnsbroker03.eset.com.

My ISP is AT&T and I use their DNS servers. Routing in the U.S. stayed pretty much through AT&T servers. Then to servers in Germany, Slovakia, and finally an Eset server.

BTW - I never once have had an issue with LiveGrid connectivity.

Link to comment
Share on other sites

34 minutes ago, itman said:

Yesterday I did a tracert on a domain Eset uses for its mPico updates, etc., h1-epnsbroker03.eset.com.

My ISP is AT&T and I use their DNS servers. Routing in the U.S. stayed pretty much through AT&T servers. Then to servers in Germany, Slovakia, and finally an Eset server.

BTW - I never once have had an issue with LiveGrid connectivity.

Maybe ESET will bring up some servers in the US to handle some of this communication given their large number of customers in the States.

Edited by kingoftheworld
Link to comment
Share on other sites

Below is actual tracert output to an Eset LiveGrid server. I would say any routing through Russian servers would be suspect:

Quote

Tracing route to h1-c06-b.eset.com [91.228.166.52]
over a maximum of 30 hops:

  1     4 ms     4 ms     4 ms  homeportal [192.168.1.XXX]
  2     7 ms     6 ms     7 ms  XXX-XXX-XXX-X.lightspeed.bcvloh.sbcglobal.net [XXX.XXX.XXX.X] =====> AT&T
  3     7 ms     7 ms     7 ms  71.151.85.132  ==============> AT&T
  4     *        *        *     Request timed out.
  5    23 ms    22 ms    22 ms  32.130.17.83   ==============> AT&T
  6    25 ms    26 ms    26 ms  cl2oh22crs.ip.att.net [12.122.2.126]
  7    27 ms    23 ms    20 ms  phlpa22crs.ip.att.net [12.122.2.230]
  8    28 ms    26 ms    26 ms  phlpa21crs.ip.att.net [12.122.3.225]
  9    23 ms    20 ms    19 ms  n54ny21crs.ip.att.net [12.122.2.149]
 10    22 ms    19 ms    21 ms  n54ny402igs.ip.att.net [12.122.131.101]
 11    20 ms    20 ms    20 ms  192.205.34.182 =========> AT&T
 12   140 ms   124 ms   122 ms  217.239.41.158 =========> The IP number is in Germany. It is hosted by Deutsche Telekom AG
 13   129 ms   128 ms   128 ms  80.150.170.82  =========> The IP number is in Germany. It is hosted by Deutsche Telekom AG
 14   124 ms   124 ms   124 ms  st-static-bckb-22.213-81-252.telecom.sk [213.81.252.22] ====> The IP number is in Slovak Republic
 15     *        *        *     Request timed out.
 16     *        *        *     Request timed out.
 17   124 ms   124 ms   124 ms  h1-c06-b.eset.com [91.228.166.52]

Trace complete.                            

 

Edited by itman
Link to comment
Share on other sites

17 hours ago, itman said:

Below is actual tracert output to an Eset LiveGrid server. I would say any routing through Russian servers would be suspect:

Thanks for posting.  I am not seeing the same hops through Russia on my test machine that I previously saw.  LiveGrid still isn't happy, but I will deal with it on Monday.

Link to comment
Share on other sites

On 1/1/2022 at 4:13 PM, itman said:

Below is actual tracert output to an Eset LiveGrid server. I would say any routing through Russian servers would be suspect:

I get basically the same thing:

image.png.523da6a14e32b5b9df0d4b711919d2b1.png

Link to comment
Share on other sites

  • Administrators

The correct way is to tracert c.eset.com. Do not use a particular server which is located in EU.

Link to comment
Share on other sites

33 minutes ago, Marcos said:

The correct way is to tracert c.eset.com. Do not use a particular server which is located in EU.

Thank you.  Just ran on one of my impacted machines and it successfully makes it to h1-c04-s.eset.com [91.228.165.44].  I am leaning back toward the problem being with this version as another machine with the issue running 8.1 was upgraded to v9 and the problem goes away.

Link to comment
Share on other sites

43 minutes ago, Marcos said:

The correct way is to tracert c.eset.com

Parallels my previous posted tracert output with the exception of connection to Slovakia Telecom routing to Eset server:

Quote

C:\Users\xxxxxx>tracert c.eset.com

Tracing route to c.cwip.eset.com [91.228.166.45]
over a maximum of 30 hops:

  1     4 ms     4 ms     4 ms  homeportal [192.168.1.XXX]
  2     7 ms     7 ms     7 ms  XXX-XXX-XXX-1.lightspeed.bcvloh.sbcglobal.net [XXX.XXX.XXX.1]
  3     7 ms     7 ms     8 ms  71.151.85.132
  4     *        *        *     Request timed out.
  5    29 ms    27 ms    26 ms  32.130.17.83
  6    26 ms    25 ms    27 ms  cl2oh22crs.ip.att.net [12.122.2.126]
  7    26 ms    26 ms    27 ms  phlpa22crs.ip.att.net [12.122.2.230]
  8    20 ms    21 ms    21 ms  phlpa21crs.ip.att.net [12.122.3.225]
  9    22 ms    27 ms    24 ms  n54ny21crs.ip.att.net [12.122.2.149]
 10    22 ms    21 ms    20 ms  n54ny402igs.ip.att.net [12.122.131.101]
 11    21 ms    20 ms    20 ms  192.205.34.182
 12   116 ms   116 ms   116 ms  pd9ef315a.dip0.t-ipconnect.de [217.239.49.90] ==> The IP number is in Germany. It is hosted by Deutsche Telekom AG
 13   121 ms   121 ms   123 ms  80.150.170.82 ==> The IP number is in Germany. It is hosted by Deutsche Telekom AG
 14     *        *        *     Request timed out.
 15     *        *        *     Request timed out.
 16     *        *        *     Request timed out.
 17   117 ms   116 ms   116 ms  h1-c01-b.eset.com [91.228.166.45]

Trace complete.

Edited by itman
Link to comment
Share on other sites

Mine is similar as well.

Tracing route to c.cwip.eset.com [91.228.166.45]
over a maximum of 30 hops:

  1     3 ms     1 ms    <1 ms  ip63-247-101-97.Houston.iapc.net [63.247.101.97]
  2    13 ms    12 ms     9 ms  ip63-247-104-101.Houston.iapc.net [63.247.104.101]
  3    23 ms    11 ms    19 ms  core1-v2-0.Houston.iapc.net [63.247.97.23]
  4     9 ms    12 ms    11 ms  63.247.97.14
  5    11 ms    18 ms     9 ms  6-1-5.ear1.Houston1.Level3.net [4.30.110.49]
  6    10 ms    12 ms    15 ms  ae-5.bear2.Houston1.Level3.net [4.69.216.142]
  7     *        *        *     Request timed out.
  8    35 ms    35 ms    35 ms  80.150.170.13
  9   584 ms   144 ms   138 ms  pd9ef315a.dip0.t-ipconnect.de [217.239.49.90]
 10   256 ms   136 ms   136 ms  80.150.170.82
 11   136 ms   135 ms   139 ms  st-static-bckb-112.213-81-254.telecom.sk [213.81.254.112]
 12     *        *        *     Request timed out.
 13     *        *        *     Request timed out.
 14   137 ms   136 ms   136 ms  h1-c01-b.eset.com [91.228.166.45]

Trace complete.

Link to comment
Share on other sites

41 minutes ago, LesRMed said:

Mine is similar as well.

The interesting part of the tracert output is we are both traversing through the same Deutsche Telekom AG server from the U.S..

Given is we are using different ISPs. This leads me to speculate that for those in the U.S. having LiveGrid connectivity issue, the source of the issue lies with their Internet provider or the DNS servers they are using.

Link to comment
Share on other sites

Another point to note is the AT&T server used for forwarding traffic to Europe is one shared by Deutsche Telekom AG:

Eset_Duetsch.thumb.png.17e7d1530b24788652b3cef541911477.png

Link to comment
Share on other sites

Here are my traceroutes along with some logs.  It seems that the connection to 91.228.165.44 is much worst.

Endpoint 1

C:\temp>tracert c.eset.com

Tracing route to c.cwip.eset.com [38.90.226.12]
over a maximum of 30 hops:

  1     *        *        *     Request timed out.
  2     *        *        *     Request timed out.
  3    26 ms    20 ms    29 ms  nwblwihed11-lag11-433.network.tds.net [184.61.161.97] 
  4    30 ms    29 ms    28 ms  nwblwidst52-tg0-0-2-0.network.tds.net [64.50.229.61] 
  5    36 ms    36 ms    30 ms  chi-b23-link.ip.twelve99.net [80.239.135.66] 
  6     *        *        *     Request timed out.
  7    32 ms    28 ms    24 ms  be2766.ccr42.ord01.atlas.cogentco.com [154.54.46.177] 
  8    51 ms    45 ms    46 ms  be2832.ccr22.mci01.atlas.cogentco.com [154.54.44.169] 
  9    54 ms    47 ms    46 ms  be3036.ccr22.den01.atlas.cogentco.com [154.54.31.89] 
 10    76 ms    73 ms    68 ms  be3047.ccr21.elp01.atlas.cogentco.com [154.54.1.125] 
 11    67 ms    69 ms    68 ms  be2930.ccr32.phx01.atlas.cogentco.com [154.54.42.77] 
 12    85 ms    80 ms    82 ms  be2941.rcr52.san01.atlas.cogentco.com [154.54.41.33] 
 13    82 ms    82 ms    91 ms  te0-0-0-35.nr61.b036483-1.san01.atlas.cogentco.com [154.24.24.186] 
 14    85 ms    80 ms    84 ms  38.88.58.18 
 15    84 ms    83 ms    96 ms  38-90-226-12.ptr.eset.com [38.90.226.12] 

Trace complete.

C:\temp>tracert c.eset.com

Tracing route to c.cwip.eset.com [91.228.165.44]
over a maximum of 30 hops:

  1     *        *        *     Request timed out.
  2     *        *        *     Request timed out.
  3    21 ms    21 ms    21 ms  nwblwihed11-lag11-433.network.tds.net [184.61.161.97] 
  4    31 ms    30 ms    31 ms  nwblwidst52-tg0-0-2-0.network.tds.net [64.50.229.61] 
  5    29 ms    26 ms    30 ms  chi-b23-link.ip.twelve99.net [80.239.135.66] 
  6    42 ms    48 ms    44 ms  nyk-bb2-link.ip.twelve99.net [62.115.137.58] 
  7   120 ms   118 ms   120 ms  ldn-bb1-link.ip.twelve99.net [62.115.113.21] 
  8     *      140 ms     *     prs-bb1-link.ip.twelve99.net [62.115.135.25] 
  9   142 ms   140 ms   138 ms  ffm-bb1-link.ip.twelve99.net [62.115.123.12] 
 10   139 ms   146 ms   143 ms  win-bb3-link.ip.twelve99.net [62.115.137.203] 
 11   145 ms   142 ms   141 ms  win-b2-link.ip.twelve99.net [62.115.114.185] 
 12   165 ms   145 ms   143 ms  87.128.239.252 
 13   141 ms   222 ms   141 ms  80.150.170.82 
 14   150 ms   146 ms   177 ms  st-static-bckb-22.213-81-252.telecom.sk [213.81.252.22] 
 15     *        *        *     Request timed out.
 16     *        *        *     Request timed out.
 17   146 ms   148 ms   146 ms  h1-c04-s.eset.com [91.228.165.44] 

Trace complete.
 

Endpoint 2:

C:\temp>tracert c.eset.com

Tracing route to c.cwip.eset.com [38.90.226.12]
over a maximum of 30 hops:

  1     *        *        *     Request timed out.
  2     *        *        *     Request timed out.
  3    41 ms    42 ms    55 ms  nwblwihed11-lag11-433.network.tds.net [184.61.161.97] 
  4    40 ms    41 ms    33 ms  nwblwidst52-tg0-0-2-0.network.tds.net [64.50.229.61] 
  5    35 ms    36 ms    34 ms  chi-b23-link.ip.twelve99.net [80.239.135.66] 
  6     *        *        *     Request timed out.
  7    34 ms    34 ms    34 ms  be2766.ccr42.ord01.atlas.cogentco.com [154.54.46.177] 
  8    56 ms    56 ms    60 ms  be2832.ccr22.mci01.atlas.cogentco.com [154.54.44.169] 
  9    57 ms    58 ms    63 ms  be3036.ccr22.den01.atlas.cogentco.com [154.54.31.89] 
 10    84 ms    85 ms    82 ms  be3047.ccr21.elp01.atlas.cogentco.com [154.54.1.125] 
 11    85 ms    79 ms    82 ms  be2930.ccr32.phx01.atlas.cogentco.com [154.54.42.77] 
 12    92 ms    91 ms    91 ms  be2941.rcr52.san01.atlas.cogentco.com [154.54.41.33] 
 13    91 ms    87 ms    93 ms  te0-0-0-35.nr61.b036483-1.san01.atlas.cogentco.com [154.24.24.186] 
 14   110 ms   105 ms   111 ms  38.88.58.18 
 15   105 ms    88 ms    92 ms  38-90-226-12.ptr.eset.com [38.90.226.12] 

Trace complete.

C:\temp>tracert c.eset.com

Tracing route to c.cwip.eset.com [91.228.165.44]
over a maximum of 30 hops:

  1     *        *        *     Request timed out.
  2     *        *        *     Request timed out.
  3    40 ms    34 ms    33 ms  nwblwihed11-lag11-433.network.tds.net [184.61.161.97] 
  4    43 ms    41 ms    35 ms  nwblwidst52-tg0-0-2-0.network.tds.net [64.50.229.61] 
  5    43 ms    44 ms    59 ms  chi-b23-link.ip.twelve99.net [80.239.135.66] 
  6    52 ms    60 ms    59 ms  nyk-bb2-link.ip.twelve99.net [62.115.137.58] 
  7   126 ms   132 ms   124 ms  ldn-bb1-link.ip.twelve99.net [62.115.113.21] 
  8     *      150 ms   152 ms  prs-bb1-link.ip.twelve99.net [62.115.135.25] 
  9   149 ms   146 ms   147 ms  ffm-bb1-link.ip.twelve99.net [62.115.123.12] 
 10   149 ms   149 ms   151 ms  win-bb3-link.ip.twelve99.net [62.115.137.203] 
 11   151 ms   148 ms   150 ms  win-b2-link.ip.twelve99.net [62.115.114.185] 
 12   152 ms   146 ms   159 ms  87.128.239.252 
 13   150 ms   218 ms   153 ms  80.150.170.82 
 14   152 ms   153 ms   156 ms  st-static-bckb-22.213-81-252.telecom.sk [213.81.252.22] 
 15     *        *        *     Request timed out.
 16     *        *        *     Request timed out.
 17   156 ms   154 ms   152 ms  h1-c04-s.eset.com [91.228.165.44] 

Trace complete.

C:\temp>

2yn0vn2 Joshcollected_eset_logs.zip 399qnq2 collected_eset_logs.zip

Link to comment
Share on other sites

  • Administrators

It doesn't matter if you are routed to US or EU LG servers since the traffic is very low, typically a few bytes or kB. If that mattered, US users would be constantly getting LG communication errors which is not the case. Most of LG servers are in EU which explains why you are routed there quite often. We'll need advanced logs from time when a LG communication error actually occurs.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...