esets_daemon freeze

[CircleSquare 0]

16 minutes ago, Peter Randziak said:

Hello guys,

we are working on the issue with a high priority.

So far it seems that the freeze occurs in Database module (last release was in August) utilized by the Anti-spam engine and it seems, that the particular issue was there before as well. The mystery remains why it started to manifest just recently. The developer reproduced the freeze and works on fixing it.

 

We are sorry for the inconvenience caused.

Kind Regards, P.R.

tanks for feedback, so maybe the problem is delivered with an signature update from anti-spam module?

in our case it happens with 4.5.7.0 without any changes off software.

 

Greetings

[Peter Randziak 944]

Currently we do not know why it started to manifest just recently.

The database module updates are being distributed along with Detection engine, Rapid response, Advanced antispam and many other modules.

[Peter Randziak 944]

Hello guys,

We have an update from the development team.

They were able to fix two issues in the Database module ( fixed version  is 1095), but they are not 100% sure, if it will solve all the issues reported, so please take this as a first attempt to fix it.
If you are willing to try it, stop the esets daemon rename the em022_32.dat (to fox example em022_32.backup) and place there the em022_32.dat version 1095, make sure to keep permissions of the original em022_32.dat to prevent further issues and that start the esets daemon back.
The testing module is available at  http://ftp.nod.sk/~randziak/Database_module_1095.7z - packed with 7zip, no encryption.

 

In case of any feedback please let us know.
 

Thank you, P.R.

[miro 0]

Hello Peter, thanks for the update.

So now i will be beta tester for a paid software in production environment. Does not look professional, sorry it is not acceptable.

Would you mind refunding our purchase and giving free license as compensation?

[Peter Randziak 944]

Hello @miro,

we do not offer this as an official solution, just as a quick fix for those who are willing to try it and can try it.

The fix will be distributed via standard channel once confirmed and properly tested.

When it comes to the compensation you can contact your reseller as this his in his competences.

Regards, P.R.

[CircleSquare 0]

3 hours ago, Peter Randziak said:

Hello guys,

We have an update from the development team.

They were able to fix two issues in the Database module ( fixed version  is 1095), but they are not 100% sure, if it will solve all the issues reported, so please take this as a first attempt to fix it.
If you are willing to try it, stop the esets daemon rename the em022_32.dat (to fox example em022_32.backup) and place there the em022_32.dat version 1095, make sure to keep permissions of the original em022_32.dat to prevent further issues and that start the esets daemon back.
The testing module is available at  hxxp://ftp.nod.sk/~randziak/Database_module_1095.7z - packed with 7zip, no encryption.

 

In case of any feedback please let us know.
 

Thank you, P.R.

Hi Randziak,

 

tanks for the update, you mean to replace "em022_32.dat" files under "/var/opt/eset/esets/lib/" ?

we plan to run a test next Friday when workload goes down a bit.

 

Greetings

[Peter Randziak 944]

Hello @CircleSquare,

yes exactly, you need to replace the em022_32.dat located at /var/opt/eset/esets/lib/ with the one provided (version 1095).

We have few deployments already using it, but so far no feedback, which might be promising.

Regards, P.R.

[Michal Noga 0]

4 hours ago, Peter Randziak said:

Hello guys,

We have an update from the development team.

They were able to fix two issues in the Database module ( fixed version  is 1095), but they are not 100% sure, if it will solve all the issues reported, so please take this as a first attempt to fix it.
If you are willing to try it, stop the esets daemon rename the em022_32.dat (to fox example em022_32.backup) and place there the em022_32.dat version 1095, make sure to keep permissions of the original em022_32.dat to prevent further issues and that start the esets daemon back.
The testing module is available at  hxxp://ftp.nod.sk/~randziak/Database_module_1095.7z - packed with 7zip, no encryption.

 

In case of any feedback please let us know.
 

Thank you, P.R.

Hi Peter, we have running ESET version 4.5.9 with replaced em022_32.dat since this morning. We will see. 

BR

Michal

Edited December 6, 2017 by Michal Noga
typo's

[FHolzer 0]

Ty Peter!
installed it and will test it.

appreciate your work - and dont worry, the best (and every) software has bugs

hope we figured the problem - my customers already go crazy ^^

for the record:

[root@mail sbin]# ./esets_update --verbose
Update is not necessary - the installed virus signature database is current.
ESETS Update utility
+-+--------------------+------------------------+------------------------+
| | Module             | Available version      | Installed version      |
+-+--------------------+------------------------+------------------------+
| | loader             |        1069 (20161122) |        1069 (20161122) |
| | perseus            |        1532 (20171031) |        1532 (20171031) |
| | engine             |       16530 (20171206) |       16530 (20171206) |
| | archiver           |        1270 (20171113) |        1270 (20171113) |
| | heuristic          |        1184 (20171110) |        1184 (20171110) |
| | cleaner            |        1150 (20171031) |        1150 (20171031) |
| | horus              |        6556 (20171206) |        6556 (20171206) |
| | dblite             |        1095 (20171206) |        1095 (20171206) |
+-+--------------------+------------------------+------------------------+

 

Edited December 6, 2017 by FHolzer

[Peter Randziak 944]

Hello guys,

thank you for your support and willingness to help.

So far still no negative feedback, we test the module internally, should get to pre-release update channel tomorrow and to release during the next week, if no issues will be reported.

Regards, P.R.

[CircleSquare 0]

we now run an test with the new dblite version over prerelease update channel.

Quote

:/opt/eset/esets/sbin# ./esets_update --verbose

Update is not necessary - the installed virus signature database is current.

ESETS Update utility

+-+--------------------+------------------------+------------------------+

| | Module             | Available version      | Installed version      |

+-+--------------------+------------------------+------------------------+

| | loader             |        1069 (20161122) |        1069 (20161122) |

| | perseus            |        1532 (20171031) |        1532 (20171031) |

| | engine             |       16539 (20171208) |       16539 (20171208) |

| | archiver           |        1270 (20171113) |        1270 (20171113) |

| | heuristic          |        1184 (20171110) |        1184 (20171110) |

| | cleaner            |        1150 (20171031) |        1150 (20171031) |

| | horus              |        6562 (20171207) |        6562 (20171207) |

| | dblite             |        1093 (20170725) |        1093 (20170725) |

+-+--------------------+------------------------+------------------------+

 

:/opt/eset/esets/sbin# ./esets_update --prerelease-updates

Virus signature database has been updated successfully.

Installed virus signature database version 16539P (20171208)

:/opt/eset/esets/sbin# ./esets_update --verbose

Virus signature database has been updated successfully.

ESETS Update utility

+-+--------------------+------------------------+------------------------+

| | Module             | Available version      | Installed version      |

+-+--------------------+------------------------+------------------------+

| | loader             |        1069 (20161122) |        1069 (20161122) |

| | perseus            |        1532 (20171031) |      1533.2 (20171205) |

| | engine             |       16539 (20171208) |      16539P (20171208) |

| | archiver           |        1270 (20171113) |        1270 (20171113) |

| | heuristic          |        1184 (20171110) |        1184 (20171110) |

| | cleaner            |        1150 (20171031) |        1152 (20171127) |

| | horus              |        6562 (20171207) |       6562P (20171207) |

| | dblite             |        1095 (20171206) |        1095 (20171206) |

+-+--------------------+------------------------+------------------------+

 

Greetings

 

Edited December 8, 2017 by CircleSquare

[CircleSquare 0]

hi everyone, it happens again on Sunday 

Quote

delivery temporarily suspended: conversation with 127.0.0.1[127.0.0.1] timed out while receiving the initial server greeting

 

[FHolzer 0]

did you recheck you versions ? (automatically updated, etc)

[FHolzer 0]

also, de error sound different,
it was

delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:2526: Connection refused

in the past and not
 

Quote

delivery temporarily suspended: conversation with 127.0.0.1[127.0.0.1] timed out while receiving the initial server greeting

 

[CircleSquare 0]

Quote

./esets_update --verbose
Update is not necessary - the installed virus signature database is current.
ESETS Update utility
+-+--------------------+------------------------+------------------------+
| | Module             | Available version      | Installed version      |
+-+--------------------+------------------------+------------------------+
| | loader             |        1069 (20161122) |        1069 (20161122) |
| | perseus            |        1532 (20171031) |        1532 (20171031) |
| | engine             |       16552 (20171211) |       16552 (20171211) |
| | archiver           |        1270 (20171113) |        1270 (20171113) |
| | heuristic          |        1184 (20171110) |        1184 (20171110) |
| | cleaner            |        1150 (20171031) |        1150 (20171031) |
| | horus              |        6575 (20171210) |        6575 (20171210) |
| | dblite             |        1093 (20170725) |        1093 (20170725) |
+-+--------------------+------------------------+------------------------+

 

version is downgraded ...

we used the prerelease-updates setting why it would returns to regular update channel?

[FHolzer 0]

i dont know, it was just a common sys-admin thought
i didnt used the prerelease-update - i changed the files manually.
 

[CircleSquare 0]

good point

full log message:

Dec 11 06:51:58 hots postfix/smtp[24645]: E9B7D11440A6: to=<xxx@xxx.de>, relay=127.0.0.1[127.0.0.1]:2526, delay=72594, delays=72294/0.04/300/0, dsn=4.4.2, status=deferred (conversation with 127.0.0.1[127.0.0.1] timed out while receiving the initial server greeting)

same error other sound .. but after the downgrade it will be an false positive in our case

[Peter Randziak 944]

Hello guys,

today at around 10:30 CEST we have released the Database module 1095, containing fix for the reported issues.

Thank you once again for your support and help with resolving of this issue.

 

In case it happens again with the Database module 1095, do not hesitate to contact us with the core files of all esets running processes.

So far we haven't received any reports of issue reoccurrence with the fixed Database module.

Regards, P.R.

[Satoshi 0]

Hello,

Our customer called us that the freeze happened after updating to Database module 1095 again. 
The following error was written in maillog of postfix.

Dec 13 13:38:20 XXXXXX postfix/smtp[11483]: XXXXXXXX: to=<XXXX@XXXXXXXX.XX>, relay=127.0.0.1[127.0.0.1]:2526, delay=300, delays=0.05/0/300/0, dsn=4.4.2, status=deferred (conversation with 127.0.0.1[127.0.0.1] timed out while receiving the initial server greeting)

And, we confirmed in data.txt of customer that the EMSL was updated to DB module 1095 as below.
--------------------------------------------
modules:
em000_32.dat:14:version: 1069 (20161122)
em000_32.dat:15:build: 1112
em000_32.dat:17:type: loader module
em001_32.dat:5:version: 1533.2 (20171205)
em001_32.dat:6:build: 1927
em001_32.dat:8:type: perseus module
em002_32.dat:5:version: 16563 (20171212)
em002_32.dat:6:build: 35712
em002_32.dat:8:type: engine module
em003_32.dat:5:version: 1270 (20171113)
em003_32.dat:6:build: 1332
em003_32.dat:8:type: archives module
em004_32.dat:5:version: 1184 (20171110)
em004_32.dat:6:build: 1156
em004_32.dat:8:type: heuristic module
em005_32.dat:5:version: 1152 (20171127)
em005_32.dat:6:build: 1207
em005_32.dat:8:type: cleaner module
em006_32.dat:5:version: 1118 (20171108)
em006_32.dat:6:build: 1169
em006_32.dat:8:type: antistealth module
em013_32.dat:5:version: 1018 (20100812)
em013_32.dat:8:type: self-defense module
em021_32.dat:5:version: 6585 (20171212)
em021_32.dat:6:build: 12801
em021_32.dat:8:type: horus module
em022_32.dat:5:version: 1095 (20171206)
em022_32.dat:6:build: 1100
em022_32.dat:8:type: DB module
em023_32.dat:5:version: 11276 (20171212)
em023_32.dat:6:build: 11427
em023_32.dat:8:type: pegasus module
em034_32.dat:5:version: 1019 (20170825)
em034_32.dat:6:build: 1023
em034_32.dat:8:type: antistealth helper module
--------------------------------------------

We reported this case to GPC already.

Are there any other cases where this problem occurred after updating to DB module 1095?

Best regards,
Satoshi

[Peter Randziak 944]

Hello @Satoshi,

this if the first issue occurrence with DB module 1095, I'm aware of.

Can you please send me the ticket ID so we can check it with a priority?

We need core files of esets processes from the frozen state to be able to analyze the issue.

Regards, P.R.

[Satoshi 0]

Hello Peter,

thank you for your reply.
We understood that this problem has not occurred elsewhere.

My colleague reported this case as TICKET 207524.
Unfortunately, the customer did not know the way to generate core files of esets processes from the frozen state at that time.
Therefore, we have asked the customer to generate core files when freeze happened next.

Thank you.

Best regards,
Satoshi

[Peter Randziak 944]

Hello Satoshi,

to generate core files the command: gcore <pid>   can be used.
I'm afraid that without them we won't be able to find the root cause of the issue.

Once you will have the core files, do not hesitate to contact me so I will have it checked with a priority.

Regards, P.R.

[dmenl 2]

Hello,

I experienced the same issue right after the update, but it has not occured since restarting all esets services.

[Satoshi 0]

Hello dmenl,

Thank you for your information.

In case of our customer, too, this issue has not occurred since restarting all esets services yet.

 

@Peter

After updating to DB module 1095, is the restarting all esets services necessary to fix this bug?

 

Best regards,

Satoshi

Edited December 19, 2017 by Satoshi

[Peter Randziak 944]

Hello guys,

I'm not aware of esets services restart need as upon modules update, the esets daemon (in which the deadlock causing the freeze occurred) is reloaded.

Regards, P.R.