Syslog messages empty

[Enda 0]

I have syslog configured on ESMC and syslog messages are being recorded and forwarded to our SIEM but every now and then it stops working and only sends blank logs such as shown in the screenshot.

Anyone experienced this before?

[Marcos 5,074]

Please raise a ticket with your local ESET support.

[Enda 0]

We have contacted support and they said they have no experience with syslog. The error seems to be related to how era processes syslog messages. The logs are present in the web console and logs are being forwarded properly to the syslog server but the messages being written to the syslog daemon by era are empty

[Enda 0]

Turning syslog off and on in the server settings sometimes fixes this but not always. Rebooting stops it working if it is.

[Marcos 5,074]

If the local ESET support is unable to provide a solution and further troubleshooting is needed, they should contact ESET HQ. You can provide me with the ticked ID and we will inquire Irish support team about it.

[MartinK 378]

Could you please provide more details of syslog configuration? Asking, because few issues related to delimiters and handling of new line characters were identified in ESMC 7.1, but in all cases empty records were just redundant, e.i. data were not lost, it was just wrongly interpreted due to incorrect encoding.

I would also recommend to capture this with enabled full verbosity trace logging in ESMC, it will be required for analysis, especially in case it will be possible to pair it with empty records in syslog.

[Enda 0]

Please see attached my syslog config in esmc.

 

[MartinK 378]

Thanks. As JSON is used, most of the issue I mentioned are not relevant, but I would recommend to check whether swticthing to TCP and "Octet counted framing" help. TCP should help for longer messages (exceeding UDP limits) and octet counting should be helpful for parser to identify start end end of each message, especially in case it is the issue you have encountered. Just be aware that both changes has to be supported also by syslog server.

[Enda 0]

I tried switching to TCP but the logs wouldn't forward at all. The syslog application was listening on TCP but as soon as I switched back to UDP it started working again. It's got to the point now where I have to login to ESMC multiple times a day to do the following to fix the issue in advanced server settings:

1. Disable syslog

2. Save

3. Enabled syslog

4. Save

Edited May 28, 2020 by Enda

[Enda 0]

Anybody know if it's possible to do steps 1-4 via a script? If we could do that and schedule it every 15 mins it would save a lot of manual effort to fix this multiple times a day