EEI Server Purge

[johnny_marre 0]

Hi,

after installing the EEI I noticed that the DATA of mysql server 8.0 generates BIN files with a maximum size of 1048578 kb and its increasing exponentially, so I would like to know if there has a way to purge the given base or it is done automatically if there is a way to reduce the purge time.

Thanks,

 

[Marcos 4,694]

Db cleanup is triggered automatically on a daily basis.

You can set it up in the server setup where you can also see what kind of events takes most of the db:

 

How many clients report to the EEI Server? Is the db on the same server or on another one? What type of db do you use? What is the hw configuration of the server? How many detections occur on a daily basis?

Could you post a screen shot of the Server status dashboard?

[johnny_marre 0]

Hi Marcos, thanks for response,

In the server configuration we have reduced the clean events and clean detection to durations of 1 week and 1 month, for the NUMBER OF THREADS WRITING TO DATABASE we left it as is and we would like to know what it is, also for STORED IN THE DATABASE EVENTS, we have unchecked the file system box in the hope of reducing space consumption.

On the 487 machines we have available, we have deployed the EEI on 42 machines, because previously, following the deployment of the EEI on all the machines, we had a hard disk saturation problem (1Tb) after one week .

For the database it is hosted on the same server, it is a MySQL 8.0.23.0 database.

For the server configuration it is a physical server windows server 2016 Ram 16 GB, processor intel xeon x3450 core 4 thread 8.

About the number of detection we will not be able to know since we are deploying as and when.

 

[TomaszBiernat 0]

Hello, I got a few explanations for you.
Starting with your latest post, NUMBER OF THREADS WRITING TO DATABASE is a number of concurrent threads that are writing events to the database. The default value should be optimal, it can be tweaked to achieve better performance of saving events, but let's say it's more 'advanced' setting.
Reducing events and detections storing time will allow you to save some disk space, especially events storing time.
Unchecking storing filesystem events will stop recording of all file related events like write to file, file opened, etc. They will still be analyzed and exposed to rule engine that is triggering detections, but they won't be stored in a database.
Looking on screenshots that you provided, there is no abnormal situation with EIServer's work, everything looks fine.
Going back to your initial question, I see that you were asking about BIN files growing in size. These are files for MySQL's binary log mechanism (https://dev.mysql.com/doc/internals/en/binary-log-overview.html) which is used mostly for replication. It is not managed by EI in any way. EI is invoking a lot of operations on a database, and all history goes to that log, so it's growing fast, but actually it has nothing in common with EI database size. If you want to check storage used just by EI database, you can check size of enterpriseinspectordb directory size in MySQL's Data directory. So the question is, if you are using replication, and if you need that binary log enabled at all. If not, it can be disabled, otherwise it can be purged periodically.

[johnny_marre 0]

Hi Thomas, Thanks for response,

about BIN files, we are not using replication, the installation was performed by default, now the purge is necessary otherwise if we deploy EEi Agent  on all machines, we will have the hard drive full.

 

[TomaszBiernat 0]

https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html

As this article says, binary log is enabled by default since MySQL 8.0. If you want to save a disk space and not purge that log manually, you can disable it. "To disable binary logging, you can specify the --skip-log-bin or --disable-log-bin option at startup."

[johnny_marre 0]

HI Thomas,

Thanks for your support, i have add line "expire_logs_days = 4" in File My.ini, now the old BIN files have been deleted.