roga 2 Posted October 12, 2016 Posted October 12, 2016 I have a small windows network where the ERA runs on the only server (2008r2) in the network which is also a domain controller. Since upgrading to ERA 6.4 I have seen an increase in files in the following 2 directories: C:\ProgramData\Microsoft\Crypto\SystemKeys C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18 (we are talking thousands of files in each case) From a couple of google searches it appears that these may be related to https request to the server, and ESET software is mentioned as a possible cause. Is this something anyone else has seen? regards Roga
ESET Staff MartinK 384 Posted October 12, 2016 ESET Staff Posted October 12, 2016 Could you please check what process is creating those files using tools like process monitor? We have seen similar behavior - those files are created once "virtual" TLS certificates store is created, which basically happens each time AGENT connects to SERVER -> is there correlation between connection interval and creation of mentioned files? There is also possibility that not AGENT, but ESET security product (EES/EAV) is creating those files - what version are you using if any?
roga 2 Posted October 12, 2016 Author Posted October 12, 2016 Thanks for the reply. I have installed process monitor for when the next spate starts. Earlier today I updated the agent on the network to the latest version, I also updated an older EFSW from 4.5 to 6.4. So far I have had no further files created. I will update with process monitor info if files start appearing again. BTW don't know if it is related by the agent on the machine with ERA is failing to report, I think I'll start a different thread for that as possibly a red herring here.
roga 2 Posted October 13, 2016 Author Posted October 13, 2016 OK I have run process monitor and I can see that ERAAgent.exe is creating many of the files
ESET Staff MartinK 384 Posted October 13, 2016 ESET Staff Posted October 13, 2016 (edited) OK I have run process monitor and I can see that ERAAgent.exe is creating many of the files Thanks for clarifying. We are currently investigating this issue as it seems to be a regression. There is currently no known workaround, but it should be safe to delete those files created by ERAAgent. Reducing connection interval may also help in slower growth, especially in case it causes problems (i.e. insufficient disk space). Edited October 13, 2016 by MartinK
roga 2 Posted October 13, 2016 Author Posted October 13, 2016 I'm not sure it is safe to delete all of the files in those locations as a handful might have some relevance to other applications. I can see that it is probably safe to delete most after a certain date, but I wouldn't know which are the essential ones and which are produced by ESET. The problem is that the location forms part of the system state backup, this has increased our cloud backup times, and of course is pushing our quota up. If someone could tell me if there is a way of identifying which files can be deleted (e.g. by a sting in the file name) I could run a batch file to trim. BTW one side effect of me looking into those directories was that I inadvertently changed the security permissions, which had other consequence for other programs, including MS sqlexpress 2014 which is part of the ESET windows package.
jimwillsher 65 Posted October 13, 2016 Posted October 13, 2016 Interesting. I just checked C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18 and i have 37,982 files in there :-) (Windows 10, ESET 6.4 stack)
ESET Staff MartinK 384 Posted October 13, 2016 ESET Staff Posted October 13, 2016 (edited) I'm not sure it is safe to delete all of the files in those locations as a handful might have some relevance to other applications. You are right, I meant those created by ERAAgent.exe. Unfortunately I don't think there is any predictable name pattern - even content is different which makes is difficult to recognize. Edited October 13, 2016 by MartinK
roga 2 Posted October 13, 2016 Author Posted October 13, 2016 The problem only started for me with 6.4, is the problem with the agent or the server? e.g. could we uninstall latest agent and install a previous version until the problem is solved? Or would we have to roll back server, but e.g. keep latest clients?
ESET Staff MartinK 384 Posted October 14, 2016 ESET Staff Posted October 14, 2016 The problem only started for me with 6.4, is the problem with the agent or the server? e.g. could we uninstall latest agent and install a previous version until the problem is solved? Or would we have to roll back server, but e.g. keep latest clients? Agent 6.3 should work with ERA Server 6.4 without any significant issues.
roga 2 Posted October 14, 2016 Author Posted October 14, 2016 So how can I install agent 6.3? I assume the generated bat file downloads the most recent agent, and of course the machine with the problem is the ERA server so I would also need to install 6.3 on that. I need to sort this ASAP as clearly is causing me problems
ESET Staff MartinK 384 Posted October 14, 2016 ESET Staff Posted October 14, 2016 (edited) So how can I install agent 6.3? I assume the generated bat file downloads the most recent agent, and of course the machine with the problem is the ERA server so I would also need to install 6.3 on that. I need to sort this ASAP as clearly is causing me problems It don't think you need to downgrade SERVER version. Even if AGENT installed locally is older, it should work. It would be also quiet a problem to downgrade SERVER as it is not supported scenario. Regarding live installer bat file, you will have to modify URL path to use older version, i.e. use download links: hxxp://repository.eset.com/v1/com/eset/apps/business/era/agent/v6/6.3.177.0/Agent_x86.msi (checksum: ea097fa6fdd5500982b1b955898c82de0cc21769) hxxp://repository.eset.com/v1/com/eset/apps/business/era/agent/v6/6.3.177.0/Agent_x64.msi (checksum: 79c77515d410fb943e8dc86310d44f6f46bf9715) Edited October 14, 2016 by MartinK
Solution roga 2 Posted October 20, 2016 Author Solution Posted October 20, 2016 So the short answer to the problem of ERAAgent creating unnecessary files in C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18 is to uninstall ERAAgent 6.4 and install ERAAgent 6.3 Having now had to deal with this on 3 windows domains I thought I would do a write up of my experiences. NB so far all of the domains which I upgraded to 6.4 have suffered the same problems. 1) All of the clients I have looked at with ERAAgent 6.4 have thousands of files under C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18. The default setting on ERA Server is for agent to report every 60 seconds, so that is 1440 times a day the files are created, over 40,000 a month. 2) It is not always possible to know which of the files can be moved or deleted from Crypto\RSA\S-1-5-18 - on some machines essential services will have necessary files in that directory. 3) Trying to access Crypto\RSA\S-1-5-18 may cause a change in permissions, i.e. windows replacing the default permissions with permissions for the current user account. This created problems later on. NB the permissions on that directory are not the same for all hosts, the permissions on a particular host will depend on the services installed. Problems caused by changing permissions on Crypto\RSA\S-1-5-18 impacted the following services (i.e. caused failures or problems): EraAgent MS SQL Server Remote Desktop Licensing server IIS 4) After replacing ERAAgent 6.4 with 6.3 duplicate entries occurred in ERA console, which needed trimming. Conclusion The upgrade to 6.4 has caused me a lot of extra work, I have thousands of junk files on the computers under my care, which I am reluctant to delete as there is no clear way of identifying them. I also had to troubleshoot a number of problems with other services, which I now know were as a result of permission changes. In one of the earlier posts above MartinK wrote that "There is currently no known workaround", well there is which is to roll back to an earlier version of ERAAgent. If this was said at the beginning it would have saved me some time. In the mean time have ESET changed or withdrawn the problem version of ERAAgent? Or is this software with a known problem still being distributed?
jimwillsher 65 Posted October 20, 2016 Posted October 20, 2016 In seven days, my folder file count has increased from 37,982 to 43,274. I'm scared to check the 180 machines under my control...... :-) Jim
ESET Staff MichalJ 434 Posted October 20, 2016 ESET Staff Posted October 20, 2016 Hello, This issue will be resolved in the next release of ERA 6 (6.5). After carefully considering all aspects of the issue, we have not considered its severity to be at level which would require a rollback of the agent to version 6.3, or release / including upgrades of completely new agent. Till today, it was not reported as critical outside this thread. After analysis, it might generate up to 3MBs per day, with default replication interval. The benefits of newer ERA agent are still higher, than this issue. We are sorry for any inconvenience caused by this on your side.
richardw1972 1 Posted November 17, 2016 Posted November 17, 2016 I have the exact same problem, ERA version 5 and clients 5.0.2229. ekrn.exe is writing thousands of files. I have over a million. What should we do?
LittlBUGer 0 Posted November 17, 2016 Posted November 17, 2016 I can confirm this issue, or at least I believe it to be the same problem. A client of ours has ERA 6.4 on their server (Windows Server 2012) as well as several workstations. I started to move a very large file from the server to a NAS they have and while that was running I did a scan of their system using TreeSize (to find any space/storage hogs). I noticed that after the scan completed, it kept showing my total file count increasing by the hundreds every few seconds. I drilled down the list of folders and found it to be exactly what was described here, though it was increasing at a drastic rate, which probably would have ended up as hundreds of MB within an hour. I only came across this thread after doing some research online as ESET was the only thing that was recently installed or changed on the server within the past month. I decided to stop the file move I initially started and suddenly the file count in the folder went back to 'normal' and I saved a couple hundred MB of space. I can only imagine it was the ERA agent inspecting/scanning the file I was moving over the network and creating all of those files. I haven't been able to confirm this on the other 13 workstations that ESET is installed on as I haven't been at the physical location of my client yet, but will do so this weekend. If this is the case, I've been copying a good deal of data from each workstation to the NAS in terms of backups so said folder will be huge by now. Is there any ETA on version 6.5 at this time? This is a serious issue for any environment, large or small. Thank you.
Administrators Marcos 5,468 Posted November 18, 2016 Administrators Posted November 18, 2016 I have the exact same problem, ERA version 5 and clients 5.0.2229. ekrn.exe is writing thousands of files. I have over a million. What should we do? In Endpoint this was already addressed a couple of months ago in an Internet protection module update. Also Enpoint v5.0.2299 is 2 years old; the latest version is 5.0.2265. We strongly recommend upgrading to the latest version, ideally to v6.4 alongside with ERA upgrade to v6.4 (release of v6.5 with further improvements is scheduled for Jan 2017).
Dedmondson 0 Posted February 17, 2017 Posted February 17, 2017 (edited) So where this does present an issue is with virus scanners. This has increased my nightly scans by 10 fold. When is 6.5 due to release? Edited February 17, 2017 by Dedmondson
Recommended Posts