Peter stay on top of it. State clear expectations "formally" (and again DOCUMENT it).
Do not dance around, if you do, nothing will change (and your position will then be on the line).
Build your case carefully, but make it expedient as it's not a matter of if something bad will happen, but how soon will something bad happen. It WILL happen.
It sounds like a small business, but if there is any Union organization there and you have a working relationship, after talking with HR (or the owner), you might consider involving them to gain some leverage and buy-in.
I've been there and done that dozens of times.
It's never easy, but sometimes you have no choice.
There are some good free softwares out there for viewing browser history logs, and usb access logs.
I'd just make sure that keeping browser history is enforced via gpo (if you can). Then they can't delete out the logs after each use, keeping you blind to their activities. While you're at it with the GPO, lock down the browsers so they can't install extensions/addins.
You could also lock down (via ESET Device Control) exactly which (down to Serial number, but as broad as make, or model) usb keys' they're allowed to use. I'd also look to disable booting from USB via the bios, and lock the bios with a PW (if you can boot to usb you can run tails or some such with no IT visibility).
And like tom said... document, document, document... Talk to your boss, make sure you're in the clear for the 'watching'.
Is this user an Admin on the computer in question?