Well, an update on things:
Today, I tried clicking on the Start Menu link, and it no longer worked and asked me to delete it; so I did. The same thing happened with the desktop icon, so that's gone now.
Whatever happened, whatever I did last night it seems to have gone now (although I expect there'll be a couple of folders here and there). Presumably the Quarantine folder has gone now too?