restrict access to list through web part

Hello,

I am having a little trouble wrapping my head around a part of a project I am working on and was hoping someone might know a good solution or could look at what I have and say that one sentence that could bring this whole thing together.

Here is the scenario:

1. I have a list located on a SharePoint site.
2. I have a custom web part that outputs a filtered format of the list.
3. The list cannot be accessible except through the web part. (Cannot be found through search, URL, or UI)
4. Only one group has access to contribute indirectly (via web part) to the list.

People have suggested a proxy account/group, but if I had the web part place the current user in the proxy group, how do I remove them from the group once they leave the page?