Took me a while to get back to this one.
I'd like to suspect STP but I'm not getting any messages from it and there are no ports in STP blocked mode. I'm eventually going to disable STP at the router and agg layer if it doesn't cause more problems than it solves.
It's not happening 30 minutes after people leave, but that's still an interesting point and I might try to capture it somehow.
Thanks!