I have not tested this in the lab to confirm, but my recollection is that if you apply a filter to the loopback interface on a Junos device the filter will apply to "self" traffic for the device in general not just that destined to the loopback address itself.
So I believe that the drops you see should be traffic that is hitting the RVI interfaces you have the switch and not transit traffic. So there would be no user impact. And this would also be the desired behavior since you don't want this traffic (otherwise why make the filter) to hit the routing engine.