In both beta and prod, if a student who has tags is concluded, they will still show in the number count for that tag. This screenshot is from Beta using the tag filter (which is great, BTW!)
I agree the new filters are cool, I tested in prod and it updates the number of students in a tag when I remove them. Did you assign the student any work with the tag or do have any submissions tied to the tag? That may be keeping it in the background.