I swear this used to exist. I could leave a comment, see the number three next to the "bell," and open that, and remove the 1 person of the three who did not need to be notified about this specific comment in the task.
It only removed them from a notification from this one comment and didn't remove them from the entire task's comments (which you can currently do by using the 3 dots in the upper right).
This helps when trying to limit conversations and clutter for things people don't need to be involved with.