Hold it against someone
Meaning: blame somebody for doing something, to like someone less because they have done something wrong or behaved badly
Example: I lost his book, but he doesn’t hold it against me. It was his mistake, but I won't hold it against him.
Example: I lost his book, but he doesn’t hold it against me. It was his mistake, but I won't hold it against him.
