You don’t have to agree with everyone you know. You can even “agree to disagree” (which typically means “I think you’re wrong, but I don’t want to talk about this anymore”). But pronouns don’t have that option — they always have to agree with their antecedents. Luckily, grammatical agreement is different from agreeing in a conversation, so those pronouns don’t need to worry too much.
A pronoun refers back to a noun that appeared previously in the sentence or conversation.
Pronoun agreement means that a pronoun and its antecedent (the noun a pronoun replaces) agree in number and gender. If the antecedent is singular, the pronoun must be singular. If the antecedent refers to a woman, the pronoun should also refer to a woman — and so on.
Pronoun-antecedent agreement helps the reader understand what noun a pronoun is replacing. Saying “Gary and I would like to see a movie, but she is too busy tonight” could confuse a reader. (Who is “she”? What does she have to do with our plans?)
While English is typically not a gendered language, it does have one gendered element — feminine and masculine pronouns (she/her, he/him) to refer to female and male nouns. However, they/them functions as a singular gender-neutral pronoun if you don’t know a noun’s gender, or if the noun refers to a person who identifies as non-binary (neither male nor female).
Using she/her or he/him in these cases would not be correct pronoun-antecedent agreement, and could even be considered misgendering.