What is with the adage "People like that are not worth it"

They're most definitely comforting themselves, convincing themselves that these women are the exception to the rule, and giving themselves permission to change the subject and not address the concern of the short man.

The problem with the "those people aren't worth it anyway" logic is that it assumes that these people are rotten to the core, through-and-through. That's not the case though because everyone is heightist until they learn not to be. So, people who are completely well-intentioned and empathetic will still practice heightism, often without even knowing it. Heightism is a system of social indoctrination that begins very early in life and continues until someone takes the time to take a step back and reconsider what they've been indoctrinated to believe about short people. Also, heightism isn't considered a bad thing by society, so well-intentioned people have no reason to retract their heightist beliefs, thus, they will keep being heightist while being otherwise decent people.

Heightism isn't just a problem where the trashcreatures on Twitter are the sole face of heightism, they're just the loudest extremists. All of the people who subconsciously assume and doubt short people, and who give deference to taller men and view their actions positively are a much-deeper issue, and it's using oversimplified "either/or" logic to assume that heightism boils down to two clean categories, which are 1)Overtly Heightist 2) Not-heightist. This issue is much more complicated than that, and there are many layers to heightism that involve both horrible people in addition to people who are otherwise productive citizens.