The New Dictionary of Cultural Literacy, Third Edition. 2002.
get someones goat
To make someone annoyed or angry: Gavin may seem unflappable, but I know a way to get his goat. This expression comes from a tradition in horse racing. Thought to have a calming effect on high-strung thoroughbreds, a goat was placed in the horses stall on the night before the race. Unscrupulous opponents would then steal the goat in an effort to upset the horse and cause it to lose the race.