我覺得跟美國人说"You must/should go with me" is too direct, unless you are talking to close friends. Otherwise, it's somewhat impolite. Then again, it also depends on where - north, south, mid-east, or mid-west - you live. As a Taiwanese live in America, I actually had to learn to be tactful so as not to offend people.