At work, do you ever?

Talk to people who you don't have much in common with. Like people you find have a low intellect and do not carry stimulating conversation? People have told me I shouldnt not talk to certain people for this reason, but I dont have anything to say to them I find them of low quality?