Does he mean I'm untrustworthy?

if you have done nothing wrong you have nothing to worry about. maybe he was joking i wasnt their so i wouldnt know. i would deffinettly ask him what he ment by it. if you feel your stepping on egg shells when around him then he worth the hle. tell him how you feel he is only human!