They didn't realize that Joseph understood what they were saying because they were talking to him through an interpreter.