I'd chalk this up to objectification and re-ification of the "other." People are "other" when they are demonstrably different, and being poor, unemployed, et c. sure makes you different. Class and race differences, whether formalized or not, tends to exacerbate this.