I grew up in the low income part of San Jose (East side), my father lives in the same house now -- though he didn't live there when I was a kid. Like Aron said, the changes are a result of gentrification, but what is happening in that neighborhood is interesting and nuanced.
The neighborhood has had crime for decades, it was probably much worse in the 80s. During that time there was major disruption when the factory jobs started closing. My neighbors were about 50/50 factory/trade families and welfare recipients. Also, in the 80s the tech industry started to take form. So, families with 'trade' type of labor skills were getting squeezed by low employment opportunities and rising property values and started to leave.
Push forward into the 90s and the 00s and most of the need for skilled labor is gone. Meanwhile, property prices are rising and welfare reform pushes the chronically unemployed households out of ESJ. The neighborhood is still run down and no upwardly-mobile types would want to live there. So, the people who have replaced them are largely even lower income immigrants. Yet, the properties are relatively expensive and the new residents are sleep 2-3 to a room to make rent.
Going back to the neighborhood, the only people that are still there from when I was a kid are very old now. Oddly, in some ways the neighborhood is quieter and cleaner than it use to be even though I would say the average resident is poorer. The kids who are growing up there now aren't going to have a bright future unless they make a very difficult leap from low income service work to tech. What usually ends up happening is that the kids just move out of the Bay Area when they get into their early 20s.
Aaron, did you grow up in the Bay Area?