Yes, the mission district is safe during the day, with a few precautions:
don't carry your cell phone in your hand
during the day only
if you're really worried don't wear blue or red (gang colors, but honestly I've never had an issue)
It's unsavory in the mission to be sure, lots of poverty and grime, outside of some gross homeless people I don't mind going there on my own for shopping. I keep my chin up, smile at the leering men (hey, at least someone's looking!!) and enjoy it - sometimes it feels just like anywhere else but the US.
Don't miss out on La Palma on 24th, and the Victoria Pastry Co - my favorite pandaria plus they have killer coffee. Jaime the owner is using his kitchen there as an incubator for locals who need a commercial kitchen, he's all about local, sustainable and community. Love him. Also, my favorite market to shop is Casa Lucas, El Farolita's tacos el Pastor are the bomb, Dynamo Donuts (bacon! and the fleur de sel caramel). Now a bunch of hipster coffee places have opened up, right next to the markets that carry wrestling masks and pinatas. Now I want a taco!!