When Bob went on the supply run and the walkers & helicopters fell through the roof, trapping Beth's boyfriend, there was a scene on the ledge during the escape where he struggled to keep his grip on a knapsack or bag or something...turned out it was because he had booze in it. At that point, Daryl went all up in his face and one of the things Daryl said was "I should have left you alongside that road where I found you", or words to that effect. In other words, he regretted bringing Bob into the group because his alcohol addiction and attempts to hide it caused the chain of events leading to the deaths in the store.
A side note: The flashback confused me too, because I couldn't figure out how he had lost Sasha & Maggie, and his shoulder wound was missing. He also looked "fresher"...not quite as beat up, overall. Then I saw he was drinking a Nyquil knockoff in the culvert cell that he built for himself. When Daryl & Glenn confronted him I figured it out. I'm kinda slow...
