As an airline pilot, I lay over in cities all over the world, and I frequently wander around looking for what I call "urban wilderness" - little overgrown pockets where nobody goes. I have frequently stumbled into whole "villages" of homeless people along river banks, in highway medians, etc. I often smell them before I see them.
Foraging in a city would be much easier than in the country. Thrift shops, dumpsters, construction sites, outright theft, etc. Anything unsecured, as Toddy mentioned.
I think the biggest problem would be security - finding a place where you could rest easy and not worry about your things being stolen. Most homeless people have either drug addictions or mental illnesses, and they prey on each other like animals. They undoubtedly spread diseases and parasites to each other as well.
The mobility and load-carrying ability of a bicycle would be useful. Make the rounds, then skeddadle to a hidden, private camp. A bicycle would make you look at least marginally more "respectable", too - good "urban camouflage".
There's plenty of healthy wild meat in cities that's much easier to approach than it would be in other settings. Squirrels, pigeons, ducks, rats, etc. The problem is figuring out a discrete way of harvesting it. You could keep yourself well-fed with a suppressed .22 pistol, some rat traps and a few snares, at least until you got caught.
