Morton's on West Street or The Sheffield Shop on Ecclesall Road may have something to tempt you. I would try the Sheffield Shop first. They don't keep a lot of particularly high-end / esoteric stuff though, if that's what you're after, but a great shop and a nice bloke. No connection etc etc
It's not just knives though - pick up a nice hip flask while you're here too...
HTH
Rat