After 17 years in the Military and often using my Kit Bags for my own use too I've never had a problem. I don't think any official would look at someone with a Mil-Spec bag as being Sinister in the slightest. If you're dressed like Rambo you'll either just be pitied or ridiculed. If any suspicion was there it would be related to the higher possibility of 'sharps' or 'flammables' being present. If you were out to cause harm you'd be a little more inconspicuously attired after all.
Returning home from my last Tour in Afghan I landed at Brize and went straight to Heathrow for a Civi connection back home (I was based in Scotland back then) and didn't have time to change. That was awkward I'll admit. Especially having to take my helmet and body armour on as hand luggage... But, it was me who felt awkward, at no time was I harassed by airport staff.
It's all common sense really. If you're going hiking in a predominately Islamic Country, or a known trouble-spot you may want to tone it down a bit (a lot!). I personally wouldn't even consider going anywhere where I felt I could be at risk of danger. If you're worried about what bag you have then I'd not bother going there. After all, where's the fun in spending your hard earned holiday time constantly on edge?
I've been spat at and threatened in the UK whilst wearing my Uniform to and from work. I've also been invited to share dinner with locals in both Iraq and Afghanistan... Go figure.