A hori hori isn't a 'hidden' blade and I would be astonished if the police ever claimed as such. Those laws are specifically to address things like belt buckle knives, or walking sticks with swords hidden in them.
It is by default an offence to carry in a public place a bladed article of any kind other than a folding pocket knife, section 139 of the Criminal Justice Act.:
https://www.legislation.gov.uk/ukpga/1988/33/section/139
There are some statutory defences to this for someone carrying them with 'good reason'.
So by default he was committing an offence, which means the police can arrest him, backed in this instance I am sure by the fact it was reported to them and they didn't just drive past and spot him pruning some bushes. His defence would be the 'good reason' defence, which a solicitor would've advised him on and he should not have accepted the caution.
Niwaki themselves define the Hori Hori as being a blade:
https://www.niwaki.com/hori-hori/#P00442-7