From your photo it looks like foxglove to me too.If there are quite a lot of them, try lifting one with a small trowel; if it's foxglove the roots will be tough, string like, cream coloured ones, stretching out evenly around the plant. It'll bed back down again fine.
There are three other plants that come up like the one you show at this time of year; one is lemon balm (melissa) but the lemony scent is unmistakeable, campion comes up in those opening twist of leaves too, but the leaves are more pointed and smoother, the other is comfrey which is usually downy on the underside and the roots are thick.
Cheers,
Toddy