Did you get smoke? Once you've got a reasonable powder base and smoke starts, give it welly for a short burst, then stop. If an ember has formed it should now smoke without you bowing. Don't rush the next stage; if you've got an ember is should stay without fussing over it. When you're starting, cheat, have a bundle of cotton wool as your tinder. Carefully transfer the smoking ember and the powder pile to the cotton wool (trying not to separate it). Very, very gently, encourage the ember with very light blows of air - in cotton wool that should be enough and you should get a flame.
What woods are you using? I'm embarrassed to admit that I have never ever succeeded in getting a decent ember with hazel as a drill for example. I know other people have, but I haven't. The thing is hazel is quite attractive to try and use as a drill because it produces nice round stems - but it is notoriously difficult.
My favourite 'go to' wood for the drill is elder, yet other people dislike it! My point being, be prepared to try different combinations; you'll find one that works then there'll be no stopping you.