The smoke should escape via the flue - there are a couple of reasons that may stop this.  

Is your flue blocked? Your flue pipes may just need a clean? Simply give them a clean (a toilet brush is the perfect size).

The flue pipes will give the best draw once they are warm, so maybe place the flue towards the back of the stove to warm the pipes first.