It's worth checking before you request when the other swapper last logged in (it shows you this beside their user details) - if someone hasn't logged in for some months, they're may not be actively using the site any more. Then, if they don't want any of your books, they will click one of the options their end to let you know that and you will get an email to tell you. Also, the swap request will disappear from your active swaps page.
Some people don't reply straight away - they may be away, or only able to use the site at weekends etc, but it is worth being patient. How long you wait depends really on how desperate you are for the book, and how many other copies there are listed. If you've had no reply within a few days and there are other copies listed, then it may well be worth your while cancelling the first request and asking someone else. But if it's the only copy available, you may as well leave the swap for longer (the other swapper gets an automatic reminder about the swap after about 8 days). However, since new swappers are restricted on how many swaps they may request until they've got some feedback, then it may be more sensible not to leave swaps on for a week or more until you're allowed more outstanding requests.