I recently made the plunge and bagan using Zsh in lieu of Bash . I've not regretted the decision in the slightest but there have been minor annoyances that needed to be dealt with. The simplest annoyance was the special keys (i.e. delete, home, &c). The solution is quite simple and elegant but not completely obvious.
The bindings are usually read from the inputrc file (located in /etc/) by Bash but Zsh does not do this by default. There are probably more elegant solutions to this problem but a quick brute force solution is to create a bindkeys file from inputrc:
Once this file has been crafted, it's simply a matter of invoking it from your .zshrc with source ~/.zshrc-bindkeys.