Sometimes happens that establishing of SSH (or other service) connection takes a really long time. This is often caused by failed reverse DNS lookups.
That situation can occur because of network problems or can be intentional action performed by system administrator.
How can you handle with it? Simply add this line to your sshd_config file.
UseDNS no
Now SSH connection should be established significantly faster.