First, create a public/private key pair on the client that you will use to connect to the server (you will need to do this from each client machine from which you connect):

$ ssh-keygen -t rsa -b 4096

This will create two files in your (hidden) ~/.ssh directory called id_rsa and id_rsa.pub. id_rsa is your private key and id_rsa.pub is your public key.

Now set permissions on your private key:

$ chmod 700 ~/.ssh
$ chmod 600 ~/.ssh/id_rsa 

Copy the public key (id_rsa.pub) to the server and install it to the authorized_keys list:

$ cat id_rsa.pub >> ~/.ssh/authorized_keys

Note: once you've imported the public key, you can delete it from the server.

and finally set file permissions on the server:

$ chmod 700 ~/.ssh
$ chmod 600 ~/.ssh/authorized_keys

A good practice will also be to disable password authentication, forcing use of keys.

@ /etc/ssh/sshd_config

PasswordAuthentication no

References...
http://wiki.centos.org/HowTos/Network/SecuringSSH