SSH is a great way to access your server remotely using encryption. 
I've configured my SSH server to listen on a specific ip address and 
specified which users can log on. How can I add further security 
assuring only the users I specifiy can access the Linux Server using 
SSH. Answer, TCP Wrappers! Many Linux services are by default compiled 
with TCP Wrapper support. And, so is SSH when installed using the Red 
Hat Package Manager....

TCP Wrappers, written by Dr. Wietse Venema, is used to add another 
layer of security and control to commonly used services. My remote 
users have static ip addresses and I want to allow them access to SSH 
and deny others. How?? TCP Wrappers has two configuration files, 
/etc/hosts.allow and /etc/hosts.deny. We can add simple commands in 
these configuration files specifying what we want, remembering that the 
order is very important! What do I mean by order? When a Linux Service 
such as SSH is called over a network, TCP Wrappers first looks at the 
hosts.allow file, evaluates rules from top to bottom and then evaluates 
hosts.deny, top to bottom. Using hosts.allow and hosts.deny, we can 
specify the same result many different ways. Let's look at an example 
in hosts.deny:

ALL: EXCEPT sshd
sshd: ALL EXCEPT 215.34.123.55 341.234.65.11

The first line in my hosts.deny, denies all services except sshd, which 
is the daemon for the SSH Server. The second line denies all ip 
addresses except those specified. But what about the hosts.allow 
configuration file? Well, TCP Wrappers looks at hosts.allow, which in 
this case is empty, and then immediately looks at hosts.deny. TCP 
Wrappers always looks at both files, hosts.allow first and hosts.deny 
second. It's that simple. Why do we do this? Suppose someone port scans 
your Linux Server, notices that port 22 is open to the internet and 
knows of a user by social engineering. TCP Wrappers will deny the user 
the brute force attempt on the SSH Server. An added level of security 
discouraging or slowing down the 'Cracker'.