RHCE - RHEL7 - Part 2

Network Services

This is the second in a three part series covering the objectives for the Red Hat Certified Engineer (RCHE, EX300) exam.

To focus only on the topics at hand I'm disabling firewalls. If you would like to get more information about how to configure the firewall in RHEL 7 please see Part 1 of this series.

Each part will follow the objectives as they have been outlined by Red Hat.

Install the packages

The full objective is: Install the packages needed to provide the service.

This seems like a very generic objective. I'll outline a general approach to installing the packages and ensuring they are started on boot. These steps will be similar for any service you wish to install on a given system.

[root@netserv ~]# yum search ntp
[root@netserv ~]# yum install ntp

[root@netserv ~]# systemctl list-unit-files | grep ntp
ntpd.service                              disabled
ntpdate.service                           disabled     

[root@netserv ~]# systemctl enable ntpd.service 
ln -s '/usr/lib/systemd/system/ntpd.service' '/etc/systemd/system/multi-user.target.wants/ntpd.service'
[root@netserv ~]# systemctl start ntpd.service 

[root@netserv ~]# systemctl list-unit-files | grep ntp
ntpd.service                              enabled 
ntpdate.service                           disabled

Configure SELinux to support the service

Everybody's favorite thing in the world, SELinux. First, let's make sure it's on and doing its thing:

[root@netserv ~]# getenforce

We need to change it to Enforcing so we'll edit the SELinux config file to make sure we boot into Enforcing and we'll use setenforce 1 to make SELinux enforce rules during this session:

[root@netserv ~]# vim /etc/selinux/config 
[root@netserv ~]# setenforce 1

Rather than put all the particular SELinux configurations here, I'll note any needed configs in the section that pertains to that service.

Use SELinux port labeling to allow services to use non-standard ports

See the previous section.

Configure the service to start when the system is booted

See the first section

Configure the service for basic operation

Typically Red Hat only expects you to have the service installed and running with very little configuration. "Basic operation" also implies that the service should not be blocked by SELinux or firewall rules to be sure to understand all the aspects of what makes the service function.

Configure host-based and user-based security for the service

RHEL 7, as with previous versions can use either firewalls or TCP Wrapper to configure host-based security. Some services like SMB and HTTPD can make restrictions based on users. This objective basically means that you should be aware of the various ways that you can restrict the given service.


This objective covers the Apache web server. There are a few significant differences between the 2.4.x version of apache that came with RHEL 6 and the 2.4.x version that comes with RHEL 7.

[root@netserv ~]# httpd -v
Server version: Apache/2.4.6 (CentOS)
Server built:   Jul 23 2014 14:48:00

First, lets get it installed. Note that we also install the httpd-manual so we can have reference material available to us when needed:

[root@netserv ~]# yum install httpd httpd-manual -y
[root@netserv ~]# systemctl enable httpd.service 
ln -s '/usr/lib/systemd/system/httpd.service' '/etc/systemd/system/multi-user.target.wants/httpd.service'
[root@netserv ~]# systemctl start httpd.service 

Configure a VirtualHost

I'll be using my domain powertra.in for these examples.

First things first, we should look at the docs that were installed with httpd-manual. In this case I'll go the url http://powertrain/manual The instructions that we need are at the bottom of the middle column of links. We'll be setting up named based VirtualHosts.

With Apache 2.2 we had to use the NameVirtualHost directive. It looks like that directive no longer has any effect in Apache 2.4.

The NameVirtualHost directive no longer has any effect, other than to emit a warning. Any address/port combination appearing in multiple virtual hosts is implicitly treated as a name-based virtual host.

Good news for us since this is less work. We can get straight to making our VirtualHost configuration files:

[root@netserv ~]# cd /etc/httpd/conf.d/
[root@netserv conf.d]# cat powertra.in.conf 
<VirtualHost *:80>
    ServerName powertra.in
    DocumentRoot /var/www/vhosts/powertra.in

[root@netserv conf.d]# cat beta.powertra.in.conf 
<VirtualHost *:80>
    ServerName beta.powertra.in
    DocumentRoot /var/www/vhosts/beta.powertra.in	

[root@netserv conf.d]# cd /var/www/vhosts/

[root@netserv vhosts]# tree
├── beta.powertra.in
│   └── index.html
└── powertra.in
    └── index.html

2 directories, 2 files

Each of those index files has a few words to make sure they are distinct. I can now go to powertra.in and beta.powertra.in both of which resolve to the same IP address and get different content for each.

In order to get some SELinux in the mix, let's configure a site on a non-standard port. We can see the ports that SELinux is aware of using semanage:

[root@denmark ~]# semanage port -l | grep http
http_cache_port_t              tcp      8080, 8118, 8123, 10001-10010
http_cache_port_t              udp      3130
http_port_t                    tcp      80, 81, 443, 488, 8008, 8009, 8443, 9000
pegasus_http_port_t            tcp      5988
pegasus_https_port_t           tcp      5989

I'd like to use one that isn't already configured so I'll use 8125.

We can use the man pages to get the information we need, in fact it has the specific example that we need already.

[root@denmark ~]# man semanage-port
Allow Apache to listen on tcp port 81
# semanage port -a -t http_port_t -p tcp 81

We'll modify it for the port we're looking for:

[root@denmark ~]# semanage port -a -t http_port_t -p tcp 8125
[root@denmark ~]# semanage port -l | grep 8125
http_port_t                    tcp      8125, 80, 81, 443, 488, 8008, 8009, 8443, 9000
steven@jupiter ~ > curl http://powertra.in:8125
Hello from port 8125!

Now let's serve some content out of a non-standard location /website. As it turns out this is a bit more involved than it was for RHEL 6. We now need to configure some access control directives for non-standard directories. This blog was able to provide the solution when I got stuck.

[root@denmark ~]# mkdir /website
[root@denmark ~]# echo "OMG SO NON STANDARD" > /website/index.html
[root@denmark ~]# ls -Z / | grep website
drwxr-xr-x. root root unconfined_u:object_r:default_t:s0 website
[root@denmark ~]# ls -Z /website/
-rw-r--r--. root root unconfined_u:object_r:default_t:s0 index.html

Let's create the VirtualHost file for this site, note the <Directory> block which will produce 403 errors regardless of SELinux configuration if omitted:

[root@denmark httpd]# cd conf.d/
[root@denmark conf.d]# cat non.powertra.in.conf 
<VirtualHost *:80>
    ServerName non.powertra.in
    DocumentRoot /website

<Directory /website>
    Options All
    AllowOverride All
    Require all granted

And the result, a 403 (caused by SELinux):

[root@denmark ~]# curl -I http://non.powertra.in
HTTP/1.1 403 Forbidden
Date: Fri, 28 Nov 2014 03:24:48 GMT
Server: Apache/2.4.6 (CentOS)
Content-Type: text/html; charset=iso-8859-1

Let's adjust the SELinux contexts Remember to check the man page for semanage-fcontext to get the example you should use to make the change permanent, using chcon as in this example will NOT persist a reboot.

[root@denmark ~]# chcon --reference=/var/www /website/
[root@denmark ~]# ls -Z / | grep website
drwxr-xr-x. root root system_u:object_r:httpd_sys_content_t:s0 website
[root@denmark ~]# chcon --reference=/var/www/vhosts/powertra.in/index.html /website/index.html 

And now we can see our content:

[root@denmark ~]# curl non.powertra.in

Configure private directories

By 'private' I believe they mean password protected directories. Again, we'll look to the manual. We are looking for the 'Authentication and Authorization' guide which is at the top of the 3rd column.

Let's start by creating a directory with some super secret stuff inside:

[root@netserv powertra.in]# pwd
[root@netserv powertra.in]# mkdir secret
[root@netserv powertra.in]# cd secret/
[root@netserv secret]# echo "secret stuff" > index.html

Now we can create (-c) a file named passwords for apache to check against.

[root@netserv httpd]# pwd
[root@netserv httpd]# htpasswd -c passwords steven
New password: 
Re-type new password: 
Adding password for user steven

Now we'll modify the powertra.in configuration file:

[root@netserv httpd]# cd ../httpd/conf.d/
[root@netserv conf.d]# vim powertra.in.conf 
[root@netserv conf.d]# cat powertra.in.conf 
<VirtualHost *:80>
    ServerName powertra.in
    DocumentRoot /var/www/vhosts/powertra.in

<Directory /var/www/vhosts/powertra.in/secret>
    AuthType Basic
    AuthUserFile /etc/httpd/passwords
    Require user steven

Now, when I try to go to powertra.in/secret I'll be asked to provide my user name steven and password.

Deploy a basic CGI application

Again, the manual gives us everything we need to complete this objective, this time the link that we're looking for (in my case) is http://powertra.in/manual/howto/cgi.html

Really, this is very simple since apache is already configured the way you need it to be to complete this objective.

[root@netserv ~]# cd /var/www/cgi-bin/
[root@netserv cgi-bin]# ll
total 4
-rwxr-xr-x. 1 root root 76 Nov 25 04:04 first.pl
[root@netserv cgi-bin]# cat first.pl 
print "Content-type: text/html\n\n";
print "Hello, World.";

All I did was create a file in /var/www/cgi-bin/ named first.pl and made it executable. From there I can go to powertra.in/cgi-bin/first.pl and see my output, meaning that my CGI "application" has worked.

Configure group-managed content

This is essentially the same process we use in setting a password protected directory. This time around we're gonna create a group file which will allow us to use similar setting for a group of users.

First, let's add some users to the passwords file we created earlier and then add them to a groups file which defines the group name and who belongs to the group. In this case our group will be named sales:

[root@netserv httpd]# pwd
[root@netserv httpd]# htpasswd passwords tom
New password: 
Re-type new password: 
Adding password for user tom
[root@netserv httpd]# htpasswd passwords sally
New password: 
Re-type new password: 
Adding password for user sally
[root@netserv httpd]# htpasswd passwords bill
New password: 
Re-type new password: 
Adding password for user bill

[root@netserv httpd]# vim groups
[root@netserv httpd]# cat groups 
sales: tom sally bill

And let's make some content the group should have access to:

[root@netserv beta.powertra.in]# pwd
[root@netserv beta.powertra.in]# mkdir group
[root@netserv beta.powertra.in]# echo "group stuff" > group/index.html

And make our directory configuration in the beta.powertra.in conf file:

[root@netserv conf.d]# pwd
[root@netserv conf.d]# vim beta.powertra.in.conf 
[root@netserv conf.d]# cat beta.powertra.in.conf 
<VirtualHost *:80>
    ServerName beta.powertra.in
    DocumentRoot /var/www/vhosts/beta.powertra.in	

<Directory /var/www/vhosts/beta.powertra.in/group>
    AuthType Basic
    AuthUserFile /etc/httpd/passwords
    AuthGroupFile /etc/httpd/groups
    Require group sales

Now if we try going to beta.powertra.in/group/ we will be asked to provide a user name and password, we can use any that we defined in the groups file (tom, sally or bill) but the user steven would not be able to get in since he's not a member of the sales group.

Configure TLS security

For this we'll need to install some extra packages mod_ssl will allow apache to use keys and certs. We will have to generate our own self-signed certs for this example. You could use the openssl tool but that is too complex for the purposes of this test. The crypto-utils package gives us the genkey tool which makes generating a key much easier and saves time (in my opinion).

[root@netserv ~]# yum install mod_ssl crypto-utils -y
[root@netserv ~]# genkey --days 365 powertra.in

Once we have our keys created we'll have to configure the VirtualHost to use TLS. I have removed the ssl.conf file that was provided with mod_ssl So I've had to put the Listen 443 https directive here.

A side note: I originally made a cert with a key length of 512 bits, I wasn't able to test this in firefox 33 since keys smaller than 1024 bits are no longer accepted. I was able to test the site with elinks and everything worked from the server side. I made another key with length 2048 and firefox liked that one just fine.

[root@netserv conf.d]# pwd
[root@netserv conf.d]# cat powertra.in.conf 
<VirtualHost *:80>
    ServerName powertra.in
    DocumentRoot /var/www/vhosts/powertra.in

<Directory /var/www/vhosts/powertra.in/secret>
    AuthType Basic
    AuthUserFile /etc/httpd/passwords
    Require user steven

Listen 443 https

<VirtualHost *:443>
    ServerName powertra.in
    SSLEngine on
    SSLCertificateFile /etc/pki/tls/certs/powertra.in.crt
    SSLCertificateKeyFile /etc/pki/tls/private/powertra.in.key

The important thing to notice is that I have placed the Listen 443 https directive here and have told apache to ignore the default ssl.conf that gets installed with the mod_ssl package. I like to do it this way to reduce the number of places I may have to troubleshoot if a problem comes up.


Fortunately, the test doesn't require us to configure a DNS server to any great degree.

Configure a caching-only name server

Let me quote Michael Jang from his book:

When configuring a caching-only name server, the first step is to look at the default version of the /etc/named.conf configuration file. The directives in the default version of this file are organized to set up a caching-only nameserver.

There are several packages associated with DNS, to simplify things I normally install what is needed this way:

[root@netserv ~]# yum install bind*

Once it's installed we perform the normal steps to "enable" and start the service.

[root@netserv ~]# systemctl enable named.service 
ln -s '/usr/lib/systemd/system/named.service' '/etc/systemd/system/multi-user.target.wants/named.service'
[root@netserv ~]# systemctl start named.service

Troubleshoot DNS client issues

Here we should be familiar with how to use the host, dig, and nslookup commands. We should also be familiar with how the DNS system works. Remember to check firewall rules and the /etc/hosts file and /etc/nsswitch.conf files for any configurations that may cause issues.


For this section I'll be following along with this guide from unixmen.com.

Provide network shares to specific clients

Let's get NFS installed:

[root@netserv ~]# yum install nfs-utils nfs-utils-lib -y
[root@netserv ~]# for i in rpcbind nfs-server nfs-lock nfs-idmap; do systemctl enable $i; done
ln -s '/usr/lib/systemd/system/nfs-server.service' '/etc/systemd/system/nfs.target.wants/nfs-server.service'
ln -s '/usr/lib/systemd/system/nfs-idmap.service' '/etc/systemd/system/nfs.target.wants/nfs-idmap.service'
[root@netserv ~]# for i in rpcbind nfs-server nfs-lock nfs-idmap; do systemctl start $i; done
[root@netserv ~]# mkdir /var/nfs-share
[root@netserv ~]# chmod 777 /var/nfs-share/

Now, I can edit the /etc/exports file to specify the host I want to share this resource with. In this example I'm using the IP address of the tester host but I could also use a FQDN here as well.

[root@netserv ~]# vim /etc/exports
[root@netserv ~]# cat /etc/exports
[root@netserv ~]# systemctl restart nfs-server
[root@netserv ~]# touch /var/nfs-share/lolfile
[root@netserv ~]# exportfs -avr

Now we can go to our client machine and make sure that we can use the remote resource.

[root@tester ~]# ip a | grep h0$ | awk '{print $2}'
[root@tester ~]# yum install nfs-utils -y
[root@tester ~]# showmount -e
Export list for
/var/nfs-share *
[root@tester ~]# mkdir /my-mount
[root@tester ~]# mount -t nfs /my-mount/
[root@tester ~]# ls /my-mount/

Provide network shares suitable for group collaboration

CertDepot has a good guide on this objective. In general, we are going to make a directory that is owned by a particular group then export that directory.

[root@netserv ~]# mkdir /shared
[root@netserv ~]# groupadd -g 60000 sharedgrp
[root@netserv ~]# chgrp sharedgrp /shared/
[root@netserv ~]# chmod 2770 /shared/
[root@netserv ~]# vim /etc/exports
[root@netserv ~]# cat /etc/exports
[root@netserv ~]# systemctl restart nfs-server
[root@netserv ~]# exportfs -avr

Notice that we create the directory with the sticky-bit set so that any new file created in this folder maintains group ownership.

Use Kerberos to control access to NFS network shares

CertDepot has another great guide on this subject as well. We'll need a KDC (Kerberos Distribution Center) as well as an NFS server. If you want to get a KDC set up quickly and easily, take a look at freeIPA. I won't cover that process here since I'm not sure that is a part of the RHCE exam. The documentation at the freeIPA site is great and the setup shouldn't take more than a half hour at most.

After adding the principal NFS server the KDC, we'll export a directory as we would normally but this time we'll use the option sec=krb5. Using this option will force the client to authenticate using kerberos before being allowed to use the exported directory.


SAMBA or CIFS has not changed much since RHEL 6 but as with NFS we now have an objective which asks us to restrict a resource using kerberos.

Provide network shares to specific clients

This objective is identical to the RHEL 6 version. Here I'm making a share available to the user steven. That user will still be restricted by the system permissions so in order to let them write to the share, they will have to own the folder. In the next section we'll cover making a share for group work.

A gotcha that didn't make sense to me was that you need to install samba-client to get smbpasswd. This site solved that one for me.

[root@samba ~]# yum install samba samba-client -y; iptables -F
[root@samba ~]# useradd steven
[root@samba ~]# smbpasswd -a steven
[root@samba ~]# mkdir /sharedfiles
[root@samba ~]# touch /sharedfiles/lolfile
[root@samba ~]# chown steven:steven -R /sharedfiles/
[root@samba ~]# vim /etc/samba/smb.conf
[root@samba ~]# cat /etc/samba/smb.conf | grep -Ev "^[;#]|^$"
workgroup = MYGROUP
server string = Samba Server Version %v
log file = /var/log/samba/log.%m
max log size = 50
security = user
passdb backend = tdbsam
load printers = yes
cups options = raw

comment = MyShare
path = /sharedfiles
browseable = yes
writable = yes
valid users = steven

[root@samba ~]# testparm
[root@samba ~]# systemctl enable smb.service 
[root@samba ~]# systemctl start smb.service

Provide network shares suitable for group collaboration

Similar to the NFS objective, we're going to create a shared directory with the sticky bit set. I'm going to follow along with this guide from CertDepot.

[root@samba ~]# groupadd rhce
[root@samba ~]# mkdir /rhcefiles
[root@samba ~]# chgrp rhce /rhcefiles/
[root@samba ~]# ls -l / | grep rhce
drwxr-xr-x.   2 root rhce  4096 Dec  6 20:28 rhcefiles
[root@samba ~]# chmod 2775 /rhcefiles/
[root@samba ~]# ls -l / | grep rhce
drwxr-sr-x.   2 root rhce  4096 Dec  6 20:28 rhcefiles
[root@samba ~]# touch /rhcefiles/testfile
[root@samba ~]# ll /rhcefiles/
total 0
-rw-r--r--. 1 root rhce 0 Dec  6 20:30 testfile

Now we'll create a couple of users for that group and configure samba for collaboration.

[root@samba ~]# yum install samba samba-client -y  
[root@samba samba]# cat smb.conf
    workgroup = MYGROUP
    server string = Samba Server Version %v
    log file = /var/log/samba/log.%m
    max log size = 50
    security = user
    passdb backend = tdbsam
    load printers = yes
    cups options = raw
    comment = Shared Files
    path = /rhcefiles
    browseable = no
    writable = yes
    valid users = @rhce
[root@samba ~]# useradd -g rhce alice
[root@samba ~]# useradd -g rhce bob
[root@samba ~]# smbpasswd -a alice
[root@samba ~]# smbpasswd -a bob

Now on the client I can simply use cifs-utils to mount the remote share.

[root@client ~]# mkdir /sambashare
[root@client ~]# mount -t cifs // /sambashare/ -o rw,username=alice,password=redhat
[root@client ~]# cd /sambashare/
[root@client sambashare]# ls
ls: reading directory .: Permission denied

Now we're coming into some SELinux issues. Even though the user is part of the group, has authenticated and has successfully mounted the remote share, they are not allowed to do anything useful with it. Let's fix that.

An important tip for the exam is that SELinux directives for SAMBA are in the smb.conf file that ships with the package. Make sure to reference it since it speaks directly to which file context we should use for this particular case.

If you create a new directory, such as a new top-level directory, label it with samba_share_t so that SELinux allows Samba to read and write to it. Do not label system directories, such as /etc/ and /home/, with samba_share_t, as such directories should already have an SELinux label.

However, that document suggests using chcon but that is only temporary: changes made using chcon will not persist a reboot for the exam so we'll have to use semanage to make the changes permanent. Remember to check the man page for semanage-fcontext to get the example you should use to make the change permanent.

remember to run restorecon after you set the file context
   Add file-context for everything under /web
   # semanage fcontext -a -t httpd_sys_content_t "/web(/.*)?"
   # restorecon -R -v /web

So modifying the example, we end up with this:

[root@samba /]# ls -Z | grep rhce
drwxrwsr-x. root rhce unconfined_u:object_r:default_t:s0 rhcefiles
[root@samba /]# semanage fcontext -a -t samba_share_t "/rhcefiles(/.*)?"
[root@samba /]# restorecon -R /rhcefiles/
[root@samba /]# ls -Z | grep rhce
drwxrwsr-x. root rhce unconfined_u:object_r:samba_share_t:s0 rhcefiles

Now from the client we can use the resource as we would expect:

[root@client sambashare]# cd
[root@client ~]# cd /sambashare/
[root@client sambashare]# ls
[root@client sambashare]# echo lol > testfile 
[root@client sambashare]# cat testfile 

Use Kerberos to authenticate access to shared directories

The Ubuntu help forums have a good guide on the subject. The heart of it comes down to editing the smb.conf file this way:

security = ADS
encrypt passwords = yes
#Samba 3.0 requires "kerberos keytab = yes" instead of the next line.
#Samba < 3.5 might require "kerberos method = system keytab" instead of the next line.
kerberos method = secrets and keytab
password server = kdc.fdqn


For this section we'll configure postfix.

##Configure a system to forward all email to a central mail server

CertDepot has a great guide on this objective which I'll be following here. The important configuration option that we'll specify is the relayhost option in postfix's main.cf file.


Configure key-based authentication

First, I'll generate a keypair on my local machine:

steven@jupiter ~> ssh-keygen -C steven@pequeno.in
steven@jupiter ~> ls .ssh/ | grep net

Now on the server, I'll make the .ssh folder which will hold my public key.

[root@netserv ~]# mkdir .ssh; cd $_
[root@netserv .ssh]# vim authorized_keys 
[root@netserv .ssh]# cat authorized_keys 
ssh-rsa AAAAB3...iQ1 steven@pequeno.in

And now I can log into the remote machine without a password:

steven@jupiter ~> ssh root@
Last login: Wed Nov 26 00:19:33 2014 from jupiter
[root@netserv ~]# 

Configure additional options described in documentation

There are a lot of various things we can do to configure SSH, but let's discuss making our server more secure.

[root@netserv ~]# vim /etc/ssh/sshd_config 
PermitRootLogin no
PasswordAuthentication no

We should not allow the root account to be accessed locally. By disabling this we can make sure that an attacker does not have an easy target to brute force.

We should disable password authentication all together. We should also use something like fail2ban to make sure that an attacker is blocked from making too many failed attempts against the server. But even without fail2ban we can make sure passwords cannot be guessed.

Most ssh configurations will be made in this way. Read the configuration file as most options have explanations.


Synchronize time using other NTP peers

When you install NTP it will come with a set of servers already defined. Normally for practice I will comment those out and use another time server. In this example I'm using one provided by the US Naval Observatory.

[root@netserv ~]# yum install ntp -y
server navobs1.wustl.edu
[root@netserv ~]# systemctl enable ntpd.service 
ln -s '/usr/lib/systemd/system/ntpd.service' '/etc/systemd/system/multi-user.target.wants/ntpd.service'
[root@netserv ~]# systemctl start ntpd.service 
[root@netserv ~]# ntpq -p
 remote           refid      st t when poll reach   delay   offset  jitter
*navobs1.wustl.e .GPS.            1 u   37   64    1   31.348   -4.922   0.000

Fork me on GitHub