Bonded interfaces in RHEL

Here's how to create a bonded interface in RHEL 6

This bond is working in failover mode with VLAN tagging over interfaces eth0 & eth1:


ifcfg-bond0

DEVICE=bond0
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
BONDING_OPTS="mode=1 miimon=100"

ifcfg-bond0.x  (where x is vlan number)

DEVICE=bond0.x
IPADDR=y.y.y.y
NETMASK=y.y.y.y
GATEWAY=y.y.y.y
USERCTL=no
BOOTPROTO=static
ONBOOT=yes
VLAN=yes
IPV6INIT=no

ifcfg-ethx (where x is 0 or 1)

DEVICE=ethx
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no

Note that creating bonding.conf in /etc/modprobe.d is no longer required in the latest RHEL 6 releases.

Restart network services and the bond will be created. Make sure all previous non-bonded interfaces are destroyed before restarting network services otherwise they will interfere with the bond creation. A reboot is the easiest way to sort this out.

To check the status of a bonded interface:

cat /proc/net/bonding/bond0

Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)

Bonding Mode: fault-tolerance (active-backup)

Primary Slave: None

Currently Active Slave: eth1

MII Status: up

MII Polling Interval (ms): 100

Up Delay (ms): 0

Down Delay (ms): 0

Slave Interface: eth1

MII Status: up

Speed: 10000 Mbps

Duplex: full

Link Failure Count: 1

Permanent HW addr: 00:9c:02:24:7c:dc

Slave queue ID: 0

Slave Interface: eth0

MII Status: up

Speed: 10000 Mbps

Duplex: full

Link Failure Count: 0

Permanent HW addr: 00:9c:02:24:7c:d8

Slave queue ID: 0

To force service to a specific interface:

 ifenslave --change-active bond0 eth0

To remove an interface from a bond

 ifenslave -d bond0 eth0

To add an interface back into a bond

 ifenslave bond0 eth0

Linux Disk Benchmarking

A really quick way to see the write performance of your disk.....
Write a 512MB file


dd bs=1M count=512 if=/dev/zero of=test conv=fdatasync ; rm test

And it tells you how fast it did it

512+0 records in

512+0 records out

536870912 bytes (537 MB) copied, 2.74433 s, 196 MB/s

The fdatasync option forces dd to do a sync to ensure data is really written to disk before it exits which should give a more realistic figure.

Server imaging with SelfImage & Windows PE

Here's how to restore an image made with the SelfImage utility using a Windows PE boot environment. Previously I have used BartPE but were finding that I kept needing to rebuild the image to support the various different NIC & array drivers used on newer servers.

A recent WinPE enviroment has these drivers already built in. So if the server can build directly from the Windows server 2008 CD then it can use the WinPE disk without modification. I've been using WinPE version 2 for what it's worth. Version 1 was based on an earlier Windows version so doesn't have the newer drivers in it.

This post assumes you have access to a WinPE environment boot cd (or iso) and also that it has been modified to include the SelfImage utility.

• Boot into WinPE environment 
• Configure interface (n is the interface number, x is IP, y is netmask, z is gateway)
• netsh interface ip set address name="Local Area Connection n" static x.x.x.x y.y.y.y z.z.z.z 1
• Map a drive to where the image is loaded (remember the username is specified as server\usersname)
• net use e: \\server\share
• Configure a primary partition on the disk using diskpart
• select disk 0
• clean
• create partition primary size=<size in mb>
• select partition 1
• active
• assign letter=c:
• Run the selfimage tool and select the output to be \Device\Harddisk0\Partition1 C:
• Before rebooting fix the boot manager otherwise the server will blue screen
• bcdedit /set {default} device partition=c:
• bcdedit /set {default} osdevice partition=c:
• bcdedit /set {bootmgr} device partition=c:

Reboot server and the job is done.

Note that on reboot Windows may complain that it was not shutdown properly, this can be ignored and Windows booted normally. Also once logged in you may find Windows installs some more device drivers, let it finish and do one more reboot to finish the job off.

The joys of SSH tunnels

We all now how to use a simple SSH tunnel to be able to remote desktop to a machine hidden behind a firewall.

For example create a tunnel to server1 to be able to RDP onto server2. In this example we'll use port 6000 so make sure nothing is running on that port on server1 first:

• Use putty to ssh onto server1
• Within putty create a local tunnel with a source port of 6000 and a destination port of server2:3389
• Now point your rdp client to localhost:6000 and your connection to server2 magically opens

Now let's extend that to 2 tunnels. In this example we can't directly reach server2 that can reach our windows box called server3. We'll create a tunnel to server1, then another tunnel from server1 to server2 and finally server2 will create the connection to server3. Again we'll use port 6000 so check it's not being used on either server1 or server2.

• Use putty to ssh onto server1
• Within putty create a local tunnel with a source port of 6000 and a destination port of localhost:6000
• Now connect to server2 from server1 like this:

ssh -L 6000:server3:3389 server2

• Once your ssh session opens fire up remote desktop and point to localhost:6000 and your connection to server3 appears before your eyes.

SSH tunnels are truely wonderful.

How To Clear /var/adm/wtmpx

On Solaris if /var/adm/wtmpx has got so big as to break the last command or fill up all your disk space, here's how to fix it:

This will create a new wtmpx file with only the last 100 entries in it.

# /usr/lib/acct/fwtmp < /var/adm/wtmpx | tail -100 > /tmp/wtmpx.ascii
# /usr/lib/acct/fwtmp -ic < /tmp/wtmpx.ascii > /var/adm/wtmpx
# rm /tmp/wtmpx.ascii

fwtmp parses the wtpmx file and dumps out the last 100 entries in ascii to a file.
Rerun fwtmp with the -ic params will read this file and create an equivalent binary wtmpx file to replace yours.

Note: If fwtmp is not available on your system you'll need to install the system accounting packages SUNWaccu & SUNWaccr.

How To Share A Disk Using iSCSI

Here's how to share a disk using iSCSI from a RHEL 6 server.
Note that this is the most basic way of setting iSCSI up and is really only useful for familiarisation of using iSCSI clients.

• Create an LVM logical volume on the desired disk. In this example /dev/sdb 

pvcreate /dev/sdb

vgcreate vg_sdb /dev/sdb

lvcreate --extents 100%FREE --name lv_sdb vg_sbd 

• Install the iSCSI apps & daemon and start it up 

yum install scsi-target-utils

service tgtd start

chkconfig tgtd on 

• Add an entry to /etc/tgt/targets.conf to make the disk available.

<target iqn.2012.05.net.home:san.target1>

backing-store /dev/vg_sdb/lv_sdb

</target> 

• Open the iSCSI ports on the firewall

tcp / udp 3260 & tcp / udp 860

That's it, the disk should now be discoverable as iqn.2012.05.net.home:san.target1.
There is no security set on the disk so you should be able to just log into it.

Cisco Network Registrar via the CLI

Can't get the GUI running then use the command line.

/export1/nwreg2/usrbin/nrcmd 

List the zones available:

nrcmd> zone list 

List the records in the knon.dtv zone:

nrcmd> zone knon.dtv listrr 

List the records in a reverse lookup zone:

nrcmd> zone 10.in-addr.arpa listrr 

This example adds myhost.knon.dtv / 10.2.3.4.
Add the hostname to the knon.dtv zone:

nrcmd> zone knon.dtv. addhost myhost 10.2.3.4 

Add the corresponding PTR record (done automatically with the GUI):

nrcmd> zone 10.in-addr.arpa. addrr 4.3.2 PTR myhost.knon.dtv

Tip: To remove resource records

nrcmd> zone 10.in-addr.arpa. removerr 4.3.2.10.in-addr.arpa

Save your records & reload the DNS to activate:

nrcmd> save
nrcmd> dns reload

To force the secondary DNS to update the zone with your new records, log onto it and execute:

nrcmd> zone knon.dtv forceXfer secondary
nrcmd> zone 10.in-addr.arpa forceXfer secondary

 Don't forget to refresh the secondary DNS caches to pick up the new records:

rndc reload lang.dtv
rndc reload 10.in-addr.arpa

Help! I'm locked out of my Cacti server

There's two stages of locked out.

• You've forgotten just the admin password to the Cacti web interface
• You've forgotten the admin password and you've forgotten the mysql database password.

The first is relatively easy to sort

Log into the cacti database and run this

mysql -u root -p <cacti db>
update user_auth set password=md5('newadminpassword') where username='admin';

Now of course you might now know which what the cacti database is called in which case

mysql -u root -p
mysql> show databases;

Look for the database which has some snmp tables in it

mysql> connect <database>;
mysql> show tables;

Now if you can't remember the root password to your Cacti database you need to reset that first. It's not the same as the root password on the server. This is how to reset it.

Create a new ini file, let's call it mysql-ini containing

UPDATE mysql.user SET Password=PASSWORD('newpassword') WHERE User='root';
FLUSH PRIVILEGES;

Shutdown mysql if it is running and restart it directly with this ini file:

mysqld_safe --init-file=mysql-ini &

Now login with the new root password then shut down mysql again. It can now be restarted with the original start script (and ini file) and you can log in with the new root password.  

Configuring Cacti on RHEL 6

This is the Cacti installation from EPEL so enable the EPEL repo.
Also enable the rhel-6-server-optional channel in redhat.repo as it's required for php-snmp.

Install the packages cacti, mysql-server, httpd, net-snmp-utils

Turn on and chkconfig mysqld and httpd
(Amend ServerName in httpd.conf if hostname is not resolvable)

 Set cacti passwords and create the cacti database

mysqladmin -u root password rootsecretpassword
mysql -u root -p -e 'create database cacti'

mysql -u root -p
mysql> GRANT ALL ON cacti.* TO cacti@localhost IDENTIFIED BY 'cactisecretpassword';
mysql> FLUSH privileges;

Replace the default /etc/snmp/snmpd.conf with this as it's too secure.

com2sec local     localhost           public
group MyRWGroup v1         local
group MyRWGroup v2c        local
group MyRWGroup usm        local
view all    included  .1                     80
access MyRWGroup ""  any  noauth  exact   all    all    none
syslocation My House
syscontact Me

Turn on and chkconfig snmpd

Walk the snmp mib and verify it returns active interfaces; otherwise it's not configured right.

snmpwalk -v 1 -c public localhost IP-MIB::ipAdEntIfIndex

Configure the cacti database

mysql -u cacti -p  cacti < /usr/share/doc/cacti-0.8.7i/cacti.sql

 Edit /etc/cacti/db.php

$database_username = "cacti";
$database_password = "cactisecretpassword";

Edit /etc/httpd/conf.d/cacti.conf

Alias /cacti    /usr/share/cacti

<Directory /usr/share/cacti/>
        Order Deny,Allow
        Deny from all
        Allow from all
</Directory>

Restart httpd

Uncomment poller cron job in /etc/cron.d/cacti

Your Cacti server awaits:

http://yourserver/cacti  & login as admin / admin

How To Build a PXE Server

So you've created some kickstart files (here) now wouldn't it be nice to have the client net boot and the installation served up over the network to create a fully automated build.

This is how to be a build a PXE server on RHEL 6. I'm assuming the server has been setup for anonymous FTP to serve out the install media from /pub/inst and the kickstart files from /pub/ks. You could also use HTTP.

Install packages tftp-server & dhcp.

Enable TFTP server in /etc/xinetd.d/tftp

disable=no

And turn it on

service xinetd start

chkconfig xinetd on

Configure a DHCP subnet in /etc/dhcp/dhcpd.conf
( PXE specific stuff in red; next-server is the PXE server)

# Global Options
Allow booting;
Allow bootp;
authoritative;
# Subnet definition
subnet 192.168.122.0 netmask 255.255.255.0 {
option routers 192.168.122.2;
option subnet-mask 255.255.255.0;
option domain-name “example.com”;
option domain-name-servers 192.168.122.2;
default-lease-time 21600;
max-lease-time 43200;
range dynamic-bootp 192.168.122.100 192.168.122.200;
filename “pxelinux.0”;
next-server 192.168.122.2;
}

And turn it on

service dhcpd start

chkconfig dhcpd on 

Create TFTP directories and copy files

mkdir /var/lib/tftpboot/rhel6
mkdir /var/lib/tftpboot/pxelinux.cfg
cp /var/ftp/pub/inst/images/pxeboot/vmlinuz /var/lib/tftpboot/rhel6
cp /var/ftp/pub/inst/images/pxeboot/initrd.img /var/lib/tftpboot/rhel6
cp /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot/
cp /usr/share/syslinux/menu.c32 /var/lib/tftpboot/

We'll create a default client config file which is served to everyone. You can make client specific ones and place them in the same directory.
Create /var/lib/tftpbook/pxelinux.cfg/default

timeout 100
default menu.c32
menu title Boot Menu
label 1
   menu label ^ 1) Boot next available boot device
   localboot
label 2
   menu label ^ 2) RHEL 6 (with kickstart)
   kernel rhel6/vmlinuz
   append initrd=rhel6/initrd.img ks=ftp://192.168.122.2/pub/ks/server1.cfg
label 3
   menu label ^ 3) RHEL 6 (interactive)
   kernel rhel6/vmlinuz
   append initrd=rhel6/initrd.img ip=dhcp repo=ftp://192.168.122.2/pub/inst

By placing localboot as the first option we avoid an accidental build if the first boot device is set to network as we default to booting the next available device.
Option 2 is an example of a fully automated install with a kickstart file
Option 3 will deliver an interactive installation
If it's all working you'll see something like this from a PXE booting client.

Building RHEL with Kickstart Files

How to build RHEL 6 on an HP Server from DVD installation with a kickstart file on a USB key:

Assume machine has already been hardware mirrored.
USB sticks are enumerated as sda so kickstart file has to use sdb for internal disk.
Those dodgy Dektek sticks enumerate as 2 drives (sda & sdb) so use sdc for internal disk which is why example below uses sdc.

Boot machine of DVD, highlight first install / upgrade option and press tab to edit. Add this to end of line:

ks=hd:sda1:/kickstart-file.cfg

Sample kickstart here, based on an autogen'ed basic server  install. Bit's I modified are in red.

# Kickstart file automatically generated by anaconda.
#version=DEVEL
install
cdrom
lang en_GB.UTF-8
keyboard uk
network --onboot no --device eth0 --noipv4 --noipv6
network --onboot no --device eth1 --noipv4 --noipv6
network --onboot no --device eth2 --noipv4 --noipv6
network --onboot no --device eth3 --noipv4 --noipv6
network --onboot yes --device eth4 --bootproto static --ip 10.182.46.6 --netmask 255.255.255.0 --gateway 10.182.46.252 --noipv6 --hostname know-msvarnish-01.knon.dtv
network --onboot no --device eth5 --noipv4 --noipv6
rootpw  --iscrypted $6$woBbCXzYdNZUmerV$mz9mELXXNXBLL.k5MpL1S7nn7c3Dlf9i7sWUTRX7W2D0qjQY6sFBrSJA.wi2EfbL.YYz.gGf8AcwwOPEUuYgw1
firewall --service=ssh
authconfig --enableshadow --passalgo=sha512
selinux --enforcing
timezone --utc Europe/London
bootloader --location=mbr --driveorder=sdc --append=" rhgb crashkernel=auto quiet"
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
ignoredisk --drives=sda,sdb
clearpart --all --initlabel --drives=sdc
part /boot --fstype=ext4 --size=500 --ondisk=sdc
part pv.008002 --grow --size=1 --ondisk=sdc
volgroup vg_knowmsvarn --pesize=4096 pv.008002
logvol /var --fstype=ext4 --name=lv_var --vgname=vg_knowmsvarn --size=51200
logvol swap --name=lv_swap --vgname=vg_knowmsvarn --size=34288
logvol / --fstype=ext4 --name=lv_root --vgname=vg_knowmsvarn --size=100 --grow 

#repo --name="Red Hat Enterprise Linux"  --baseurl=cdrom:sr0 --cost=100 

%packages
@base
@client-mgmt-tools
@console-internet
@core
@debugging
@directory-client
@hardware-monitoring
@java-platform
@large-systems
@network-file-system-client
@performance
@perl-runtime
@server-platform
@server-policy
pax
python-dmidecode
oddjob
sgpio
certmonger
pam_krb5
krb5-workstation
perl-DBD-SQLite
%end

Need to set the language to UK.
The --initlabel to clearpart should stop it asking for confirmation of initialising a new disk
The ignoredisk param tells installer to ignore usb sticks during partitioning
Make sure the --ondisk, --drives, --driveorder all match up with the enumeration of the internal disk
The --grow option tells that logvol to fill the rest of the disk.
Comment out the repo line as it defaults to DVD and didn't seem to work with it in.