Linux | ULTips - Unix, Linux administration tips

Installing Proftpd on Red-Hat EL 5 or 6

March 8, 2012JeyLinux, Unix - Linux ...compile, ftp, make, Proftpd

Installing proftpd on Red-Hat EL 5 or 6

Proftpd is one of the best an most wildly used ftp server on unix-like platforms.
You can find proftpd already packaged for many distributions.

Packaged software are easy to install and time saving but if you want the latest version and/or a custom install with specific options you will have to go through source compiling.

First download the latest version:

# wget "ftp://ftp.solnet.ch/mirror/proftpd.org/distrib/source/proftpd-1.3.4a.tar.bz2"
Download the md5 file to check the file data integrity:
# cat proftpd-1.3.4a.tar.bz2.md5
4e3235dc1ef95d36e59721d70c5c489c proftpd-1.3.4a.tar.bz2
# md5sum proftpd-1.3.4a.tar.bz2
4e3235dc1ef95d36e59721d70c5c489c proftpd-1.3.4a.tar.bz2

Unzip and mv the sources on /usr/local/src/ :

# bunzip2 proftpd-1.3.4a.tar.bz2
# tar xvf proftpd-1.3.4a.tar.gz
# mv proftpd-1.3.4a /usr/local/src/

Check if you have gcc rpm :

# rpm -qa|grep gcc
gcc-4.4.5-6.el6.x86_64

You may need to install other packages depending on what you already installed and the the proftpd options you will choose.

# cd /usr/src/local/proftpd-1.3.4a/

Read the install documentation:

# more INSTALL

Run the configuration.
I use –prefix= force proftd to install on /usr/local/proftpd and –enable-autoshadow to be able to use standard and shadow passwords.

# ./configure --prefix=/usr/local/proftpd --enable-autoshadow
# echo $?
0

Build proftpd :

# make
echo \#define BUILD_STAMP \"`date +"%a %b %e %Y %H:%M:%S %Z"`\" > include/buildstamp.h
cd lib/ && make lib
make[1]: Entering directory `/usr/src/proftpd-1.3.4a/lib'
gcc -DHAVE_CONFIG_H -DLINUX -I.. -I../include -O2 -Wall -c pr_fnmatch.c
gcc -DHAVE_CONFIG_H -DLINUX -I.. -I../include -O2 -Wall -c sstrncpy.c
...
...
gcc -DHAVE_CONFIG_H -DLINUX -I.. -I../include -O2 -Wall -c ftpwho.c
make[1]: Leaving directory `/usr/src/proftpd-1.3.4a/utils'
gcc -L./lib -o ftpcount utils/ftpcount.o utils/scoreboard.o utils/misc.o -lsupp -lcrypt -ldl
gcc -L./lib -o ftpdctl src/ftpdctl.o src/pool.o src/str.o src/netaddr.o src/log.o src/ctrls.o -lsupp -lcrypt -ldl -L./lib/libcap -lcap
gcc -L./lib -o ftpscrub utils/ftpscrub.o utils/scoreboard.o utils/misc.o -lsupp -lcrypt -ldl
gcc -L./lib -o ftpshut utils/ftpshut.o -lsupp -lcrypt -ldl
gcc -L./lib -o ftptop utils/ftptop.o utils/scoreboard.o utils/misc.o -lsupp -lcrypt -ldl
gcc -L./lib -o ftpwho utils/ftpwho.o utils/scoreboard.o utils/misc.o -lsupp -lcrypt -ldl
[root@frsu0069 proftpd-1.3.4a]# echo $?
0

Install it:

# make install
cd lib/ && make lib
make[1]: Entering directory `/usr/src/proftpd-1.3.4a/lib'
make[1]: Nothing to be done for `lib'.
make[1]: Leaving directory `/usr/src/proftpd-1.3.4a/lib'
cd src/ && make src
make[1]: Entering directory `/usr/src/proftpd-1.3.4a/src'
make[1]: Nothing to be done for `src'.
make[1]: Leaving directory `/usr/src/proftpd-1.3.4a/src'
cd modules/ && make static
make[1]: Entering directory `/usr/src/proftpd-1.3.4a/modules'
make[1]: Leaving directory `/usr/src/proftpd-1.3.4a/modules'
test -z """" -a -z """" || (cd modules/ && make shared)
make[1]: Entering directory `/usr/src/proftpd-1.3.4a/lib/libcap'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/usr/src/proftpd-1.3.4a/lib/libcap'
...
...
cd lib/ && make install
make[1]: Entering directory `/usr/src/proftpd-1.3.4a/lib'
make[1]: Nothing to be done for `install'.
make[1]: Leaving directory `/usr/src/proftpd-1.3.4a/lib'
/usr/bin/install -c -o root -g root -m 0644 config.h /usr/local/proftpd/include/proftpd/config.h
cd include/ && make install
make[1]: Entering directory `/usr/src/proftpd-1.3.4a/include'
make[1]: Leaving directory `/usr/src/proftpd-1.3.4a/include'
/usr/bin/install -c -o root -g root -m 0644 proftpd.pc /usr/local/proftpd/lib/pkgconfig/proftpd.pc
test -z """" || (cd locale/ && make install)
[root@frsu0069 proftpd-1.3.4a]# echo $?
0

Proftpd is now installed :

# ls /usr/local/proftpd
bin etc include lib libexec sbin share var

Copy the man pages in the appropriate directories (so you can use them) :

# cd /usr/local/proftpd/share/man/man1
# cp * /usr/share/man/man1/
# cd ../man5
# cp * /usr/share/man/man5/
# cd ../man8
# cp * /usr/share/man/man8/

Test proftpd with the embedded test procedure:

# sh /usr/local/src/proftpd-1.3.4a/sample-configurations/PFTEST.install
Sample test files successfully installed in /tmp/PFTEST.

The script purpose is to create a sample configuration file “PFTEST.conf” in /tmp/PFTEST
Run proftpd :

# /usr/local/proftpd/sbin/proftpd -n -d 5 -c /tmp/PFTEST/PFTEST.conf

(-n for “don’t start as daemon” and -d 5 for debug level “medium”)
Now connect with an ftp client to port 2012 with login proftpd and password proftpd :
on the server you will something like :

# /usr/local/proftpd/sbin/proftpd -n -d 5 -c /tmp/PFTEST/PFTEST.conf
FRSU0001 proftpd[18003]: using TCP receive buffer size of 87380 bytes
FRSU0001 proftpd[18003]: using TCP send buffer size of 16384 bytes
FRSU0001 proftpd[18003]: : using 'mod_ident.c' section at line 32
FRSU0001 proftpd[18003]: : adding section for resolved path '/'
FRSU0001 proftpd[18003] 192.168.0.111:
FRSU0001 proftpd[18003] 192.168.0.111: Config for ProFTPD TEST Installation:
FRSU0001 proftpd[18003] 192.168.0.111: /
FRSU0001 proftpd[18003] 192.168.0.111: AllowOverwrite
FRSU0001 proftpd[18003] 192.168.0.111: TransferLog
FRSU0001 proftpd[18003] 192.168.0.111: RequireValidShell
FRSU0001 proftpd[18003] 192.168.0.111: UseFtpUsers
FRSU0001 proftpd[18003] 192.168.0.111: WtmpLog
FRSU0001 proftpd[18003] 192.168.0.111: Umask
FRSU0001 proftpd[18003] 192.168.0.111: DefaultServer
FRSU0001 proftpd[18003] 192.168.0.111: UserID
FRSU0001 proftpd[18003] 192.168.0.111: UserName
FRSU0001 proftpd[18003] 192.168.0.111: GroupID
FRSU0001 proftpd[18003] 192.168.0.111: GroupName
FRSU0001 proftpd[18003] 192.168.0.111: AuthUserFile
FRSU0001 proftpd[18003] 192.168.0.111: AuthGroupFile
FRSU0001 proftpd[18003] 192.168.0.111: PidFile
FRSU0001 proftpd[18003] 192.168.0.111: TransferLog
FRSU0001 proftpd[18003] 192.168.0.111: RequireValidShell
FRSU0001 proftpd[18003] 192.168.0.111: UseFtpUsers
FRSU0001 proftpd[18003] 192.168.0.111: WtmpLog
FRSU0001 proftpd[18003] 192.168.0.111: IdentLookups
FRSU0001 proftpd[18003] 192.168.0.111: Umask
FRSU0001 proftpd[18003] 192.168.0.111: ProFTPD 1.3.4a (maint) (built Thu Mar 8 2012 14:02:45 CET) standalone mode STARTUP
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): session requested from client in unknown class
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): connected - local : ::ffff:192.168.0.111:2021
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): connected - remote : 192.168.0.20:3392
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): FTP session opened.
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching PRE_CMD command 'USER proftpd' to mod_core
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching PRE_CMD command 'USER proftpd' to mod_core
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching PRE_CMD command 'USER proftpd' to mod_delay
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching PRE_CMD command 'USER proftpd' to mod_auth
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching CMD command 'USER proftpd' to mod_auth
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching POST_CMD command 'USER proftpd' to mod_delay
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching LOG_CMD command 'USER proftpd' to mod_log
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching PRE_CMD command 'PASS (hidden)' to mod_core
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching PRE_CMD command 'PASS (hidden)' to mod_core
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching PRE_CMD command 'PASS (hidden)' to mod_delay
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching PRE_CMD command 'PASS (hidden)' to mod_auth
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): dispatching CMD command 'PASS (hidden)' to mod_auth
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): user 'proftpd' authenticated by mod_auth_file.c
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]):
FRSU0001 proftpd[18006] 192.168.0.111 (192.168.0.20[192.168.0.20]): Config for ProFTPD TEST Installation:
…

Proftpd is working!
Now you just have to configure and secure Proftpd …

Glances: installing and tuning default thresholds on RedHat ES6

February 3, 2012JeyLinux, Unix - Linux ...glances, limits, linux, monitoring, redhat, thresholds, tuning

Glances is a free (under LGPL license) tool, written by Nicolas HENNION (aka Nicolargo) in Python and using libstatgrab to monitor your system.

I like the concept and it’s very easy to install and to handle.

I used to work a lot with top and sar on linux but now if I have the opportunity to install Glances, I do it.

Conceptually Glances is more like nmon on AIX, or glance on HP-UX (quite the same name, what a coincidence?).

Installation:

Download Glances (1.3.7 on this post) :

Compile glance (you will need gcc) :

# tar ztf glances-1.3.7.tar.gz
# cd glances-1.3.7/
# ./configure
# make
# make install

Then install libstatgrab and pystatgrab librairies.

# rpm -Uvh libstatgrab-0.15-1.el6.rf.x86_64.rpm
# rpm -Uvh pystatgrab-0.5-9.el6.x86_64.rpm

You will not found the 2 rpm on RH ES6 CDs but you can find them on rpm.pbone.net for instance.

Don’t try to install pystatgrab from the sources: it won’t work as it uses pkg-config and pkg-config is not able to locate the libs on RedHat (don’t now why?).

Run Glances:

# glances.py

Glances: installing and tuning

From the screenshot one can see that the server’s main statistics are available on one screen:

Cpu, load, Memory, Network, Disk I/O, Disk space, and process

For the monitoring part you have colors for current thresholds overflow and an history of the last 3 to 10 warning and critical alerts.

For more information see the man page or Nicolargo’s web site (in French):

If you want to change the default limits, you will have to edit glances.py

# locate glances.py
/usr/local/bin/glances.py
# vi /usr/local/bin/glances.py

Find glancesLimits class :

class glancesLimits():
"""
Manage the limit OK,CAREFUL,WARNING,CRITICAL for each stats
"""
# The limit list is stored in an hash table:
#  limits_list[STAT] = [ CAREFUL , WARNING , CRITICAL ]
# Exemple:
#  limits_list['STD'] = [ 50, 70 , 90 ]
 
__limits_list = {   #           CAREFUL WARNING CRITICAL
'STD':  [50,    70,     90],
'LOAD': [0.7,   1.0,    5.0]
}

And modify the default limits, for instance:

__limits_list = {   #           CAREFUL WARNING CRITICAL
'STD':  [40,    50,     70],
'LOAD': [0.2,   0.5,    1.0]

Glances: tuning default thresholds

Now you can see that while cpu% user if still lower than 90, the color is now red, meaning CRITICAL and the process who were in CAREFUL state before (<70) are now in WARNING state. Same comment on the load.

How to setup iSCSI on Linux (RedHat)

September 9, 2011JeyLinux, Unix - Linux ...initiator, IQN, iscsi, LUN, LUN masking, NAS, qnap, target, VMware, vswitch4 Comments

Definitions:

iSCSI initiator : the endpoint that initiates a iSCSI session. An iSCSI initiator sends SCSI commands over an IP network. It’s the client endpoint.

iSCSI Target : refers to a storage resource located on an iSCSI server (most of the time it’s a “storage array”). It’s the server endpoint.

LUNs (Logical Number Units): number used to identify a logical unit, which is a device addressed by the SCSI protocol (thus Fiber Channel or iSCSI). It usually represents slices of large RAID disk arrays.

IQN (iSCSI Qualified Name) : iSCSI name of the target or initiator.

On the Storage Server:

Enable and configure the iSCSI Target on your storage server.

Mine is a QNAP Turbo NAS. I’ve got 1 target with 5 LUNs configured.

SCSI Portal
 
X Enable iSCSI Target Service
 
iSCSI Service Port:           3260
 
mytarget (iqn.2004-04.com.qnap:ts-859:iscsi.mytarget.c5884b)     Connected
 
id:0 - lun1 ( 2024.00 GB)               Enabled
 
id:1 - lun2 ( 2024.00 GB)               Enabled
 
id:2 - lun3 ( 2024.00 GB)               Enabled
 
id:3 - lun4 ( 2024.00 GB)               Enabled
 
id:4 - lun5 ( 1804.13 GB)               Enabled

I have two network interfaces:

1- for QNAP management, IP : 10.0.0.5

2- iSCSI access, directly connected to the server : 192.168.0.1

For more security you can enable “LUN masking”. It will restrain iSCSI target to be accessed to only by the initiator of your client (the client initiator name IQN can be found on /etc/iscsi/initiatorname.iscsi).

On the Linux client (see tips ** for VMware configuration) :

Install “iscsi-initiator-utils” on the server that will connect to the iSCSI volume:

# rpm –Uvh iscsi-initiator-utils-6.2.0.865-6.el5.x86_64.rpm

Set up iscsi automatic start on boot and start iscsi services:

# chkconfig iscsid on
# service iscsid start
# chkconfig iscsi on
# service iscsi start

Discover your iSCSI targets:

# iscsiadm -m discovery -t st -p 192.168.0.1

In my case it will show 2 targets (one for each network connection):

192.168.0.1:3260 iqn.2004-04.com.qnap:ts-859:iscsi. mytarget.c5884b
10.0.0.5:3260 iqn.2004-04.com.qnap:ts-859:iscsi. mytarget.c5884b

I have 2 routes for the same target.

Log to the target through IP 192.168.0.1 :

# iscsiadm -m node -T iqn.2004-04.com.qnap:ts-859:iscsi.mytarget.c5884b -p 192.168.0.1 -l

Add automatic login at boot :

# iscsiadm -m node -T iqn.2004-04.com.qnap:ts-859:iscsi.mytarget.c5884b -p 192.168.0.1 --op update -n node.startup -v automatic

As I have another access to the target, I will disable it in order to not disturb the previous configuration:

# iscsiadm -m node -T iqn.2004-04.com.qnap:ts-859:iscsi.mytarget.c5884b -p 10.0.0.5 --logout
# iscsiadm -m node -T iqn.2004-04.com.qnap:ts-859:iscsi.mytarget.c5884b -p 10.0.0.5 --op update -n node.startup -v manual

At this point you will see the iSCSI LUNs as block devices on your client.

On my system the five iSCSI block devices are /dev/sdc, sdd, sde, sdf and sdg.

You will have create partitions and format them to either standard Linux partition or LVM partition.

I chose LVM because I need large file systems.

You can use parted or fdisk (if < 2To), see article: ” How To Make Partitions Larger Than 2To With Parted GPT Support“.

So here is the result:

# fdisk -l
Disk /dev/sdc: 2173.2 GB, 2173253451776 bytes
Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1      264216  2122314988+  8e  Linux LVM
Disk /dev/sdd: 2173.2 GB, 2173253451776 bytes
Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1      264216  2122314988+  8e  Linux LVM
Disk /dev/sde: 2173.2 GB, 2173253451776 bytes
Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1      264216  2122314988+  8e  Linux LVM
Disk /dev/sdf: 2173.2 GB, 2173253451776 bytes
Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1      264216  2122314988+  8e  Linux LVM
Disk /dev/sdg: 1937.1 GB, 1937169711104 bytes
Device Boot      Start         End      Blocks   Id  System
/dev/sdg1               1      235514  1891766173+  8e  Linux LVM

Then create you LVM volume group and logical volume :

# pvcreate /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1
# vgcreate -s 256M vol_vg /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1
# lvcreate -l 28672 vol_vg -n vol_lv1
# lvcreate -l 10924 vol_vg -n vol_lv2

File system creation:

# mkfs -t ext3 -b 4096 -N 100000 /dev/vol_vg/vol_lv1 -L VOL1
# mkfs -t ext3 -b 4096 -N 100000 /dev/vol_vg/vol_lv2 -L VOL2

Then mount the file systems:

# mkdir –p /VOL1 /VOL2
# mount -t ext3 /dev/vol_vg/vol_lv1 /VOL1
# mount -t ext3 /dev/vol_vg/vol_lv2 /VOL2
# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vol_vg-vol_lv1 7.0T  6.3T  395G  95% /VOL1
/dev/mapper/vol_vg-vol_lv2 2.7T  1.5T  1.2T  57% /VOL2

If you will want to automatically mount your iSCSI file systems at startup (see article: How To Use UUID And Blkid To Manage Devices).

Get the UUID for each file system:

# blkid /dev/vol_vg/vol_lv1
/dev/vol_vg/vol_lv1: LABEL="VOL1" UUID="4a496f92-6840-4736-a0d5-5b9916113835" SEC_TYPE="ext2" TYPE="ext3"
# blkid /dev/vol_vg/vol_lv2
/dev/vol_vg/vol_lv2: LABEL="VOL2" UUID="cab5e3ec-4797-4227-98e8-e9bca3c3f766" SEC_TYPE="ext2" TYPE="ext3"

The add UUIDs to /etc/fstab :

UUID=4a496f92-6840-4736-a0d5-5b9916113835       /VOL1   ext3 _netdev    0 0
UUID=cab5e3ec-4797-4227-98e8-e9bca3c3f766       /VOL2   ext3 _netdev    0 0

** Tip :

If your Linux is a VM on ESXi :

– Dedicate a network adapter to connect directly the Storage Array to the VMware server.

CAT 5e/6 RJ45 cable (through dedicated hardware switch, if needed).

– Create a “vSwitch” using the dedicated network adapter with vShere Client.

vswitch iscsi

– Add a network adapter using the new vSwitch on your virtual host configuration.

Now you have a direct iSCSI connection to you storage array. You can start the configuration.

How to make partitions larger than 2To with parted GPT support

August 16, 2011JeyLinux, Unix - Linux ...array, EFI, fdisk, GPT, lvm, mkfs, parted, partition

On Unix or Linux I never had to manage LUNs larger than 2To because using LVM I can create very large filesystems with reasonably small LUNs (in general I use LUNs from 256Go to 2To depending on purpose of the filesystem). It’s more convenient to manage “small” LUNs as you arrange them the way you want.

Nevertheless, low cost storage DAS arrays (understand array without SAN like eSATA arrays) sometimes comes with very basic option RAID level, number of disks, period.

In this case I have a SAS array with two RAID 5 volumes, 23To each and I want to make 1 big filesystem with the 2 disks.

Usually I choose fdisk to create partitions but fdisk will fail to manage volumes greater than 2To (MBR limitations).

GPT/EFI partitions:

In order to create larger partitions you have to use GPT (GUID partition table) and EFI partitions.

http://en.wikipedia.org/wiki/GUID_Partition_Table

http://en.wikipedia.org/wiki/EFI_System_partition

GPT support must be included in the kernel in order to work with EFI partitions.

A lot of professional Linux distributions embed GPT support by default.

GNU Parted is GPT compatible, so this is how to proceed.

Parted:

Create the two 23To partitions with parted:

# parted /dev/sdc mklabel gpt
# parted /dev/sdc mkpart primary 1 -1
# parted /dev/sdd mklabel gpt
# parted /dev/sdd mkpart primary 1 -1

(-1 indicate the end of the disk).

# parted dev/sdc print
Model: DELL PERC 6/E Adapter (scsi)
Disk /dev/sdc: 24.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End     Size    File system  Name     Flags
1      1049kB  24.0TB  24.0TB               primary
 
# parted /dev/sdd print
Model: DELL PERC 6/E Adapter (scsi)
Disk /dev/sdd: 24.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End     Size    File system  Name     Flags
1      1049kB  24.0TB  24.0TB               primary

LVM and xfs :

Use LVM to aggregate the 2 partitions:

# pvcreate /dev/sdc1 /dev/sdd1
# vgcreate vg_data /dev/sdc1 /dev/sdd1
# lvcreate -l 100%FREE -n lvdata1 vg_data

And create the filesystem :

# mkfs -t xfs /dev/vg_data/lvdata1
# df -h /dev/mapper/vg_data-lvdata1
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/ vg_data-lvdata1
44T  534G   44T   2% /backupst1

I had to use xfs (you will need a license on some professional Linux distributions) because ext4 is limited to 16To filesystems.

UUID : doc en français

August 12, 2011JeyLinux, Unix - Linux ...blkid, label, linux, ubuntu, uuid

Pour revenir sur l’article “How To Use UUID And Blkid To Manage Devices” mais en français cette fois-ci, j’ai trouvé des bonnes explications sur le lien suivant :

http://doc.ubuntu-fr.org/uuid_et_label

C’est une doc Ubuntu mais les explications et la plupart des commandes fonctionnent sur d’autres Linux.