Linux Hints, Tips, and Tricks

Running 32-bit Cisco AnyConnect VPN Client 2.2 on 64-bit Ubuntu 8.10 desktop will result in some errors if certain 32-bit shared libraries are missing:

# ./vpnsetup.sh
Installing Cisco AnyConnect VPN Client …
Extracting installation files to /tmp/vpn.k13598…
ciscovpn/
ciscovpn/vpn_install.sh
ciscovpn/vpnagentd
ciscovpn/vpnagentd_init
ciscovpn/vpn_uninstall.sh
ciscovpn/libssl.so.0.9.8
ciscovpn/libcrypto.so.0.9.8
ciscovpn/vpnui
ciscovpn/anyconnect.desktop
ciscovpn/vpn
ciscovpn/pixmaps/
ciscovpn/pixmaps/company-logo.png
ciscovpn/pixmaps/cvc-about.png
ciscovpn/pixmaps/cvc-connect.png
ciscovpn/pixmaps/cvc-disconnect.png
ciscovpn/pixmaps/cvc-info.png
ciscovpn/pixmaps/systray_connected.png
ciscovpn/pixmaps/systray_disconnecting.png
ciscovpn/pixmaps/systray_notconnected.png
ciscovpn/pixmaps/systray_reconnecting.png
ciscovpn/pixmaps/vpnui48.png
ciscovpn/VPNManifest.dat
ciscovpn/vpndownloader.sh
ciscovpn/update.txt
ciscovpn/license.txt
Starting the VPN agent…
/etc/init.d/vpnagentd_init: 68: /opt/cisco/vpn/bin/vpnagentd: not found
#

$ /opt/cisco/vpn/bin/vpn
Cisco AnyConnect VPN Client (version 2.2.0136).

Copyright (c) 2004 - 2008 Cisco Systems, Inc.
All Rights Reserved.

  >> warning: No profile is available.  Please enter host to “Connect to”.
  >> state: Disconnected
  >> notice: VPN Service is available.
  >> registered with local VPN subsystem.
  >> state: Disconnected
VPN> connect xx.xx.xx.xx
  >> contacting host (xx.xx.xx.xx) for login information…
  >> notice: Contacting xx.xx.xx.xx.
  >> warning: Unable to process response from xx.xx.xx.xx.
  >> error: Connection attempt has failed due to server certificate problem.
  >> state: Disconnected
VPN> exit
goodbye…
$

Here are the steps I executed on my Ubuntu desktop to resolve this issue:

Download and install getlibs:

# wget http://www.boundlesssupremacy.com/Cappy/getlibs/getlibs-all.deb
# dpkg -i getlibs-all.deb

Install 32-bit shared libraries:

# getlibs /opt/cisco/vpn/bin/vpn
# getlibs libsqlite3.so.0

Create some symbolic links:

# mkdir /usr/local/firefox
# cd /usr/local/firefox
# ln -s /usr/lib32/libnss3.so
# ln -s /usr/lib32/libplc4.so
# ln -s /usr/lib32/libnspr4.so
# ln -s /usr/lib32/libsmime3.so

Now re-run vpnsetup.sh and launch vpn:

# ./vpnsetup.sh
$ /opt/cisco/vpn/bin/vpn

When a Linux system has been running for a while memory fragmentation can increase which depends heavily on the nature of the applications that are running on it. The more processes allocate and free memory, the quicker memory becomes fragmented. And the kernel may not always be able to defragment enough memory for a requested size on time. If that happens, applications may not be able to allocate larger contiguous chunks of memory even though there is enough free memory available. Starting with the 2.6 kernel, i.e. RHEL4 and SLES9, memory management has improved tremendously and memory fragmentation has become less of an issue.

To see memory fragmentation you can use the magic SysRq key. Simply execute the following command:

# echo m > /proc/sysrq-trigger

This command will dump current memory information to /var/log/messages. Here is an example of a RHEL3 32-bit system:

Jul 23 20:19:30 localhost kernel: 0*4kB 0*8kB 0*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1952kB)
Jul 23 20:19:30 localhost kernel: 1395*4kB 355*8kB 209*16kB 15*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 12244kB)
Jul 23 20:19:31 localhost kernel: 1479*4kB 673*8kB 205*16kB 73*32kB 21*64kB 847*128kB 473*256kB 92*512kB 164*1024kB 64*2048kB 28*4096kB = 708564kB)

The first line shows DMA memory fragmentation. The second line shows Low Memory fragmentation and the third line shows High Memory fragmentation. The output shows memory fragmentation in the Low Memory area. But there are many large memory chunks available in the High Memory area, e.g. 28 4MB.

If memory information was not dumped to /var/log/messages, then SysRq was not enabled. You can enable SysRq by setting sysrq to 1:

# echo 1 > /proc/sys/kernel/sysrq

Starting with the 2.6 kernel, i.e. RHEL4 and SLES9, you don’t need SysRq to dump memory information. You can simply check /proc/buddyinfo for memory fragmentation.

Here is the output of a 64-bit server running the 2.6 kernel:

# cat /proc/buddyinfo
Node 0, zone DMA 5 4 3 4 3 2 1 0 1 1 2
Node 0, zone Normal 1046 527 128 36 17 5 26 40 13 16 94
# echo m > /proc/sysrq-trigger
# grep Normal /var/log/messages | tail -1
Jul 23 21:42:26 localhost kernel: Normal: 1046*4kB 529*8kB 129*16kB 36*32kB 17*64kB 5*128kB 26*256kB 40*512kB 13*1024kB 16*2048kB 94*4096kB = 471600kB
#

In this example I used SysRq again to show what each number in /proc/buddyinfo is referring to.

To do simple calculations in bash, the ((…)) command can be used:

$ echo $(( 10 + 10 ))
20

To do floating point or more complex math, try the bc command:

$ echo "scale=5; 5/3" | bc
1.66666

To convert hex values to decimal, the ((…)) or bc command can be used:

$ echo $((0xff))
255
$ echo 'obase=10; ibase=16; FF' | bc
255

Note that the hex value must be in uppercase letters if you use the bc command.

To convert decimal to hex:

$ echo 'obase=16; ibase=10; 255' | bc
FF

To convert hex to binary:

$ echo 'obase=2; ibase=16; FF' | bc
11111111

To do unit conversions, you can use the units command:

$ units -t '1mile' 'km'
1.609344

Or to do definition lookups:

$ units -t '1 googol'
        Definition: 1e+100

To find out what a particular driver/module does, the modinfo command can be used in many cases:

# modinfo -d e1000
Intel(R) PRO/1000 Network Driver
#

# modinfo -d hangcheck-timer
Hangcheck-timer detects when the system has gone out to lunch past a certain margin.
#

To get a list of parameters supported by a kernel driver/module, the modinfo -p command will usually provide the information:

# modinfo -p e1000
debug:Debug level (0=none,…,16=all)
InterruptThrottleRate:Interrupt Throttling Rate
RxAbsIntDelay:Receive Absolute Interrupt Delay
RxIntDelay:Receive Interrupt Delay
TxAbsIntDelay:Transmit Absolute Interrupt Delay
TxIntDelay:Transmit Interrupt Delay
XsumRX:Disable or enable Receive Checksum offload
FlowControl:Flow Control setting
AutoNeg:Advertised auto-negotiation setting
Duplex:Duplex setting
Speed:Speed setting
RxDescriptors:Number of receive descriptors
TxDescriptors:Number of transmit descriptors
#
# modinfo -p hangcheck-timer
hangcheck_dump_tasks:If nonzero, the machine will dump the system task state when the timer margin is exceeded.
hangcheck_reboot:If nonzero, the machine will reboot when the timer margin is exceeded.
hangcheck_margin:If the hangcheck timer has been delayed more than hangcheck_margin seconds, the driver will fire.
hangcheck_tick:Timer delay.
#

To set parameters during module loads, you can add entries to /etc/modprobe.conf on RHEL or /etc/modprobe.conf.local on SLES. For example:

options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180

To load the module and see the new settings, run:

# modprobe -v hangcheck-timer
insmod /lib/modules/2.6.9-22.EL/kernel/drivers/char/hangcheck-timer.ko hangcheck_tick=20 hangcheck_margin=280
#
# dmesg | tail -1
Hangcheck: starting hangcheck timer 0.5.0 (tick is 30 seconds, margin is 180 seconds).
#

The newly loaded module will be at the top of the lsmod list:

# lsmod | head -2
Module Size Used by
hangcheck_timer 3289 0
#

To unload the module, run:

# rmmod hangcheck-timer

To rename all files in a directory and add a new extension the xargs command can be used:

ls | xargs -t -i mv {} {}.old

xargs reads each item from the ls ouput and executes the mv command. The ‘-i’ option tells xargs to replace ‘{}’ with the name of each item. The ‘-t’ option instructs xargs to print the command before executing it.

To rename a subset of files, specify the file names with the ls command:

ls *.log | xargs -t -i mv {} {}.old

Or to add a current timestamp extension you may want to use the date command similar to this one:

ls *.log | xargs -t -i mv {} {}.`date +%F-%H:%M:%S`

The extension will look like “.2006-08-10-19:37:16″.

If you want to rename the extension of files, try the rename command:

rename .log .log_archive.`date +%F-%H:%M:%S` *

This command replaces the first occurrence of ‘.log’ in the name by .log_archive.`date +%F-%H:%M:%S`.

The following command replaces .htm extensions with .html for all files that start with “project*”:

rename .htm .html project*

To do a simple and quick network performance test the ftp command can be used.

FTP on Linux and other Unix systems allows you to pass shell commands to the ftp client by using the pipe symbol ‘|’ as the first character of the file name. With this feature you can send a very large file to a remote host using /dev/zero as input and /dev/null as output.

Example:

ftp> put "|dd if=/dev/zero bs=1M count=100" /dev/null

This command transfers a large file without involving the disk and without having to cache the file in memory. If you use a large file on a disk it might become a bottleneck. In this example, “|dd if=/dev/zero bs=1M count=100″ becomes the input file. Since a dd command without the “of=” paramater prints the content of the file to standard output (stdout), the ftp client can read the output and pass it on to the remote file which is /dev/null on the remote host.

To retrieve information on system’s hardware like vendor, manufacturer, product, S/N, etc. the following command can be used:

dmidecode

The dmidecode command reads the information from the system BIOS, see also http://www.nongnu.org/dmidecode/.

There are a few other commands you might want to check out which list installed hardware components:

dmesg
lsdev
lshal
lspci
lsusb
lsscsi

Beginning with the 2.6 kernel you can get lots of information from /sys. For example, to get information on an Emulex HBA:

# ls /sys/class/scsi_host/host1/
board_mode     lpfc_cr_delay            lpfc_poll             option_rom_version
board_online   lpfc_drvr_version        lpfc_poll_tmo         portnum
cmd_per_lun    lpfc_fcp_class           lpfc_scan_down        proc_name
ctlreg         lpfc_fdmi_on             lpfc_topology         programtype
device         lpfc_hba_queue_depth     lpfc_use_adisc        scan
fwrev          lpfc_link_speed          management_version    serialnum
hdw            lpfc_log_verbose         mbox                  sg_tablesize
host_busy      lpfc_lun_queue_depth     modeldesc             state
info           lpfc_max_luns            modelname             uevent
lpfc_ack0      lpfc_multi_ring_support  nport_evt_cnt         unchecked_isa_dma
lpfc_cr_count  lpfc_nodev_tmo           num_discovered_ports  unique_id
#

Sometimes it can be difficult to debug scripts. For example, a script only fails if it’s being executed by an application and you have no way of telling the application how the script should be executed to redirect the output. Or you simply don’t want to redirect the output of the script each time you execute it.

Adding the following lines at the beginning of the script can be very useful:

export PS4='$0.$LINENO+ '
exec > /tmp/script.log
exec 2>&1
set -x

Example:

 cat test
#!/bin/bash
export PS4='$0.$LINENO+ '
exec > /tmp/script.log
exec 2>&1
set -x
ls -ld /etc
ls -ld /boot
echo "This is a test"

$ ./test
$ cat /tmp/script.log
./test.6+ ls -ld /etc
drwxr-xr-x 83 root root 7512 2006-07-22 16:49 /etc
./test.7+ ls -ld /boot
drwxr-xr-x 5 root root 1960 2006-07-22 15:30 /boot
./test.8+ echo 'This is a test'
This is a test
$

These lines will turn on debugging and all information will be redirected to the log file. So you won’t have to redirect the output each time you run the script, e.g. “./script > /tmp/script.log 2>&1″. In some cases you can’t do that if the script is invoked by an application.

The PS4 builtin shell variable describes the prompt seen in debug mode. The $0 variable stands for the name of the script file itself. $LINENO shows the current line number within the script. The exec command redirects I/O streams. The first exec command redirects stdout stream 1 to /tmp/script.log. 2>&1 redirects stderr stream 2 to stdout stream 1. And “set -x” enables debugging.