HowTo: Use cat Command In Linux / UNIX

How do I use cat command in Linux? How do I use cat command in UNIX? How can I use cat command in UNIX or Linux shell scripts?

The cat command is considered as one of the most frequently used commands on Linux or UNIX like operating systems.
It can be used for the following purposes under UNIX or Linux:

• Display text files on screen.
• Copy text files.
• Combine text files.
• Create new text files.

cat command Syntax

The syntax is as follows:

 
cat filename
cat options filename
cat file1 file2
cat file1 file2 > newcombinedfile
 

Displaying The Contents of Files

To read or read the contents of files, enter:
$ cat /etc/passwd
The above command will display the contents of a file named /etc/passwd. By default cat will send output to the monitor screen. But, you can redirect from the screen to another command or file using redirection operator as follows:
$ cat /etc/passwd > /tmp/test.txt
In the above example, the output from cat command is written to /tmp/text.txt file instead of being displayed on the monitor screen. You can view /tmp/text.txt using cat command itself:
$ cat /tmp/test.txt

Concatenate files

Concatenation means putting multiple file contents together. The original file or files are not modified or deleted. In this example, cat will concatenate copies of the contents of the three files /etc/hosts, /etc/resolv.conf, and /etc/fstab:
$ cat /etc/hosts /etc/resolv.conf /etc/fstab
You can redirect the output as follows using shell standard output redirection:
$ cat /etc/hosts /etc/resolv.conf /etc/fstab > /tmp/outputs.txt
$ cat /tmp/outputs.txt
You can also use a pipe to filter data. In this example send output of cat to the less command using a shell pipe as the file is too large for all of the text to fit on the screen at a time:
$ cat /etc/passwd | less

How Do I Create a File?

You can use cat command for file creation. To create a file called foo.txt, enter:
$ cat > foo.txt
Sample outputs:

This is a test.

To save and exit press the CONTROL and d keys (CTRL+D). Please note that if a file named foo.txt already exists, it will be overwritten. You can append the output to the same file using >> operator:
$ cat >> bar.txt
The existing bar.txt file is preserved, and any new text is added to the end of the existing file called bar.txt. To save and exit press the CONTROL and d keys (CTRL+D).

How Do I Copy File?

The cat command can also be used to create a new file and transfer to it the data from an existing file. To make copy of
$ cat oldfile.txt > newfile.txt
To output file1's contents, then standard input, then file2's contents, enter:
$ cat file1 - file2
A hyphen indicates that input is taken from the keyboard. In this example, to create a new file file2 that consists of text typed in from the keyboard followed by the contents of file1, enter:
$ cat - file1 > file2

cat command options

To number non-blank output lines, enter (only works with GNU cat command version):
$ cat -b /etc/passwd
Sample outputs:

     1 root:x:0:0:root:/root:/bin/bash
     2 daemon:x:1:1:daemon:/usr/sbin:/bin/sh
     3 bin:x:2:2:bin:/bin:/bin/sh
     4 sys:x:3:3:sys:/dev:/bin/sh
     5 sync:x:4:65534:sync:/bin:/bin/sync
     6 games:x:5:60:games:/usr/games:/bin/sh
     7 man:x:6:12:man:/var/cache/man:/bin/sh
     8 lp:x:7:7:lp:/var/spool/lpd:/bin/sh
     9 mail:x:8:8:mail:/var/mail:/bin/sh
    10 news:x:9:9:news:/var/spool/news:/bin/sh

To number all output lines, enter (GNU cat version only):
$ cat -n /etc/passwd
To squeeze multiple adjacent blank lines, enter (GNU cat version only):
$ cat -s /etc/passwd
To display all nonprinting characters as if they were visible, except for tabs and the end of line character, enter (GNU cat version only):
$ cat -v filename

cat Command Abuse

The main purpose of cat is to catenate files. If it's only one file, concatenating it with nothing at all is a waste of time, and costs you a process. For example,
$ cat /proc/cpuinfo | grep model
Can be used as follows:
$ grep model /proc/cpuinfo
Another example,
cat filename | sed -e 'commands' -e 'commands2'
Can be used as follows which is cheaper:
sed sed -e 'commands' -e 'commands2' filename

Linux: Howto Make a Directory Command

How do I make directory under Linux operating systems using the command prompt (bash shell)?

You need to use the mkdir command to create new folders or directories under Linux operating systems.

mkdir command Syntax

mkdir command has the following syntax:

 
mkdir dirname
mkdir dirname1 dirname2
mkdir -p dir1/dir2/dir3
 

Examples

The following command would create a directory called foo:
$ mkdir foo
To list directories, enter:
$ ls
$ ls -l
The following command would create two directories within the current directory:
$ mkdir tom jerry
$ ls -l
The -p option allows you to create parent directories as needed (if parent do not already exits). For example, you can create the following directory structure:
$ mkdir -p ~/public_html/images/trip

Transfer Files From One UNIX Server To Another Server Using Windows / Linux Desktop

How do I securely transfer files from one UNIX / Linux server to another UNIX server using Windows or Linux desktop clients without using ftp client?

You need to use secure sftp or scp client for Windows XP / Vista / 7. Under Linux or Apple Mac OS X desktop you can use regular OpenSSH scp / sftp client to transfer files.

Windows SSH Client

You can use free SFTP, FTP and SCP client for Windows called putty or winscp.

Fig.01: Winscp transfer files from one UNIX server to another (image credit Winscp website)

Linux / UNIX / OS X SSH scp Client Examples

Use the following command from the server to which you want the files to go. In this example, transfer all files (/var/www/html) from remote server called server1 to local directory called /backup:
scp -r user@server1:/var/www/html/ /backup
In the following example, transfer all files (/var/www/html) from remote server called server1 to another server called server2:
scp -r user@server1:/var/www/html/ user@server2:/var/www/html/

Say hello to rsync

I recommend using rsync command which will only push or download updated files. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. In this example, copy files from remote server called server1 into /backup directory:
rsync -avz -e ssh user@server1:/var/www/html /backup

HowTo: Linux Rename Files

How do I rename a file under Linux operating systems using command line (bash shell prompt)?

You need to use the mv command to rename files or directories under Linux operating systems. The same command is also used to move files to different directories.

The mv Command Syntax

The syntax is as follows:

 
mv  source target
 
mv [Options]  source target
 

Take: Rename a File Under Linux

Rename a file called curriculum-vitae.txt as resume.txt, enter:
$ mv curriculum-vitae.txt resume.txt
If the target file (resume.txt) is located in the same directory as the source file (curriculum-vitae.txt), then the source file (curriculum-vitae.txt) can only be renamed.

Rename / Move Confirmation (Interactive) Option

You can force to display prompt before overwriting files. The -i option enables interactive option. So if file or directories with the same name already exists in the destination directory, mv will prompt the user:
$ mv -i file2.txt /tmp/
Sample outputs:

mv: overwrite `/tmp/file2.txt'? 

Move / Rename Verbose Option

Pass the -v option to display the name of each file before renaming and/or moving it:
$ mv -v file3.txt /tmp/
Sample outputs:

`file3.txt' -> `/tmp/file3.txt'

To rename file3.txt as file10.bak, enter:
$ mv -v file3.txt file10.bak
Sample outputs:

`file3.txt' -> `file10.bak'

Backup a File

To make a backup of each existing destination file pass the -b option. This option will tell mv to make a backup copy of each file that may be overwritten or removed:
$ touch file10.txt
$ mv -v -b file10.txt /tmp
Sample outputs:

`file10.txt' -> `/tmp/file10.txt'

To view a backup file called file10.bak, enter:
$ ls
Sample outputs:

file10.bak

Moving A file

In this example, move a file called file1.txt to /tmp/ directory, enter:
$ mv file1.txt /tmp/

Wildcards

In this example, move all files and directories, including all the contents of those directories, from the current directory to the directory /home/newdir:
# cd /home/olddir/
# mv * /home/newdir/
Please note that the asterisk (symbol) is nothing but a shell wildcard character that represents all files.

Other mv Command Options

From the mv command man page:

       --backup[=CONTROL]
              make a backup of each existing destination file
      -f, --force
              do not prompt before overwriting
       -i, --interactive
              prompt before overwrite
       -n, --no-clobber
              do not overwrite an existing file
       If you specify more than one of -i, -f, -n, only the final one takes effect.
       --strip-trailing-slashes
              remove any trailing slashes from each SOURCE argument
       -S, --suffix=SUFFIX
              override the usual backup suffix
       -t, --target-directory=DIRECTORY
              move all SOURCE arguments into DIRECTORY
       -T, --no-target-directory
              treat DEST as a normal file
       -u, --update
              move only when the SOURCE file is newer than the destination file or when the destination file is missing
       -v, --verbose
              explain what is being done
       --help display this help and exit
       --version
              output version information and exit

Linux: Set OR Change The Library Path

I've compile and installed a library at /usr/local/lib/libapp2.so -> libapp2.so.1.4.3. How do I set the Library path under Linux operating systems?

You need to use ldconfig config file and ldconfig command which creates the necessary links and cache to the most recent shared libraries found in the directories specified on the command line, in the file /etc/ld.so.conf, and in the trusted directories such as /lib64 or /usr/lib64 (/lib or /usr/lib on 32 bit systems). The /etc/ld.so.conf contains lib settings which can be used to add or delete paths. However, you need to simply drop your config file in /etc/ld.so.conf.d/ directory and it will be used by /sbin/ldconfig to configure dynamic linker run time bindings.

Add Your Path

Create a file called /etc/ld.so.conf.d/myapp.conf:
# vi /etc/ld.so.conf.d/myapp.conf
Add the following path:

/usr/local/lib

Save and close the file.

Activate Your Library Path

You must run the following command to activate path:
# ldconfig

Verify Your New Library Path

# ldconfig -v | less
OR
# ldconfig -v | grep /usr/local/lib
Sample outputs:

/usr/local/lib:
        libGeoIP.so.1 -> libGeoIP.so.1.4.6
        libGeoIPUpdate.so.0 -> libGeoIPUpdate.so.0.0.0
/usr/lib64/mysql:
        libmysqlclient_r.so.15 -> libmysqlclient_r.so.15.0.0
        libmysqlclient.so.15 -> libmysqlclient.so.15.0.0
/lib:
        libutil.so.1 -> libutil-2.5.so

How Do I Delete The Library Path?

Simply, delete the file:
# rm /etc/ld.so.conf.d/myapp.conf
# ldconfig

How Do I Edit The Library Path?

Simply edit the file and reload the changes:
# vi /etc/ld.so.conf.d/myapp.conf
# ldconfig

How Do I Compile Program With Shared Libs And GNU GCC?

You can use the following gcc
$ gcc -Wl,-R/path/to/lib -I/path/to/include -L/path/to/lib -o myAppName mycode.c -llibapp2
See gcc man page for further details.

Further readings:

man ldconfig
man ld
man gcc

Linux: Setup SSH To Tunnel VNC Traffic Though Internet

I have UNIX workstation and Linux server at work and Ubuntu desktop at home. It is recommended that I use ssh tunnel the VNC protocol for secure communication. How do I access my UNIX workstation desktop here at home over the Internet using ssh?

VNC can be accessed over the Internet using ssh protocol. This will improve security as traffic will be encrypted.

Sample Setup

                                       pc22.nixcraft.in
 +----------------+                   +-----------------+
 | Laptop @ Home  |---> Internet ---> | UNIX PC @ Work  |
 +----------------+                   +-----------------+
                                       vncserver port 5901
OR
 +----------------+                   +-----------------+
 | Laptop @ Home  |---> Internet ---> | Router/firewall |
 +----------------+                   | port forwarding |
                                      +-----------------+
                                            | fw.nixcraft.in ssh/tcp22
                                            |
                                          ///\\\
                                          //   \\
                                          /      \
                                       +-----------------+
                                       | UNIX/Linux /OS X|
                                       | at work         |
                                       +-----------------+
                                       pc22.nixcraft.in
                                       vncserver @ port 5901

Your work pc may be directly connected to the Internet. Otherwise most offices have a router and firewall installed. You need to make sure your firewall allows port forwarding for TCP port 22 to your UNIX / OS X / workstation or server called pc22.nixcraft.in. It works as follows:

 +------------+
 | SSH Client |-------> Internet ssh connection with encryption
 | with       |                         |
 | vncviewer  |                         |
 +------------+                         |
                                       \|/
                             +------------------------+
                             | SSH server at port 22  |
                             | Vncserver at port 5501 |
                             +------------------------+

You connect from your local ssh client (localhost) to your ssh server (pc22.nixcraft.in) installed at your work UNIX pc with port forwarding at router / firewall level. I'm assuming that port forwarding is correctly configured at your office. Now, open the terminal and type the following command:

ssh  -N -f -L 5000:localhost:5901 vivek@pc22.nixcraft.i

The above command will start an ssh connection to pc22.nixcraft.in and also listen on port 5000 on localhost and forward vnc connection to port 5901 on pc22.nixcraft.in. Usaully local and remote port numbers are same to avoid confusion:

ssh  -N -f -L 5901:localhost:5901 vivek@pc22.nixcraft.i

Now, you can use vncviewer at your home as follows:
vncviewer localhost:5901
You can also use GUI tool and set VNC server location to localhost:5901

Fig.01: Setup ssh to tunnel VNC traffic over the Internet

Once connected you will get desktop login windows or last session window as follows:

Fig.02: VNC in action and more secure using SSH

HowTo: UNIX / Linux Open TCP / UDP Ports

How do I open the TCP or UDP ports under UNIX / Linux like operating systems?

A port is an application-specific or process-specific software construct serving as a communications endpoint and it is identified by its number such as TCP port number 80 . It is used by TCP and UDP of the Internet Protocol Suite. A port number is a 16-bit unsigned integer, thus ranging from 0 to 65535.

        UNIX / Linux
    +------------------+
    | Networking stack |
    |      eth0        |
    +------------------+
           |
    +------------------+
    |  Apache process  |--> Binding port 80 @ 202.54.1.1 IP
    +------------------+

In the above example Apache process associates its input and output channel file descriptors (fd) with a port number 80 and an IP address 202.54.1.1. This is known as binding. It is used to send and receive web pages via UNIX / Linux operating system's networking stack (software). In other words communication is done using application ports. When you start the Apache you open port 80 for communication. Common services such as web, mail, pop3 et all use use specifically reserved, well-known port numbers for receiving service requests from client hosts. The well-known ports are defined the Internet Assigned Numbers Authority (IANA). Type the following command to see list well-known of TCP and UDP port numbers:
$ less /etc/services
grep -w 80 /etc/services
Sample outputs:

www  80/tcp  http  # WorldWideWeb HTTP
www  80/udp    # HyperText Transfer Protocol

Privileged Ports

Typically port number less than 1024 are used by well know network servers such as Apache. Under UNIX and Linux like oses root (super user) privileges are required to open privileged ports. Almost all clients uses a high port numbers for short term use. This is also known as an ephemeral port. For example Apache use TCP port 80

  Server                         Client w/ Firefox
 +----------+                    +----------------+
 | Apache   |                    | connects using |
 | TCP Port |                    | an ephemeral   |
 | 80 @     |<-----> eth0 <----> | port #         |
 |202.54.1.2|                    | 46025          |
 +----------+                    +----------------+

The port numbers are divided into three ranges:

1. Well Known Ports: those from 0 through 1023.
2. Registered Ports: those from 1024 through 49151
3. Dynamic and/or Private Ports: those from 49152 through 65535

You can increase local port range by typing the following command (Linux specific example):
# echo 1024 65535 > /proc/sys/net/ipv4/ip_local_port_range
You can also increase or decrease socket timeout (Linux specific example):
# echo 2000 > /proc/sys/net/ipv4/tcp_keepalive_time

Common Well Known Port Numbers

The following are used by UNIX / Windows / Linux / BSD / OS X and all other server operating systems or network devices (see /etc/services file):

• 21: FTP Server
• 22: SSH Server (remote login)
• 25: SMTP (mail server)
• 53: Domain Name System (Bind 9 server)
• 80: World Wide Web (HTTPD server)
• 110: POP3 mail server
• 143: IMAP mail server
• 443: HTTP over Transport Layer Security/Secure Sockets Layer (HTTPDS server)
• 445: microsoft-ds, Server Message Block over TCP

How Do I See Open Ports and Socket Information Under UNIX or Linux?

You can use the netstat command:
# netstat -tulpn
FreeBSD specific example:
# sockstat -l
To list open IPv4 connections use the lsof command:
# lsof -Pnl +M -i4
The ss command is used to dump socket statistics. It allows showing information similar to netstat command. It can display more TCP and state information than other tools
# ss -s
# ss -l
# ss -pl
# ss -o state established '( dport = :smtp or sport = :smtp )'

Examples

Each TCP or UDP port is opened using a UNIX service or daemon such as Apache web server. You can also write a program using C, C++, Perl, Shell or Bash to open any port. You can also use utilities such as nc command .

Apache Server Example (open TCP port 80)

Start the Apache web server under FreeBSD as follows to open TCP port 80:
# /usr/local/etc/rc.d/apache22 forcestart
OR
# /usr/local/etc/rc.d/apache22 start
To displays listening sockets (open ports) under FreeBSD, enter:
# sockstat -l
OR
# netstat -nat | grep LISTEN
You should see port 80 opened under FreeBSD. Under CentOS or Redhat (RHEL) Linux, you can open port 80 using the following commands:
# service httpd start
# chkconfig httpd on
# netstat -tulpn | grep :80

Firewall Configuration

All port numbers are encoded in the transport protocol packet header, and they can be read by other components of the network stack such as firewall. Firewall can be used for port forwarding or denying access to open port. For example, block an abusing IP address called 1.2.3.4 using UNIX firewall. In other words, Apache port is open but it may be blocked by UNIX (pf) or Linux (iptables) firewall. You also need to open port at firewall level. In this example, open tcp port 80 using Linux iptables firewall tool:
# /sbin/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
# service iptables save

See also:

1. CentOS / Redhat Linux Iptables Firewall Configuration Tutorial
2. Redhat / CentOS / Fedora Linux Open Port
3. FreeBSD Setting up Firewall using IPFW
4. OpenBSD PF Firewall Script – /etc/pf.conf File

nc Command Example

The nc (or netcat utility) is used for just about anything under the sun involving TCP or UDP. It can open TCP connections, send UDP packets, listen on arbitrary TCP and UDP ports, do port scanning, and deal with both IPv4 and IPv6. In this example, open port 5000 using nc command:
$ nc -l 5000
On a second console or from a second UNIX / Linux machine, connect to the machine and port being listened on:
$ nc localhost 5000
OR
$ nc unix.system.ip.here 5000
In this example, send data from one computer to another:
$ nc -l 5555 > output.txt
Using a second machine, connect to the listening nc process (@ port 5555), feeding it the file which is to be transferred:
$ nc your.unix.systems.ip.here 5555 < input.txt
You can run netstat command to view open ports:
$ netstat -a
$ netstat -nat | grep LISTEN
Sample outputs:

tcp4       0      0  *.5555                 *.*                    LISTEN
tcp4       0      0  10.1.3.29.53           *.*                    LISTEN
tcp4       0      0  192.168.56.1.53        *.*                    LISTEN
tcp4       0      0  115.242.47.238.53      *.*                    LISTEN
tcp4       0      0  127.0.0.1.953          *.*                    LISTEN
tcp4       0      0  127.0.0.1.53           *.*                    LISTEN
tcp4       0      0  127.0.0.1.631          *.*                    LISTEN
tcp6       0      0  ::1.631                *.*                    LISTEN

Python Example

Create a file called echo_server.py:

#!/usr/bin/python
 
# Demo server to open port 8888
# Modified from Python tutorial docs
import socket
 
HOST = '127.0.0.1'       # Hostname to bind
PORT = 8888              # Open non-privileged port 8888
 
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
while 1:
    data = conn.recv(1024)
    if not data: break
    conn.send(data)
conn.close()
 

Create a file called echo_client.py:

#!/usr/bin/python
 
# Demo client program
# Modified from Python tutorial docs
import socket
 
HOST = '127.0.0.1'     # Set the remote host, for testing it is localhost
PORT = 8000            # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.send('Where there is love there is life')
data = s.recv(1024)
s.close()
print 'Received', repr(data)
 

Save and close the file. Run it as follows:
$ chmod +x *.py
Start server, enter:
$ ./echo_server.py
$ netstat -nat | grep LISTEN
On a second console connect to the localhost and port being listened on using echo_client.py:
$ ./echo_client.py