Common Series - Linux Common Commands

The Common Series is a record of frequently used programming tools. This article is the first in the Common Series, used to document commonly used Linux commands during daily development for easy reference when needed.

View Directory#

ls#

# Go back to the previous directory
cd - 

# Show all files
ls -a

# Show files in the directory sorted by modification time
ls -lht

tree#

Display the file structure of the current directory

# Display the file structure of the current directory
tree -FC /
  -a: Show hidden files
  -C: Color display
  -L 2: Show only 2 levels
  -F: Display a \ after directories; display executable files with *; similar functionality to ls -F

Modify File Permissions#

# Modify file permissions
chmod

# Add write permission to xxx, commonly used to grant permissions to scripts that need to run
chmod +x xxx

# Add read permission to xxx
chmod +r xxx

# Change xxx's permissions to 777, making the file readable, writable, and executable by everyone
chmod 777 xxx

# Recursively modify the permissions of the entire folder
chmod 777 xxx -R

View File Content#

more#

# Browse file content
more <filename>  
Enter: Next line
Space: Next page # Page progress increases by 36%
b: Previous page
q: Exit

less#

# Similar to more, with more features
less  
Enter: Next line
y: Previous line
Page Down: Next page
Page Up: Previous page
q: Exit

tail#

# Display the last 3 lines of the file
tail -3 <filename>   

# Commonly used to observe the log writing process
tail -f -n 1000 <filename>

wc#

# Output results in the order of line count, word count, byte count
wc

wc -l: Count lines
wc -w: Count words
wc -c: Count bytes

grep#

# Filter out the given match_pattern in the file
grep match_pattern <file>
Common parameters
-c Count the number of occurrences of the text in the file
-n Print the line number of matches
-i Ignore case when searching

# Read several lines of data from stdin, if a line contains xxx, output that line. --color represents highlighting
grep xxx --color   

# Recursively search text in multi-level directories (a programmer's favorite for searching code)
grep "class" . -R -n

# Find used commands
history | grep <command>

xargs#

xargs can convert input data into command line parameters for specific commands; thus, it can be combined with many commands for use. For example, grep, find；

xargs parameter explanation

-d Define the delimiter (default is space; the delimiter for multiple lines is n)
-n Specify output as multiple lines
-I {} Specify the replacement string, which will be replaced during xargs expansion; used when the command to be executed requires multiple parameters
-0: Specify 0 as the input delimiter

Convert multi-line output to single-line output
cat file.txt | xargs

Convert single-line to multi-line output, -n: Specify the number of fields displayed per line
cat single.txt | xargs -n 3

cat file.txt | xargs -I {} ./command.sh -p {} -1

cut#

Used to remove part of the content from each line

# Split a line of content, read multiple lines of data from stdin
Example:
echo $PATH | cut -d ':' -f 3,5 Outputs the 3rd and 5th columns of PATH split by :
echo $PATH | cut -d ':' -f 3-5 Outputs the 3rd to 5th columns of PATH split by :
echo $PATH | cut -c 3,5 Outputs the 3rd and 5th characters of PATH
echo $PATH | cut -c 3-5 Outputs the 3rd to 5th characters of PATH

Find Files#

locate#

Find files by pathname, the search range is in the /var/lib/mlocate/ database, stored in memory, updated once a day, so locate cannot search for newly created files; you can use updatedb to immediately update the database.

If the system does not come with the locate command, you can install it using yum install mlocate -y, and after installation, execute the updatedb command

# Find files with zip in their names
locate zip

find command#

Find files by filename

# Search for all *.py files in a certain file path
find /path/to/directory/ -name '*.py' 

# Case-insensitive search
find /home -iname "*.txt" 

# Find all files ending with .txt and .pdf in the current directory and subdirectories
find . -name "*.txt" -o -name "*.pdf"  

# View the number of files in the current directory
find ./ | wc -l   

# Search by type, only list all directories, -type f for files / l for symbolic links / d for directories
find . -type d -print

# Limit the size of the files to search, +1M means searching for files larger than 1M
find ./ -type f -name "*.py" -size +1M

# Delete files that meet the criteria, the following command can delete files with the .bak suffix in the current directory
find ./ -type f -name "*.bak" -delete

sed Text Replacement#

sed is generally used for text replacement, operating line by line

The basic working method of sed is:

Read the file into memory line by line (this memory is also called pattern space)
Operate on that line using each script of sed
Output the line after processing

Format

sed ‘operation command’ one or more files

sed's replacement command:
sed 's/old_string/new_string/' filename

sed with multiple instructions
sed -e 's/old_string/new_string/' -e 's/old_string/new_string/' filename

First occurrence replacement
sed 's/text/replace_text/' file   // Replace the first matching text in each line

Global replacement
sed 's/text/replace_text/g' file

By default, after replacement, output the replaced content; if you need to directly replace the original file, use -i:
sed -i 's/text/replace_text/g' file

Remove blank lines
sed '/^$/d' file

The first matching parenthesis content is referenced using marker 1
sed 's/hello\([0-9]\)/\1/' file

Bash Keyboard Shortcuts#

# Move the cursor to the first character of the line
Ctl + a

# Move the cursor to the end of the line
Ctl + e

# Delete all characters from the cursor to the beginning of the line
Ctl + u   

# Commonly used: 
# Delete the input content
Ctl + e + Ctl + u

Archiving and Backup#

gzip#

Compress a single or multiple files

gzip foo.text

gunzip#

Decompress files

# Compress, generating foo.txt.gz
gunzip foo.text

zip#

Compress and archive into a .zip file

zip -r text.zip text

unzip#

Decompress .zip files

unzip text.zip

tar#

Archive multiple files or directories

# Pack and compress with gzip
tar -zcvf <compressed_file_name> <files_to_compress> 
z: Call the gzip compression command for compression
c: Pack files
v: Show the running process
f: Specify the filename

# Decompress
tar -zxvf test.tar.gz -C /usr** (-C indicates the specified decompression location)
x: Represents unpacking

View Disk Space#

# View disk space usage size: -h: human-readable, displays results in an easy-to-read format
df -hT

# View the space occupied by the current directory, -s recursively shows the size of the entire directory
du -sh

# View the sizes of all subfolders in the current directory sorted:
for i in `ls`; do du -sh $i; done | sort
Or:
du -sh `ls` | sort

Query Processes#

ps#

# Query information about running processes:
# VSZ - Virtual memory size RSS - The amount of RAM used by the process
ps -aux/ ps -ef

top#

# Display process information and update in real-time
top
After opening, press M: Sort by memory usage
After opening, press P: Sort by CPU usage, view the processes using the most CPU and memory in the system
After opening, press i: Make top not display any idle or zombie processes
After opening, press q: Exit

lsof#

# View the status of processes occupying ports:
lsof -i:3306

# View the files opened by the process of user username
lsof -u username

# Query the files currently opened by the init process
lsof -c init

Others#

# Query process ID (suitable for remembering only part of the process field)
pgrep -l <process_name>

# View the process occupying the port
netstat -anp | grep port

Terminate Processes#

# Kill the process with the specified PID (PID is Process ID)
kill PID

# Kill related processes
kill -9 3434

# Kill job (job is job number)
kill %job

Analyze Thread Stack#

Use the command pmap to output the memory status of the process, which can be used to analyze the thread stack;

pmap PID

Performance Monitoring#

Monitor CPU#

# View CPU usage
sar -u

eg:
$sar -u 1 2
The two parameters behind indicate the monitoring frequency, for example, in the example, 1 and 2 indicate sampling once per second, a total of 2 times;

# View average CPU load, sar specified -q can view the number of processes in the running queue, the size of processes on the system, average load, etc.;
sar -q 1 2

Query Memory#

# View memory usage status sar specified -r can view memory usage status;
sar -rh 1 2

# View memory usage
free -mh

Query Port Occupation#

# View the current system port usage
netstat -an

# Show processes and occupied ports
netstat -ntlp

Query Page Swapping#

View the status of page swapping. When page swapping occurs, the server's throughput will drop significantly; if the server is in poor condition and it is suspected that page swapping occurs due to insufficient memory, you can use the command sar -W to confirm whether a large amount of swapping has occurred;

sar -W 1 3

Comprehensive Application#

When sar is unavailable in the system, you can use the following tools as alternatives: vmstat under Linux, prstat under Unix systems

eg: View CPU, memory, and usage: vmstat n m (n is the monitoring frequency, m is the number of monitoring times)

vmstat 1 3
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
0  0  86560  42300   9752  63556    0    1     1     1    0    0  0  0 99  0
1  0  86560  39936   9764  63544    0    0     0    52   66   95  5  0 95  0
0  0  86560  42168   9772  63556    0    0     0    20  127  231 13  2 84  0

Use the watch tool to monitor changes. When continuous monitoring of a certain data change in an application is needed, the watch tool can meet the requirements; after executing the watch command, it will enter an interface that outputs the currently monitored data, and once the data changes, it will highlight the change;

eg: Monitor memory changes while operating redis:

watch -d -n 1 './redis-cli info | grep memory'
(The following is the interface content in the watch tool, once memory changes, it will highlight the change in real-time)
Every 1.0s: ./redis-cli info | grep memory                                                                  Mon Apr 28 16:10:36 2014

used_memory:45157376
used_memory_human:43.07M
used_memory_rss:47628288
used_memory_peak:49686080
used_memory_peak_human:47.38M

ping#

Send ICMP packets to a specified host to confirm network connectivity with the host. The sending interval is 1 s, and you can interrupt the sending with Ctrl + C. A normal network will show a 0% packet loss rate.

# Check if connected to the internet
ping www.baidu.com

traceroute#

Lists all hops from the local network to the destination host

# It will display the hostnames, IP addresses of the routers passed, and three round-trip time samples from local to that router
traceroute -I www.baidu.com

ip#

A multifunctional network configuration tool used to check the system's network interfaces and routing tables

It will display multiple network interfaces, common interfaces:
- lo: Loopback interface, a virtual interface used by the system to "talk to itself"
- eth0: Ethernet interface
If the first line of the interface contains the word UP, it indicates that the interface is enabled
The inet field on the 3rd line is the IP address of that network interface

# 
~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:16:3e:0e:a0:8b brd ff:ff:ff:ff:ff:ff
    inet 172.30.140.229/20 brd 172.30.143.255 scope global dynamic eth0
       valid_lft 310094870sec preferred_lft 310094870sec
    inet6 fe80::216:3eff:fe0e:a08b/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:53:b2:cd:34 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

netstat#

Used to check various network settings and statistics, requires installation of net-tools yum install net-tools

# Check the system's network interfaces
netstat -ie

# View all network connections
netstat -nt

ssh#

Used for communication between the local host and the remote host

# You need to enter the login password for the ubuntu user on the remote-sys host
ssh ubuntu@remote-sys

Other Common Commands#

# Commonly used with alias command, there should be no spaces around the = sign, and the aliased command needs to be enclosed in single quotes
alias ll='ls -alh'

# View the difference between two files:
diff file1 file2

# View the value of the SVN environment variable:
env | grep SVN