Linux provides a powerful arsenal of tools for file manipulation and analysis. This guide presents fundamental commands that every IT professional and cybersecurity specialist should master.
Binary Analysis and Reverse Engineering
Exploring Binary Files
All files in the system are stored as binary data on the hard drive. To examine this data in hexadecimal format:
hexdump file.bin # Basic hex dump
hexdump -C file.bin # Canonical format (hex + ASCII)
hexdump -n 100 file.bin # Display only first 100 bytes
The strings Command in Cybersecurity
The strings command is a fundamental tool in reverse engineering and malware analysis:
strings executable_file # Extract readable strings
strings -n 8 file.bin # Show strings of 8+ characters only
strings -o file.bin # Show offset of each string
Practical Example:
strings -n 6 /usr/bin/passwd | grep -i password
This command extracts all readable character sequences from a binary file, making it invaluable for discovering embedded text, URLs, or configuration data in executables.
File Redirection and Output Management
Creating and Redirecting Output
The > symbol redirects output to a file:
echo "catarina-is-amazing" > truths.log # Overwrite file
echo "additional-truth" >> truths.log # Append to file
ls -la > directory_listing.txt # Save command output
Error Handling with Redirection
Linux uses different output streams. Stream 2 represents errors:
command 2>/dev/null # Suppress errors only
command >/dev/null 2>&1 # Suppress all output
find / -name "*.txt" 2>/dev/null # Find files, ignore permission errors
Log File Management
Navigating Log Files
System logs are typically stored in /var/log:
cd /var/log # Navigate to log directory
ls -la /var/log # List all log files with details
du -sh /var/log/* # Show size of each log file
Removing Files
To delete files safely:
rm filename # Delete single file
rm -i filename # Interactive deletion (ask confirmation)
rm -r directory/ # Delete directory recursively
rm *.log # Delete all .log files
Text Processing and Analysis
Viewing File Content
First Lines with head:
head file.txt # Shows first 10 lines
head -n 5 file.txt # Shows first 5 lines
head -c 100 file.txt # Shows first 100 characters
Last Lines with tail:
tail file.txt # Shows last 10 lines
tail -n 5 file.txt # Shows last 5 lines
tail -f file.txt # Follow file updates (real-time)
sudo tail -f /var/log/syslog # Monitor system logs in real-time
Reverse Display with tac:
tac file.txt # Displays file content in reverse order
Line Numbering
nl file.txt # Adds line numbers to output
nl -b a file.txt # Number all lines (including empty)
nl -w 3 file.txt # Set number width to 3 characters
Automation and Bulk Operations
Loop-Based File Generation
Create multiple log entries efficiently:
# Basic loop - generates 50 numbered events
for i in {1..50}; do echo "Event $i" >> events.log; done
# Loop with timestamps
for i in {1..10}; do echo "$(date): Event $i" >> timestamped_events.log; done
# Generate test data
for i in {1..100}; do echo "User$i,[email protected],$(( RANDOM % 100 + 18 ))" >> users.csv; done
# Create multiple files
for i in {1..5}; do echo "Content for file $i" > "test_file_$i.txt"; done
File Permissions and Security
Finding Files by Permissions
find . -perm 777 -type f # Files with full permissions
find . -perm 755 -type d # Directories with standard permissions
find /home -perm 666 -type f 2>/dev/null # Files readable/writable by all
find . -name "*.sh" -perm +111 # Executable shell scripts
Permission Number Breakdown:
- 7 = rwx (read, write, execute)
- 6 = rw- (read, write)
- 5 = r-x (read, execute)
- 4 = r– (read only)
Advanced Text Processing
Combining Commands: cat and grep
# Basic usage
cat file.txt | grep "pattern" # Filter content by pattern
cat *.log | grep "ERROR" # Find errors in all log files
cat file.txt | grep -i "case_insensitive" # Case-insensitive search
cat file.txt | grep -v "exclude_this" # Invert match (exclude pattern)
cat file.txt | grep -n "pattern" # Show line numbers
# Advanced grep flags
grep -r "pattern" /var/log/ # Recursive search in directory
grep -A 3 "ERROR" logfile.txt # Show 3 lines after match
grep -B 2 "ERROR" logfile.txt # Show 2 lines before match
grep -w "word" file.txt # Match whole words only
Practical Examples:
# Log analysis
cat /var/log/auth.log | grep "Failed password" | head -10
ps aux | grep "apache" # Find Apache processes
cat /etc/passwd | grep -E "/bin/(bash|sh)$" # Find users with shell access
Stream Editing with sed
The sed command is essential for large-scale file modifications:
# Basic substitution
sed 's/old/new/' file.txt # Replace first occurrence per line
sed 's/old/new/g' file.txt # Replace all occurrences (global)
sed 's/old/new/gi' file.txt # Global, case-insensitive
# Line operations
sed '5d' file.txt # Delete line 5
sed '/pattern/d' file.txt # Delete lines matching pattern
sed '/^$/d' file.txt # Delete empty lines
# In-place editing
sed -i 's/old/new/g' file.txt # Modify file directly
sed -i.bak 's/old/new/g' file.txt # Create backup before modifying
Practical Examples:
# Configuration file editing
sed -i 's/DEBUG/INFO/g' application.conf # Change log level
sed 's/^/[PROCESSED] /' logfile.txt # Add prefix to each line
sed 's/[[:space:]]*$//' file.txt # Remove trailing whitespace
Regular Expressions
Regular expressions (regex) provide powerful pattern matching:
# Basic patterns
grep "^start" file.txt # Lines starting with "start"
grep "end$" file.txt # Lines ending with "end"
grep "[0-9]" file.txt # Any digit
grep "[a-zA-Z]" file.txt # Any letter
# Advanced patterns
grep -E "word1|word2" file.txt # Either word1 or word2
grep -E "[0-9]{3}" file.txt # Exactly 3 digits
grep -E "\b[0-9]{1,3}(\.[0-9]{1,3}){3}\b" file.txt # IP addresses
Conclusion
These Linux commands form the foundation of effective file management and analysis. Whether you’re troubleshooting system issues, analyzing logs, or conducting security investigations, mastering these tools will significantly enhance your productivity and capabilities.
Remember to always test commands in a safe environment before applying them to production systems, especially when dealing with file modifications and deletions.
