Post

Linux Plus-Managing Files

Posted Updated
By Anthony Yeung 8 min read

image

Managing Linux Files

1
2
3
4

tty
ls -l

ls -l $(tty)

Working with Files and Directories

  • FHS (File Hierarchy Standard)
  • Changing Directories
  • Listing Directory and File Metadata
  • Deleting Directories

Install Tree

1

sudo apt-get install tree

1

tree -L 1 /

Output

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

eung@PB-3157:/$ tree -L 1 /
/
├── bin -> usr/bin
├── boot
├── dev
├── etc
├── home
├── init
├── lib -> usr/lib
├── lib32 -> usr/lib32
├── lib64 -> usr/lib64
├── libx32 -> usr/libx32
├── lost+found
├── media
├── mnt
├── opt
├── proc
├── root
├── run
├── sbin -> usr/sbin
├── snap
├── srv
├── sys
├── tmp
├── usr
└── var

23 directories, 1 file

1

ls - ld /dev

Output

1
2

 ls -ld /dev
drwxr-xr-x 16 root root 3560 Jul 30 13:20 /dev

Go back to the previous directory

1

cd -

or

1

echo $OLDPWD

Create a Directory

1
2

mkdir test
ls -ld test

Output

1

drwxr-xr-x 2 user1 user1 4096 Jul 31 21:51 test

List of parent directory

1
2

mkdir -p test/d1/d2
tree test

Output

1
2
3
4
5
6

tree test
test
└── d1
    └── d2

2 directories, 0 files

Create a list of directories

1
2

mkdir test/d{10..19}
tree test

Output

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

test
├── d1
│   └── d2
├── d10
├── d11
├── d12
├── d13
├── d14
├── d15
├── d16
├── d17
├── d18
└── d19

12 directories, 0 files

Delete all the directories within and include test directory

1

rm -rf test

Output

1
2
3
4

tree test
test  [error opening dir]

0 directories, 0 files

Creating a new file

  • The touch command is often used to create new empty files but there is more to it
  • Changes both acess and modified times, change will always update
1

touch file1
  • Changes access time
    1

    touch -a file1
  • Changes modified time
    1

    -touch -m file1

Looking into the file details - Timestamp

Field

  • Access - Last read of the file
  • Modify - Data last modified
  • Change - Metadata last changed
1
2
3
4

stat file1
stat -c %x file1 #Access
stat -c %x file1 #Modify
stat -c %c file1 #Change

Archive Mode (-a)

To maintain metadata when copying files, we can use the option -a or –archive, ownership, permissions and time stamps are all copied with the file.

-a is equals to -dR –preserve=all

Copy a file

1

cp file1 file2

To copy and maintain the metadata

1

cp -a file1 file3

Understanding File Metadata

  • Sorting files my last modified time
  • Using stat
  • Using touch
  • Preserving metadata
1

ls -ltr

  • -r - reverse sorting

  • File filename -
    1

    file profile.d

  • stat . - It will bring up the last argument.

  • stat filename - Show detail of the file
1
2

touch file1
stat file1

Output

1
2
3
4
5
6
7
8

  File: file1
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 820h/2080d      Inode: 5636        Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  ayeung)   Gid: ( 1000/  ayeung)
Access: 2024-08-07 14:19:27.133723941 -0700
Modify: 2024-08-07 14:19:27.133723941 -0700
Change: 2024-08-07 14:19:27.133723941 -0700
 Birth: 2024-08-07 14:19:27.133723941 -0700

IF you use the cat file1, the Access (Time) will be changed.

1
2
3
4
5
6
7
8
9

/etc$ stat profile.d
  File: profile.d
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 820h/2080d      Inode: 635         Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-08-06 12:25:28.211466106 -0700
Modify: 2024-08-07 10:39:34.476028582 -0700
Change: 2024-08-07 10:39:34.476028582 -0700
 Birth: 2024-08-07 14:19:27.133723941 -0700

The Modify and Change time will be changed once using echo command as follow

1
2

echo hello >> file1
stat file1

Only the Change time will upated after using chmod 770 file1

1
2

chmod 770 file1
stat file1

Output

1
2
3
4
5
6
7
8

  File: file1
  Size: 6               Blocks: 8          IO Block: 4096   regular file
Device: 820h/2080d      Inode: 5636        Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  ayeung)   Gid: ( 1000/  ayeung)
Access: 2024-08-07 14:19:27.133723941 -0700
Modify: 2024-08-07 14:20:07.433701948 -0700
Change: 2024-08-07 14:20:35.433701948 -0700
 Birth: 2024-08-07 14:19:27.133723941 -0700

Using stat -c individual elements of the metadata,

%x - option - is the file was last read %y - option - is the last modified time %z - option - is the last change time

1

stat -c %x file1

Remember if we use the cp copy command, the metadata of the copied file (file2) will not be the same as the original file, file1.

1

stat file1

1
2

cp file1 file2
stat file2

Using mv command (rename or move), only will change the Change of the new file

In this case, file1 renamed as file4, only the Change time is updated

1

mv file1 file4

Using Editor in Linux

When working at the Linux CLI, people will soon settle on their editor. The editor nano has a short learning curve while vim is more powerful, the learning curve is longer.

Install vim

1

sudo dnf install -y vim-enhanced nano rsync git-core

Install nano

1

sudo nano /etc/ssh/ssjd_config

Install vim

1

sudo vim /etc/ssh/sshd_config

Using the Stream Editor

To script or automate edits, always the goal in DevOps, we can look at sed, the Linux stream editor

The Ubuntu Vagrant system does not endable password-based SSH authentication by default

  • E - enhanced regular expression that allows to use the parentheses
  • i - means inplace edit
  • s - within the string that we pass through to sed is then saying substitute, and the text that we want to look for to substitute is, then (PasswordAuthentication)no.
  • The parentheses allow me to use groups, so I can then reference the replacement text being the first group, ie. PasswordAuthentication, and configuring it to be yes
1
2

sudo sshd -T | grep passwordauthentication
sudo sed -Ei 's/(PasswordAuthentication) no/\1 yes/' /etcssh/sshd_config

Enable password

1

sudo systemctl restart sshd

For more commands using VIM editor, use VIM Tutor

1

vimtutor

To use Ubuntu SSH server using password based authentication, we need to enable it

1

sudo sshd -T | grep passwordauthentication

1

sudo grep -i PasswordAuthentication /etc/ssh/ssh_config

Output

1

PasswordAuthentication yes

If Password Authentication was set to No, use the following commands to enable password authentication

1

sudo sed -Ei 's/PasswordAuthentication no/PassowrdAuthentication yes/'

or using regular expression to enable password authentication.

1

sudo sed -Ei 's/(PasswordAuthentication) no/\1 yes/' /etc/ssh/sshd_config

Once the password authentication enabled, we need to restart the systemctl. And we can double check if password authentication has been enabled.

1
2

sudo systemctl restart sshd
sudo sshd -T | grep passwordauthentication

Remote Copy

Whilst scp is ok to copy single file to remote systems, rsync is better equipped to copy complete directory structures and keep them up to data

1
2
3
4

sudo useradd -m user1
sudo passwd user1
scp file1 user1@192.168.33.13

Look for all HTML files and copy those int our docs directory.

1
2
3

mkdir docs
find /usr/share/doc -name '*.html'  -exec cp {} docs/ \;
rsync -ave ssh docs user1@192.168.33.13:

Creating SSH Git Repositories

Working in DevOps, you will certainly come across the VCS (Version Control System) git, allow colaboration in development. Rather than using a hosting system such as GitHub or GitLabs we can creat our own SSH Host. Ubuntu

1
2

su - user1 && mkdir project1 && cd project1
git init --bare

Alma

1
2
3
4
5
6
7

git clone user1@193.168.33.13:/home/user1project1
cd prject1 && vim my.sh && chmod 755 my.sh
git add .
git config --global user.email "your@example.com"
git config --global user.name "Your name"
git commit -m "initial commit"
git push origin master

Opensuse

1
2

sudo zypper in -y git-core
git clone user1@192.168.33.13:/home/user1/project1

remote copy

1
2

sudo useradd -m user1
sudo passwd user1

1

scp file1 users1@192.168.33.13:

1
2
3
4

sudo rsync
find /usr/share/doc -name '*.html' -exec cp {} docs/ \;
ls docs
rsync -ave ssh docs user1@192.168.33.13:

Objectives

  • File Compression Using Compression Tools
  • Archiving Using cpio

Creating TAR Files

The command tar can be used to create file archives. Althouth, Tape Archives, they are more commonly used in standard filesystems. By default, a TAR file is not compressed but may appear to be a slightly small size than the original content. This is due to the more efficient use of blocks in the filesystem and not compression

Find the size of our etc directory

  • du - disk utilization
  • s - summary
  • h - human readable
1
2

sudo du -sh /etc
22M /etc

Create archive with a specify the file to back up

  • -c - create
    1
2
3
4

    sudo tar -cf etc.tar /etc
tar: Removing leading '/' from member names
ls -lh etc.tar
-rw-r--r--. 1 root root 21M Jan 18 10:49 etc.tar

    Note: The most common block size is 4K on Intel based systems. Each new file goes to a new block potentially wasting space

Tar Operations

  • -c - Create
    1

    tar -cf file.tar file1 file2
  • -t - Table of Contents or –verify
    1

    tar -tf file.tar
  • -x - Extract
    1

    tar -xf file.tar

More Examples

We will investigate the basic tar operations

For summary, look at the disk usage of a directory

1

du -s

or To overcome permission on files

1
2
3

sudo !!
sudo du -sh /etc 
23M /etc

1
2

sudo tar -cf etc.tar /etc
tar: Removing leading `/' from member names

To test (verify) the archive,

  • -t or –verify
    1

    sudo tar -tf etc.tar /etc
Linux
Linux
This post is licensed under CC BY 4.0 by the author.