Create a filesystem report with Python

In many cases, data accumulates over time, and a lot of this data is not being accessed at all, sometimes for years. Accordingly, it would be great if you could create a report that shows which data, the last access date, its size and its owner.

With a few lines of Python, you can very efficiently traverse (I named it traverse.py) any mounted filesystem and save to a CSV file that has the following set of data:

  • Filename (with absolute path)
  • Last Access Date & Time
  • File Size (in bytes)
  • UID or username of the owner
import os, sys, time
import pathlib #pathlib only needed if fileowner will be a string not UID

for root, directories, filenames in os.walk(sys.argv[1]): #Loop through the filesystem or folder passed as an argument
    for filename in filenames:
        try:
            fullpath = os.path.join(root, filename) #Get full path of the file
            timelastaccessed = time.ctime(os.stat(fullpath).st_atime) #Get last access time
            filesize = os.stat(fullpath).st_size #Get file size
            fileowner = os.stat(fullpath).st_uid #Get file owner as a UID
            #fileowner = pathlib.Path(fullpath).owner() #Get file owner as a string, much slower
            print(fullpath, ",", timelastaccessed, ",", filesize, ",", fileowner, sep='')  #Format to output in CSV
        except OSError:
            print("Path does not exist or is inaccessible") #In case the file was inaccessible
        except UnicodeEncodeError:
            print("Encoding Error") #In case of files with names that throw a UnicodeEncodeError exception

To execute, just run:

python3 traverse.py filesystem_or_folder >> output.csv

The above outputs the UID, but if you want to output the username itself, just comment the fileowner = os.stat(xxxxx) and uncomment the next line that uses pathlib

You can then import that CSV file into any available analytics platform like PowerBI and do your magic!

That’s it! Enjoy!

Posted in Linux | Tagged , , , , | Leave a comment

Script to count number of logged in users to OpenVPN

To get that information you need to query your openvpn-status.log file, which usually resides in /var/log/openvpn/openvpn-status.log.

A normal cat will show the following:

OpenVPN CLIENT LIST
Updated,Wed Mar 5 11:57:51 2019
Common Name,Real Address,Bytes Received,Bytes Sent,Connected Since
someuser,someip:55530,100686479,319036434,Tue Mar 5 03:22:04 2019
someuser,someip:58064,23120958,141190549,Tue Mar 5 12:30:05 2019
someuser,someip:7115,18011423,65882394,Tue Mar 5 14:26:00 2019
someuser,someip:4463,79682826,444263153,Tue Mar 5 08:01:52 2019
ROUTING TABLE
Virtual Address,Common Name,Real Address,Last Ref
xxx.xxx.xxx.xxx,someuser,someip:47080,Tue Mar 5 12:51:44 2019
GLOBAL STATS
Max bcast/mcast queue length,46
END

The command to get number of connected users at any given time is:

echo $((sed '/ROUTING TABLE/Q' /var/log/openvpn/openvpn-status.log | wc -l-3)) "users connected on" date +"%m-%d-%Y %H:%M"
4 users connected on 03-05-2019 13:23 #Just an example, not real value

This basically parses the text till reaching the line with “ROUTING TABLE”, then counts the lines and subtracts 3 from them due to the first unneeded 3 lines

That’s it! Enjoy!

Sources:

Posted in Linux | Tagged , , | Leave a comment

What to do when ctrl+c not killing your telnet command

To check if ports are open or not, we usually do the following:

telnet IP PORT

For example:

telnet 192.168.100.5 25

Sometimes after seeing your result, you keep pressing ctrl+c, ctrl+x or ctrl+\ to force quit it, but it just doesn’t! It seems silly I know and I feel a bit stupid for writing this, but it was driving me crazy at some point!

Anyway, just press ctrl+5 or ctrl+] and then type quit, that’s it!! This works the same way on Linux, MAC and Windows.

user@server:~$ telnet xxx.xx.xxx.xxx 25
Trying xxx.xx.xxx.xxx…
Connected to xxx.xx.xxx.xxx.
Escape character is '^]'.
220 *
^C^C^C^C^C^C^C^C^C^C^C^C^C
^C^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z
^\^\^\^\^\^\^\^\^\^\^\^\^\
^]
telnet> quit
Connection closed.

Sources:

https://superuser.com/questions/486496/how-do-i-exit-telnet

Posted in Linux | Tagged , | Leave a comment

How to reset display settings for KDE Plasma

In case your display arrangement or settings, e.g. when you have 3 screens and you are for some reason unable to reorganize them or change their settings, just reset your display settings:

rm -rf .local/share/kscreen

Then just log out and back in.

That’s it! Enjoy!

Sources:

Posted in Linux | Tagged , , , | Leave a comment

How to force resync between PBIS and AD on Linux

In case you made a change on your AD, such as modifying groups or changing attributes of a user, it sometimes take a little while to reflect that on the Linux server that authenticates against AD via PBIS.

The best solution in that case is to clear the pbis cache and force pbis to sync immediately via this command:

sudo /opt/pbis/bin/ad-cache --delete-all

Enjoy!

Posted in Linux | Tagged , , , , | Leave a comment

How to change max open files for nginx server on Ubuntu

The following error occurred a lot in /var/log/nginx/error.log:

Too many open files

This means that the nginx master process reached the maximum number of open files which was set to the default of 4096. (To check it, simply cat /proc/pid/limits, where pid is the process id)

To resolve, this, edit /lib/systemd/system/nginx.service and add the following line:

LimitNOFILE=30000 #change it according to your needs

Then reload the systemctl daemon:

sudo systemctl daemon-reload

and finally restart the nginx service:

sudo service nginx restart

That’s it, the error is gone!

Tested on Ubuntu 16.04 with nginx 1.17, should work with any other OS using systemd.

Enjoy!

Sources:

Posted in Linux, Ubuntu | Tagged , , , , , | Leave a comment

Add a new HDD to a VM without rebooting on Ubuntu server

Firstly, install scsitools package:

sudo apt install scsitools

Then run the command rescan-scsi-bus:

sudo rescan-scsi-bus

That’s it, your new virtual HDD has now been detected.

Another way would be to do it with a loop and without the need for that package:

for host in ls /sys/class/scsi_host/; do
echo "- - -" > /sys/class/scsi_host/${host}/scan;
done

Enjoy!

Sources:

Posted in Linux | Tagged , , , | Leave a comment