Trick grep / egrep to always display filename

If you use grep / egrep on a single file, it will display the matched patterns, but not the filename nor the line number.

This can be boring in case you encapsulate a grep / egrep call inside i.e a find:

find ./mydir -type f -name '*.msg' -exec grep 'mysearch' {} \;

In that case it will only print the match and you will not know in which file.

The trick is to add a dummy filename as a second file to search. Doing so will trick grep / egrep into thinking you are searching in multiples files, thus displaying the filename on each match.
/dev/null is the perfect target for the second dummy file as it will never match anything.

Example:

find ./mydir -type f -name ‘*.msg’ -exec grep ‘mysearch’ {} /dev/null \;

Bonus / reminder:

In grep / egrep use the -n option to also display the match line number

Example:

find ./mydir -type f -name ‘*.msg’ -exec grep -n ‘mysearch’ {} /dev/null \;

#sysadmin #tips #tricks #grep #egrep

KAFKA

KAFKA is in all mouths these days. So I share here, also as a reminder for me, a couple of #KAFKA ressources that looked helpful to me.

TLDR:

Do not use KAFKA for:

-“Little” Data flows (overkill)

-Streaming ETL (handling transformation is a hassle)

-Store and process large files (images, videos, proprietary files, etc.)

https://www.kai-waehner.de/blog/2020/08/07/apache-kafka-handling-large-messages-and-files-for-image-video-audio-processing/

https://www.upsolver.com/blog/apache-kafka-use-cases-when-to-use-not

https://kafka.apache.org/intro

https://docs.confluent.io/kafka-clients/librdkafka/current/overview.html

https://docs.confluent.io/platform/current/clients/librdkafka/html/md_INTRODUCTION.html

Cargo Cult Sytem Administrator

And some other types too.

Fun article to read here: https://blog.lastinfirstout.net/2009/11/cargo-cult-system-administration.html

One particularly right: “Asserting that [Technology O] or [Platform L] or [Methodology A] is inherently superior to all others and blindly applying it to all problems. When you make such claims, are you applying science or religion?”

“It’s easy to fall into cargo cult mode.
Just re-boot it, it’ll be fine.” – Michael Janke

Clean cache to reclaim memory without a reboot

Tip submitted by my friend J.K when we were in need of a way to recover reserved cache space without booting.


In root:

PageCache only:

sync ; echo 1 > /proc/sys/vm/drop_caches

Dentries and inodes:

sync ; echo 2 > /proc/sys/vm/drop_caches

PageCache, Dentries and inodes:
(not recommanded in production as it forces a full cache rebuilt. you may still need it):

sync ; echo 3 > /proc/sys/vm/drop_caches

In case you need to do it in a sudo:

echo 3 | sudo tee /proc/sys/vm/drop_caches

Syslog, rsyslogd: change output format to display log level and facilities

If ever you need to sort log messages by their log level yo may need to change the default output format of the syslog.
For rsyslog it’s located in /etc/rsyslogd.conf. Add these lines after the line “$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat” :

$template precise,”%timegenerated% %HOSTNAME% {%syslogpriority%,%syslogfacility%} %syslogtag% %msg%\n”
$ActionFileDefaultTemplate precise

Then restart rsyslogd:
sudo service rsyslog restart

#rsyslogd #syslog #log #loglevel #facilities