My text editor of choice when I'm in Linux-land is Vim. One of its most powerful features is the ability to use regular expressions when searching. This capability makes hunting through giant log files so easy. However, I ran into a problem today that I wasn't sure how to work around.
I was looking through a large log file, trying to find the lines that included the text Internal Server Error. Making this difficult, however, were hundreds of such entries that were calling a known failing case. I wanted to weed out these known cases, and with regular expressions, I was able to. The known case I wanted to ignore looked something like the following:
[02/Feb/2023 16:06:14] ERROR [log.py:224] Internal Server Error: /api/v1/some_bad_endpoint/
This endpoint is the only one in the
/api root, so I wanted to ignore
/api as a part of my search. The magic is through negative look-ahead assertions, the syntax for which I was unfamiliar (
\@! being the key). My search command in Vim ended up being:
/Internal Server Error: \/\(api\)\@!
Using this regular expression helped me jump to every instance of the Internal Server Error text that wasn't a call to the known-failing
/api root. So handy!
Gary10:51 AM on Feb 11, 2023
Without vim or if the logs were really large or compressed, I'd use a pipeline with grep.
zcat log.gz | grep "Internal Server Error" | grep -v /api | less
Jonah4:16 PM on Feb 12, 2023
Yes, a pipeline is probably the easiest solution. Good tip.