Reviewing your logs sounds boring but it’s a necessity. You can easily have an ssh session open in a small putty window while tailing a log while doing other work and if you glance at it once in a while you might see interesting things.
For example, hacker losers of the world like to dump files onto web servers. Those files might do other things like create back doors onto the site so they can upload more hacker loser files.
Here’s an example of a line I saw on an Apache server the other day:
220.127.116.11 – – [05/Sep/2016:09:56:24 -0400] “GET /wp-content/themes/twentytwelve/search.php HTTP/1.1″ 404 – “-” “Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:29.0) Gecko/20100101 Firefox/29.0”
Hmmmmm…what’s interesting about this you might ask?
Well for starters it doesn’t look normal. Firefox 29 is OLD…yes, they’re up to something like version 48 at the time of this post. I believe that around version 29 they had automatic updating implemented so it’s unusual to see such an old browser tapping the server directly to a search.php file. Also, I did a whois lookup on that IP address. It’s a French IP from a SAS provider (a.k.a VPS – Virtual Private Server) which has become the plague of the Internet allowing the hacker loser community to either stand their own infrastructure up and then blow it away when they’re caught or hack weak targets from companies who don’t secure those VPS’s and trust the owners to do it (yeah like that’s going to happen!).
Looking through the log and grepping for the “X11” string which I thought was unique I saw this:
18.104.22.168 – – [04/Sep/2016:21:47:36 -0400] “POST /fqdnwashere.com/wp/wp-
A similar string in the log but this time it was a “POST” not a “GET”. The similar Firefox 29 shows up again. I looked at that file. Yup…it was a hacker loser file that got uploaded to that website and every time it was tapped it sent out SPAM emails. So the hosted website was spamming thanks to a malicious PHP file uploaded and then the zombie servers are instructed to tap it with a “POST” request. The POST request does something (I don’t know what) to send out spam emails from random firstname.lastname@example.org website (by the way that domain was edited out and I called it fqdnwashere.com).
I remediated the situation by removing that start.php file and replacing it with my own file. That way the zombie servers still hit it, get their “200” success code and I can keep adding them to my block list. I notified a few ISP’s and hosts who’s servers are zombie slaves of some hacker loser (a.k.a a “bot” or “bot net”) but didn’t have time to tell everyone.
What I did then was added anyone hitting the “start.php” file among a few other things to my counter bot or anti-hacker bot that adds those IP’s to the block list we maintain.
It pays to pay attention to your logs, that’s what they’re there for! Check it or wreck it.
This was the start of Whackers For Hackers IPBL. Hackers attacking our infrastructure continually.