heWhat do you do if you suspect your server (VPS or dedicated) has been compromised? If you are a customer, you have the option to leverage our team to perform the incident response on your behalf. What if you want to do an investigation on your own?
In this post, we will talk about Rootcheck, an open source command line tool that looks for indicators of compromise on Linux or BSD systems. It looks for known backdoors, kernel-level rootkits, malware and insecure configuration settings. It performs a few tests that will certainly help you during your hack investigation.
Rootcheck is included as part of OSSEC (which we also recommend when running your own server). It provides continuous endpoint monitoring, alerting and visibility into your environment. However, if you have not been running OSSEC or are just inspecting a server to see the current state, Rootcheck can be executed on demand and will provide a baseline view of the system.
Getting started with Rootcheck
Rootcheck is packaged separately and can be downloaded directly from our code repository or from here to get the latest stable version:
$ wget http://dcid.me/ossec-packages/rootcheck-latest.tar.gz
After downloading Rootcheck, untar the package, run the install script and get it running:
$ tar -zxvf rootcheck-latest.tar.gz
$ cd *rootcheck*
$ sudo ./install.sh
Once the installation is finished, you will see a file called “rootcheck” created in your current directory. Execute that file to get the rootcheck audit started:
$ sudo ./rootcheck
..
** Starting Rootcheck v2016-02 **
** http://dcid.me/rootcheck/ **Be patient, it may take a few minutes to complete…
[INFO]: Starting rootcheck scan.
[OK]: No presence of public rootkits detected. Analyzed 269 files.
[OK]: No binaries with any trojan detected. Analyzed 76 files.
You should see the same message as above, informing you that Rootcheck has started and is scanning your server. If any rootkits are identified, you will be alerted. For example, if you have an out-of-date CMS (i.e., WordPress, Drupal or Joomla), the following alerts will be displayed:
[INFO]: System Audit: Web vulnerability – Outdated Drupal 7 installation. File: /var/www/vhosts/site.com/includes/bootstrap.inc. Reference: http://sucuri.net/latest-versions .
[INFO]: System Audit: Web vulnerability – Outdated Drupal 7 installation. File: /var/www/vhosts/site2.com/includes/bootstrap.inc. Reference: http://sucuri.net/latest-versions .
If it finds web-based malware, like the Filesman backdoor, it will provide the following alert:
[INFO]: System Audit: Web malware – Backdoors / Web based malware found – Filesman shell. File: /var/www/vhosts/site.com/lic.class.php.
Additionally, as part of the audit, Rootcheck will parse through your web logs on the server and identify the latest login attempts and their location and display it as such:
[INFO]: Latest successful logins to the server:
root |28 |
srcip: ‘192.x.x.x / USA / New Jersey’
srcip: ’75.x.x.x / USA / California’
srcip: ‘170.x.x.x / USA / Utah’
admin |1 |
srcip: ‘132.x.x.x / Russia’
Try it out and let us know how it goes!
Always awesome software from you Daniel. How about another writeup version that includes first archiving an image of the system, then rootcheck the system from ISO and read-only filesystem mounts?
Thanks! That is the ideal process, but wouldn’t work very well with rootcheck. Most of the checks require a live “compromised” system as it tries to analyze the ports, files, kernel and the relation between them.
Thank you Daniel. One of your more valuable posts IMHO.
Thanks! 🙂
Hi Daniel, do you think is useful to run it , via cron, once a day to check if the server is fine?
Thanks
In that case, it might be better to run OSSEC directly. OSSEC automates rootcheck on a pre-defined schedule and you get the log monitoring + integrity checking with it.
Step by step doc on how to install it:
http://dcid.me/texts/my-ossec-setup-manual.html
Blless you Sir for all your good works…we are in your wake and we appreciate you prolly more than you realise. Just wanna say ‘Thank you’
Hi Daniel, great post, for the alerting system above, i didn’t see the alert level/risk, will you provide this feature for later development or you actually didn’t want to? Because when i want to integrate it with SIEM for example, it will be a great information to know the risk/level for every alert we received
Thanks
Hi Daniel, thanks for the great article. I ran rootcheck on a test server left untouched for months and found this:
[FAILED]: Trojaned version of file ‘/bin/passwd’ detected.
[FAILED]: Trojaned version of file ‘/usr/bin/passwd’ detected.
Should I simply nuke both files?
@pageii:disqus That’s interesting. I tested on a few cpanel servers and didn’t get this error. Mind sending me an email (dcid @ thisdomain) so we can investigate?
The trojan item pointed to a necessary symlink file at /usr/local/cpanel/bin/jail_safe_passwd. I’ve emailed you the details. Hope this info was of some help.
Why should i use rootcheck instead of maldet or rkhunter? Does it bring extra features ? Does it excell somewhere else? Should it instead by used complementary with the other opensource tools i’ve mentioned?