Over the past few weeks our Security Operation Center (SOC) has been seeing some unique and very suspicious requests to some of our servers. At first we thought it was a Distributed Denial of Service (DDoS) attack, mainly due to the high concentration of requests (thousands per second). Looking further however, it actually seems like some DNS resolver was broken and consequently redirecting all of their users traffic to us.
Here are some example requests, see for yourself:
GET /100004020560199/picture HTTP/1.1 Host: graph.facebook.com
or
GET /extstyle.css HTTP/1.1 Host: static.ak.facebook.com User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36
or
GET /wp-content/themes/vip/nesn/images/nesn_favicon_128.png Host: s0.wp.com
If you do not understand why these requests are interesting, look at the Host: header. It means someone tried to visit static.ak.facebook.com or s0.wp.com and somehow reached our servers. Since we do not host Facebook or WordPress.com, it generates an error on our end.
These were not the only domains that were trying to reach us. Just in the last day, we received requests for: farm4.static.flickr.com, 24.media.tumblr.com, cdn11.optimecdn.com, l.longtailvideo.com, platform.twitter.com, media-cdn.tripadvisor.com, analytics.twitter.com, m.facebook.com, graph.facebook.com, assets.zendesk.com and many other domains.
Why is Facebook, Twitter, WordPress and Zendesk pointing to Sucuri’s Website Firewall (CloudProxy)?
That was a little mystery that took us a bit of time to understand. At first, we thought it was just a new form of DDoS trying to use random domains names to evade our detection.
However, the request headers looked very legitimate. Even via passive fingerprinting, we were able to properly tie the operating system to the browser and the user agent. It also didn’t look like a DDoS botnet that we could identify. To our surprise, it seemed like real browser requests from valid users.
There was just one catch: all requests were coming from China.
We shared our logs and finding with multiple peers in the security community and the consensus was that these requests were not malicious per se. It seemed as if the Great Chinese Firewall was mis-configured, instead of blocking the requests to certain sites, it was redirecting, to us at that.
So if a specific site was blocked, the requests to graph.facebook.com also got blocked and redirected to us. Same for Twitter, Zendesk or Tumblr.
This explains why most of the requests were actually for CDN, images or API files.
Why is the Chinese Firewall Doing That?
That’s a good question and one we do not know the answer to. We can speculate that it is a bug on their end, but can’t be sure. It actually seems similar to the issue that TorrentFreak reported with Pirate bay requests being redirected to random IP addresses.
However, instead of Pirate Bay, it is happening with the most popular platforms and CDNs out there to some of our IP addresses. These “fake” attempts generate thousands of requests per second from thousands of different Chinese IP addresses. It would certainly be enough to DDoS most servers.
Is anyone else seeing something similar?
Full List of Domains
If anyone is curious, these are all the domains that reached us just today:
host: “10.media.tumblr.com”,
host: “11.media.tumblr.com”,
host: “12.media.tumblr.com”,
host: “13c3a.http.cdn.softlayer.net”,
host: “15.media.tumblr.com”,
host: “16.media.tumblr.com”,
host: “17.media.tumblr.com”,
host: “18.media.tumblr.com”,
host: “1.media.tumblr.com”,
host: “20.media.tumblr.com”,
host: “22.media.tumblr.com”,
host: “23.media.tumblr.com”,
host: “24.media.tumblr.com”,
host: “26.media.tumblr.com”,
host: “28.media.tumblr.com”,
host: “29.media.tumblr.com”,
host: “2.bp.blogspot.com”,
host: “2-edge-chat.facebook.com”,
host: “2.media.tumblr.com”,
host: “30.media.tumblr.com”,
host: “3-edge-chat.facebook.com”,
host: “3.media.tumblr.com”,
host: “4.media.tumblr.com”,
host: “5.media.tumblr.com”,
host: “6.media.tumblr.com”,
host: “7.media.tumblr.com”,
host: “8.media.tumblr.com”,
host: “987hh.com-www.45878.com”,
host: “9.media.tumblr.com”,
host: “a1.dspnimg.com”,
host: “a248.e.akamai.net”,
host: “a4.ec-images.myspacecdn.com”,
host: “abs.twimg.com”,
host: “accounts.youtube.com”,
host: “ad-audit.tubemogul.com”,
host: “a.deviantart.net”,
host: “adn.6638.edgecastcdn.net”,
host: “ads.exoclick.com”,
host: “ads.gayfriendfinder.com”,
host: “ads.w55c.net”,
host: “am.6park.com”,
host: “analytics.twitter.com”,
host: “api.facebook.com”,
host: “apis.google.com”,
host: “api.twitter.com”,
host: “apps.facebook.com”,
host: “assets1.whicdn.com”,
host: “assets2.whicdn.com”,
host: “assets3.whicdn.com”,
host: “assets.crucial.com”,
host: “assets.mltd.com”,
host: “assets.modelmayhem.com”,
host: “assets.zendesk.com”,
host: “badge.facebook.com”,
host: “banners.videosecrets.com”,
host: “bbc6.global.ssl.fastly.net”,
host: “blogs.ocweekly.com”,
host: “bp0.blogger.com”,
host: “b.s-static.ak.facebook.com”,
host: “cache.armorgames.com”,
host: “cache.blogads.com”,
host: “cdn03.cdn.justjaredjr.com”,
host: “cdn11.optimecdn.com”,
host: “cdn1.barong.inxy-host.com”,
host: “cdn1.editmysite.com”,
host: “cdn1.nudevector.com”,
host: “cdn1.sidhe.co.nz”,
host: “cdn2.editmysite.com”,
host: “cdn2.search.xxx”,
host: “cdn3.aptoide.com”,
host: “cdn3.everyjoe.com”,
host: “cdn3.howtogeek.com”,
host: “cdn5.howtogeek.com”,
host: “cdn.adgear.com”,
host: “cdn.androidcommunity.com”,
host: “cdn.api.twitter.com”,
host: “cdn.collider.com”,
host: “cdn.c.photoshelter.com”,
host: “cdn.easyhotpics.com”,
host: “cdnedge.vinsolutions.com”,
host: “cdn.epom.com”,
host: “cdn.everyjoe.com”,
host: “cdn-frm-sg.wargaming.net”,
host: “cdn.gayboysbox.com”,
host: “cdn.gaycnn.com”,
host: “cdn.gaydudestube.net”,
host: “cdn.gq.com.tw.s3-ap-northeast-1.amazonaws.com”,
host: “cdng.vpnpie.biz”,
host: “cdnimages.gayhits.com”,
host: “cdn-images.mailchimp.com”,
host: “cdn.lfstmedia.com”,
host: “cdn-marketools.plus500.com”,
host: “cdn-mkt.wooga.com”,
host: “cdn.nitrome.com”,
host: “cdn.nudevector.com”,
host: “cdn.porn-lab.com”,
host: “cdn.pornvideospider.com”,
host: “cdn.ps.teenmodels.com”,
host: “cdn.recruitnet.co”,
host: “cdn.sidhe.co.nz”,
host: “cdn.slashgear.com”,
host: “cdn.soundstagedirect.com”,
host: “cdn.tubeporndiet.com”,
host: “cdn.usablenet.com”,
host: “cdn.xgaybox.com”,
host: “cms.myspacecdn.com”,
host: “cn.epochtimes.com”,
host: “cn.wsj.com”,
host: “comps.fotosearch.com”,
host: “content.onhotels.com”,
host: “css.c.photoshelter.com”,
host: “csync.flickr.com”,
host: “cti.w55c.net”,
host: “dc108.4shared.com”,
host: “dc200.4shared.com”,
host: “dc204.4shared.com”,
host: “dc205.4shared.com”,
host: “dc219.4shared.com”,
host: “dc265.4shared.com”,
host: “dc317.4shared.com”,
host: “dc327.4shared.com”,
host: “dc335.4shared.com”,
host: “dc644.4shared.com”,
host: “dc672.4shared.com”,
host: “dc733.4shared.com”,
host: “dingo.care2.com”,
host: “dl6.offercdn.com”,
host: “dl-web.dropbox.com”,
host: “docs.google.com”,
host: “drive.google.com”,
host: “e1.static.hoptopboy.com”,
host: “ecdn.liveclicker.net”,
host: “eg-img.agoda.net”,
host: “eg.img.agoda.net”,
host: “emoneycreater.appspot.com”,
host: “farm1.static.flickr.com”,
host: “farm2.static.flickr.com”,
host: “farm3.static.flickr.com”,
host: “farm4.static.flickr.com”,
host: “farm5.static.flickr.com”,
host: “farm6.static.flickr.com”,
host: “farm7.static.flickr.com”,
host: “farm8.static.flickr.com”,
host: “farm9.static.flickr.com”,
host: “fbstatic-a.akamaihd.net”,
host: “galleries.payserve.com”,
host: “gamemedia.armorgames.com”,
host: “gamerch-static-contents-gz.s3-ap-northeast-1.amazonaws.com”,
host: “graph.facebook.com”,
host: “graphics2.asiafind.com”,
host: “graphics2.asiafriendfinder.com”,
host: “graphics.alt.com”,
host: “graphics.cams.com”,
host: “graphics.outpersonals.com”,
host: “graphics.pop6.com”,
host: “graphics.streamray.com”,
host: “gs1.wpc.edgecastcdn.net”,
host: “i0.wp.com”,
host: “i1.sndcdn.com”,
host: “ia902706.us.archive.org”,
host: “icdn2.digitaltrends.com”,
host: “icdn5.digitaltrends.com”,
host: “icdn6.digitaltrends.com”,
host: “imagena1.lacoste.com”,
host: “imagena2.lacoste.com”,
host: “images.contactmusic.com”,
host: “images.goodsmile.info”,
host: “images.mrskincash.com”,
host: “images.neopets.com”,
host: “images.popin.cc”,
host: “img1.zergnet.com”,
host: “img2.zergnet.com”,
host: “img3.zergnet.com”,
host: “img4.zergnet.com”,
host: “img.docstoccdn.com”,
host: “img.elle.co.jp”,
host: “img.epochtimes.com”,
host: “img.fatxxxtube.com”,
host: “img.kanzhongguo.com”,
host: “img.muji.net”,
host: “img.qz.com”,
host: “img.secretchina.com”,
host: “imgs.ntdtv.com”,
host: “imgx3.dditscdn.com”,
host: “img.youtube.com”,
host: “i.utdstc.com”,
host: “livepassdl.conviva.com”,
host: “l.longtailvideo.com”,
host: “m1.aboluowang.com”,
host: “massmedia-cdn.wistia.com”,
host: “media1.break.com”,
host: “media.247sports.com”,
host: “media-cache-ec0.pinimg.com”,
host: “media-cache-ec2.pinimg.com”,
host: “media-cache-ec4.pinimg.com”,
host: “media.cathkidston.com”,
host: “media-cdn.tripadvisor.com”,
host: “media.dermstore.com”,
host: “media.livepromotools.com”,
host: “media.mademan.com”,
host: “media.sfweekly.com”,
host: “media.skincarerx.com”,
host: “m.facebook.com”,
host: “mobapi.bloomberg.com”,
host: “mobile.twitter.com”,
host: “mzstatic.playhaven.com”,
host: “p1.zdassets.com”,
host: “passets-cdn.pinterest.com”,
host: “pbs.twimg.com”,
host: “photos-a.pe.facebook.com”,
host: “photos.modelmayhem.com”,
host: “photos.pop6.com”,
host: “piclist.pop6.com”,
host: “pic.pimg.tw”,
host: “p.jwpcdn.com”,
host: “platform.twitter.com”,
host: “playstationna.i.lithium.com”,
host: “plus.google.com”,
host: “pmcdn.staticpmrk.com”,
host: “public.oneallcdn.com”,
host: “p.vitalmx.com”,
host: “q-ec.bstatic.com”,
host: “quests.armorgames.com”,
host: “r2—sn-nx57yn7s.googlevideo.com:443”,
host: “r6—sn-5uaeznze.googlevideo.com”,
host: “r6—sn-i3b7rnee.googlevideo.com”,
host: “r7—sn-jc47eu7l.googlevideo.com”,
host: “rc-regkeytool.appspot.com:443”,
host: “realtime.services.disqus.com”,
host: “s0.wp.com”,
host: “s1.dmcdn.net”,
host: “s1.hubimg.com”,
host: “s1.wp.com”,
host: “s2.dmcdn.net”,
host: “s2.wp.com”,
host: “s3-ec.buzzfed.com”,
host: “s3.hubimg.com”,
host: “s3.pimg.tw”,
host: “s7.pimg.tw”,
host: “s9.pimg.tw”,
host: “s9.thisnext.com”,
host: “s.cdn.gaiaonline.com”,
host: “s.gravatar.com”,
host: “s.pimg.tw”,
host: “sr.photos3.fotosearch.com”,
host: “s-static.ak.facebook.com”,
host: “static02-ec-vn.zalora.com”,
host: “static1.businessinsider.com”,
host: “static2.businessinsider.com”,
host: “static2.docstoccdn.com”,
host: “static3.businessinsider.com”,
host: “static4.businessinsider.com”,
host: “static5.businessinsider.com”,
host: “static6.businessinsider.com”,
host: “staticd.cdn.adblade.com”,
host: “static.exoclick.com”,
host: “static.libsyn.com”,
host: “static.linkbucks.com”,
host: “static.miniclipcdn.com”,
host: “static.movideo.com”,
host: “staticna2.lacoste.com”,
host: “static.payserve.com”,
host: “staticx2.dditscdn.com”,
host: “staticx3.dditscdn.com”,
host: “staticx4.dditscdn.com”,
host: “s.utdstc.com”,
host: “s.wordpress.org”,
host: “s.xe.com”,
host: “sync.graph.bluecava.com”,
host: “s.youtube.com”,
host: “t.6park.com”,
host: “tags.crwdcntrl.net”,
host: “th00.deviantart.net”,
host: “th01.deviantart.net”,
host: “th02.deviantart.net”,
host: “th03.deviantart.net”,
host: “th05.deviantart.net”,
host: “th06.deviantart.net”,
host: “th07.deviantart.net”,
host: “th08.deviantart.net”,
host: “th-th.facebook.com”,
host: “thumbs.3jizz.com”,
host: “ukcdn.usablenet.com”,
host: “use.typekit.com”,
host: “vdc-img-0.ig1-cdn.com”,
host: “video.ap.ntdtv.com”,
host: “wac.24ba.edgecastcdn.net”,
host: “wac.450f.edgecastcdn.net”,
host: “wac.76ff.edgecastcdn.net”,
host: “widgets.twimg.com”,
host: “wpc.a818.edgecastcdn.net”,
host: “wprp.zemanta.com”,
host: “www.open.com.hk”,
host: “www.secretchina.com”,
host: “www.stratfor.com”,
host: “www.youtube.com”,
host: “www.youtube-nocookie.com”,
host: “x.myspacecdn.com”,
It’s not a bug, it’s a new feature of GFW.
At the beginning, the IPs used for DNS poisoning are limited, like this https://github.com/ihacku/gfw_dns_resolver/blob/master/gfw_dns_resolver.c
After this change, GFW use random ips, so those tools which rely on IP blacklist to detect if one domain is poisoned or not will no longer works.
But the battle will never end https://github.com/clowwindy/ChinaDNS
Except that, I’ve got an idea.
As you know some ips have already been block by GFW, you can get one from any big vps provider.
So why not use GFW as a free DDoS firewall to block DDoS attack from China? It might works.
Accept
I think I’m seeing this kind of attack on one website on my server. All URL’s are with XXX words (almost all) and when I take the site down the server is ok. When I put this site back up the server goes down. I have tested blocking *google* as a referrer and that seems to do the trick. The server did not go down and in just a matter of seconds 243 hits were blocked on the site. But of course I don’t want to block Google as a referrer to my site. But the access log on the server shows me that many of the referring urls are google.co.tr or similar…
Newsflash…
My site came under this same type attack, with them pointing to me as thepiratebay.org . 99.9%+ of the hits were coming from china. So I went about blocking countries. Eventually the DDoS slowed to almost nothing.
So I decided to open the door again, country by country. firstly JUST to Russia, and BAM! The attack from China re-amplified. If this were anything but a botnet under the control of Russian masters, the attack couldn’t have even seen my server was back up (I was absorbing and not replying to packets from blocked countries). The level of amplification was 2+ orders of magnitude, so I am sure this was the defining point. By 2+ orders of magnitude, I mean from 1 connection every second, to over 100+. I believe the traffic to be faked, generated by bots, for these reasons…
1. If it were bittorrent traffic, deflection by giving it the 404 response would have quenched it at the source.
2. If it were bittorrent traffic, the other counter I did, saying the client was tracker banned, should have source quenched it.
3. The vast majority of the clients were appearing as “bittorrent” without the normal version number that should have been appended after the word bittorrent.
4. Simple “stealth” blocking had the effect of dropping the traffic to manageable levels. It should not have re-amplified when the attacking IPs were still blocked.
5. It is well known that DNS can be ignored, and a fake URL put into an http: header, to make it look like you are actually being pointed at by nameservers, when you are not.
What we have here is a botnet, paid for by Russians, operating out of China. This is nothing new.
Sincerely,
Zaphod
I know there’s some people in China put a list of hosts to their computer host file to bypass the GFW and DNS poison. This might be related to your problem.
This is probably not the case, but you know how Comcast injects a message into user’s HTTP request if they hit the 3 strikes rule? What if a China ISP just injects a page request to a specific URL for millions of there users? X_X
Are you using CloudProxy I think you can put your fears aside if you secure your site with Sucuri’s fantastic WAF. I am saying this as an unpaid but happy customer that has seen cloud proxy eliminate what would normally be a huge issue.
Tom
There are some interesting characteristics in traffic patters:
– list of affected IP addresses is not random (the list of affected IPs is pretty much static)
– the traffic only comes in certain hours to the affected IPs
– the affected IPs are typically hostings (i.e. no ADSL or otherwise home addresses)
– different IPs get different shares of traffic
– and many more!
I’m a security researcher writing an extended article about this.
I’d be interested to speak with people who are affected by this kind of “bittorrent DDoS”. The magazine I’m writing the article for is willing to cover some of the costs related to this DDoS (your hosting cost, compensate for your time) if you help us track this attack better – please contact me at tchm at virtall dot com for details.
Thanks for suggesting to block Russia. We’ve been dealing two weeks with this same issue now. Things got better manageable after blocking Chinese traffic, but it wasn’t the real solution as traffic was still there only had our firewall to handle this instead of our application servers.
After I read your suggestion to block Russian traffic as this might be where the botnet is controlled from I decided to add Russia to our blocklist as well. And within minutes most of the Chinese traffic disappeared.
Thanks for confirming my suspicions, and confirming my advice. I am sure some on here have been trying to cloak this by “sharing” incorrect experiences.
Smells like the beginning of WWIII to me.