GoDaddy store your passwords in clear-text and may try to SSH to your VPS without permission

*UPDATE: I just got off the phone with Neil Warner, GoDaddy’s CSO (Chief Security Officer) and he explained the situation to me. Check it out: GoDaddy Security update

I have been a GoDaddy user for a while and never had problems with them. In fact, differently than some people, I had great support and service from them.

However, one recent situation is making me change my mind about them…

I have my domains and a bunch of VPS (virtual private servers) with GoDaddy and one of those servers is/was hosting the Sucuri’s official site.

I am a bit paranoid about security and on all my servers I switch the SSHD port to a different one and restrict to only a few IP addresses. On the offical SSH port (tcp 22), I install a honeypot to detect ssh scans and which passwords/users they use (you can see some of my analysis in this post: Honeypot analysis – Looking at SSH scans)

Anyway, early this year I started posting information about web-based malware and a few days after I did that, I saw on my honeypot logs:

Jan 8 06:55:28 d1 sshd[27670]: Failed password for [mygodaddyuser] from 64.202.160.65 port 49271 ssh2
Jan 8 06:55:30 d1 sshd[27670]: Failed password for [mygodaddyuser] from 64.202.160.65 port 49271 ssh2
Jan 8 06:56:38 d1 sshd[28528]: User root from nat-64-202-160-65.ip.secureserver.net not allowed because listed in DenyUsers
Jan 8 06:56:38 d1 sshd[28528]: Failed none for invalid user root from 64.202.160.65 port 50727 ssh2
Jan 8 06:56:53 d1 sshd[28528]: Failed password for invalid user root from 64.202.160.65 port 50727 ssh2
Jan 8 06:56:55 d1 sshd[28528]: Failed password for invalid user root from 64.202.160.65 port 50727 ssh2

And checking my honeypot logs, I saw:

Jan 8 06:55:28 d1 sshd[27670]: hh: user: [mygodaddyuser]|pass: [MYGODADDYPASS]
Jan 8 06:55:30 d1 sshd[27670]: hh: user: [mygodaddyuser]|pass: [MYGODADDYPREVIOUSPASS]
Jan 8 06:56:53 d1 sshd[28528]: hh: user: root|pass: [MYGODADDYPASS]

I was shocked! My first thought was that someone had stolen my GoDaddy password (that I use to login to their web page) and even my previous password! (I had changed my password a few weeks before that).

I quickly ran and started a panic mode incident response, changed passwords and started to look how I got hacked and what was going on, when I decided to look at the IP address that tried to access my box:

$ whois 64.202.160.65
[Querying whois.arin.net]
[whois.arin.net]

OrgName: GoDaddy.com, Inc.
OrgID: GODAD
Address: 14455 N Hayden Road
Address: Suite 226
City: Scottsdale
StateProv: AZ
PostalCode: 85260
Country: US

NetRange: 64.202.160.0 – 64.202.191.255
CIDR: 64.202.160.0/19
NetName: GO-DADDY-SOFTWARE-INC
NetHandle: NET-64-202-160-0-1
Parent: NET-64-0-0-0-0
NetType: Direct Allocation
NameServer: CNS1.SECURESERVER.NET
NameServer: CNS2.SECURESERVER.NET
NameServer: CNS3.SECURESERVER.NET
Comment:
RegDate: 2002-10-22
Updated: 2007-06-14

Hum.. It came from Godaddy’s own network. I was about to send an email to abuse@godaddy.com, whem I got this email:

It has come to our attention that the [your site name] may be infected by malware. We would like to investigate this matter further, however the login credentials we have on file for your server do not allow us access to the server. In order for us to proceed to investigate the possible infection, we require that you provide the proper login credentials to access your server with administrative rights within 48 hours or by January 10th @ 2 pm MST (GMT -0700) by using our “Password Sync” option, or your server will be suspended. To update the logon information, please follow these steps:

Log into your account.
Click on the ‘My Account’ link.
Click on the ‘Dedicated/Virtual Dedicated Servers’ link.
Select the server you need to update the log on information for.
Click on the ‘Open Manager’ link.
Click on the Support: Sync Passwords button.
Enter the current SSH and root information and save the information.

WTF!WTF!WTF! Yes, I cursed them for a while! Why?

  1. They tried to SSH to my “private” server without my authorization!
  2. They wanted my ROOT password and SSH access!
  3. They HAD MY MAIN GODADDY PASSWORD (AND PREVIOUS ONE) in CLEAR-TEXT!
  4. They almost gave me a heart attack

I don’t know if anyone find that horrifying, but I do! I would understand storing the initial password for the server in clear-text or something like that. But the main password from my GoDaddy account? Giving their admins access to them so they can SSH to my box? Keeping my old password in clear-text too? SSHing to my box without asking my first? Wow….

The end of the story… After I calmed down, I contacted them and explained about my web-based malware security research and told that I would not give anyone SSH access. If they really required that I would switched providers. They did some investigation, apologized and let me stay… How nice they are…

Scan your website for free:
About David Dede

David Dede is a Security Researcher in the SucuriLabs group. He spends most of his time dissecting vulnerabilities and security issues. You won't find him on Twitter because he is paranoid about privacy.

  • http://www.infosanity.co.uk Andrew Waite

    wow, you might want to look for another provider anyway? Nice to see honeypot technologies showing providing value in the real world. Thanks for sharing.

    On the plus side, I didn't know that GoDaddy provided free incident handling as part of their basic services…

  • http://www.blogger.com/profile/02943450415559789239 steveshead

    Just knowing that they store passwords in clear text is enough for me to drop them like a hot brick. It only takes one rule to be "bent" to bring the house down.

    Thanks for sharing though – great story.

    Steve

  • Anonymous

    Maybe they just use their own encryption – nothing tells you they have the clear text password and nothing tells you they were able to decrypt it somehow, should it have been salted.

  • KlfJoat

    WOW. I switched operating systems from Windows to Linux for less than this! When Microsoft demonstrated that they could arbitrarily change a setting on my computer without my consent, I stopped using them (back in 2005). But if THIS happened with my registrar, I would EXPLODE!!!

    Gandi.net is a French registrar that I've used with positive results, and have heard many positive reviews. Check out their TOS, you might be pleased by some of the provisions.

  • http://id.sidneysm.com/ Sidney San Martín

    Anonymous,

    If the password can be accessed at will, I don't care whether it's encrypted or written on a piece of paper in someone's desk drawer.

  • Anonymous

    Total boycott of everything GoDaddy from now on. Fucking wankers.

  • Anonymous

    @ Andrew Waite

    Honeypot "technologies"…….

    In other news; blacksmith creates horseshoe with hammer and anvil "technology".

  • http://www.blogger.com/profile/09819173281750148724 budu

    Thanks for the tip, this is an eye opener!

  • Anonymous

    namecheap.com is a good place for registering domains. If you want to find a hosting provider go to-

    webhostingtalk.com

  • http://www.shqiperia.com mandi

    Most of the providers do store your passwords! So does ThePlanet for example!

  • Anonymous

    Way to fuck up.

    Also, I don´t think they wanted to search for malware or something. My guess is they say that to newbie users to get the root password.
    (I may be wrong, but I´m too paranoid to care.)

  • http://zgware.com Zach W

    There is nothing ok about that. In service of security and redundancy you should split your providers. This is ridiculous.

  • http://www.blogger.com/profile/14980808976404159238 http://sucuri.net

    mandi: uff.. that's a relief … If theplanet does that, they do it wrong too. If they access a "private" server without permission that is wrong too.

    Like my mother always said: Just because someone else is doing something, doesn't mean it is right and you should do it too.

  • http://www.blogger.com/profile/10876995057940083020 Justin

    The permission is probably buried somewhere in the TOS. Not that that makes it any better.

  • Anonymous

    linode ..

  • http://www.blogger.com/profile/06658170385477434704 Colin

    This is probably the worst privacy/security violation that I have seen from a registrar ever.

  • Anonymous

    Ok did anyone miss the point that they own the servers/ service. They have to right to know what you are doing on their equipment. You are just paying a rental fee.

    • Adrian

      From your logic, a landlord has the right to barge in to your rented home at any time.

  • http://d0s.eu d0s

    Nice catch!
    There's an eye-opener if there ever was one.
    I host with OVH.co.uk (servers in France) and use enom.com for all my domain registrations.

    d0s

  • Anonymous

    Just a thought — you should research encryption if you want to be a security researcher. The most likely situation is GoDaddy is not storing your password in cleartext. Rather, they are likely storing it encrypted and decrypting when they use it.

    Not understanding that encrypted != cleartext is a problem for security researchers. In lay terms, encryption means that information is jumbled but can be recovered.

  • http://d0s.eu d0s

    Anonymous, are you for real?

  • Anonymous

    Just another thought. All of you non-security researchers who claim that it is acceptable that anyone has access to your previous and current passwords need to go read a book about security. Encrypted or not, it's reversible to plaintext! How much do you think those encryption keys would be worth?

    The pedantic anonymous above is a fool, but thanks for the lesson Professor.

  • http://www.blogger.com/profile/02673698256953352045 Dave

    @Anonymous

    Who cares? Storing the passwords encrypted is barely better than storing it in clear text. The point is that someone has access to your password whenever they want.

  • Anonymous

    First: Thanks for allowing anonymous comments!

    I think [February 24, 2010 11:27 AM] is *partially* right. If you own the company that leases VPS's, and provides bandwidth, there are certain laws/regulations that you must be careful to abide by. Why? Because things like this happen:
    - http://www.wired.com/threatlevel/2009/04/company-caught/
    - http://tech.slashdot.org/article.pl?sid=09/04/03/231220
    - http://www.poligazette.com/2009/04/05/the-unusual-fbi-raid-of-a-dallas-datacenter/

    Having a system that you are (potentially) responsible for go rogue puts you in a sticky situation. (See at least one URL referenced above).

    This is *not* to say that GoDaddy did the *right* thing!

    Your post raises questions that I think GoDaddy should answer, at least to you, the customer:

    - What, specifically, caused them to single our your server as potentially effected by malware?

    - Why, specifically, didn't they at least *attempt* to notify the server/domain owner *before* attempting to connect to it, even assuming they (thought they) needed to take immediate action? [Is your contact information up to date?]

    - What, specifically, was an engineer or automated program going to do once it logged into your server?

    I think it was wrong for them not to attempt to notify you, and I agree with your points that pretty much every way they handled this was bad practice. I have to wonder what the "EULA" or TOS says, though. Have a spare copy?

    And now for something off-topic:

    Unless it's a truly co-located box that you simply pay bandwidth for, I doubt you retain any right to run old/insecure operating systems on their network, and I'm sure they have provisions against that sort of thing in their TOS. Not that you were doing that, but obviously they have a system in place to detect suspicious activity.

  • http://www.packetport.net Mephux

    lol nextgenhacker101 – In the above posts.. http://www.youtube.com/watch?v=SXmv8quf_xM lol

  • http://www.blogger.com/profile/04722333831771899337 Uber

    You have yet, however, to prove your claim that it was indeed GoDaddy's staff attempting this. Are the email headers from GoDaddy? Have they admitted to it? All I see right now is that *someone* got your credentials and *someone* could have spoofed an email. Are there not many thousands of hosts whose source IP address arises from the GoDaddy domain? Yours does, does it not?

  • Anonymous

    Thanks for posting this, very serious indeed, I have a few domains with them and I will be gone soon enough!

  • http://www.blogger.com/profile/14980808976404159238 http://sucuri.net

    Uber: You make some good points. But that email came from them and we even discussed it by phone after (I called them). They said that they indeed tried to access, but because I had changed the passwords they couldn't access to make sure it was malware-safe.

  • Maddy

    ^^^^
    What an enlightenment !

    @ author : Thanks for sharing !
    I never used GoDaddy
    reason is the BobParsons pic on the homepage :P

  • Anonymous

    Millions of dollars spent on Superbowl ads wasted. They should have spent the money on hiring people that can help setup a non-janky operation

  • Shaker

    I got rid of my GD VPS after they broke my machine with an upgrade and then wanted me to pay for them to fix it.

    The concern about them having your login credentials shows your lack of experience with hosting/colocation however. EVERY host I know of requires that they retain root access to a machine on THEIR network. If you didn't realize that before you didn't read your contract.

    • Adrian

      I'm thinking that too. But if it was NOT in the contract, it's basically breaking and entering.

  • http://www.blogger.com/profile/04722333831771899337 Uber

    Well, if they admitted they had attempted it, I'd drop them like a hot potato. Too, bad, I like Danika…

  • Anonymous

    LOUD NOISES!

  • Stefan

    What Software do you run on 22 as honeypot ?

  • Anonymous

    I'm not quite sure why you're surprised. Is this the first time you've leased a VPS/dedicated box?

    Many providers have this same business practice of keeping VPS passwords (I doubt they store *ALL* passwords in plaintext) so that they can login if need be. This is why most companies ask you for your root password when you setup your box. Now "if need be" falls under a lot of scenarios. The most obvious is for the less technical users who rent a VPS and then run into trouble, email support asking them why X doesn't work anymore. Rather than going through the motions of having the user send a password over email (potentially in plaintext over a wire), they store a copy in what I imagine is a relatively safe location (aka. NOT over a wire) and use that. This seems reasonable to me. There are also cases where the user ..forgets a root password. This might not happen to you, but it surely saves a lot of trouble for those that it does happen to.

    The best part of all of this is that you somehow think a password is really keeping godaddy from your files. They have physical access to your disks, and you're surprised that they can access your data? If you don't *trust* your provider with treating your data fairly, you have bigger issues than them knowing your password.

    Finally, I'd suggest that if you plan on moving away from GoDaddy you do some better research. Realize that there's a good chance your new provider will be doing exactly the same thing.

  • Anonymous

    @Shaker Go find a decent webhosting company. I've been working in the industry for several years now and I can assure you that the decent ones don't insist on root access, nor require any records of passwords to be kept up to date, and nor will they randomly log in to boxes without seeking user permission first. There are bits of various safe-harbor legal protections that get thrown away if they do.

    What they do usually explicitly state is that they will shut down or suspend service under various circumstances, the wording changes from company-to-company but usually covers detrimental service impact, hacking etc. etc.

    Why would a hosting company ever need your actual root password? Should they ever really need access to your machine they'll just boot it off a live CD and change the password, or similar.

  • http://www.infiltrated.net sil

    Anonymous is on point with this statement: "In lay terms, encryption means that information is jumbled but can be recovered."

    http://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis

    I'm so glad we have security experts!

  • http://godaddy.com Nick Fuller

    We received a complaint about the server in question and attempted to conduct an investigation into the machine. We will not go into details of the complaint or investigation publicly, but Go Daddy Chief Security Officer Neil Warner discussed the issue with the customer.

    For the record, Go Daddy does NOT store passwords in plain text format. All customer passwords are stored in a secure, encrypted location with extremely limited access.

    A select few trusted employees have access to the passwords when they are conducting an investigation or supporting a customer. In order to retrieve a password, the trusted employee must provide a written justification that is logged into the Go Daddy system. These logs are routinely reviewed by a separate audit team.

    We take security very seriously at Go Daddy, both for our customers and our network. If Go Daddy receives a complaint or notices abnormal behavior from a customer server, an investigation is conducted. We provide incident response as a courtesy to our customers for no charge.

    Nick Fuller
    GoDaddy.com
    Communications Manager

  • Anonymous

    I'm @Anonymous from above. Couple of points:

    1) Most/all shared & dedicated hosts keep your password, as someone else indicated above. This is widespread practice, used at (among other places) Rackspace and The Planet. These are all legitimate vendors, and you are overreacting.

    2) If your material is so sensitive that the host should not have access, you need to control the hardware. You are a security researcher, so you know that giving someone physical access to a machine (i.e. by renting it from someone else, in their data center, managed by their employees) is one step away from giving them passwords. Do you think their Linux console single-user mode doesn't work? Or that they can't access the disk directly? Or that they have other means of accessing your data if they wanted? Do you really think they couldn't derive your password from the passwd file on the filesystem if they wanted to? And was it the password or the box or the data on that box you're trying to protect (hint: these result in different security protocols)? Either you trust your vendor or you don't, but whether they are stealing your password is the least of your worries.

    3) I apologize for being pedantic, but if you want to be a security researcher you must be precise when making claims about the security of others' systems. Crying wolf is very bad when you are in this field, and imprecision is making some false accusations in this case. Besides, a lot of security is theory — who could know or derive what, and by what means. It's pedantic stuff, security, when you get it right.

    4) I'm not addressing at all whether the host had an appropriate need to be on your box at that time. They manage millions of websites, so it's most likely that the TOS you agreed to specify that they can login to your box under the current conditions for a specific purpose.

  • Anonymous

    It's a VPS, they can access the data without knowing any passwords…

  • Anonymous

    You made Securitymonkey's blog and he has an interesting point. Is godaddy snapshotting these virtual machines before accessing them? There are forensic evidence considerations if the machine is being used in a crime.

    http://it.toolbox.com/blogs/securitymonkey/godaddy-has-my-passwords-37130

  • Anonymous

    First of all, this is common. You are one customer, consider how many they have that pay for servers and know nothing about running them. You secured your box, they attempted to access it to ensure you had a problem before wasting your time and theirs, and when they didn't have access they contacted you.

    Simple procedure. They have tons of vps hacked or even purchased with intent to spam/botnet/DOS etc and it makse their network look bad from the outside while causing massive problems to their customers.

    Most customers keep their passwords on file with GoDaddy and then expect them to fix problems that would normally be the customers issue. Hell some customers in this business turn their problems into the Hosts issue saying they arn't being proactive.

    Your server didn't fall into this category and they informed you of the problem. Tho I would assume they received an abuse compliant regarding one of your ips unless they also run their own honeypots and noticed unusual activity on their vps network.

    Just remember with VPS servers, the network is virtual also and extremely easy to capture all network traffic ;)

  • Anonymous

    I had read on a mozilla blog somewhere that godaddy also does not do credit card communication for their websites using encryption either.

  • Anonymous

    They use OpenVZ anyway, your VPS is an easily accessible subdirectory on one of their servers ;)

  • Anonymous

    So Neil called you? How was it talking to the anti-christ? Did you feel really smart after speaking with him, cause most do!

  • Anonymous

    @ Anonymous at 7:40PM

    That would be a MAJOR problem, as there are policies and/or laws that require the use of encryption when transmitting card details on the wire.

  • Anonymous

    Try rimuhosting they don't demand a root password they just won't help you fix a problem that requires root access

  • Anonymous

    Actually Rimuhosting are awesome. I used them for years, and ran into a few of the guys there at a Linux conference recently.

    Their support is top notch, and they never fuff around with your VPS without asking. Worst case, they'll ask you to setup an account with their SSH key and sudo access for remote support.

  • Anonymous

    Can't read the left half of your post because of some stupid fucking google ad box overlapping. nice site design loser.

  • Anonymous

    You should use adblock and noscript. No ad boxes in sight.

  • Anonymous

    @d0s:
    OVH stores their ssh-key on your box to provide them with access ;) (check your authorized_keys's)

  • Anonymous

    Another anonymous poster here saying "get off your high horse". If you were in my data center and refused to facilitate access to the dedicated server you are leasing from us when something needs to be investigated, I'd shut your box down and reset the root password myself.

    Yes, you would be more than welcome to go find another provider.

    For colo, I wouldn't touch your server without consent but I would disconnect it from the network if I suspected compromise.

    Get over yourselves. Your hosting provider has a reputation to uphold, and as the owner of the address blocks that you operate from, they have a responsibility. It's the rest of their customers that will suffer if your compromised server is getting subnets put on block lists.

  • http://www.blogger.com/profile/03672219245225897410 trig-

    Anyone that has a hosted VPS running on the Virtuozzo infrastructure can have their VPS accessed without a password by the hosting team without permission..

    The hosting company would simply enter; vzctrl enter VE_ID

    This would give them full root access without bash history, you can do the same on OpenVZ.. Naturally they can also open /vz/root/VE_ID/root/_

    If you have Plesk, your password is stored in /etc/psa/.psa.shadow

  • Anonymous

    Why anyone would use any service of GoDaddy baffles me. These people are hacks (not hackers), thieves and domain hijackers.

  • http://scrottie.livejournal.com/ scrottie

    I'm working for yet-another-Xen-instance-provider as a programmer. Like yours, they offer a "managed" service. I don't know anything about GoDaddy's product, but the guys I'm working for have clearly labeled "managed" and "unmanaged" ones. These guys were doing managed dedicated hosts before this. Most people do like this a lot — if something isn't how they want it, they call up and someone logs in and changes for them. They're exploring ways to keep passwords in sync. The (numerous, automated) administration options that don't require shutting the instance down (and directly mounting the filesystem) require ssh'ing in as root (also automated). Backups use your root password. If you colo and people have physical access, you're extending a measure of trust to the colo. But being on a VPS goes further. I'm not sure how I feel about all of that. I guess I still want my own hardware even though the VPS thing is great for a lot of people.

  • http://www.theplanet.com Tomy Durden

    The Planet does store server passwords, but it's stored using a two-way encryption. I am not able to divulge the details on the encryption schemes or methods that we use.

    When a technician needs a password, they click a button which decrypts the password. It also logs the IP and credentials of the technician.

    The customer is given the option to keep the password updated or not. The SLA is affected to some degree if the password is not up to date.

    Keep in mind, with physical access, one doesn't need the password. One can simply boot into single user mode, or in the case of Windows, use BartPE to reset the password.

    Our technicians will only log into a server if it's explicitly requested either by ticket, escalation procedures, or at the proper and justified request from law enforcement.

  • http://www.technopotomus.com Lance

    Thanks for the heads up. Never thought much about godaddy before, but now you've pointed out a sure security flaw everyone should be aware of. And not just of godaddy, but any other hosting solutions that may not be responsible.

    Amazing they wanted access to your private network. I would be just as irate.

  • Anonymous

    Someone got first but I'll say it again:

    linode.com

  • Anonymous

    I have never had a problem with linode.com doing anything like this. I'm not sure if your VPS is considered managed or not, but if not, I would drop them like a hot potato.

    As it is Godaddy lost me as a customer several years ago due to their abusive customer service reps.

  • Anonymous

    Switch to Linode :D.

  • Anonymous

    Former GoDaddy Investigator here. Did you read your Terms of Service. GoDaddy reserves the right to access your dedicated hosting box at any time for any reason and yes they do encrypt customer passwords. What you experienced is not out of the norm. They've got thousands of dedicated hosts all crammed together on the same network and are charging you a fraction of what it would cost for you to have your own truly private box on your own truly private network. The reason they wanted to investigate your box is because it strange or large amounts of traffic was detected to or from you box. Granted, it's legitimate… but they won't know that without investigating.

    There are a lot of bad people out there who use stolen credit card numbers to get dedicated hosting boxes and use them for malicious purposes. Any not having any right to investigate, would result in absolute chaos. Imagine if they didn't investigate anything. I can guarantee your box would be under a DoS attack from another dedicated host right now, or the network would be completely congested with port scan, or your box could be infected or compromised leading to other customers getting infected or compromised. It's for the greater good my friend. A word to the wise, be nice to them and explain what the traffic is… There are no-name hosting providers that don't give a ****. Those hosting providers typically can't brag about 99.9% uptime because so often they have botnets and infections running wild. Think on that a bit…

  • http://kristianpaul.org paul

    i moved to gandi since i discover bad issues from godaddy reading one day in nmap.org, after this i think i was totally righ migrating from then

  • Anonymous

    http://tools.softsutra.com/locip/index.php >> Get Ip Address & Location Of A Person….Best Tool I Found To Get Some One's Computer Information… X

  • Pingback: 文件同步让生活更轻松(1) | 坚果铺子博客()

  • Pingback: Experiences with using GoDaddy, Linux Web Hosting | The (Unorganized) Musings of a Computer Scientist()