How to Restore Your Hacked Site | Modules Unraveled

tl;dr

Rollback a server backup (files and database) from before October 15th 2014.

No server backup?

  1. Run "git status" to find new and modified files.
    • Delete new files
    • Checkout modified files
  2. Thouroughly check files directory for anything unusual.
  3. Make sure the .htaccess file in the files directory restricts code execution
  4. Restore database from pre Oct. 15th backup
  5. Update Drupal Core to latest release

... Read on for details...

I think I might have been hacked. What do I do?

Hi, this is Brian Lewis with Modules Unraveled.

As you probably already know, there was a huge security fix released for Drupal 7 on October 15th (SA-CORE-2014-005). The patch to update Drupal is actually quite small, but the implications of not updating your site are massive. As a matter of fact, if you haven't already updated your site, chances are you have already been hacked. There were automated programs systematically attacking Drupal sites hours after the fix was released. In this video I'm going to show you how to find out whether or not your site has been hacked. And if so, I'll walk you through what you need to do now, to reduce the damage done.

There are two ways to find out whether your site has been hacked. With "git status" and by searching the database.

  • Run "git status" inside Drupal root
    • This will show us any files that have been modified since our last commit. On the live server, there shouldn't be any, so anything listed here, I know is a result of being hacked.
    • This is a huge reason you should be using version control on your site. If you're not, you can try to re-download every module, theme and library you have and download a fresh copy of the version of Drupal core that you had before the attack and replace all of those on your server. I'm hesitant to recommend this as a full fix though, because there may be hidden files, or files in places you don't think to look. Really, my recommendation is a full re-install. If you're in this situation, I'm sure you don't want to hear that, but I hope this gives you a reason to look into Git.
  • Search for "file_put_contents" in database
    • If there is a result. You've been hacked.
    • Click "Browse".
    • Click the "BLOB" link under "access_arguments". This should download a file to your local machine.
    • Open that file with a text editor.
    • Notice that only one file is listed. There may be others that need to be deleted.
  • If there are no extra files in your git repo, and no results in database search. You're not hacked. Update Drupal Core now! Or at least do the hotfix mentioned here as a temporary measure.
  • Delete/checkout all files listed by "git status" (Also check your files directory. The files directory should not be in Git, but that means there's no easy way to view new and modified files, but they could have been placed there. By default, the .htaccess file that is in that directory prevents php code from being executed, but Michael said he has seen an attack that modified that .htaccess file. So, you need to check your site.)
  • Restore Database (Otherwise thouroughly check Users, Node, etc.)
  • Install latest Drupal Core update

Recap:

  1. Run "git status" to find new and modified files.
    • Delete new files
    • Checkout modified files
  2. Thouroughly check files directory for anything unusual.
  3. Make sure the .htaccess file in the files directory restricts code execution
  4. Restore database from pre Oct. 15th backup
  5. Update Drupal Core to latest release

Updates:

  1. Drupal security team member Greg Knaddison (greggles) wrote up a great guide on what to do when you get hacked. He includes things I didn't mention like making a forensic copy of your site to inspect later, and notifying site stakeholders. You can read that here.

Comments

"If there are 0 results, you are fine" is misleading. There are exploits that do not leave a trace in the menu_router table.

Thanks for the tip on exploits that touch files directory and its htaccess file. I'll add this to the workflow chart; http://drupal.geek.nz/blog/your-drupal-website-has-backdoor

Thanks Bevan, I didn't realize that. Crazy stuff!

first: i hope these script kiddys die
thank you for that article. here is what i found out. even if there is nothing in the db there have been files modified in my /modules directory. they also modified core files like trigger.api and added weird stuff to it.

The term "script kiddies" is probably far from accurate. Given the exploits, uses and behaviour, most seem to be professionals exploiters. For example, I have not heard of any reported defacements.

Yeah, I think he was just expressing his frustration through name calling. We can all fall into that sometimes. :)

Just a heads up. You can hide modification from showing up in 'git status' by setting the "assume unchanged"-flag. To verify the integrity of a git workspace you also need to look for those files.

See this stack overflow question.

Good call! I don't use that particular option in Git, so I didn't mention it, but is a totally valid point!

Brian,

You missed a few steps. Since this vulnerability gives apache permissions, you need to empty /tmp

rm -rf /tmp

You also need to check all folders that apache has write permission to for code files of any kind that might run on your server. PHP, Bash, Perl if your server has it installed....

Next, you can't use git to check for changed files because it's trivial to tell git not to track a file. You must use something like tripwire, or https://www.drupal.org/project/hacked

Finally, you must check the Drupal permission system including, but not limited to:
New users with existing admin roles
New users with new admin roles.
New permissions modules (like Organic Groups) adding new permissions to Drupal.