• How is it funded?
• Who is on the team?

• Roadmap for Drupal

Jibran Ijaz ‏@JibranIjaz
What will happen to the workflow initiative if Drupal shop(s) will stop supporting it? #MUP160

Jibran Ijaz ‏‏@JibranIjaz
How can we make sure that this'll be a community driven initiative not specific to someones requirements. #MUP160

Steve Persch ‏@stevector
Do you anticipate support for making/deploying configuration changes through workspaces? #MUP160

Steve Persch ‏@stevector
@JibranIjaz @dickolsson I'd like part of the answer to be consensus formed around Behat-executable stories/scenarios. #MUP160

Justin Ludwig @jludwig89
Can the Workflow Initiative fit into a multi-site install, where content is to be deployed to more than one site? #MUP160

• I’m glad you asked. Since transitioning to BlackMesh this past August, my focus has changed to automating the development workflow process, with a special focus of course on Drupal. I’ve been working on a project we call Cascade which leverages two open source tools, Jenkins and Ansible. A lot of Drupal developers aren’t also sysadmins - so they aren’t involved with, or maybe don’t have permissions, to take code live - they may also not have access to copy the database down for development purposes. Cascade automates this so those with the right permissions can do this with a click of a button.

• Lets start with Jenkins - it is what is termed a Continuous Integration server - or CI for short. Other CI’s include Gitlab CI or Bamboo from Atlassian, but Jenkins is open source and has been around a long time. In simplest terms, its like a cross between a cron and a job queue that’s integrated with revision control. Think of it this way - you have multiple developers working on a project, all committing their code to Git or SVN. Hopefully you have also written some tests. Think SimpleTest for Drupal 7 or PHPUnit for Drupal 8. When each developer commits their work, you want to know if it breaks any of the tests. If it does, you don’t want their work merged or deployed. If it does pass the tests, you may want the code automatically deployed to staging, for example. Jenkins manages all that nicely.

• Sure. Well, most small shops don’t have continuous integration set up, but its the best example. I guess I should define continuous integration - continuous integration is where developers commit their code multiple times a day and each time that code is run against all the tests you’ve written for the project. So for example, Drupal 7 ships with a bunch of SimpleTests - so assume you’re working on a project and you want to make sure that you haven’t broken any of Drupal 7’s internals. Do you want to know that as soon as you make the mistake or do you want to find that out in a couple of weeks when you are done the project? Well, with continuous integration set up, every few hours you will commit your code and Jenkins will automatically run all the Drupal SimpleTests, emailing you a report of the tests. Within a few hours of doing your work you’ll get a report of all the tests that failed - and all you had to do is commit your code. Jenkins did the rest. Of course, you would have had to set Jenkins up, but in the long run, its easier to fix mistakes soon after you make them then to go debugging code you haven’t looked at in two weeks.

• CI is usually associated with development, but maybe you want to make sure that your CSS has access to the right assets - in that case, a themer would like to catch those mistakes in a staging environment earlier rather than later. So I could make the case for a themer, but generally speaking, yes, its usually something thats usually associated with development.

• Normally Jenkins is installed on a remote machine somewhere in your development environment. I’ve installed it on a VM before, so that could work too - but then you start to lose the value of Jenkins because you have to make sure the VM is running, has internet access, etc. Jenkins is something you want to set up and forget. It just does its thing.

• If you’ve ever heard of Chef or Puppet, it is essentially an automation tool. Immagine that we wanted to automate the setting up of our staging environment every weekend. Think about the steps involved. First, we have to copy the production database to our staging database, and then we have to check out the latest version of our staging branch. We may also need to get the latest copy of the files directory, run drush to clear the drupal cache, restart varnish and apache or use a different settings.php file. These are steps that are repetitive and that you could spend your Monday mornings doing, or you can create whats called an Ansible playbook to do it for you.

• For the most part yes. If you made any configuration changes, like Views or new content types that aren’t in a feature, you will still have to add those manually. But otherwise, yes. The repetitive tasks are what are scripted. If there is anything unique, like creating a new account with the right permissions, that may not be worth your time to script since its a one time thing, you may not want that in your ansible playbook.

• The big difference between Ansible and its alternatives is that Ansible doesn’t require a client script. So for example, with puppet or chef, the tasks are executed by a script running on each machine. Ansible, on the other hand, is triggered from a centralized source, say your laptop or an admin machine, which then logs into other machines to execute the commands remotely. This was an important distinction for us because it meant that we didn’t have to install all the puppet or chef dependencies on all our servers.

• Exactly. As long as you can connect via ssh, you’re good. Actually, you also need at least Python 2.6 as well, but thats on most distro’s by default. For logging in, ansible’s configuration files allow you to specify the port, username and password of the remote server, or an ssh key of course. It can even su or sudo on the remote machines. What's also cool is that Ansible can run asynchronously - so if you have multiple staging machines the commands will run on each at the same time. Now, its nice if all the remote machines are the same OS - but for advanced users, you can even specify how commands are run slightly differently for different OS’s.

• There are two big benefits as I see them, and another not so apparent. First, a lot of these tasks are repetitive. And things like copying a database may take a bit of time. Or merging code. Or running tests. Etc. Anything that you can automate means time you can spend on other things. Second, not everyone is as experienced - or maybe they don’t have the permissions - to execute all the tasks. You don’t want mistakes, you don’t want to give everyone permissions. So you figure out the best way to do it and then you automate it. The last reason is not as obvious. I think a lot of times we hack things together in our development environments to get it working - but then may run into issues later on. We don’t spend the extra time because its temporary. By spending a little extra time getting it right, we have created a reusable pattern that we can use on our next project. By encapsulating our best practices, we not only have a quicker setup, but we have a better one too.

• Exactly. And I’ll add, ansible can be used for each of installation, configuration, and orchestration. The examples we’ve talked about so far are orchestration - moving databases, code, etc. It can also be used to install Apache, Mysql, Mongodb, etc. Any set of system commands that are repeatable.

• Exactly. And throw in Vagrant into the mix and now you can do that on your local machines using Virtual machines. Immagine spinning up a brand new VM and within a few clicks you have your entire development environment with a fresh drupal install all ready for you on a VM.

• I think all three benefit from automation. For example, in a previous life where I didn’t use Ansible, my themer was insanely good at theming, but when it came to running commands remotely on a server to check out his work, he was a fish out of water. I wish I had written an Ansible playbook so that he could check his code out onto staging. Or even better, if I had set up Jenkins to run an Ansible playbook to automatically check it out his work each time he committed. He wouldn’t have had to wait on me, sometimes a few days if I was not around. That said, he would not have been able create the ansible playbook.
• As for who is using Ansible, well, Twitter does - they use it to roll out updates across their hundreds of machines. And of course BlackMesh, the hosting company I work for, also does. The product Cascade I mentioned uses ansible and Jenkins to do a lot of the things we talked about today, only we set it up so you don’t have to.