Profiler Builder

• What is Profiler Builder?
  • Profiler Builder was created cause I’m lazy and wanted to just build a site, then figure out the profile, not build both at the same time. My work on profiler builder started to lead me toward the notion that install profiles and distributions can be more of a pain then they are worth, hence recipes.
• Is there anything that Profiler Builder doesn’t catch?
• How are you using Profiler Builder?

Drush Recipes

• What is the Drush Recipes plugin
  • Drush Recipes is a series of drush calls chained together in a lightweight command-file, similar to chef and it’s recipes / roles structure.
• Why do this instead of just using standard drush calls?
  • You might need to mess around w/ it to get a sense of some of the things you can do with it since it’s a lot more then just chain automation as it supports branching path logic, automatic recipe authoring, drush commandline recording to author recipes, the ability to take two sites and engineer the difference between them (as drush calls), remote loading of recipes, etc.
• I use TextExpander to do this, so I just type a shortcut, and my commands are filled in. How is using Drush Recipes different?
• Use cases for this?
  • First-time site builds
  • Development/Testing
  • drush ddt
  • drup
  • dvr
  • Chain them together (reference)
  • dwr - Interactive (What theme do you want to install?)
  • Madlib (Tokenize Drush commands)
  • Reference Make files
• Where can we see your recipes and contribute our own?

(Part of Intro)

• What is the most recent project you have been working on?
  • I’m glad you asked. Since transitioning to BlackMesh this past August, my focus has changed to automating the development workflow process, with a special focus of course on Drupal. I’ve been working on a project we call Cascade which leverages two open source tools, Jenkins and Ansible. A lot of Drupal developers aren’t also sysadmins - so they aren’t involved with, or maybe don’t have permissions, to take code live - they may also not have access to copy the database down for development purposes. Cascade automates this so those with the right permissions can do this with a click of a button.

Automation Tools

• You mentioned Ansible and Jenkins - I’ve heard of Jenkins, but haven’t used it, and I haven’t even heard of Ansible. So, can you start out by explaining what they are?
  • Lets start with Jenkins - it is what is termed a Continuous Integration server - or CI for short. Other CI’s include Gitlab CI or Bamboo from Atlassian, but Jenkins is open source and has been around a long time. In simplest terms, its like a cross between a cron and a job queue that’s integrated with revision control. Think of it this way - you have multiple developers working on a project, all committing their code to Git or SVN. Hopefully you have also written some tests. Think SimpleTest for Drupal 7 or PHPUnit for Drupal 8. When each developer commits their work, you want to know if it breaks any of the tests. If it does, you don’t want their work merged or deployed. If it does pass the tests, you may want the code automatically deployed to staging, for example. Jenkins manages all that nicely.
• Okay, I’m sure the more savvy listeners followed that perfectly. Unfortunately, I’m a little slow sometimes. :) Can you give me a concrete example of a standard operation you might perform with Jenkins?
  • Sure. Well, most small shops don’t have continuous integration set up, but its the best example. I guess I should define continuous integration - continuous integration is where developers commit their code multiple times a day and each time that code is run against all the tests you’ve written for the project. So for example, Drupal 7 ships with a bunch of SimpleTests - so assume you’re working on a project and you want to make sure that you haven’t broken any of Drupal 7’s internals. Do you want to know that as soon as you make the mistake or do you want to find that out in a couple of weeks when you are done the project? Well, with continuous integration set up, every few hours you will commit your code and Jenkins will automatically run all the Drupal SimpleTests, emailing you a report of the tests. Within a few hours of doing your work you’ll get a report of all the tests that failed - and all you had to do is commit your code. Jenkins did the rest. Of course, you would have had to set Jenkins up, but in the long run, its easier to fix mistakes soon after you make them then to go debugging code you haven’t looked at in two weeks.
• Okay, that sounds good. As a site-builder, it sounds like Jenkins is most useful for developers writing custom modules. Is that a fair assessment?
  • CI is usually associated with development, but maybe you want to make sure that your CSS has access to the right assets - in that case, a themer would like to catch those mistakes in a staging environment earlier rather than later. So I could make the case for a themer, but generally speaking, yes, its usually something thats usually associated with development.
• Is this something that you install on your local machine? Remote machines? All of the above?
  • Normally Jenkins is installed on a remote machine somewhere in your development environment. I’ve installed it on a VM before, so that could work too - but then you start to lose the value of Jenkins because you have to make sure the VM is running, has internet access, etc. Jenkins is something you want to set up and forget. It just does its thing.
• So now, let’s talk about Ansible. What is it?
  • If you’ve ever heard of Chef or Puppet, it is essentially an automation tool. Immagine that we wanted to automate the setting up of our staging environment every weekend. Think about the steps involved. First, we have to copy the production database to our staging database, and then we have to check out the latest version of our staging branch. We may also need to get the latest copy of the files directory, run drush to clear the drupal cache, restart varnish and apache or use a different settings.php file. These are steps that are repetitive and that you could spend your Monday mornings doing, or you can create whats called an Ansible playbook to do it for you.
• Interesting. So, if I’ve got a site that I’m working on, and a client that needs to review it (let’s go with your example of weekly), instead of finalizing all of my changes and updates on Friday afternoon, copying them to the staging server for review, clearing the cache, restarting varnish and apache and whatever else I need to do to get it ready for review, I can just run a single command and Ansible will do all of that for me?
  • For the most part yes. If you made any configuration changes, like Views or new content types that aren’t in a feature, you will still have to add those manually. But otherwise, yes. The repetitive tasks are what are scripted. If there is anything unique, like creating a new account with the right permissions, that may not be worth your time to script since its a one time thing, you may not want that in your ansible playbook.
• Okay, so you mentioned Chef and Puppet, and I’ve heard of those, but not Ansible - why do you prefer Ansible over the others?
  • The big difference between Ansible and its alternatives is that Ansible doesn’t require a client script. So for example, with puppet or chef, the tasks are executed by a script running on each machine. Ansible, on the other hand, is triggered from a centralized source, say your laptop or an admin machine, which then logs into other machines to execute the commands remotely. This was an important distinction for us because it meant that we didn’t have to install all the puppet or chef dependencies on all our servers.
• Oh, okay. So, if I have multiple sites running, whether they’re on the same server, the same hosting company, or even different hosting companies, I just install Ansible on my local (or admin machine) and everything is performed from there?
  • Exactly. As long as you can connect via ssh, you’re good. Actually, you also need at least Python 2.6 as well, but thats on most distro’s by default. For logging in, ansible’s configuration files allow you to specify the port, username and password of the remote server, or an ssh key of course. It can even su or sudo on the remote machines. What's also cool is that Ansible can run asynchronously - so if you have multiple staging machines the commands will run on each at the same time. Now, its nice if all the remote machines are the same OS - but for advanced users, you can even specify how commands are run slightly differently for different OS’s.

Use Cases

• Okay, I can already see huge benefits of utilizing these tools. But, I’d love to get your opinion on what the benefits are for Developers/Site-builders/Themers?
  • There are two big benefits as I see them, and another not so apparent. First, a lot of these tasks are repetitive. And things like copying a database may take a bit of time. Or merging code. Or running tests. Etc. Anything that you can automate means time you can spend on other things. Second, not everyone is as experienced - or maybe they don’t have the permissions - to execute all the tasks. You don’t want mistakes, you don’t want to give everyone permissions. So you figure out the best way to do it and then you automate it. The last reason is not as obvious. I think a lot of times we hack things together in our development environments to get it working - but then may run into issues later on. We don’t spend the extra time because its temporary. By spending a little extra time getting it right, we have created a reusable pattern that we can use on our next project. By encapsulating our best practices, we not only have a quicker setup, but we have a better one too.
• Perfect. So, save time by automating tasks like copying a database. Prevent mistakes by limiting who has permissions to execute tasks, and automating them so that even those who do have permission can’t introduce user error. And by setting up a process that uses best practices, creating new environments is faster, and better than if I had to try to remember all of the steps myself.
  • Exactly. And I’ll add, ansible can be used for each of installation, configuration, and orchestration. The examples we’ve talked about so far are orchestration - moving databases, code, etc. It can also be used to install Apache, Mysql, Mongodb, etc. Any set of system commands that are repeatable.
• Oh... So if you’ve got a server that you have full access to, you could actually wipe and rebuild the entire server AND Drupal site? We’re not limited to just configuring the Drupal site?
  • Exactly. And throw in Vagrant into the mix and now you can do that on your local machines using Virtual machines. Immagine spinning up a brand new VM and within a few clicks you have your entire development environment with a fresh drupal install all ready for you on a VM.
• Now, I do wonder who this is more geared toward. Developers, Site-builders or Themers. I understand that each of them can use these, and would probably help them all with their daily tasks, but who do you see benefiting the most from these tools. Or, do you have examples of people in each category that you know of that are using them?
  • I think all three benefit from automation. For example, in a previous life where I didn’t use Ansible, my themer was insanely good at theming, but when it came to running commands remotely on a server to check out his work, he was a fish out of water. I wish I had written an Ansible playbook so that he could check his code out onto staging. Or even better, if I had set up Jenkins to run an Ansible playbook to automatically check it out his work each time he committed. He wouldn’t have had to wait on me, sometimes a few days if I was not around. That said, he would not have been able create the ansible playbook.
  • As for who is using Ansible, well, Twitter does - they use it to roll out updates across their hundreds of machines. And of course BlackMesh, the hosting company I work for, also does. The product Cascade I mentioned uses ansible and Jenkins to do a lot of the things we talked about today, only we set it up so you don’t have to.

Z-Ray

• So, let’s start out with the basics. What exactly is Z-Ray?

  • So Z-Ray is a cool new tech that we introduced just a few months back which gives developers deep insight into all the PHP elements constructing their page, including Drupal-specific elements.
  • It’s basically a toolbar that’s displayed right in front of you in your browser. No code changes needed. You don’t have to configure anything. Just open your app in a browser and you’ll see Z-Ray at the bottom of the page!

• How does is work? Is there a module that you have to install on your site?

  • No. It’s not a module. Without going into too much detail: Z-Ray collects info while your app is being processed on the server side, and once the request is completed, Z-Ray’s JavaScript code is injected into the response with all the collected data.
  • There are other mechanisms at work, such as Ajax support, but as a whole that’s all there is to it. It’s also the limit of my technical understanding of how it works :-)

• So what info does Z-Ray display? What are it’s main features?
  Well. There’re so many features in Z-Ray, and I don’t think we have the time to go over them all, but to mention just a few.

  • Z-Ray gives you info on SQL queries. You’ll see a query’s bound value, the result of the query, how long the query took, and how many rows in your code are affected by the query.
    You can even see a backtrace of the query to get the larger picture on how your code was executed.
  • Z-Ray also gives you all the errors generated by PHP itself, or those created by its core functions. You can also see errors for silenced expressions. Exceptions are also displayed.
  • What do we have for Drupal devs? Z-Ray will give you a list of loaded Drupal modules with invoked actions and hooks, a list of called Drupal functions, a list of used Drupal forms on the page, and some general user info.
  • We’re especially excited about Z-Ray Live! Until now we’ve spoken about using Z-Ray in a browser, right? But what if you’re developing APIs or a mobile app? No browser there. So Z-Ray Live! is a new feature accessible via a dedicated page in the Zend Server UI, with an embedded Z-Ray.
    So as soon as this page is loaded, Z-Ray records and displays any request made to the web server, never mind where its coming from - whether from a browser, a mobile device or a web-service client.
  • One of the coolest things about Z-Ray is that you can plug in your own customized extension. Even people in Zend itself have begun developing their own extensions so its pretty viral.
    By the way, all the code for the Drupal extension is available on Github, so feel free to fork it and send us a pull request.
  • There’s integration with IDEs, session and cookie data, request info, and so much more to talk about.

• Is Z-Ray just for development? Or should it be used in production too?

  • Z-Ray was designed to be used both in dev and prod. While in development it works on every request, in production you can manually use Z-Ray using specially created access tokens. And it also periodically saves snapshots for important URLs - like the slowest requests on your web server, most time consuming requests, and so on. And again - with no changes to your code and no real implication on end-user experience or server performance.

• OK, if I want to give it a shot, what does the installation process look like?

  • Z-Ray’s bundled with Zend Server, so to use Z-Ray you would need to download and install Zend Server - a total no brainer. Just like installing any other PHP stack.

• So, how do you see Z-Ray helping Drupal developers?

  • At Zend we like to talk about left-shifting. This basically means that Z-Ray helps developers hit issues very early in the development cycle and way before going to staging and production.
  • We all know that getting clarity on all the Drupal elements working under the hood is extremely hard and takes loads of time. So at the end the day we believe that Z-Ray gives Drupal devs the visibility they need to properly profile their apps, identify bugs very early, and troubleshoot them.

Entity Pilot

• What is Entity Pilot?
  • I’ve been working on Entity Pilot since February, and have slowly been working through my backlog of features - but now its ready for Beta testers so that’s why I here on the show.
• How does Entity Pilot work?
  • The basic premise is you create your content like normal and then create a new departure. You can add the content to your departure in logical groups, or you can create one departure for each piece of content - its a pretty flexible workflow. So if you were working on a new product launch you might create all of that content on your staging site. You’d be able to see what the site will look like with the new content, preview the front-page and landing pages etc.
• You’re using airport terminology, like “baggage”, “departure” and “arrival”. Can you break those down, and explain what each one entails?
  • Passengers
  • Baggage
  • Departure - When you create the departure the baggage handler service takes care of adding the dependencies - so if you create a node, any terms or images it requires, or the author account are automatically added as baggage.
  • Arrival - On another site, you setup your Entity Pilot account and then create a new Arrival. This presents you with the list of your flights that exist in Entity Pilot for your account. After selecting the flight for the arrival, you move to the approval stage.
  • Customs - The approval stage presents you with a list of content on the incoming flight. Each item can be previewed and if it matches existing content on the site the administrator is able to view a diff of the changplanet-drupal the desired items to import are selected and imported either immediately, or via background processing.
• How does this work in a team?
• Talk about security
• Pricing

Use Cases

• For marketing: Prepare content for a product launch on the staging site. Land the content on the live site on launch day.
• For editors: Share content between your sites. Write content once and adapt to any site in your network.
• For developers: Deploy content when you deploy code. Use real content not lorem ipsum. Solid APIs to integrate into your custom deployment workflow or code.

Performance Optimization

• Before we dive deep into Varnish, I’d like to get a feel for the various performance improvements anyone can make to speed up their Drupal. What’s the process you think through when optimizing a site?
  • DB Tuning
  • Boost
  • Memcache
  • Redis
  • APC
  • Varnish
  • Module Choices!

Varnish

• What exactly is Varnish?
• When researching Drupal performance optimization, I came across a lot of references to APC and Varnish. What is the difference?
• Is this for anonymous or authenticated traffic?
• Is the Varnish module required to utilize Varnish with Drupal?
• What are the steps needed to install and utilize Varnish? (Broad terms, not actual terminal commands)
• Does SSL affect Varnish?
• What doesn’t Varnish do? (What needs to be done by Drupal, or other software instead?)
• How does Varnish affect a dev/staging/live workflow? Does Varnish need to be instlaled on the local machine?

Deployotron

• What is Deployotron?
  It’s a deployment tool.
  A low-key approach to easy and safe deployment.
• When did you start developing it?
  Dec. 2013
• Why did you feel it was necessary to add another layer on top of Drush?
  Needed simple, safe, easy to grok deployment process. Not a Swiss army knife. Other solutions proved to be either too complex/feature-rich or require 3rd party software beyond a simple drush command.
• How does it work?
  Deployotron is implemented as a drush command.
  When deploying it runs a set of “actions” that does the different steps of deploying.
• What kinds of actions can you perform with Deployotron?
• Is this extendable? Can other developers add their own commands to be performed during deployment?
• Is there a way to configure deployment per environment? So that certain commands are always performed on Staging but not on Live.
• What needs to be setup before you can use Deployotron?
  Deployotron requires drush and the appropriate ssh public keys on the target servers.
  And Git.
• How does it improve the deployment workflow?
  Easy and fast deployment to other enviroments from the command line.
  Quick to set up for a new project.
• What if you mess up? Is there a rollback feature?
  There’s an OMG command which will import a dump and reset the codebase to the one that was deployed at the time.
• How does this integrate with other tools like Capistrano, Aegir, etc.
  It doesn’t. Deployotron was meant to be more of an alternative; a simpler one.
• So, is this aimed at people hosting on shared hosts? Or do you need a VPS at minimum?
  Not aimed at shared hosts, as such, but it does support it, given that you have ssh access and drush availability of course.

Use Cases

• How are you guys using it at Reload?
  Very carefully :-)
  With careless abandon.
• Do you know of anyone else using it?
  No, not specificly, but there was some interest at the DrupalHagen DrupalCamp.

CINC

• What is CINC?
• How is it different from Features or Configuration Management?
• Is it something you use on an ongoing basis? Or is it just for the initial site setup?
• What types of configuration can you manage with CINC?
• What if you already have a content type created, and you want to add a field to the content type?
  • How does that affect existing content, and new content.
• What about the reverse? Can you remove a field?
  • What happens to the data that is already in the database?
• Can you undo configuration that you’ve created with CINC?
• How do you prevent site admins from disabling the module and deleting their content types?
• CINC YAML
• CINC & Features
• CINC & Drupal 8 Config API
• cinc.io
• Sheet2Module
• How do you see CINC working in a headless Drupal setting?

Use Cases

• Create dozens of fields quickly.
• Add a field to a content type after an existing field.
• Update configuration only if it still matches the default settings.
• How do you use this in a dev/staging/production
• Have you noticed any improved feedback, improvements to your workflow while using CINC?
• If people want to jump in and help development or work on new features what should they do?

Dutch Drupal Foundation

• What is the Dutch Drupal Foundation?
  • Dutch Drupal Foundation supports Dutch Drupal Community. We’re groing. Trying to handle work in a professional way in a voluntary way. Lead double lives: daytime job versus community commitments.
• How is this different from the Drupal Association?
• How has that been going for you?
• Would you recommend other regional areas start something similar?

Events

• What events have you organized recently?
  • Drupal Events in Dutch speaking part of EU (Belgium and Netherlands) became big. E.g. DrupalJam (350ppl), Drupal Training Day (250 students). Relatively large events become routine. Now ppl spreading cross border on tour to share experience
• Tell me about DrupalJam. What is it, and how long have you been putting it on?
  • DrupalJam largest event in Netherlands. Started small in 2007 with 30 persons and some pizza boxes in a basement. Now 350 ppl / €25K. Outgrown study dorm and pizza’s. Also started Drupal Training Day (largest worldwide) and Drupal Splash Awards. Lot of work and commitment. Getting more serious. Growing pains expected
• What are some difference in paid events vs. free events?
• What has the attendance been like for free events?
• Where does the income come from?
• How do you get sponsors?
• When do you recommend charging a ticket price to attendees?
• How have you seen your community transition from coders who just come to code for fun, to people who are making their living using Drupal?
  • Come for Drupal, stay for code. Sounds fun. It is. But business side of things and commitment to making this work professionally is different cake.
  • Balance between business side and community side, which is a recurring topic for everything we do in the community. Drupal has become enterprise platform, community still many smaller agencies w/ different values (see incline on Gartner oct. 2014)
• What does the Drupal landscape look like in europe
  • Pan EU events: Frontend United, CxO, Drupal Government Days Brussels 2009
  • DrupalEU