Dropfort

• What is Dropfort?
  • Dropfort is a suite of tools to develop and manage Drupal applications.

On the development side, it integrates with GitHub and GitLab to track commits, issues and tags. It then packages releases based on those tags and lets you share those releases with your Drupal sites. Same way you tag and create releases on Drupal.org. The only difference is the released modules are private. Meaning a site that wants to use those modules needs to authenticate to download them.

For example, if you want to download a custom module to your site from dropfort, you can just do a “drush dl mymodule --source=https://app.dropfort.com/fserver/release-history”. Dropfort generates the same XML data for its modules as does Drupal.org for contrib modules. Meaning the Update module works with Dropfort, all your Drush commands work and Drush make works too. It’s all pretty seamless. The only difference with our XML is that it's not publicly available. Your site has to be allowed to see the update feed which is what you configure in the Dropfort web app itself.

The other half of Dropfort are the operations or “ops” tools. You connect your sites to Dropfort using the Dropfort Update module (which is available on Drupal.org) and it will start doing a few things. The most obvious is tracking the update status of your site. Being a Drupal shop, we monitor a few dozen Drupal sites at once and so logging into each site to see what modules need updating and the status of those sites is time consuming. What Dropfort let’s us do is see all those sites in one dashboard. I can login and see the update status and status report data from all the sites in one place. Dropfort then uses this data to generate some graphs about your sites. For example it can tell you how many dev modules you’re using across all your sites, it shows you a list of what sites have security updates and it does some fancy calculations to grade your site’s health as well. Lots of metrics to know what’s going on and how things are changing over time.

The last part, and this is what I’ve been working on mostly these last few months, is an Environment manager. It’s still pretty fresh and there are some rough edges but it does work. You can create a set of environments (dev, testing or production) to store machine configurations to both do your development and run your Drupal applications. You can say “I want a server running apache, with MySQL and the Commerce Kickstart distro” on it. Then you can either download a Vagrant file which will provision a vm, or you can download a docker container which will do the same. Or you can run a bash script on an existing machine to link the server to Dropfort and have that configuration deployed. It’s pretty neat stuff. Basically any server anywhere can be turned into a managed Drupal cloud.

Like I said, there’s a whole suite of stuff in here.

• Why did you create Dropfort? Where did it start?

It all really came from using Feature Server for Drupal 6. When we incorporated Coldfront in 2011 and really turned it into a full time job, we wanted a way to distribute code to all our clients (even though at the time I think we only had one). We setup FServer to deploy code to your client sites. But manually creating the releases and the pages and stuff was kind of a pain. So we came up with a special commit syntax we could add to Subversion (this was before Git was a big thing). So could make our svn commit and in the message we’d write [release:full] and some post-commit scripts would run on the svn server. They’d look at the commit message and then take the code, create a tgz file, create a tag, commit the tag and then upload the tgz file to FServer using some REST web service endpoints we create on a Drupal 6 site with Services. That would create the release page, the release notes, add the files and generate the update XML. It was pretty well a mini Drupal.org but with Subversion (instead of CVS which D.O was still using at the time). It actually worked really well. So well in fact that the University of Ottawa asked if we could install a version for them to manage their Drupal stuff (which they’re actually still using today until their git migration is done). That’s when the lightbulb went off I guess. We had built this stuff for us but as it turns out, other people want to deploy custom modules too! Who’da thunk it?

And that’s when the idea for a more “web app” version of our SVN workflow came to be. At the time I thought “Yeah we can totally rewrite this in no time, I give is 6 months and we’ll have a web app ready to go”. That was 3 years ago I think? It took a bit longer than expected. Mostly I kept adding features… Yay scope creep. But now we’ve got a pretty awesome suite of tools and we’re focusing on the polish now. I’ve been told I’m not allowed to make feature requests for at least a month. We’ll see how that goes.

• Can this be used to compete with Drupal.org? Meaning can people share public releases here instead of on d.o?

Nope. Public modules should be on Drupal.org. No module can be downloaded from Dropfort without authenticating first. We don’t want to supersede d.o in any way. We actually looked into writing a feature to automatically move a project from bieng private to a public one on d.o but since Drupal.org doesnt’ have an API we couldn’t do that. But yeah, this is for privately distributed modules only.

• Does the monitoring dashboard check both the private projects as well as public ones on d.o?

  • Yes

• What did you build it with? Is Dropfort open-source?

The original SVN workflow using Subversion, CLI PHP, Drupal 6, 2 custom modules and some REST / Services stuff.

Dropfort uses Git, Drupal 7, Services, about 20 or so custom modules, Puppet and our Drupal 7 port of FServer. So Dropfort is a Drupal application itself. We actually use Dropfort to manage Dropfort. Meaning we track it’s own updates and status using itself, and it packages releases for itself. A little inceptiony but it works.

Most of the parts which make up Dropfort are open. Some of the custom modules aren’t openly available. But that’s mostly because we don’t have the bandwidth to help and support the distro on D.o. Especially the stuff involving setting up a Puppet master. We’d spend more of our time debugging that than actually making things work. Doesn’t mean we won’t share everything eventually, just not right now.

• What’s the plan for Dropfort? Is it a paid service or is it free?

Right now it’s free to use. The main reason for that is we haven’t written the commerce component yet so we can’t actually charge for it so… yeah. But we’re looking at different ways of monetizing. It’s tricky cause we want people to use it but at the same time we don’t really know what people will use. There’s such a variety of things in there it’s tough to decide what should be charged for. For example Github is pretty straightforward in their pay structure. If the code is open, your repo is free. If your code is closed, you pay for the repo. For us, I think it’s a question of usage. We’re leaning towards all tools are free to use with any account, it’s just a question of how much storage or how many sites you’re using. But regardless, anyone who uses it now is free to use it as much as they want. And we’ll have some special plans for early adopters as a thanks for their feedback. More than likely a bunch of free stuff.

• How does this compare to other tools like Platform.sh, Pantheon, Acquia Dev Cloud?

The big difference is that they’re primarily a hosting platform. Dropfort is a management platform. You can connect a Pantheon site or an Acquia Dev cloud site to Dropfort and use most of the features no problem. You’d probably skip the release packaging stuff and environment management (for now) but the stats tracking and collaboration tools would work just fine. Dropfort doesn’t care where or how you run your Drupal site. As long as it can reach the internet, you can use Dropfort.

But you can use Dropfort with GitHub or GitLab or neither. You can use Vagrant or Docker or both. We do our best to integrate with anything which might make building Drupal application easier. It’s all about choice.

As for the hosting side of things, we give you tools to deploy your own server or cloud of servers. Meaning you can run an optimized network of Drupal web servers on whatever provider you want. It’s a philosophical difference. We let you host your code and sites wherever you want whereas with others you live on their machines. Which can have a lot of advantages and for the majority of folks out there, that’s fine with them.

But for us we’ve found it difficult for some enterprises here in Canada to get hosting on services in the US which are bound by the Patriot Act. We have FIPA, the Freedom of Information and Privacy Act which states that we can’t share information about users unless the user has explicitly allowed that agency access. The Patriot Act is pretty much the exact opposite of that. So we figured we’d bring most if not all of the advantages of a cloud solution (the optimized configuration, automated deployments / scaling, generated environments) to anyone’s infrastructure. You just supply the hardware, Dropfort does the rest.

I see it as just another option in how you can host your Drupal site. You can choose how much or how little you want to be involved in managing the hosting environment. Whichever way works best for you is the one you should go with.

• How does this handle dev/staging/live scenarios?
• How about local?

Use Cases

• Let’s talk about the current use cases for Dropfort.
  • Managing several sites in one place
  • Create custom, shareable development environments
  • Create releases of projects destined for more than one application
• Why would you use Dropfort instead of just Git to manage deployments?
  • We use Drush Make for just about everything. We control our releases using Drush make. We apply patches with Drush make. We really like Drush make. And we really don’t like merge conflicts. The number of times I’ve come into a project where the entirety of Drupal core and all the contrib modules are in a single repo with a team of people trying to all work on it at once has taught me that’s not the way to work. Treat your projects like d.o does, as self contained sets of functionality. Use make files to build your application and drush to manage updates. This is how Drupal is designed to work. Drupal is a collection of modules. When you all of a sudden lump it all together into a single repo you’re breaking how Drupal was meant to be managed.
• Can you explain a bit more about how Drush Make works?
• You just mentioned automated updates.
• What’s in the near, and far future for Dropfort?

Training

While at DrupalCon recently, someone mentioned my recent podcasts about training and hiring Drupal talent, and mentioned that Cameron had been a part of a somewhat unique program to train new talent through a two month training program, with the intent to hire some of them. I was intrigued (and based on the great response of recent episodes on similar topics) I decided to get ahold of him, so here we are.

• What company were you working for when you did this? And what was your role there?
  • ProPeople (Now FFW)
• How did the idea come about?
  • We were inspired by a similar program they did in the Moldova office.
• Who were you targeting?
  • Current college students, and recent grads
• What did you hope to accomplish?
  • There were two goals. One: We wanted to be able to hire people in the Boise area. Two: To get our name out there, to university students, as well as the broader tech community in the Boise area so that when people were looking for work in the future, they’d look at ProPeople.
  • We weren’t looking for Drupal experts, we wanted them to come out with the minimum ability necessary to become a competent member of a team within ProPeople
• How many people did you have participating at the beginning?
  • 70-80 applied
  • 12 were accepted (2 dropped out)
• How many did you end up hiring?
  • 1 full time junior developer
  • 1 contractor
• How did you promote it?
  • Craigslist, Boise State University CS department, Random encounter at the coffee shop
• What did they participants do?
  • Clone Netflix and Basecamp
  • Wanted them to work as a team
  • Learn Features (getting config into code - turns out it’s more difficult than we thought)
  • Issues lead to Skype calls to talk about how something might be done better
• What kind of interaction did you have with the participants?
  • Daily checking call with each member
  • Weekly check-in call with each team

Should others do this?

• Was there enough of an ROI that you think Drupal shops should be doing more of this?
  • Fairly expensive in number of dollars spent (About $4,000 - from that came one junior developer, and one consultant, who have been there ever since). Compare that to a recruitment agency, where you can pay well over $4,000 each.
  • Great to see who is actually going to go out and learn what they need to know in order to accomplish their project vs. someone you might hire, and then later find out that they’re not a good fit.
• Do you have any suggestions for companies that are interested in looking into running a program like this?
  • If you’re going to do this, you need to have someone committed to running the program. It’s not a no-commitment project. Having more than one would be even better.
  • One reason we chose to go the training route, is because ProPeople didn’t have a standard classroom style training program. I’d recommend having more structured content: What knowledge we want them to come out of the program with. The hands on stuff is great, but throwing someone into the deep end is a bit hit or miss. (Use Blink Reactions stuff)

NodeSquirrel

• What is NodeSquirrel?
  • Backup and Migrate (over 300,000 sites using it)
  • Only cloud backup-as-a-service for Drupal
  • Backup database and Files
  • Every site should have automatic off-site backups configured.
• That leads great into the pantheon acquisition, because now everyone can create free off-site backups with NodeSquirrel.

Pantheon Acquisition

• How did the aqcuisition come about?
  • Drew - BadCamp October 2014. Sitting at a Pantheon party, ended up talking to Zack about running a services firm and product at the same time. The excitement, issues etc. involved with that. And it happened.
• Did you pitch it to him? Did he pitch it to you?
  • Me.
• So, what’s in it for them? Why did Pantheon buy NodeSquirrel?
  • They want to build great tools that developers can love and trust and use.
• What’s new now that it’s been acquired by Pantheon?
  • It’s FREE! (There’s a new free tier)
  • It’s not a trial. It’s always free.
  • It’s not just for Pantheon customers.
  • We have access to Pantheon’s resources to further the development and continue improving support.
  • It’s been a side project from the start, but now that it’s backed by a real product company, we’re officially here for the long-haul. So, if you’ve been holding off because you didn’t want to risk storing your backups with something that might not be there in a year, you can rest assured that we’ll be here.
  • (Ronan) Now I’m able to dedicate more time and effort to the Backup and Migrate module (instead of only using my free time), which helps the community as a whole.
• What happens to current users?
  • Higher storage limits
  • Might even be able to downgrade and pay less.
  • Won’t lose any features. Still use local backup, etc. (except maybe email)
• What’s in the future?
  • Drupal 8
  • Wordpress
  • More features for both Backup and Migrate and NodeSquirrel
  • Bringing Pantheon features to NodeSquirrel and NodeSquirrel features to Pantheon (Notes)

Writing a Book for D8

• What’s it like writing a book for a piece of software that isn’t even officially released yet?
• How long did the writing process take?
  • Packt publishing sent us a proposal to write this book in December of 2013. We got started quickly, sending them an outline of the chapters and an estimated page count in the same month. The original estimated page count was 150, it turned out to be around 120. We received a pretty strict time line, having to finish a chapter every two weeks, starting in December of 2013.
  • We managed to finish most chapters in two weeks, but some of the longer ones took a little longer since we also started one of our biggest projects we had had until then, also in January. That was pretty tough because that project took up way more than a regular full time job, so we ended up having to write all of the chapters late at night and on the weekends. In May, all of our chapters then went to the editors and we didn’t hear back from the publisher for a really long time.
  • We also told them that we will have to rewrite a lot of the chapters since there was so much work in progress with the Configuration Management Initiative and they were changing a lot about how it worked, like going from the file based default to the database default. I think it was in January of 2015 when chapters came back with some feedback and we started rewriting every chapter, which was pretty painful at the time. We were able to update some of the documentation at drupal.org with the changes we found. It felt good to contribution at least a small part, when with our project and the book we had no time left to contribute code to Drupal 8 like we usually do.
  • We spent around 40 days on the book between the two of us.
  • In December, Packt asked the first publisher to review the book. We had recommended them one of our team members at undpaul, Tom, who has a similar amount of Drupal knowledge as Stefan. We really wanted to have someone from CMI to review the book, like Greg Dunlap. They had turned down reviewing the book after the first chapters were written, because too much would still change. Then after the changes went in we kept recommending Greg but I never heard anything back, maybe he was busy or they didn’t bother to ask. At the beginning of this year they told us the book was planned to be published by March. We recommended waiting because we didn’t expect a release candidate before the European Drupalcon and we would have rather had someone like Greg take the time to review, but Packt had another opinion :) Since most of CMI changes were finished, we didn’t feel too uncomfortable about the time of publishing, and it was also kind of nice to finally be done with this thing :) So it took a little over a year from start to finish. It was published on March 24th.
• Do you expect to need to rewrite anything between now and when 8.0 is released?

The Book: Drupal 8 Configuration Management

• What do you cover in the book?
  • We start of with a basic introduction to what Configuration Management in Drupal means, because it is a thing in Software development in general, that doesn’t usually refer to what it is in Drupal, where it basically just means that configuration is saved in files which makes deployment easier. In the first chapters, we make sure the reader understands what Configuration Management means and why version control is so important. We mention some best practices and then show how to use it for non-coders as well, since there’s a nice backend non-technical folks can use, even if you don’t use version control (which of course we don’t recommend). We also have a part that describes how managing configuration works in Drupal 7 (Features!) and then dive into code examples, explaining schema files, showing how to add configuration to a custom module, how to upgrade Drupal 7 variables to the new system and cover configuration management for multilingual sites.
• Who is the target audience of the book?
• Why did you decide to write about Configuration Management?
  • We have used Features to deploy configuration changes for a very long time, I don’t recall not using it since we started the company 5 years ago. We have talked about it at several DrupalCamps and Drupal User Groups and always tried to convince everyone to use it. We were really excited about the Configuration Management Initiative and thought it was a very good fit for us.
• Before we started recording, you mentioned that there is a companion website to the book. Can you talk about what content we’ll find there, and what purpose that serves?
• Are you building any sites in D8 at Undpaul?

d8upgrade.org

• What is d8upgrade.org?
• When did you put the site together?
  • Today
• How were you offering the service before the site was up?
  • Talk about MVP
• Why did you guys create this service?
  • Build a more direct communication channel for module maintainers - Mailing list (module devs talk to people using module
  • Initiative to get out the catch 22 - modules aren’t ready, so you can’t upgrade, you don’t upgrade so the modules don’t get money to be upgraded.
• Does it cost anything?
  • Nope - it’s free
• Are you planning to monitize the site in any way?
  • We expect a lot of site owners will be looking for a way to upgrade their website, if they don’t yet have an agency, we can help them find a good agency. If this evolves how we imagine, this could fuel the sales pipeline for our consultancy and a few selected partners. We are not going to go hardcore salesy on this, but put in links for module work and 2for1 discounts and hopefully some of those will convert into customers for us or our partners.
• How does it work? What do I have to do to track a site?
  • As simple as possible, conscious choice to not make it a module - just copy paste your module page
• How many people have signed up so far?
  • We’ve got a total of 73 sites now, some from the same people - after Drupalcon LA we expect that to become a few hundred.
• What do you have planned for the future of the site?
  • The community misses a permission marketing channel for module developers. We would like to let site owners sign up for important information about the modules they use through a push communication channel. There is a lot of talk in the community that we are not getting any support from site owners for the development of D8, but we’ve made it almost impossible for them to stay up to date. There is no way they are going to start reading the issue queues of modules on regular basis.
• I took a look at the site and saw something called 2for1 - Give to Get. What’s that?
  • This is part of the strategy we want to use to monetise the platform: we offer our customers a 2 dollar discount on their future D8 project for every dollar they invest in helping Drupal 8 get out of the door. This gets us more business and brings outside funding into the community. For too long we’ve been giving money from one developer’s pocket to the next developer’s pocket. We hope we can change that this way. We are inviting other consultancies to join us in this campaign, if you want to join, you can become a partner in the campaign.

Special Thanks!

Thank you FourKitchens for the DrupalCon Ticket and Kalamuna for helping offset my travel expenses! You guys (and gals) are all awesome!

How to Select Modules

• So, let’s start out by talking about the genesis of your session. What made you think this topic needed to be covered?
• What’s the problem with just installing another module?
• What if I don’t program?
  • Reuse modules
  • Push back on requirements
  • Make sure the cost of adding another module is not just the cost of the time it takes to install it
  • Simple modules really aren’t programming
• What are the potential problems with custom code?
• How do you determine if you should install a module or write some custom code?

Specific Modules You Can (or Should) Avoid

Not really telling people to avoid specific modules more about thinking twice when they select them.
* Entityform vs. Webform
* Entity View Modes
* Page title
* Commerce Custom Order Status

AcquiaU

While this episode might end up sounding like a giant advertisement for AcquiaU, it’s really not intended to be. I wanted to have you on to talk about the concept of, how a company that hires Drupal developers, can and should go about training them before they are hired.

In order to set the backdrop for the rest of our conversation, I’d like to quote a bit from the AcquiaU website:

The challenge the community is facing is one of supply and demand. Simply put, there just aren’t enough people to fill the needs. At any given time in the past 6 months, job aggregator Indeed.com has over 2,500 open position across the US for Drupal talent.

How do we close the gap? Find the people with the right passion and grow their talent from the inside-out. We're not looking for people with years and years of Drupal experience. We're looking for people who are curious, motivated, determined, and who can inspire a little crazy in us all. At Acquia, culture and a person's POTENTIAL to contribute and grow with us matters. A lot. These are the underpinnings of a successful candidate.

What I love about that is that you’re not looking for senior level developers with 5+ years experience. Because you’re not going to find them. They all already have jobs.

Mike and I ranted about that in the last podcast, so I won’t rehash it here, but what we boiled it down to is that Drupal shops need to create a talent pipeline for recruitment, which, as I understand it, is essentially what AcquiaU is for Acquia.

Ok, with all that said, I’ll shut up now, and let you do the talking.

• Can you give us your description of what AcquiaU is?
  • The program is 14-weeks of hands-on training in Drupal, Acquia Products, related web technologies, and professional development skills like team building, leadership, and communication skills. We spend the first 6 weeks in a classroom environment, which is a combination of lecture, group projects, individual assignments, and self-paced learning. The most recent graduates’ project was to redesign the program’s website, u.acquia.com Each participant is assigned an Acquia Mentor who is there to not just be a buddy, but to help from a technical perspective. The next 8 weeks are spent with job rotations where they work with our customer facing Professional Services developers and customer support. Each person is assigned a client team and works side-by-side on real projects. You might think it is like any other tech bootcamp out there but we differentiate ourselves in a couple of key areas. First, we make sure we have an open job opportunity for each person who joins the program and second, we pay people to learn. Many other bootcamps have a high cost- on average up to $10,000 and while they help with job placement, I can’t say how many have jobs lined up for graduates BEFORE they join the program.
• How do you select your candidates? Or can anyone join the class?
  • We have a rigorous screening process and look for people with 2-3 years of technology experience, but who might not be able to get a job with a development shop. A lot of times, this level of talent is overlooked because companies don’t have the internal mechanisms to train, mentor, and coach junior level talent. They are already stretched thin and want new hires to hit the ground running at a fairly high level of proficiency.
• What types of skills do you teach?
  • We dive deep into Drupal and other web tech skills like Drush, GitHub, and Agile and a dive into our own products and services. Helping people become well rounded also means that we do workshops in team building, communication skills, and presentation skills. The next session will have an engineering focus so we will be digging into LAMP stack and web architecture.
• Do the students have any obligation to Acquia at the end of the program? (Like they have to work for you for a given time period after the program?)
  • People are hired on as temp employees and we really hope they have had a great experience and want to stay on. The program’s goal is to hire them at Acquia or with one of our partners.
• What percentage of students would you say you hire on average?
  • So far we have a 90% hire rate. The goal for 2016 is to expand the program and hire more people into other Drupal shops
• Do you have information about those that you don’t hire? Do you know if they’re employed somewhere else? Or did they decide Drupal wasn’t for them?

Expanding the Concept

• Now that you have a few classes under your belt, is this something you think other shops should look into doing?
  • People have asked me this and I think they should think about what the end goal is. Our program is not to just train more people for Acquia, but to give back to the Drupal community by creating a long-form drupal training program with learning paths and a structured hands-on curriculum.
• We were talking before we started recording about this idea. Mike had mentioned that shops should create a talent pipeline. And while I agree with that in theory, what that means is that the shops first have to develop a training program, and one or more people who are skilled at both Drupal and teaching in a way that doesn’t alienate the trainee. From your perspective, how would you respond to that?
  • Having a talent pipleline means that you have a people development strategy that aligns to your business strategy, and that you have launched that people plan long before you launch the business strategy. Most companies play catch-up and are more reactive than proactive. Being proactive means you’re looking ahead 2-4 years out and making plans for your people.
• If there is a shop owner out there listening right now, what would your advice be on how to go about creating a program like this?
  • I think you really have to be prepared to commit. Budgets need to allow for hiring junior talent, the business needs to be ready to bring in this level of need. It takes a lot of planning to launch a program like this. For companies that can’t support hiring 5-10 junior level talent, they should start out with a smaller number. A really strong learning program doesn’t just focus on the skills, but on different ways that people will need to learn and being able to translate really complex ideas into ways that different people will relate to. If you’ve ever heard about the adult learning cycle and experiential learning, we know that people tend to be most successful learning new skills when they can reach back into their own experience and apply them to the new content. Being a really strong developer doesn’t always mean that you can tap into other peoples’ experiences and make it relevant to them now. So when you look at creating that pipeline and having junior level talent come on board, you also have to figure out the most effective way to do it.

The current job market in Drupal

• What does the job market look like in the Drupal space?
• Let’s talk about the pipeline idea.
  • Experienced developers vs. Junior developers

What to do to start your career

• If you’re completely new what do you do?
• If you’re a hobbyist, what do you do?
• How do you get experience?

From the employers perspective

• What should job descriptions look like?

Questions from Twitter

• Paul Booker
  How should a small business, non-profit, charity, .. find a drupal developer when they need something doing?
• Damien McKenna
  For someone who knows nothing about web development, what's their best path forward?
• Chris Hall
  When do you know to ask for/be a Drupal Dev vs a PHP Dev with Drupal experience? Should/is there be a difference?