Blog - thoughts and ramblings...

Drupal Geek: Using Drush to Pull Latest Production Website Back to a Development Environment

In this blog, I'll describe how to leverage drush, the DRUpal SHell, to pull a production system back to a development environment. Couple of assumptions here:

  1. Drush is installed and available from the command line. This is a topic for another blog but, in essence, you download the module from http://drupal.org/project/drush, expand the tarball and put it in your path.
  2. You are using SSH private/public keys to login to the production server. Once again, this is beyond the scope of this blog post but this is a good place to learn how to do it: http://remysharp.com/2007/01/22/ssh-without-a-password/
  3. Your development environment web server and database are already set up, meaning that the directory for the dev site already exists and the server points to it. Also, the database exists.

Step 1. Create a drush alias file.

Drush has the ability to use alias files to refer to specific drupal site installations. Instead of having to ssh to a server, change directory to the specific drupal sites directory and then run drush, I can just run something like "drush @mysite ..." and then the rest of the command. The drush alias file should be stored in your home directory's .drush folder. Here is what I would do on a new Mac machine:

If it doesn't already exist, create a .drush folder in your home directory

mkdir ~/.drush

Now, with your favorite text editor (I switch back and forth between using Coda and Textmate), create a new file. The file MUST end with "aliases.drushrc.php". You can choose to prefix it any way you like using ordinary file naming conventions. For example, let's do an alias file for the example.com website. I would likely name the file "example.aliases.drushrc.php" like so using Textmate:

mate ~/.drush/example.aliases.drushrc.php

or using vi:

vi ~/.drush/example.aliases.drushrc.php

Assuming your development environment is local, enter the following:

< ?php //note that the space before the question mark should be removed...couldn't post to blog without it :-(
$aliases['example-local'] = array(
  'uri' => 'default', // The Drupal multi-site folder under <drupalroot>/sites/
  'root' => '/Users/bjones/Sites/example/httpdocs', // This is your website root folder
);
$aliases['example-prod'] = array( 
  'uri' => 'default',
  'root' => '/var/www/example.com/httpdocs', //Path to website root on your production server
  'remote-host' => 'www.example.com', // url to your production web server accessible via ssh
  'remote-user' => 'bjones', // your username on that server. Remember that you should have ssh keys for logging in
  'path-aliases' => array(
      '%dump' => '/Users/bjones/.drush/example-dump.sql', //this keeps you from having to specify dump files
   ),

);
?>

The $aliases lines have arbitrary names that we're going to use now

Step 2. Pull the files

From the command line on your dev server, enter:

drush rsync @example-prod @example-local

Note that the first alias is the source site and the second one is the destination site. Drush will use the information in the example.aliases.drushrc.php file to know where to get the source files and where to put them on the destination site.

Your dev machine will ask you to confirm that you're going to destroy your local version of the directory and replace it with the production site. The very cool thing about this process is that drush (rsync really) will only pull what has changed on the production server to the development machine. One gotcha for this process can be if you don't have full write permissions on the development directory.

Note: drush rsync does not pull settings.php files. This is very cool in that your development environment can have different database login information than your production system and drush rsync will not overwrite the dev environment setup.

Step 3. Pull the database

Again from the command line on your dev server, enter:

drush sql-sync @example-prod @example-local

Again the system will let you know that armageddon is raining down on your poor development environment. This time, the developemnt database will be replaced with the production database. Assuming that's ok with you, confirm.

Conclusion

That's pretty much it. Once you setup your alias files for your different sites, you'll discover that pull prod back to dev to make a change is two command line entries: "drush rsync ..." and "drush sql-sync ..." It makes doing updates to a dev environment so much easier (and encourages you to NOT make changes directly to prod).

Microsoft Acquiring Skype: Fears of History Repeating Itself

The news came out this morning that Microsoft had, in fact, agreed to purchase Skype for $8.5B. I suppose that I've known all along that Skype would not go on as-is forever. My hope was that it would go IPO and remain committed and focused to what it does really well. With the news this morning, I'm filled with more than a bit of trepidation. You see we've gone through this once before with a product called Groove.

Back in 2005-06, we purchased a software product from a company called Groove Networks. Ray Ozzie was the founder and brainiac behind it. In simple terms, it was peer to peer software that allowed us to share files, projects and other tools on any computer we set up without a central server...sort of. There actually was a central server that Groove Networks controlled that made the whole thing tick. Then, Microsoft bought Ray Ozzie...I mean bought Groove Networks and made Ray Ozzie the big man on campus.

Groove was "integrated" into Microsoft's Office server stack meaning that the free standing product went away. For folks that love Microsoft products, it was a bonus. We were Microsoft centric at the time but considering all options with the coming release of Vista. The bigger problem was that during the transition our project management tool that was built in Groove wouldn't work with any new releases of Groove. As we added people, we couldn't add more Groove licenses and, even if we could, our core project management tool wouldn't work on the new version (which is all you could get). FAIL.

That was the straw that broke the camels back for me. I purchased my first Mac (having never personally logged into a mac in my life) in January 2006. I transitioned literally overnight and have never gone back.

Here's my fear. Skype is unique. It's the only solution I've found that meets all of our real-time web communication needs. We use it for instant messaging, voice, group video...even phone calls to the telephone people. :-) It works really well. It's what allows us to run our business with folks working from home. It empowers us to collaborate with clients spread all over the United States. Now it's in the hands of Microsoft which has a penchant for making you drink the whole cup of kool-aid if you want any of the tools inside the cup.

I feel like it's 2005 again. Google Talk is looking pretty good right now.

Saving the planet (with ColdFusion) is just part of the job

So we've been the primary web partner for the Student Conservation Association (SCA) since just after we spun off of IniNet back in late 2002. For me, working with them has been a point of great pride. It resonates with me on two fronts...1) they are mission driven and 2) they work with young people to help prepare them for life. They put (mostly) college interns and high school volunteers out into parks across the country to "serve the land." That is, they do things to make the parks better and they've been doing it since 1957. They were green before being green was cool.

Over the years I've observed something that always bothered them. They were a group bent on preserving our earth yet realized that printing ungodly amounts of paper was a necessary evil to putting better than 4000 students into the field each year. In fact, more than 25,000 documents needed to be printed, signed and tracked each year. That's a lot of paper.

This year we embarked upon an ambitious plan to minimize, if not eliminate, that paper. This year we implemented an e-signature system into their web-based student tracking system called MAX. Built on a ColdFusion (CF) foundation, MAX is the central repository of all SCA's internship and volunteer positions that matched students with needs nationwide. It tracks everything from intern requisitions to paycards to name tags for uniforms. Back in 2005, it completely transformed SCA's means of doing business. The best part of choosing CF as the application platform? It's able to grow with their needs change. In this case, we empowered SCA to better pursue its mission by making the volunteer/intern selection process paperless.

Admittedly we do most of our applications in Drupal these days but we still have ColdFusion expertise on staff just so we can do cool projects like this. We continue to be proud to call SCA our clients (and our friends) and we're glad they let us take part in supporting their mission.

Bad blind dates and a good book...

So, I'm the new Support Manager here at Lucidus. When I started diving into this role I asked Mr.G what exactly it was that I was going to do. Simply enough.. "Customer Service, yada, yada, yada…I’m sending you a book."

A book? Really, Mr.G?

Sure enough, three days later I get this thin, 6x8, hardbound book delivered to me via a nice postal worker. (I was excited that I was thought of highly enough that he splurged for hardbound. w00t!) The name of the book was Raving Fans, which, it turns out, is exactly what Mr.G wanted me to produce.

So what is a Raving Fan? Well, I guess it is easier to start off with what it isn't. Most people are not raving fans and it is due highly to the lack of customer service we have become accustomed to. We expect crappy service because that is the norm. Sure, we all want awesome service, but we set our sights frightfully low. It's kind of like when you go out on the 5th blind date your best friend set you up with. When we get anything marginally exceeding what we have had the first four dates, we're blown away. Still, four Mr. Wrongs don't make a Right.

I know that there are some places out there that have amazing customer service and, as a consumer, I absolutely flock back to them. It can be done. I've been a Raving Fan before and I know what it's like. So, who is a Raving Fan? Hopefully you will be and I know I am going to do everything in my power (well, as much as Mr. G will give me… wait.. I'm the Support Manager.. that should be more than enough power to take over the world!… I mean the company!… I mean my position..) to make you one. I want you to be able to shout from the rooftop that you love our web development company. You know how those people on TV say they love their bank? I want better!

I'm here to listen and make sure you are taken care of. I truly look forward to working with all of our customers old and new. So, let me help you become a Raving Fan of Lucidus.

Eating Our Own Dogfood: How Lucidus uses Drupal (and Open Atrium) to collaborate on projects.

So we've been using a web-based system called Golaborate to track our work and collaborate with customers on projects for a couple years now. It's built in Drupal and based on packaged installation called Open Atrium. Open Atrium is the best example I know of a Drupal "product" that really gives you pre-configured functionality out of the box. It includes private areas for each project with tools for sharing notes and documents, posting project assets, monitoring project schedule and tracking tasks.

The coolest part is the fact that, like Drupal itself, we can extend it to add whatever functionality we want.

We've added our own "features" to the system to track business development leads and provide a shared company address book. Using common modules in Drupal like Views, we were able to create a feature for global case management that helps our developers get their own work queue and stay on top of things. The global reporting tool also provides us a case scoreboard for knowing how many items are open and what's most important. We built each of the tools in under a day using the framework provided and have been able to refine them over time.

Most recently, we added the TimeTracker module to the system and now we're able to track our time against each individual "case" to know how much time we spend on support on a ticket-by-ticket basis. Next up, we're working on a feature that will capture user stories, otherwise known as our feature list. All of our web projects begin with a strategic engagement where we analyze the project and deliver the "blueprints" for the project including design comps, wireframes and a sitemap. We also create a "feature list" that includes all of the possible things we could build into the site and a fixed price quote for each one. Our clients choose the features they want and then we build the site.

This new Golaborate tool will allow us to create the feature list within the system, empower our clients to approve the features they want and then let us schedule the work into one of our two week development cycles. From there, the options are dizzying. We could automate project scheduling (to a degree), generate delivery calendars, track time against the features, note progress on each feature within comments attached...my head is spinning! (Don't tell my finance folks but we could even automate invoicing and payment through there).

The best part? All within a single system built in Drupal. The fact is that the longer we use Drupal (five years now), the more it keeps proving itself to be just the swiss army knife we need for our projects.

What does Drupal have to do with the Apple App Store (and iPhone)?

So there's no question that the Apple iPhone has revolutionized the traditional phone and really brought the concept of the little personal data assistant (PDA) into the main stream connected with the phone. The thing is that there have been lots of PDA's and even "smartphones" over the years that never quite gained acceptance. Some people contend it's the user interface that makes the iPhone so cool. There's no doubt in my head that the iPhone interface (like all Apple interfaces) is an actually joyful experience. But I don't buy it as the reason everyone started using it. The reason for the rise of the iPhone is the App Store. The fact that you can a) find thousands of apps that you could be interested in, b) see what other people have to say about them and c) install them seamlessly is what sets apart the iPhone from predecessors. Simply stated, "it's the apps stupid."

What does this have to do with Drupal? Drupal is a great content management system but what really makes it shine are the modules. They're sort of like the apps on an iPhone and they all can be found at http://drupal.org. Some of the great advantages to this include:

  • We don't have to go googling the entire Internet to find a decent module. Every module we use is housed right there on the project website.
  • All of the modules are listed in the same format which makes it easy to find documentation, search to see if anyone has the same issue with a module we're having and even contact the maintainer of the module with a question.
  • We're better able to judge if a module should be considered for our standard Drupal implementation. With more than 6,000 modules available, that's a really important need. The site shows how many sites have implemented each individual module. We can see how many bugs are in the issue queue and also tell if the module is actively maintained.
  • All of the revisions of that module are stored and we can see what changed in the code from version to version. We can even automagically pull the latest version using a utility called Drush.
  • Perhaps the best part? They're all free. The Drupal site requires that every module on there be governed by the GNU Public License (GPL). That basically means it's free for us to use, distribute and extend so long as we also keep it under the GPL. That is a HUGE difference from our days using Joomla where it seemed that every new module (called a "component" in Joomla jargon) wanted to get into us for more money and no ability to extend. In short, Drupal is a great software to update your own website but there are lots of great software packages for doing that (including Joomla). One thing that sets Drupal apart is the "app store" of modules maintained on the drupal.org website which makes it easy to find, evaluate and contribute to free modules that extend your website in literally thousand of ways. Thank you Dries.

Drupal Image Management gets help from the White House!

IMCE is an unfortunately cryptic name for a very cool tool that lets ordinary business users easily transfer photos from their computers to pages within their Drupal-based website. (Let's just call it the "image tool" for the rest of this explanation). The image tool shows you the folders on your server just like Windows Explorer or Mac's Finder and lets you add new files, resize images or insert one into your web page. The tool has been around for a few years and Lucidus installs it with each website we build.

The reason we're pretty excited of late with the image tool is that a number of improvements to the interface have been made, making it WAY more intuitive to use. For example, we used to insert an image into the web page by clicking the image tool button, selecting an image and clicking "Send to TinyMCE." What the heck is TinyMCE? (see the end of this post to discover what TinyMCE stands for). :-) Now, the button has been relabeled to "Insert File". Simple changes make a big difference.

One very cool tidbit of additional information is where the upgrades have largely come from. The Obama Administration decided to leverage Drupal for http://whitehouse.gov when they took office and their web development staff have been contributing back to the open source community ever since. One of the things they've helped a great deal with is the image tool. BTW, who knew the President had a web development staff?

p.s. For those who care, TinyMCE is the rich text editor we include with Drupal sites that makes editing web pages as easy as writing an email. It also suffers from an unfortunately cryptic name.

Kari and Andrea switching seats on the bus

Over the past several years, our clients have enjoyed interaction with Kari Fischer, our Support Manager. She's been a One-Woman-Army of Awesome when it comes to support management and training. As you get to know Kari, you come to realize she's all about learning new things...and getting in touch with her inner geek.

Almost two years ago, we added Andrea King to the team as our Accounting Clerk and Contracts Administrator. From the very beginning, Andrea did not hesitate to boldly go where few accounting clerks have gone before; she dove head first into learning the inner workings of development and support. It wasn't long before she began to show a deeper interest in helping out the geek team.

To allow Kari and Andrea to continue growing, it was obvious that we had to make a change. So, as of March 1, Kari passed the torch on to Andrea as our new Support Manager, and Kari will be moving into a Software Development position. We're wicked excited about the change and know that our customers will be too.

Important note for our customers: If you usually send emails directly to Kari for support, now is a good time to switch to support@lucidus.net.

Lucidus is going VIRTUAL!

I'm proud to announce that Lucidus is going virtual. What that means is that effective December 1st, we have empowered all of our employees to work "remotely", i.e. from home.

Since 2006, we've had staff working from home in different parts of the country. For the past six months, half our staff has worked from home full time and the rest have worked from home at least 20% of the time. It wasn't easy at first; we had to learn (and create) best practices for making everyone feel part of the team. We've worked hard to become experts in working as remote teams, leveraging the best of web technology to stay "connected" all day long.

Our folks have come to rave about the freedom (and focus) of working from home. The proof is in the results. We're turning out our best web projects ever, have reduced our resolution times on tickets to an all-time low and are enjoying our best year since 2006.

Of course our deep, long-standing relationship with our marketing partner, Communicators Group, will continue. We’ll have a presence with them at 9 Church Street in Keene for collaboration on projects, meetings, etc. I guess that makes us mostly virtual.

Scrolling ≠ Bad

Great post this week on Boxes and Arrows about the usability of scrolling. Change comes hard and many folks still have "Users don't like to scroll" in their heads. Milissa Tarquini squashes this in her post Blasting the Myth of the Fold

The biggest lesson to be learned here is that if you use visual cues (such as cut-off images and text) and compelling content, users will scroll to see all of it. The next great frontier in web page design has to be bottom of the page.

If you content is worth it users with scroll. The hard part is building/writing/gathering that content. In my experience is is often as hard and time consuming as is the technical work web development to display the content.

Pages