I hate Tim Sutton in the same way I hate Steven Singer. I hate Tim because he actually IMPLEMENTS some of the things on my list of ‘shit to tinker with when you get some free time’ instead of just TALKING about them. He and Pepijn Bruienne have been working on some code for Veewee that will allow you to automate the creation of OS X VMs in VMware Fusion. For those who are interested, you can check out the pull request containing this code and comment/help them out. For everyone else, read on and entertain the idea…

Prerequisites:

1. OS X
2. VMware Fusion
3. Git
4. Ruby 1.9.3 and rbenv
5. Mountain Lion (10.8) installer application

OS X

This walkthrough assumes that you’re running OS X on your development workstation. That’s what I use, and it’s the workflow I know.

VMware Fusion

If you’ve used Veewee before, odds are good that you MIGHT have used it to build baseboxes for Vagrant and/or Virtualbox. Veewee + Vagrant is a post for ANOTHER time, but in short it’s an awesome workflow for testing automation like Puppet on Linux or Windows. When I originally tried using Veewee to build OS X VMs, I had absentmindedly tried to do this using Virtualbox…which isn’t supported. As such, VMware Fusion is the only virtualization platform (as of this posting date) that is supported with this method. I’m using VMware Fusion 5 pro, but YMMV.

Git

Because the code that supports OS X only exists in Tim’s fork of Veewee (as of this posting date), we’re going to have to install/use Veewee from source. That introduces a bit more complexity, but hold my hand – we’ll get through this together. Git comes with the XCode Command Line Tools or can be installed with native packages.

Ruby 1.9.3 and rbenv

Veewee uses the gssapi gem which requires Ruby 1.9.1 or higher. The problem, though, is that the version of Ruby that comes with OS X is 1.8.7. There are typically two camps when it comes to getting a development version of Ruby on OS X: rbenv or RVM. I recommend rbenv because it doesn’t screw with your path and it’s a bit more lightweight, so that’s the path I’m going to take in this writeup. Octopress has instructions for getting rbenv on your machine – so make sure to check those out for this step. The instructions describe using rbenv to install Ruby version 1.9.3 – and that’s the version we’ll use here.

Mountain Lion installer application

This workflow supports creating a VM for Mountain Lion (10.8), but it SHOULD also work for Lion (10.7). The final piece of our whole puzzle is the installer application from the App Store. Get that application somehow and drop it in the /Applications directory (that’s where the App store puts it by default). We REALLY only need a single disk image from the installer, and we’ll get that next.

Some assembly required…

Now that you have all the pieces, let’s tie this FrankenVM up with some code and ugly bash, shall we? Splendid.

Copy out the installation disk image

In the last step, we downloaded the Install OS X Mountain Lion.app file from the App Store to /Applications, but we’ll want to extract the main installation disk image somewhere where we can work with it. I’m going to make a copy on the Desktop so we don’t screw up the main installer:

1

$ cp /Applications/Install\ OS\ X\ Mountain\ Lion.app/Contents/SharedSupport/InstallESD.dmg ~/Desktop

Beautiful. This should take a minute as it’s a sizeable file, but in the end you’ll have the installation disk image on your Desktop. For now this is fine, but we’ll be revisiting it later…

Clone Tim’s fork of Veewee

As if the writing of this blog post, the code you need is ONLY in Tim’s fork, so let’s pull that down to somewhere where we can work with it:

1
2
3
4
5
6

## I prefer to work with code out of a 'src' directory in my home directory
$ mkdir ~/src
$ cd ~/src

$ git clone http://github.com/timsutton/veewee
$ cd ~/src/veewee

Install Gems for veewee

We now have the Veewee source in ~/src/veewee, but we need to ensure all the Rubygems necessary to make Veewee work have been installed. We’re going to do this with Bundler. Let’s switch to Ruby 1.9.3 and get Bundler installed:

1
2
3

$ cd ~/src/veewee
$ rbenv local 1.9.3
$ gem install bundler

Next, let’s use bundler to install the rest of the gems we need to use Veewee:

1

$ bundle install

Once that command completes, Bundler will have installed all the necessary gems for Veewee and we can move on.

Define your new VM

Veewee has templates for most operatingsystems that can be used to spin up a ‘vanilla’ VM. Tim’s code provides a template called ‘OSX-10.8.2’ containing the necessary scaffolding for building a vanilla 10.8.2 VM. Let’s create a new VM project based on this template called ‘osx-vm’ with the following:

1
2

$ cd ~/src/veewee
$ bundle exec veewee fusion define 'osx-vm' 'OSX-10.8.2'

This will create definitions/osx-vm inside the Veewee directory with the template code from templates/OSX-10.8.2. We’re almost ready to let Veewee create our VM, but we need an installation ‘ISO’ first…

Prepare an ‘ISO’ for OS X

The prepare_veewee_iso.sh script in Veewee’s templates/OSX-10.8.2/prepare_veewee_iso directory provides awesome detail as to why we can’t use the vanilla InstallESD.dmg file to install 10.8 in our new VM. Feel free to open that file and read through the detail, or check out the details online for more information. Let’s use that script and prepare an installation ‘ISO’ for our new VM:

1
2
3

$ cd ~/src/veewee
$ mkdir iso
$ sudo templates/OSX-10.8.2/prepare_veewee_iso/prepare_veewee_iso.sh ~/Desktop/InstallESD.dmg iso/

You’ll need to be root to do this, but the script should handle everything necessary to prepare the ISO and drop it into the iso directory we created.

Define any additional post-installation tasks

Veewee supports post-installation tasks through the postinstall.sh script in the definitions/osx-vm folder. By default, this script will install VMware tools, setup Vagrant keys, download the XCode Command Line Tools, and install Puppet via Rubygems. Because this is all outlined in the postinstall.sh script, you’re free to modify this code or add your own steps. Here’s the current postinstall.sh script as of this posting:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

date > /etc/vagrant_box_build_time
OSX_VERS=$(sw_vers -productVersion | awk -F "." '{print $2}')
# Install VMware tools if we were built with VMware
if [ -e .vmfusion_version ]; then
  TMPMOUNT=`/usr/bin/mktemp -d /tmp/vmware-tools.XXXX`
  hdiutil attach darwin.iso -mountpoint "$TMPMOUNT"
  installer -pkg "$TMPMOUNT/Install VMware Tools.app/Contents/Resources/VMware Tools.pkg" -target /
  # This usually fails
  hdiutil detach "$TMPMOUNT"
  rm -rf "$TMPMOUNT"
fi

# Installing vagrant keys
mkdir /Users/vagrant/.ssh
chmod 700 /Users/vagrant/.ssh
curl -k 'https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub' > /Users/vagrant/.ssh/authorized_keys
chmod 600 /Users/vagrant/.ssh/authorized_keys
chown -R vagrant /Users/vagrant/.ssh

# Get Xcode CLI tools for Lion (at least to build Chef)
# https://devimages.apple.com.edgekey.net/downloads/xcode/simulators/index-3905972D-B609-49CE-8D06-51ADC78E07BC.dvtdownloadableindex
TOOLS=clitools.dmg
if [ "$OSX_VERS" -eq 7 ]; then
  DMGURL=http://devimages.apple.com/downloads/xcode/command_line_tools_for_xcode_os_x_lion_november_2012.dmg
elif [ "$OSX_VERS" -eq 8 ]; then
  DMGURL=http://devimages.apple.com/downloads/xcode/command_line_tools_for_xcode_os_x_mountain_lion_november_2012.dmg
fi
curl "$DMGURL" -o "$TOOLS"
TMPMOUNT=`/usr/bin/mktemp -d /tmp/clitools.XXXX`
hdiutil attach "$TOOLS" -mountpoint "$TMPMOUNT"
installer -pkg "$(find $TMPMOUNT -name '*.mpkg')" -target /
hdiutil detach "$TMPMOUNT"
rm -rf "$TMPMOUNT"
rm "$TOOLS"

# Get gems - we should really be installing rvm instead, since we can't even compile Chef or have a Ruby dev environment..
gem update --system
gem install puppet --no-ri --no-rdoc
# gem install chef --no-ri --no-rdoc
exit

Build the VM

Everything we’ve done has all been for this step. With the ISO built, the VM defined, and all necessary Gems installed, we can finally BUILD the vm with the following command:

1
2

$ cd ~/src/veewee
$ bundle exec veewee fusion build osx-vm

This process takes the longest – you should see VMware Fusion fire up, a new VM get created, and follow the process of OS X being installed into the VM. When it completes, your VM will have been created. Just like most Vagrant workflows, the resultant vm will have a vagrant user whose password is also vagrant. Feel free to login and ensure that everything looks good.

Now what?

At this point, I would snapshot the VM before making any changes. Because Virtualbox isn’t yet supported with Veewee for building OS X VMs (and Vagrant doesn’t currently include VMware Fusion support for its workflow), this VM isn’t going to fit into a Vagrant workflow…yet. What you have, though, is a vanilla OS X VM that can be built on-demand (or reverted to a snapshot) to test whatever configuration changes you need to make (all 11.36 Gb worth of it).

As you would expect, this is all pretty experimental for the moment. If you’re a Ruby developer who needs an OS X VM for testing purposes but have never managed OS X and it’s ‘quirky’ imaging process, this workflow is for you. For everyone else, it’s probably an academic proof-of-concept that’s more interesting from the point of view of “Look what you can do” versus “Let’s make this my primary testing workflow.”

Credit goes to Patrick Dubois for creating and managing the Veewee project, and to Tim Sutton and Pepijn Bruienne – like I mentioned before – for the work they’ve done on this. You can speak with them directly in the ##osx-server chatroom on Freenode, or by checking them out on Twitter.

Masochists are everywhere. Managing a system by hand appeals to a certain subset of the population, but I joined Puppet Labs because I was a fan of automating the shit out of menial tasks. I have no burning desire to handcraft an artisanal blog made out of organic bits. I tried web development at one point in my life and learned an important lesson:

I am not a web developer

What I DO want is to have a platform to share information and code I’ve accumulated along the way that:

1. Doesn’t look like hell
2. Doesn’t take forever to update
3. Doesn’t require me being online to write a post
4. Allows me to post code that’s syntactically highlighted and easy to copy/paste
5. Accepts Markdown
6. Fits into my DVCS workflow
7. Is free – because screw paying for A BLOG

Seriously, is this too much to ask?

Originally I waded the waters of Posterous but found that not only did it have awkward Markdown syntax, but staging of blog posts was cumbersome. Also, while there WAS gist integration, the code you posted looked like crap. That sucks because most of what I post has code attached.

Others have sold their soul for Wordpress/Drupal/Bumblefuck CMS (whether hosted or unhosted platforms), but it still felt like too much process and not enough action for my taste.

Github to the rescue…again

The answer, it turns out, was right in front of my face the whole time. Github Pages has always been available to host web content or Jekyll sites – I’m just late to the damn party.

Hosting static web content was out; like I said before, I don’t really want to ‘manage’ this thing. Jekyll, however, intrigued me. Jekyll is essentially a static site generator that allows you to throw Markdown at it and get static content in the end. There are even Jekyll bootstrap projects aimed at making it (even more) stupid simple. I tried plain vanilla Jekyll and realized that I didn’t really want/need that level of control (again, I’m not into web development). I pulled down Jekyll Bootstrap, but this time it felt a little TOO ‘template-y’. Next, I pulled down an awesome Jekyll template called Left by Zach Holman (seen, conveniently, in the picture on the left). I REALLY liked the look of Left, but was stuck with Jekyll’s code formatting that was…less than ideal. Jekyll is pluggable (and people have made plugins to fix this sort of thing), but I still didn’t have enough experience at that time to be able to deal with the plugins in an intelligent manner.

Octopress = Jekyll + Plugins + <3

During my plugin party, I discovered Octopress, which basically had EVERYTHING I wanted wrapped with a templated bow. I loved the way it rendered code, it supported things like fenced code blocks , and it seemed REALLY simple to update and deploy code. The thing I DIDN’T like was that NEARLY EVERY DAMN OCTOPRESS SITE LOOKS EXACTLY THE SAME! I know I said that I’m not a web developer and didn’t want to tweak styles a great bit, but damn – couldn’t I get a BIT of differentiation? That can be done, but you’re still editing styles (so some CSS is necessary – but not much). After evaluating all three options, I opted for Octopress.

Documentation? Who knew!?

Octopress.org has GREAT documentation for getting setup with an Octopress blog. They even have documentation for setting up Ruby 1.9.3 on rbenv or RVM (I recommend rbenv for reasons beyond this blog), so make sure to check it out if you’re unfamiliar with either. To not reinvent that documented wheel, make sure to check out that site to get Octopress setup (I recommend cloning all repositories to somewhere in your home directory like ~/src or ~/repos, but other than that their docs are solid). Normally I post the dirty technical details, but the point of this post is to outline WHY I made the decision I did and WHY it works for ME.

Pretty code is pretty

I’m not gonna lie – syntactical highlighting with the Solarized theme was pretty sexy. Let’s look at some Ruby:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

def create
    if resource[:value_type] == :boolean
      unless resource[:value].first.to_s =~ /(true|false)/i
        raise Puppet::Error, "Valid boolean values are 'true' or 'false', you specified '#{resource[:value].first}'"
      end
    end

    if File.file? resource[:path]
      plist = read_plist_file(resource[:path])
    else
      plist = OSX::NSMutableDictionary.alloc.init
    end

    case resource[:value_type]
    when :integer
      plist_value = Integer(resource[:value].first)
    when :boolean
      if resource[:value].to_s =~ /false/i
        plist_value = false
      else
        plist_value = true
      end
    when :hash
      plist_value = resource[:value].first
    else
      plist_value = resource[:value]
    end

    plist[resource[:key]] = plist_value

    write_plist_file(plist, resource[:path])
  end

How about some Puppet code?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

property_list_key { 'simple':
  ensure => present,
  path => '/tmp/com.puppetlabs.puppet',
  key    => 'simple',
  value  => 'value',
}

property_list_key { 'boolean':
  ensure     => present,
  path       => '/tmp/com.puppetlabs.puppet',
  key        => 'boolean',
  value      => false,
  value_type => 'boolean',
}

property_list_key { 'hashtest':
  ensure     => present,
  path       => '/tmp/com.puppetlabs.puppet',
  key        => 'hashtest',
  value      => { 'key' => 'value' },
  value_type => 'hash'
}

Doesn’t that look awesome?

For a blog with a ton of code, something like this is pretty damn important.

Markdown formatting

Markdown is becoming more prolific as a lightweight way to format text. If you’ve used Github to comment on code, then you’ve probably used markdown syntax at some point in your life. You’ll notice the recurring goal of being quick and precise, and markdown really ‘does it for me’.

Workflow++

It’s really liberating to write a blog post in (insert favorite text editor). Travelling as often as I do, I’m usually jotting down notes in vi because I’m in a plane or somewhere without internet access. Octopress not only lets you write offline, but it will let you generate and preview your site locally while being offline. That’s totally handy. My workflow typically looks like this:

• Generate a post

To start a new post, you can create a new file by hand, or use the provided Rakefile to scaffold it out for you (NOTE: to see all commands available to the Rakefile, you can run rake -T). Here’s how to scaffold with rake:

1
2

$ rake 'new_post["Title of my post"]'
$ vi source/_posts/2013-01-17-title-of-my-post.markdown

• Edit the post file
• Generate the site content

Remember that Octopress/Jekyll generates static content to be uploaded to your host. With every change, you’ll need to re-generate the content. Yeah, that kinda sucks, but that action has been abstracted away in the Rakefile with the following command:

1

$ rake generate

• Display and view the site

1

$ rake preview

The following command will serve up your page over Webrick in the terminal, just navigate to http://localhost:4000 in your browser to see a local copy of how your site will look once deployed. Once you’re done, just do a Control+c from your terminal to cancel the process.

• Edit/generate/preview until done
• Commit your code and deploy

Because everything is a series of flat-files, you can use git to keep it all under version control. Did you make a mistake? Revert to a previous commit. Deploying to your blog is similarly easy, as you’ll see in the next steps

Be a unique snowflake

I used this article to help change the color scheme of my blog, and checked out a list of other Octopress sites to steal a couple of other tweaks. That’s all I needed for customization, but if you need more knobs they’re there for you to twist.

Hosted by your pals at Github

Github Pages is a free service for any Github user (again free to join) and is an EXCELLENT FIT for an Octopress blog. As you would expect, Octopress has great documentation for enabling deployment to Github Pages. If you have your own custom domain, you can STILL use Github Pages (hint, the instructions are on that page too).

Fuck it. Ship it.

The act of updating a blog has GOT to be frictionless. Is this simple enough for you:

1

rake deploy

Yep. That’s all it takes to deploy code to your blog once you’ve setup Github Pages. Forget about logging in, wading through a GUI, cutting/pasting content, FTPing things up to a host, and any of that crap. I’ve done that. ‘That’ sucks.

Write your own damn blog

You now have no excuse to NOT share all the cool things you’re doing. Seriously, if you can EDIT A TEXT FILE then you can ‘maintain’ a blog.

JAMF’s NetSUS Appliance – Netboot in a Box

Today, JAMF released a new appliance VM based on Ubuntu 10.04 LTS (Lucid) that, for once, provides an ‘out of the box’ implementation of Netboot and Software Update Service WITHOUT requiring OS X hardware (based on Reposado) and other open-source technologies.

For those people who are relatively new to Linux, I thought I’d provide an easy walkthrough of how to get started using this VM on your Laptop. I’m going to demo this using VMware Fusion instead of VirtualBox mainly because I prefer how Fusion handles Networking. I’ve used many VMs, and Fusion always tends to be much more solid and stable for me. If you don’t have Fusion or don’t WANT to use Fusion, feel free to use VirtualBox.

Download and Convert the OVA File

The appliance can be downloaded directly from JAMF and comes as a VirtualBox OVA file. Rich Trouton has provided a great walkthrough on converting the OVA file to a VMX file, so I’ll link you to that post (since he did it so well).

Start up the VM

Once you have your VMX file, you can open it up with Fusion and import it into your Virtual Machine Library. There is one change we’ll want to make, so click on the VM and open its Settings panel (this is different from Fusion 3.0 and 4.0 – I’ll be describing the 4.0 method). In the Settings panel, you’ll want to select the Network Adapter icon and make sure the radio button next to “Share the Mac’s network connection (NAT)” is selected. This will create a local IP address so we can contact it directly from our Laptop. Once you’ve done that, start the VM and get to the login screen (you’ll need to click the Ok button through the message about logging into the NetSUS instance). Feel free to login with the username ‘shelluser’ and the password ‘shelluser’. You should see a Message of the Day splash with one of the sections being “IP address for eth0:”. Note the IP address, we’ll need it in the next section (if you don’t see this, just run ifconfig from the command line and look for the IP address for the eth0 adapter – it should be in the format of 192.168.x.x).

Prepare to Break Free!

It really sucks to work with VMs from WITHIN the VM window for various reasons (lack of Copy/Paste, loss of mouse, etc…), so we’re going to setup a host entry on your local laptop so we can SSH into the VM as we need. I’m going to give my VM a hostname of ‘netsus.puppetlabs.vm’ but you can name it whatever you want. Let’s first edit the /etc/hosts file on your laptop (NOT within the VM, this is ON your laptop) by running the following from the command line:

1

sudo vim /etc/hosts

Feel free to use pico/nano to edit /etc/hosts in case you don’t have any experience with vim. We want to add the following line in the /etc/hosts file:

1

192.168.217.154 netsus.puppetlabs.vm

NOTE SUBSTITUTE 192.168.217.154 WITH THE IP ADDRESS YOU GOT FROM THE PREVIOUS STEP! That’s the IP address that was assigned on MY laptop, and it will definitely be different from the IP address you got on YOUR computer. Also, you don’t HAVE to use ‘netsus.puppetlabs.vm’ – you can assign it any name you want. Save and close /etc/hosts and then let’s test connectivity from the command line by doing a ping:

1

ping 192.168.217.154

As long as we have connection, we’re fine (remember, substitute your IP Address for mine). If you DON’T have connectivity, make sure your VM is running and review your /etc/hosts file to make sure the changes were saved (also, check the IP address on your VM again).

Snapshot Time

The beauty of Fusion is that you can take snapshots and revert to them should you mess things up. Let’s take a snapshot now in case we screw things up in the following steps. I’m using VMware Fusion 4.0, so I take a snapshot by clicking on the VM window, clicking on the Virtual Machine menu at the top, and then down to Snapshots. From there, you can click the ‘Take Snapshot’ button and let it do its thing. When it’s done, hit the close button and return to the VM

Enable SSH and Login

By default, the VM doesn’t have the SSHD process running, so we can’t yet SSH in from our laptop. Enable this by doing the following from within the VM:

1

sudo apt-get install openssh-server

Hit “Y” when it asks if it should download and install. When it completes, switch back to your LAPTOP (i.e. NOT inside the VM), open up Terminal, and do the following:

1

ssh shelluser@netsus.puppetlabs.vm

Type yes to add the SSH key to your known_hosts file and use the password ‘shelluser’ to login. Tada! Now you can work directly from Terminal and enable copy/pasting directly from your Terminal window. You can always type exit to close the ssh connection when you’re done.

Install Puppet

So, it wouldn’t be one of my write-ups if I DIDN’T get Puppet installed. Actually, if you’ve only managed Macs, then you might not have seen the benefit of Puppet before. Now that we’re on Linux, you’ll definitely see the benefit of Puppet (especially if you’re not familiar with the service/package manipulation commands). Puppet can help abstract that for you, so we’re going to enable it.

The version of Puppet that’s in the main package repository is quite old (0.25 as opposed to the current 2.7.10 version we support). Let’s add Puppet Labs’ Apt Repository to the list of repositories queried so we can get a recent version of Puppet. Do that by opening up the /etc/apt/sources.list file for editing with the following:

1

sudo vim /etc/apt/sources.list

If it prompts you for your password, go ahead and enter it. Also, if you’re not familiar with vi or vim, I’ll try and help you with the basics. Arrow down to the line JUST above the line that begins with “deb http://” and press the i button on your keyboard to insert the following lines:

1
2

deb http://apt.puppetlabs.com/ubuntu lucid main
deb-src http://apt.puppetlabs.com/ubuntu lucid main

When you’re done, hit the escape key and press ZZ (hold shift and hit Z twice, you want two capital-Z’s) to save and close the file. This will add Puppet Labs’ repository to the list of repos queried.

Next, we’ll need to add Puppet Labs’ GPG key to the system so we can validate the packages that are downloaded. From the terminal, execute the following:

1

 gpg --recv-key 4BD6EC30

The first time you run it, it will bother you about needing to create the keyserver file and won’t run correctly. Let’s execute it correctly to receive the key:

1

 gpg --recv-key 4BD6EC30

Great, we should have the key, now we want to load it. Do that with this command:

1

gpg -a --export 4BD6EC30 | sudo apt-key add -

Finally, let’s update apt so that it knows about the repository. Do that with this command:

1

sudo apt-get update

When it finishes running, you can finally install Puppet with the following command:

1

sudo apt-get install puppet

Type “Y” when it asks you if it should download and install all the dependencies. That’s it! Puppet should be installed! Run the following to see what version we’ve installed:

1

puppet --version

As of this writing, 2.7.10 is the current version. As long as you don’t have version 0.25 you should be good!

Snapshots away

It would be wise to take another snapshot at this point. Don’t worry, I’ll wait.

Inspect the System

Puppet lets us see what’s been installed on the VM. Take a look at all the packages on the system by doing the following:

1

sudo puppet resource package

Type your password when prompted and Puppet will return you with a list of all packages on the system and the versions that are running. Similarly, you can see what services are running with the following:

1

sudo puppet resource service

Notice that sshd is running and other services like netatalk (that enables AFP on Linux) should also be running. Sweet. We’ll come back to Puppet later.

Login to the Web GUI

If your VM is running, you should be able to open a web browser on your Laptop and navigate to the following address:

1

https://netsus.puppetlabs.vm

Note the httpS as we’ll need to use SSL to get into the Web GUI. When it asks if you want to accept the self-signed cert, make sure to choose Yes. You can login with the user ‘webadmin’ and the password ‘webadmin’

Once you’re logged in, feel free to read through JAMF’s instructions for enabling their services. They do a much better job than I about walking you through that.

Access to the Raw Files

Back in the Terminal window, you can access all the PHP/Shell magic that comprises the Web GUI in the /var/www/webadmin directory. You can poke around through all the files (and use sudo to open some of the key bits) at your leisure.

Note that /etc/dhcpd.conf is the main configuration file for the DHCPD service and it’s got some special bits to accommodate Apple Hardware.

Also, /var/appliance has some magic (and the /var/appliance/configurefornetboot script has some sed statements for fixing a stock dhcpd.conf file and enabling those Apple Hardware bits).

More Later?

That’s it for now – I just wanted to get something online to show people how to setup and poke around in the VM. Later I can show you how to manipulate and manage the VM with Puppet, which is pretty fun. Your VM should get you started and let you play around locally, but if you want to test out Netboot and the like then you’ll need an externally-accessible IP address (as apposed to the 192.168.x.x IP, which is only accessible from your Laptop). You can change the settings in VMware Fusion to enable that (like we did in step 2).

Hope this helped you out!

As a former Mac Sysadmin, I frequently felt like I had one toe in the world in which Linux Sysadmins frequently found themselves, but was surrounded by a culture of GUI-Clickery that either feared, scorned, or flat-out avoided tools that were command-line based (or, as many will point out, used the GUI (Graphic User Interface) tools because ‘they worked’ and that was all they needed). This sucked because Linux Sysadmins have some tools that, as a current colleague would say, ‘are kind-of awesome.’

One of those tools is Git, which is a Distributed Version Control System (or, DVCS). Git, in a TOTAL nutshell, is a tool that allows you to create multiple ‘what-if scenarios’ with very little cost and the ability to save yourself from…well, yourself.

Yes – You’re Going to Use the Command Line

Like I mentioned before, the GUI is ingrained in the culture of the Mac. It wasn’t until OS X that Mac Sysadmins finally had a proper command-line (which I will refer to as the CLI, which is short for Command Line Interface. If you’re not familiar with that terminology, I’m speaking about the interface you get when you run /Applications/Utilities/Terminal.app), so we’re a bit ‘new’ to the whole idea. I can help you if you fear or avoid the CLI; there’s always room for learning and I’ll try to hold your hand as much as possible. For those that downright refuse to use the CLI…well, I’m sorry?

What IS Git?

Git is relatively new in the DVCS world (being initially released in 2005). Developers have been using DVCS for MANY years with tools like CVS, Subversion, Git, Mercurial, and others. For a team of developers who ALL need to make contributions to a SINGLE project or code base, it’s vital that they have a way to modify a file without breaking something that someone else is doing. That’s what DVCS tools do – allow you to take something as large as a code base like OS X 10.7 (or something as small as a directory with a couple of files) and make changes without affecting everyone else who’s working on the SAME code base (or files in the directory).

So why am I choosing to talk about Git instead of, say, Subversion? That’s easy: lightweight branching and merging. Git allows you to easily create a ‘topic branch’ (or, using the metaphor above, a ‘what-if scenario’), make changes, and see if your changes are favorable or if they downright stink. If your changes are desirable, you can then ‘merge’ your topic branch back in with your master branch (or, the original state the files were in when you created the topic branch in the first place). If the changes don’t work well, you can just delete the topic branch and ‘no-harm, no-foul’; your original files remain unchanged and in the pristine condition that they existed when you first created the topic branch. True, Subversion could do the same thing, but Git allows MANY people to do the same thing and then EASILY merge in EVERYONE’S changes without having to fight Subversion (just trust me on this if you’ve never used Subversion).

And why am I choosing to talk about Git instead of Mercurial? Easy – Git is what I learned :) As I understand it Mercurial functions similarly, and I truly don’t have the experience with Mercurial to tell you the differences. I’m talking about Git because Git is what I use and Git is what my company uses.

So what does this have to do with me as a Mac Sysadmin?

Good question. Mac Sysadmins are more frequently finding themselves having to use the CLI to tune their clients and servers. Also, many Mac Sysadmins are finding that certain CLI tools are working better for their teams (in terms of simplicity, responsibility, and visibility). As they dip their toes into this CLI world, they’re finding that they want the same features they had in the GUI: the ability to do a ‘save-as’, to create ‘versions’, and to have the ability to have a rollback function similar to Time Machine. Git gives you this through the CLI (or, yes, even through the GUI as I will mention later in the article).

If you’re ready to take that step then follow me…

A note on Git and Github

You might have heard of Github. Github is a free (they have a free tier and plenty of subscription tiers of service) service that allows you to ‘push’ your git repositories up to them for storage in ‘the cloud’. Github is basically centralized-storage for your git repositories and is NOT NECESSARY for you to use git as a tool. In reality, you CAN use Github or setup something like Gitolite on a local server in your environment to get most of the benefits of Github without having to push your repositories to Github’s servers.

The bottom line: Github and git (the tool) are two different things. Github didn’t create git, and git doesn’t need Github to work properly. Keep that in mind as you read through.

Installing Git

Git can be installed in a variety of ways. First and foremost, you can download the latest package from Git’s Googlecode Page. You can also choose to install git through Macports or Homebrew as long as you have them installed (both of which require the Developer Tools…but so does The Luggage, the tool we’ll be using later). Simply download the package from Git’s Googlecode Page, install it, and you should be good to go!

If you’re using Macports, make sure the Developer Tools are installed, Macports is installed, and then do the following from the command line (PLEASE NOTE: The dollar sign is a prompt – you SHOULDN’T type it. Lines BEGINNING with a dollar sign, or a prompt, should be typed by you. Lines that DON’T begin with a prompt represent the output you receive from typing the command. If you don’t see a line that DOESN’T begin with a prompt, then I’m not listing the output. Got it? Good!):

1

$ sudo port install git-core

If you’re using Homebrew, make sure the Developer Tools are installed, Homebrew is installed, and then do the following:

1

$ sudo brew install git

To check to see where and what version of git is installed, execute the following commands :

1
2

$ which git
$ git --version

You should receive the path to where git is installed after you run the first command, and you should receive the version of git that’s currently installed on your system. Any version of Git greater-than or equal to 1.7 is probably ideal, though most of the commands we’ll be using will probably work just fine with previous versions.

Your first Git repo

I find it’s easier to understand Git if I describe an actual scenario instead of talking in terms of ‘git can do…’. I’m going to talk about using git with The Luggage. I’ve written about The Luggage in several articles on this site, so feel free to familiarize yourself with it if you haven’t used the tool before. As a quick review, The Luggage is a tool that lets you create plain text Makefiles that can be used to build packages to install files, deploy scripts, and basically distribute ‘things’ to the users in your organization. Auditing HOW a package is made can be incredibly difficult for a team of sysadmins, but The Luggage helps you with this by using plain text Makefiles. Git will give you the ability to track WHO made changes, WHEN they were made, and ROLLBACK the changes if you notice something undesirable happening. Git and The Luggage are a perfect combo if you’re a sysadmin.

Incidentally, The Luggage exists as a Github repository, so we have to CLONE it (or download it) from Github first before we can start playing with it. I recommend creating a directory in your home directory (/Users/-username-) and cloning your repositories there – that way you retain ownership of the files and can work on them at will. I put all my git repositories in ~/src, but if you’re using something like Mobile Home Directories or even Network Home Directories, then you’ll want to make sure that you’re not syncing these directories back to some central location and burning up your quota in some fashion. Let’s create our repositories directory and clone The Luggage source code with these steps:

1
2
3

$ mkdir -p ~/src
$ cd ~/src
$ git clone git://github.com/unixorn/luggage.git

Doing this will create the ~/src directory where we’re putting all our repositories and download The Luggage source code to ~/src/luggage. As you might be able to infer, git clone will clone an existing git repository from its location on another server (we call that the remote location, or just remote for short) down to a local directory. This is usually how many sysadmins first encounter git – by needing to download something from Github or a remote server. Let’s look and see what git knows about this repo by doing the following:

1
2
3

$ git remote -v
origin  git://github.com/unixorn/luggage.git (fetch)
origin  git://github.com/unixorn/luggage.git (push)

This command will list all the remote repositories that we’re tracking. By default, you will get a remote by the name of ‘origin’ that lists the location where you originally cloned the repository (in our case, this is Github).

Why is it listed twice? Well, with git, you can pull down, or fetch, changes from a remote repository as well as push up changes from your LOCAL repository to the remote repository. Frequently these paths will be the same, but in our case these links are to the READ-ONLY version of this repository, so trying to push changes will always fail because we don’t have access. We’ll talk later about how you can push up changes, but for now let’s create our OWN repository instead of working with someone ELSE’S.

Creating your OWN git repo

The Luggage has a directory called Examples, but it’s absent of any actual examples. We want to create our own directory where we can store our Luggage Examples. For the purposes of this demo, I’m going to create a directory called luggage_examples that will contain our…well… Luggage examples:

1
2

$ cd ~/src
$ mkdir luggage_examples

Now that we have a directory, we need to make sure that git is tracking our changes so that we can create those ‘what if’ scenarios and save our work.

1
2
3

$ cd ~/src/luggage_examples
$ git init
Initialized empty Git repository in /Users/gary/src/luggage_examples/.git/

Git created a new empty repository at ~/src/luggage_examples and created a .git directory inside luggage_examples. This .git directory is where git stores all its dynamic data that’s pertinent to your repository, so MAKE SURE not to delete it or modify any files in there (without knowing what you’re doing).

That’s it! We’ve created a repository! Now let’s actually see how git works locally on your machine.

Working with Files

As I mentioned before, you don’t NEED Github to use git. Git functions as a mechanism to ‘version’ your files so you can rollback to a previous version or create our ‘what-if’ scenarios and then accept (merge) or deny (destroy) them depending on our needs. You can easily create a git repository on your local computer ONLY, but of course you lose the ability to restore your work if your hard drive malfunctions. This is what Github gives us, and we’ll look at that functionality later.

The example I’m going to use is based on the previous blog post I wrote on Using the Google Mac Sysops Crankd Code Let’s create a directory for this example and create a starter Makefile:

1
2
3

$ mkdir -p ~/src/luggage_examples/crankd_google
$ cd ~/src/luggage_examples/crankd_google
$ vim Makefile

I use vi or vim to edit files from the Terminal. Feel free to use any text editor you like (nano, emacs, Textmate, TextWrangler, etc…), but CREATE a file called ‘Makefile’ (note the capital M) and put it in ~/src/luggage_examples/crankd_google. Here’s what I’ll put into this file:

1
2
3
4
5
6
7
8
9

    # Title:       Crankd-Google Example
    # Author:      Gary Larizza
    # Description: Something so Marnin doesn't kill me w/ postinstalls :)

    include /Users/gary/src/luggage/luggage.make

    TITLE=Crankd-Google
    REVERSE_DOMAIN=com.googlecode
    PAYLOAD=pack-crankd

Great, so we have our skeleton of a file! The file is saved in our folder, but we haven’t actually told git to track the file (and so git is AWARE that the file EXISTS, but it isn’t tracking its changes…yet). This is a good time to explain the three states in which files can exist in a git repository.

The Three Stages of Files: Working Tree, Index, and Repository

Git has three ‘locations’ by which a file can exist (even though it technically exists, in our case, in ~/src/luggage_examples/crankd_google). These three locations are called the working tree, the index, and the repository

The Working Tree

When you put a file in a directory that’s being managed by git, but you haven’t told git to track the files changes, then this file is said to be in the working tree. Our file we just created, Makefile, exists in the working tree currently. Git knows that the file EXISTS, but it hasn’t saved the file’s ‘state’ so we can rollback the file if need be.

The Index

The index is also known as the cache or the staging area, and it’s a place to put files before you’re ready to make a ‘commit’ (which is something of a ‘milestone’ or a version). A commit is a representation of a point in time in the history of your git repository. Think of it as a snapshot of your repository – ‘this is how our files looked at this specific time’. With commits, you can always rollback to a specific ‘commit’ (or even advance to a future commit if you had rolled-back to an earlier commit). In order to rollback to a specific version of a file, though, you need to make a commit, and in order to make a commit you need to tell git what FILES will comprise our commit. As I mentioned, a commit is just a point in time…but it doesn’t HAVE to be a point in time for a SINGLE file. A commit can represent a specific point in time for a NUMBER of files all at once. To make life easier for everyone, though, if you make a commit comprised of multiple files, then you want to make sure that the changes to the multiple files all represent a single functional change (for example, say we wanted to change a file that existed in our crankd_google folder that was being copied into a package, but we ALSO had to change the Makefile to change WHERE this file was to be put in the final package. These two changes are related and comprise a single functional change, so you would want to make a commit with the changes to both files at once.).

Our Makefile exists in the working tree currently, but we want to put it in the index because we’re ready to make a commit (no, our Makefile functionally doesn’t DO anything yet, but we want to make a commit so we can rollback to this point in time should we mess up anything). Understand that the locations of the ‘working tree’ and ‘index’ are relative to git ONLY. Yes, this file lives in ~/src/luggage_examples, and it will CONTINUE to exist in that directory (as far as the OS X filesystem is concerned), but to git it currently exists in the working tree. How do we know that the file is in the working tree and NOT in the index? Use the git status command to show you that info:

1
2
3
4
5
6
7
8
9
10
11

$ git status

# On branch master
#
# Initial commit
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
# Makefile
nothing added to commit but untracked files present (use "git add" to track)

Anything listed under ‘Untracked files:’ is in the working tree. You can add a file to the index by doing the following:

1
2
3
4
5
6
7
8
9
10
11
12

$ git add Makefile
$ git status

# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached ..." to unstage)
#
# new file:   Makefile
#

Great, the file is in the index! Notice that the file is listed under ‘Changes to be committed:’ meaning that it’s in the index but NOT YET committed (or, ‘in the repository’).

The Repository

The Repository is where your files exist once they’ve been made part of a commit. As I mentioned before, files must be committed before we can roll back their states (and their states can only be rolled back to individual commits). You can never rollback a file’s state UNLESS it’s tied to a commit. If you made a commit to a file last week and haven’t made another since, but wanted to revert a file to how it existed three days ago, you would be out of luck. Git ONLY rolls back to specific commits (this is why it’s important to make frequent commits). Before we make a commit, however, we need to make sure to identify ourself to git (if you haven’t previously set this up). The user.email and user.name settings are the most important settings as they will be used to trace back what code you committed. If you’re not sure of how git is setup, use the following command to list the git config settings:

1
2
3

$ git config -l
user.name=Gary Larizza
user.email=gary@puppetlabs.com

Should the user.name and user.email settings NOT be configured, you can configure them with the following commands:

1
2

$ git config user.name 'Your name'
$ git config user.email your@email.com

Now that you’ve been identified (so we can lay the blame on you in the future), let’s actually commit some code. The git commit command allows git to commit all staged files to the repository in one fell swoop.

Every commit has three things: the list of files and CONTENT of the files that should be committed, a SHA1 hash value representing the commit itself (essentially, a unique identifier for the commit so it can be tracked), and the commit message describing the what and why of the commit.

Creating a commit message seems insignificant but should NOT be taken lightly. Great commit messages usually consist of a title line containing no more than 60 characters followed by a paragraph describing the changes. Because many people may potentially need to trace your code and commits, it’s important to list how your code functioned previously, the reasoning behind the change, and what specifically was changed. Don’t skimp with your commit messages! Always assume that the next person to maintain your code owns many dangerous weapons and knows where you sleep, so do that person a favor and be as kind to them (in the form of great commit messages) as possible!

Let’s begin the process of making a commit by issuing the following command:

1

git commit

Executing git commit will drop you into the default text editor (which is vim by default – you can change this by running git config core.editor /path/to/your/editor before doing git commit) and show the following:

1
2
3
4
5
6
7
8
9
10
11

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached ..." to unstage)
#
# new file:   Makefile
#

Note that everything that begins with a hash ( # ) is solely for YOUR benefit and will be stripped out of the commit message, so don’t worry about it being included. Here’s an example of a commit message I would use in this instance:

1
2
3
4
5

Initial commit for crankd_google code

Previously we didn't have an example for creating a package of the 
Google Mac Sysops code.  This commit adds the shell for the Makefile and
includes the luggage.make file in ~/src/luggage.

Great! When you save and close your editor, you should see something that looks like this:

1
2
3

[master (root-commit) f423ac6] Initial commit for crankd_google code
 1 files changed, 9 insertions(+), 0 deletions(-)
 create mode 100644 Makefile

This tells us that we made a commit on the master branch with a SHA1 hash ID beginning in f423ac6 being attributed to this commit. In most cases, the first 7 digits of the hash are all you need for uniqueness when you refer to this commit, and that’s usually all that git will provide you in THIS instance. YOUR SHA1 hash won’t match the hash provided above (hence a unique ID), so don’t panic if it doesn’t match up for you. And that’s it! We’ve done it! Now what?

Examining your Work

Git tracks the ENTIRE history of a repository, and thus will track any and every commit made. To display this history, run the following command:

1
2
3
4
5
6
7
8
9
10

$ git log
commit f423ac6e9548d93f53b15d1154723c5abcd69738
Author: Gary Larizza 
Date:   Sun Jan 22 23:27:31 2012 -0500

    Initial commit for crankd_google code

    Previously we didn't have an example for creating a package of the
    Google Mac Sysops code.  This commit adds the shell for the Makefile and
    includes the luggage.make file in ~/src/luggage.

By default, git log will show ALL information about ALL of your commits (including the full commit message). There’s a LARGE number of arguments that git log will accept that allows you to format the output for ANY purpose. One of the common lists of arguments that I like to employ is the following (long) command:

1
2

$ git log --pretty=format:'%C(yellow)%h%C(reset) %s %C(cyan)%cr%C(reset) %C(blue)%an%C(reset) %C(green)%d%C(reset)' --graph --date-order
* f423ac6 Initial commit for crankd_google code 10 minutes ago Gary Larizza  (HEAD, master)

This format both colorizes the output for readability and also shows a nice graph of commits as you do things like branching and merging. Here’s an example of this command (with line length chopped for brevity) being run on a repository with many commits:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

* 72a2fe0 (#5445) Create /var/lib/puppet in OS X Package 26 hours ago Gary Larizza  (HEAD, gary/bug/2.7.x/5445_crea
*   c6667c5 Merge pull request #324 from glarizza/bug/2.7.x/2273_launchd_dashw 12 days ago Daniel Pittman  (origin/
|\  
* \   bc6c642 Merge pull request #318 from glarizza/bug/2.7.x/11202_apple_rake 12 days ago Michael Stahnke 
|\ \  
| * | 6c14a28 Build a Rake task for building Apple Packages 12 days ago Gary Larizza  (gary/bug/2.7.x/11202_apple_r
* | |   5ec5657 Merge pull request #127 from adrienthebo/ticket/2.7.x/8341-prevent_duplicate_loading_of_facts 12 da
|\ \ \  
* \ \ \   450da6a Merge pull request #92 from duritong/tickets/2.7.x/1886 12 days ago Daniel Pittman 
|\ \ \ \  
| | | | * d5bef5e (#2773) Use launchctl load -w in launchd provider 12 days ago Gary Larizza  (gary/bug/2.7.x/2273_
* | | | |   1cb2b6a Merge branch 'ticket/2.7.x/11714-envpuppet' into 2.7.x 12 days ago Josh Cooper 
|\ \ \ \ \  
| |_|_|_|/  
|/| | | |   
| * | | | 5accc69 (#11714) Use **%~dp0** to resolve bat file's install directory 12 days ago Josh Cooper 
* | | | |   fe79537 Merge pull request #323 from glarizza/bug/2.7.x/fix_launchd_spec 12 days ago Jeff McCune 
|\ \ \ \ \  
| * | | | | c865a80 Clean up launchd spec tests 12 days ago Gary Larizza  (gary/bug/2.7.x/fix_launchd_spec, bug/2.7
|/ / / / /

As you can see, git log is powerful and handy to list out what’s happened to your repository. You’ll also notice that it lists WHO made the changes, which is handy if you have MANY people contributing to the same code base.

That’s it for now

Hey , what gives?! We haven’t really DONE anything! Git has a tremendous amount of power, and it’s important to know WHAT’S happening before I can progress and show you the power of git. This article is ALREADY quite long, so I’ve broken it up for ‘easier’ reading. Rest assured, in parts 2 and beyond we will modify files, create topic branches, merge topic branches, use Github (for backup AND to allow others to contribute to your project), and much more. Stay tuned for the following article(s) and as always, put any questions in the comments or email me directly!

On December 21st, the MacOps team at Google released a Googlecode repository containing a couple of scripts that they use for managing Macs. The scripts provided interface with crankd (available in the Pymacadmin code suite) and Facter, which are a couple of tools with which you MIGHT not be familiar.

I saw a couple of questions on the ##osx-server IRC channel about how these scripts work, and I decided to do a quick writeup on how you would implement the scripts in your environment. This is meant to be a walkthrough to getting their scripts up-and-running, but please READ EVERYTHING FIRST before implementing.

Prepare Crankd:

Crankd is a vital part of the Google MacOps puzzle. Crankd essentially executes Python code in response to System/Network events on your system. I have MUCH MORE information on crankd in a guide I posted a while back, but I’m going to give you the ‘quick version’ for those looking to get started quickly:

• Clone or download the Pymacadmin Github repo. If you have git installed you can accomplish that by doing the following from the command line:

1

git clone https://github.com/acdha/pymacadmin.git

If you DON’T have git on your machine just visit https://github.com/acdha/pymacadmin, click on the Downloads tab, and download the .tar.gz or .zip version of the repo. Next, double-click on the resultant file to expand it into a folder.

• Change to the pymacadmin folder and run the install-crankd.sh script which will install crankd.py into /usr/local/sbin and create /Library/Application Support/crankd Once you’ve changed to the pymacadmin source directory, run the following from the command line:

1

sudo ./install-crankd.sh

OR…

If you already have a Puppet environment configured, I’ve written a module that will install crankd for you. The upside is that this module should work out of the box, but the downside is that I’ve copied the source to the module’s files/ directory. Since Pymacadmin hasn’t been changed since 2009, though, this shouldn’t pose a big issue. Simply copy this module to your modulepath and declare it with the following line in your node declarations:

1

include crankd

Download the Google-macops source code

You can download the source by using subversion with the following command:

1

svn checkout http://google-macops.googlecode.com/svn/trunk/ google-macops-read-only

This will create a folder called ‘google-macops-read-only’ in the current directory.

Copy the Python crankd files

Change to the directory where you downloaded the Google-macops source code and you will notice two folders: crankd and facter. Change to the crankd directory and copy all of the python files to /Library/Application Support/crankd with the following command from the command line:

1

sudo cp *.py /Library/Application\ Support/crankd/

This copies two files into /Library/Application Support/crankd: ApplicationUsage.py and NSWorkspaceHandler.py. The ApplicationUsage.py file contains most of the magic. In a nutshell, it will create a SQLite database file called /var/db/application_usage.sqlite containing information on applications that are launched and quit (we will inspect this information later), and it will update the database any time an application is (you guessed it) launched or quit. Crankd provides the mechanism by which the code is run any time an application is launched/quit and these Python files actually update the database with the pertinent information.

Copy the provided crankd plist to /Library/Preferences

Changing to the crankd directory in the google-macops-read-only directory you should find a file called ‘com.googlecode.pymacadmin.crankd.plist’ which is a sample plist for utilizing Google’s code that tracks application usage via crankd. It’s up to you to merge these Python methods into your existing crankd setup, or utilize the plist they’ve provided. The plist essentially says that any time an application is LAUNCHED, you should call the ‘OnApplicationLaunch’ method in the NSWorkspaceHandler class, and any time an application is QUIT, you should call the ‘OnApplicationQuit’ method in the NSWorkspaceHandler class. Let’s accept these conditions and copy their plist into /Library/Preferences (assuming that we don’t have an existing crankd setup) with the following command:

1

cp com.googlecode.pymacadmin.crankd.plist /Library/Preferences

Test out crankd from the command line

With crankd.py installed, the Python methods copied to /Library/Application Support/crankd, and the com.googlecode.pymacadmin.crankd.plist plist copied to /Library/Preferences, we can finally test out the code to see how it works. Start crankd with the following command:

1

sudo /usr/local/sbin/crankd.py --config=/Library/Preferences/com.googlecode.pymacadmin.crankd.plist

After entering your password, you should see the following in your terminal:

1
2

INFO: Loading configuration from /Library/Preferences/com.googlecode.pymacadmin.crankd.plist
INFO: Listening for these NSWorkspace notifications:  NSWorkspaceWillLaunchApplicationNotification, NSWorkspaceDidTerminateApplicationNotification

Now, let’s launch a couple of applications and see how we track their usage through the sqlite database. Do this by starting Safari and TextEdit, and then closing them (note that you MUST start them fresh – if they’re already opened make sure to close them, then open them, and THEN close them again). You should see something that looks like the following in the terminal:

1
2
3
4

INFO: Application Launched: bundle_id: com.apple.Safari version: 6534.52.7 path: /Applications/Safari.app
INFO: Application Launched: bundle_id: com.apple.TextEdit version: 264 path: /Applications/TextEdit.app
INFO: Application Quit: bundle_id: com.apple.TextEdit version: 264 path: /Applications/TextEdit.app
INFO: Application Quit: bundle_id: com.apple.Safari version: 6534.52.7 path: /Applications/Safari.app

Finally, you can quit crankd from the terminal by holding the control button down and pressing the ‘c’ key on your keyboard.

Inspect the application_usage.sqlite database

If you’re proficient in SQL, feel free to browse through the contents of the database. To keep things simple, let’s download an app called SQLite Database Browser from the following link. Launch SQLite Database Browser, go to File –> Open Database, hold the Shift and Command buttons down on your keyboard and then press the ‘g’ button to bring up the ‘Go to the folder:’ dialog box, type in /var/db/application_usage.sqlite in the box and press return, and then finally click the open button to open the database. If you click the ‘Browse Data’ tab near the top of the window, you should be able to see the raw data in the database. It should list the last time (in epoch time), bundle id, version, path, and number of times that the application has been launched (note that you can use an online utility to convert from epoch time to regular time).

Inspect the Facter fact

If you’re not familiar with Facter, you should click on the link and check it out. Facter is an awesome binary designed to gather information about your system and report it back in the form of a ‘fact’, which is essentially a key => value pair (for example, the ‘operatingsystem’ fact will return a value of ‘Darwin’ on Macs running OS X or ‘RedHat’ on machines running RedHat Linux). Google has provided a custom Facter fact that will report a fact NAME that corresponds with an application you’ve launched on your system (such as ‘app_lastrun_com.apple.safari’ or ‘app_lastrun_com.apple.textedit’) and a VALUE with the epoch time or the last time that application was run. The output will ultimately look like this:

1
2

app_lastrun_com.apple.safari => 1325203354
app_lastrun_com.apple.textedit => 1325203357

NOTE: This Facter fact requires the SQLite database that is created by the crankd code from above, so make sure you’ve followed the previous steps or you might have problems getting fact values.

We’ll need to install Facter. You can download a copy of Facter from Puppet Labs directly. Simply download the DMG, expand it, and run the package installer inside.

To test out the fact, we will need the full path to the folder containing the google-macops code we downloaded in step two above. I downloaded the source to a path of /Users/gary/Desktop/google-macops-read-only and so that’s the path I’m going to use in the next step.

Facter is designed to work with Puppet, but you can also run it independently. Facter is ALSO designed to be plug-able with custom facts, and that’s what Google has shipped us. In order to TEST these custom facts, we need to set the RUBYLIB environment variable (as Facter will look for a custom facts inside a ‘facter’ directory that’s located inside the path returned by the RUBYLIB environment variable). RUBYLIB should be set to the path of the downloaded google-macops source, so in my case I would run the following from the command line:

1

export RUBYLIB=/Users/gary/Desktop/google-macops-read-only

NOTE: make sure RUBYLIB is in capital letters. If you downloaded the google-macops source to a different path, substitute your path for ‘/Users/gary/Desktop/google-macops-read-only’.

Next, we need to make sure we have the ‘sqlite3’ ruby library installed on our system as the custom fact requires this library for correct operation. One way to install sqlite3 is by using Rubygems with the following command from the command line:

1

sudo gem install sqlite3

NOTE: If you have troubles using Rubygems to install sqlite3, you can either build it from source (which is beyond the scope of this article), or you can use Macports (if you have it installed) to install it with the following command:

1

sudo port install sqlite3

The last step is to actually run Facter to test out the fact. You may have a problem TESTING this fact if you installed sqlite3 via Rubygems – the reason is because the custom fact doesn’t actually load Rubygems before it tries to load sqlite3 (this isn’t a problem if you use the fact with Puppet as Puppet loads Rubygems. It’s ALSO not a problem if you’re running version 1.9 or greater of ruby, as you no longer need to load Rubygems BEFORE trying to include a ruby library with ruby 1.9 or greater). To remedy this problem, let’s add a single line to the custom fact. Notice the line that says the following:

1

require 'sqlite3'

This line is where the custom fact loads the sqlite3 library. Now, the line BEFORE this line we need to add a single line:

1

require 'rubygems'

This line will load Rubygems so that it knows where to access the sqlite3 library. Note that this line is unnecessary if you use this custom fact with Puppet as, like I mentioned before, Puppet already loads Rubygems. For the purpose of our testing, though, we will need this line.

Finally, let’s test the fact by running Facter from the command line:

1

facter

You SHOULD get a list of ALL your Facter facts, but up at the top you should have a couple of facts that begin with ‘app_lastrun_’ and then end with the bundle_id of the application that was launched. These are the custom facts that have been created.

You’ll note that the time reported is in epoch time. If you’d prefer a date/time string that’s more legible, you’ll need to modify the apps.rb file in the facter directory. Line 39 in the file (or, line 40 if you added the require ‘rubygems’ line previously) should say ‘appdate’ but change it to say the following:

1

Time.at(appdate.to_i)

Running facter again (like above) will give you a more legible date:

1
2

app_lastrun_com.apple.safari => Thu Dec 29 19:02:34 -0500 2011
app_lastrun_com.apple.textedit => Thu Dec 29 19:02:37 -0500 2011

???

Profit!

Now that you know how the code works, you’re free to modify, extend, and improve it as you see fit! The one thing that’s missing here is a LaunchDaemon that keeps crankd running in the background. Since you’re maintaining your environment, I’ll leave that up to you (though I walk you through this step in ‘Step 5. A LaunchDaemon to keep crankd running’ in my previous crankd writeup). Feel free to check out my other writeups on Facter, Puppet, and Crankd to see how I used to manage hundreds of Macs in an educational environment!

As of version 2.6, Puppet introduced a feature called “Run Stages” that will allow you to better control the order that Puppet configures your system. The problem with Run Stages (as of right now) is that there’s not that much good Documentation out there. Hopefully this document helps someone else out there who wants to setup Run Stages in their own environment.

A NOTE ON SYNTAX!!!

One of THE MOST DIFFICULT concepts to understand for puppet newbies is that these two things are identical:

1
2

include foo
class { 'foo': }

Both of these lines INVOKE the ‘foo’ class for use on a particular node. The problem is that the second method (the parameterized class method) looks nearly IDENTICAL to the method you would use to DECLARE the foo class in foo.pp. The only difference is that the class name is now INSIDE the curly braces (versus being OUTSIDE the braces when you DECLARE the class in your foo.pp file). It is VERY important that you distinguish the difference between DECLARING a class and using a parameterized class to INCLUDE the class in your site.pp file. I will point this out later too…

A Linux Example – Repositories

I would bet that the resource with the most “require” and “before” dependencies in the Linux world (Well, in the RHEL world anyways) would be the yumrepo. I’m sure most of us have many declared yumrepo resources in our manifests that each have their own tangled web of dependencies. My goal was to create a “repo” class that would have all of my repositories and use a Run Stage to ensure that my repo class was installed prior to any other package installs. Let’s look at some code:

1
2
3
4
5
6
7
8

class general::repo {
  yumrepo { 'huronrepo':
    descr    => 'Huron Repository',
    enabled  =>  '1',
    gpgcheck =>  '0',
    baseurl  =>  'http://10.13.0.6/huronrepo',
  }
}

This is my repo subclass based on my general base class. I have a single repository, but could easily have another 5 or 10 as need arises (I’m keeping things simple for demo purposes). Let’s look at another class:

1
2
3
4
5
6
7
8
9

class general::centos {
  include general::repo

  package { 'mcollective':
    name    => "mcollective",
    ensure  => 'present',
    require => Package['stomp'],
  }
}

Here’s a centos subclass off of the general base class. We’re including our repo class and declaring a single package that requires our “huronrepo” yumrepo. If we only needed to install a single package, we could use the “require” parameter; but if you assume that we will eventually need to install MULTIPLE packages, then this logic doesn’t scale. Just to keep things consistent, here’s my general base class:

1
2
3
4
5
6

class general {
  case $::operatingsystem {
    'CentOS': { include general::centos }
    'Darwin': { include general::osx }
  }
}

You can see that we include the general::centos class as long as Puppet is running on a CentOS box. There’s one final piece to this puzzle – it’s the actual declaration and assignment of the Run Stages. I’m doing this in my site.pp file. Here’s a snippit of what’s in my site.pp file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# /etc/puppet/manifests/site.pp

Exec { path => "/usr/bin:/usr/sbin:/bin:/sbin" }

# Run Stages
stage { 'pre':
  before => Stage["main"],
}

# Node Declaration
node 'server.whomever.com' {
  class { 'general::repos':
    stage => 'pre'
  }
}

Here’s where we’ve declared a “pre” stage that needs to run before the “main” stage. We don’t have to declare the “main” stage because Puppet uses that stage by default. Next, we’re including the general::repos class inside our ‘server.whomever.com’ node declaration and assigning it to the “pre” stage (which means that everything in that class will get configured prior to our “main” stage). Remember that you can declare as many stages as you need and chain them all to setup the order that you want, but that can become very unmanageable very quickly. I find it ideal to setup a ‘pre’ stage for repo setup, and then setting up dependencies within class files.

The World is Your Stage

This might not be the “best” way to use Run Stages, but it works for me. Hopefully this little writeup cements the concept for you too.

With the current movement in the Mac Community toward modular imaging strategies, there’s a spike in the need for properly formed package installers. Apple’s package format is well documented for its benefits and flaws and there are a string of applications that will help you create your perfect package (from Apple’s Packagemaker to the many third-party applications). While all the various package creation applications will ultimately create a desirable package in the end, very few of them have the triple-threat of easy code-review, replication, and portability. This is where The Luggage comes in.

The Luggage is a packaging system based on GNU’s make command that is found on most every *nix-based operating system (OS X included). Joe Block, the author, open-sourced the project based on an internal Google tool for creating packages with Makefiles. By using Makefiles, every member of your team can quickly glance at the file and see exactly what is being inserted into a package. Changes can be made and committed quickly, errors can be squashed without tearing apart .pkg files, and you can reproduce packages quickly and efficiently without wading through many GUI windows.

Why do I need to Package?

Many vendors already ship properly formatted package installers for their software – and for that we thank them. There are still a couple of major software vendors that choose to use unique third-party package “wrappers” to install their software. While this is fine if you only ever need to install that software on one machine, it makes software deployment…difficult. Because of this, systems administrators need to re-package this software into a proper package installer that will deploy and install silently.

If you’re a fan of open-source software, you will find that many projects do not offer their software in Apple-friendly packages. The Luggage will help you wrap their source files into a package format that can be distributed to your machines.

Finally, you may have a whole bevy of scripts that you use for configuration/customization of your clients. These scripts can easily be deployed via ARD or wrapped into a payload-free package with a single postinstall script. The Luggage will help you keep track of all your scripts and package them for distribution.

As I’ve said before, there are other third-party applications out there that will create a package to your needs. Many will use snapshotting (or fseventsd monitoring) to create a package based on what’s changed on your system. While this is lightning fast (in most cases), you will need to redo this whole process if something needs to be changed in the resultant package.

How do we use The Luggage?

As we said before, The Luggage is just a giant Makefile. Make has its own unique language, so you need to obey its syntax and formatting standards. I’ve linked to the GNU make manual here so you can get a quick overview of how it works (WARNING, it’s quite large), but this guide will cover all the basics you need to know to get started.

Note that while it is NOT NECESSARY TO COMPLETELY UNDERSTAND MAKE TO BEGIN USING THE LUGGAGE, it will help you out tremendously as you start to create complicated packages if you DO understand make’s nuances. This article may be long, but that’s only to make sure that the reader understands what is going on in the background.

The luggage.make File

The base of everything you will do with The Luggage is a file called luggage.make. It is by no means definitive, and you’re encouraged to add to it should you encounter a situation where a recipe doesn’t exist for a directory into which you want to install files (Don’t worry, we’ll get into this later), but it does serve as the basis for all packages you’re going to be creating.

At the top of the file are all the variable declarations. Anything that begins with SOMETHING=somethingelse is setting a variable that we will encounter later. Many of these variables (such as PACKAGEMAKER, WORK_D, CP, INSTALL, and so on) are paths to various commands that we will need in our rules (setting absolute paths to common commands saves typing later and helps us avoid errors with things like PATH environment variables). Dereferencing these variables in a Makefile is done with a syntax that looks like this –> ${CP} (this outputs the value of the CP variable, which is actually /bin/cp). Note that you DO NOT use quotes when you set a variable (i.e. if you want to set the path to CP you do it with CP=/bin/cp and NOT by doing CP=“/bin/cp”) – if you DO use quotes, they will be included in the value of that variable (which will cause you all kinds of problems).

Next, we have the target stanzas. A target (or a rule – I will use the word rule throughout the article) is setup to look like this:

1
2
3

        do-something: l_usr_bin
        @-echo "Commands are entered here"
        @-echo "Everything below our rule is executed"

In this case, the target, or rule, is called do-something and has dependencies based on ANOTHER rule that’s called l_usr_bin. Below this line is called our recipe, and right now there are two echo commands. If the above code was in a blank text file called Makefile we could execute the two echo commands by running the following command from the command line:

1

    make do-something

This would in turn echo the two lines of text below the do-something rule (This is not totally true – since we haven’t defined a rule for l_usr_bin it would probably error out, but I’ve kept that dependency in the above example to show you how a rule works.). Looking specifically at luggage.make, the target stanzas define the behavior for The Luggage’s various behaviors (make pkg, make dmg, make zip, and so on). Note that since many of these rules have dependencies on OTHER rules, a simple command of make pkg will trigger.

Next, you will encounter the Target directory rules. This is the part of the luggage.make file that you may need to edit. Joe has done a great job of defining the most popular directories into which you will install files/applications/etc. but it would be impossible for him to define EVERY location that you could possibly need. Here is the structure of creating a Directory Rule:

1
2
3
4

        l_etc_puppet: l_etc
        @sudo mkdir -p ${WORK_D}/etc/puppet
        @sudo chown -R root:wheel ${WORK_D}/etc/puppet
        @sudo chmod -R 755 ${WORK_D}/etc/puppet

This rule will create an /etc/puppet directory in Luggage’s working directory (The variable WORK_D will be used frequently – it’s the temporary directory that The Luggage creates to simulate the target volume onto which it’s installing files. Everything following ${WORK_D} will be the EXACT PATH into which files will be installed on the machine that runs your package.) These Directory Rules become very important when we create custom Makefiles because they serve as dependencies which create the directories in The Luggage’s Working Directory. In a nutshell, if you’re installing files into a directory with The Luggage, you need to have a Directory Rule that creates this directory first. Bottom Line: if you don’t FIND a Directory Rule for a directory where you will be installing files, then you’ll need to create one.

Finally are the File Packaging Rules. These are handy shortcut rules that will keep your makefiles very short and readable. In a nutshell, Joe has defined some of the most common directories to which files are installed and created one-line commands that will install specific files into those directories. For example, say you were creating a custom Makefile in a directory that also had a launchd plist called com.foo.bar.plist in it. If ALL you needed to do was create a package that installed that launchd plist into the /Library/LaunchDaemons directory you could setup your Makefile like this:

1
2
3
4
5

include /usr/local/share/luggage/luggage.make

TITLE=install_foo_launchd_plist
REVERSE_DOMAIN=com.foo
PAYLOAD=pack-Library-LaunchDaemons-com.foo.bar.plist

The PAYLOAD variable tells The Luggage which rules to execute. In this case, it’s executing the File Packaging Rule for launchd plists (pack-Library-LaunchDaemons-%) that creates the ${WORK_D}/Library/LaunchDaemons directory, installs the com.foo.bar.plist file into it, sets the correct permissions, and then executes the Packagemaker command line tool to create your package installer. Congratulations, you created a package in 5 lines of code!

Preparing for using The Luggage

The Luggage is fairly self contained, but it DOES use Apple’s Packagemaker command line tool – and the way to install THAT is to download and install the Developer Tools for your computer’s OS version (There are different Developer Tools packages for 10.6, 10.5, and so on). The Developer Tools can be downloaded from Apple’s Developer Site, but you must create a (free) developer account first. If you’re a Mac sysadmin, you should already have all of this.

Once the Developer Tools have been installed you will then need to install The Luggage. You can use The Luggage to create an installer package for The Luggage (Weird, yes) by first downloading the source code from Github. Just click on the big Downloads button in the upper right corner of the screen and download a zip version of the files. From there, double-click on the downloaded zip file to open it, open up Terminal and change to the folder that was created which contains The Luggage’s source code, and execute the following command:

1

    sudo make pkg

This will create an installer package which can be run. This package installs The Luggage into the /usr/local/share/luggage directory (Don’t believe me? Check the Makefile for your self!). You’re now ready to use The Luggage!

Before we get into the package creation examples, we need to talk about directory structure and version control systems (svn, hg, git, etc…). When you create new packages with The Luggage, it’s necessary to create a new Makefile in a new directory. I like to use the /usr/local/share/luggage/examples directory to create my packages. The easiest way to begin a new package is to copy a directory that contains a working Makefile and simply tailor it to your needs. Unless your job is SOLELY packaging (or you’re a unix graybeard), you’re NOT going to remember all the syntax and nuances of make. Don’t re-create the wheel – just copy and edit.

Next, you’re going to want to backup your files and/or have a versioning system. Version control systems (like Subversion, Mercurial, Git, and the like) are becoming increasingly popular with the current DevOps movement, so it might be a good idea to start playing with one NOW while you’re learning a new skill! If you’re TOTALLY new to this concept, I recommend using git, but it’s entirely up to you. I maintain a git repository of my Luggage fork that’s open to anyone to review and borrow. If you decide to use something like Git, well then GOOD ON YA, MAN! If not, make sure you have a backup of your Makefiles. They’re small files; make two backups :)

Package Creation Examples

Now that we’ve described how make works, how luggage.make works, given a quick example of how to create a package in 5 lines, and had the installation/backup talk let’s walk you through some basic package creation examples. We’ll create packages that install an application, create a package that installs printers and executes a postinstall script, create a package that installs source code in many directories, and finally create a complex Makefile that uses variables and bash commands.

An Application Package

The most common packages will simply install an application file into the /Applications folder on your computer. Since .app files are actually bundles (a directory that contains subdirectories of executable code), we will need to use tar to wrap these bundles into a single file. From there we can have The Luggage untar the application into the correct folder. Joe’s Firefox example does this, but he also has an added step of curling the tarred/bzipped application from a web server. I’m going to skip that step (for now) so you understand how the basic process works, but it’s a best practice to use the curl method so you can keep your applications up-to-date on your fileserver without having to copy the new files to your luggage directory every time.

Let’s first create the /usr/local/share/luggage/examples/calculator directory and then make a copy of /Applications/Calculator.app into the /usr/local/share/luggage/examples/calculator directory. Next, lets open Terminal and change to our /usr/local/share/luggage/examples/calculator directory. Finally, run the following command to tar.bz2 our Calculator.app:

1

    tar cvfj Calculator.app.tar.bz2 Calculator.app

If you did it correctly, it should create the Calculator.app.tar.bz2 file that we need for The Luggage. Make sure this file is in the SAME directory (and level – so don’t create subfolders) as the Makefile we’re going to create. You can actually delete the Calculator.app copy that we created – we won’t need it. From there, our Makefile is simple – it should look like this:

1
2
3
4
5

include /usr/local/share/luggage/luggage.make

TITLE=Calculator_Install
REVERSE_DOMAIN=com.yourdomain
PAYLOAD=unbz2-applications-Calculator.app

That’s it! The PAYLOAD variable is the magic variable here – it contains the rule(s) that are to be executed in creating our package. The unbz2-applications-% File Packaging Rule is defined in our luggage.make file (which we’ve included at the top of our Makefile), so we don’t NEED anything else in our Makefile. MAKE SURE that the capitalization and spelling of Calculator.app in “unbz2-applications-Calculator.app” and your “Calculator.app.tar.bz2” files are IDENTICAL or this package will NOT be created (make relies on this).

UPDATED: An easier Application Package

Joe has actually created a Ruby script that will perform all of the actions outlined in the previous example for you. It’s called app2luggage.rb and it can save you a few steps if you know how to use it. Let’s take a look and see what we need to use it.

The first thing you will need to do is install the trollop rubygem as that’s what app2luggage.rb uses to parse its options. You can do this with the following command:

1

    sudo gem install trollop

Next, make sure the app2luggage.rb script exists in your /usr/local/share/luggage directory. Finally, let’s run the command with the following arguments:

1

    sudo /usr/local/share/luggage/app2luggage.rb -a /Applications/Calculator.app -i /usr/local/share/luggage/examples/Calculator_Application/ -l /usr/local/share/luggage/luggage.make -r com.huronhs -p Calculator_Application

Here’s what each argument means:

1. -a is the path to the Application we want to tar up and install with The Luggage – we’re using the path to Calculator.app for now
2. -i is the path to the folder that app2luggage.rb will create that contains our Makefile and tarred up application. This folder SHOULD NOT EXIST or app2luggage.rb will exit (so as not to overwrite your data).
3. -l is the path to our luggage.make file
4. -r is the reverse domain for our organization
5. -p is the package id for the Makefile

There are other arguments available, simply run app2luggage.rb with the —help argument to see them all. Once app2luggage.rb runs successfully, it will create the directory you specified in the -i argument and populate it with a Makefile and the tarred up application. The only thing left to do is to make your pkg, zip, or dmg.

Installing a printer with a preinstall script

One of the most popular solutions I offer with using The Luggage is to create a package that will install a printer and then install a pre-configured PPD file that sets all the initial settings for the printer. I do this with Puppet currently, and I know it’s popular in Munki too. You can also optionally install a specific driver (if the drivers for your printer aren’t already on your machines). For those people who like to skip ahead, I have this example on my luggage repo.

This Makefile demonstrates the use of a preinstall/preflight script (which isn’t actually installed into the payload of the package). The Luggage has a special packaging rule for this called pack-script-% that works so long as the name of your script corresponds exactly with what you write in the PAYLOAD variable. This file also demonstrates the use of multiple rules in the PAYLOAD variable by using the \ character. While this isn’t difficult to understand, it is a necessary syntax.

My preflight script is right here for those who are interested. It’s copied into the directory we create that contains our makefile and I name it preflight. Note that it contains variables that need to be changed (all of which are at the top of the script) before you deploy this package.

Next, we need to get a .ppd file that contains all the configuration data for our printer. The easiest way to do this is to install your printer on a demo machine using the EXACT SAME SETTINGS that will be configured in your script (protip: actually RUN the script on your computer first), configure it how you want (number of trays, memory settings, type of finisher, etc…), and then open the /etc/cups/ppd directory on your model computer. Inside should be a .ppd file for your printer containing all the settings you’ve just configured. Copy this .ppd file into the folder that contains your Makefile and preflight script. Mine (in this example) will be called psm_HHS_Office_9050.ppd

Now, let’s look at our Makefile:

1
2
3
4
5
6
7
8
9
10
11
12
13

    include /usr/local/share/luggage/luggage.make

    TITLE=HHS_Main_Office_9050_Managed_Installer
    REVERSE_DOMAIN=com.huronhs
    PAYLOAD=\
        pack-hp-ppd \
        pack-script-preflight

    pack-hp-ppd: l_etc_cups_ppd
      @sudo ${CP} ./psm_HHS_Office_9050.ppd ${WORK_D}/etc/cups/ppd/psm_HHS_Office_9050.ppd
      @sudo chmod 644 ${WORK_D}/etc/cups/ppd/psm_HHS_Office_9050.ppd
      @sudo chown root:_lp ${WORK_D}/etc/cups/ppd/psm_HHS_Office_9050.ppd

The first thing you should notice is that our PAYLOAD variable starts with a \ and has three lines. The \ signifies that the contents of our variable spans multiple lines. In reality, the value of PAYLOAD is actually “pack-hp-ppd pack-script-preflight” but it’s formatted so it’s easier to read. Next, notice that the pack-script-preflight rule contains the word “preflight” after pack-script-. This means that our script must be named preflight (EXACTLY – case is sensitive here). Finally, we’ve also specified a pack-hp-ppd rule. Since luggage.make DOES NOT define this rule, we’ve defined it in our Makefile.

The pack-hp-ppd rule has a dependency on the l_etc_cups_ppd rule. This rule IS defined in luggage.make (well, it is for me – I can’t remember if I created it or not. If it isn’t there for you, then you’ll need to create it using the other folder creation rules as a guide) – and it creates the /etc/cups/ppd folder structure that we need in our package. Lets look at the three lines which are called our recipe*:

1
2
3

@sudo ${CP} ./psm_HHS_Office_9050.ppd ${WORK_D}/etc/cups/ppd/psm_HHS_Office_9050.ppd
@sudo chmod 644 ${WORK_D}/etc/cups/ppd/psm_HHS_Office_9050.ppd
@sudo chown root:_lp ${WORK_D}/etc/cups/ppd/psm_HHS_Office_9050.ppd

The first line references the CP variable (if you remember – its value is /bin/cp) to copy the psm_HHS_Office_9050.ppd file from the directory that contains our Makefile into Luggage’s working directory/etc/cups/ppd. The second line sets its mode to 644 (to match the mode for all .ppd files in that directory), and the third line sets the owner to root and the group to _lp (note that this group is ONLY available in 10.5 and 10.6 machines – so you’ll need another package for 10.4 clients or below).

That’s it! The resultant package will run the preflight script and then install the correct .ppd file after the script is run. This package will successfully install and configure your printer in one fell swoop.

Installing files into multiple directories

So far our Makefiles have been pretty easy. In fact, most of them have been a couple of lines. Lets look at a package that installs files into multiple locations. There’s a cool open source tool called Angelia that was created by R.I Pienaar. I use it to send alerts to my iPhone and to also process Nagios Alerts. The problem is that it’s only packaged for Linux machines. Since I manage Mac machines, I thought I’d lend R.I. a hand and create a package installer for it. This package is also on my luggage repo if you want to work ahead. Let’s look at the Makefile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

    #  Angelia Package Creation:

    include /usr/local/share/luggage/luggage.make

    TITLE=Angelia_Installer
    REVERSE_DOMAIN=com.huronhs
    PAYLOAD=\
        pack-angelia-etc \
        pack-angelia-binaries \
        pack-angelia-lib \
        pack-angelia-spool \
        pack-angelia-log \
        pack-angelia-launchd


    pack-angelia-launchd: l_Library_LaunchDaemons
      @sudo ${CP} net.devco.angelia.plist ${WORK_D}/Library/LaunchDaemons
      @sudo chmod -R 644 ${WORK_D}/Library/LaunchDaemons

    pack-angelia-binaries: l_usr_sbin
      @sudo ${CP} ./angelia-send.rb ${WORK_D}/usr/sbin/angelia-send
      @sudo ${CP} ./angelia-spoold.rb ${WORK_D}/usr/sbin/angelia-spoold
      @sudo ${CP} ./angelia-nagios-send.rb ${WORK_D}/usr/sbin/angelia-nagios-send
      @sudo chmod -R 755 ${WORK_D}/usr/sbin

    pack-angelia-lib: l_Library_Ruby_Site_1_8
      @sudo ${CP} -R ./angelia ${WORK_D}/Library/Ruby/Site/1.8
      @sudo ${CP} ./angelia.rb ${WORK_D}/Library/Ruby/Site/1.8
      @sudo chmod -R 755 ${WORK_D}/Library/Ruby/Site/1.8

    pack-angelia-etc: l_etc_angelia
      @sudo ${CP} -R ./templates ${WORK_D}/etc/angelia
      @sudo ${CP} ./angelia.cfg ${WORK_D}/etc/angelia
      @sudo ${CP} ./COPYING ${WORK_D}/etc/angelia
      @sudo ${CP} ./README.markdown ${WORK_D}/etc/angelia
      @sudo chmod -R 755 ${WORK_D}/etc/angelia

    pack-angelia-log: l_var_log_angelia
      @sudo touch .create
    pack-angelia-spool: l_var_spool_angelia
      @sudo touch .create

It’s definitely longer, but I don’t think it’s any more complex than the examples we’ve seen before. The PAYLOAD variable spans multiple lines using the \ character and I’ve defined all the custom rules. All of the rules depend on folder rules in my luggage.make file (which were created if they didn’t exist before), and the recipes inside those rules are extremely simple to navigate (mainly copying files and changing permissions). Creating a script that makes this package would be QUITE A BIT longer than this Makefile, and the Makefile itself took less than 3 minutes to create. The only caveat is that we need to make sure all the files/directories we’re copying into Luggage’s WORK_D directory are in the folder that contains the Makefile. Because of this, anytime Angelia is updated I will need to copy over new files.

It would be much easier to create a Makefile that downloaded the current version of all of these files before it copied them into our package…

Creating a more complex Makefile

The basis of this example was my thought that it would be MUCH easier for me to have a Makefile that downloaded the newest versions of the files that I need before it copies them into the package. Make supports this through its support of shell commands and variables – it only requires you to code it up. Let’s look at a package I created for Marionette-Collective.

Marionette-Collective is awesome software (also created by R.I Pienaar and now owned by Puppet Labs) that allows you to communicate with multiple nodes at once. They DID have a script that created their packages, but I wanted to create a Makefile that would create a package of ANY version of their software. Let’s look at the Makefile first and I’ll break it down:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

    #  Example:  A dynamic installer for Marionette-Collective
    #
    #  Author:  Gary Larizza
    #  Created: 12/17/2010
    #  Last Modified: 12/18/2010
    #
    #  Description:  This Makefile will download the version of MCollective specified
    #    in the PACKAGE_VERSION variable from the Puppet Labs website, untar it,
    #    and then install the source files into their Mac-specific locations.  
    #    The MAJOR and MINOR versions must be specified for the Info.plist file
    #    that Packagemaker requires, but I use awk on the PACKAGE_VERSION to 
    #    get these.  See inline comments.
    #
    include /usr/local/share/luggage/luggage.make


    # Luggage Variables:
    #    If the TYPE variable isn't specified via the CLI, we will install everything
    #    into the resultant package
    TITLE=MCollective_Installer_Full
    REVERSE_DOMAIN=com.puppetlabs
    PAYLOAD=\
        unpack-mc-${MCFILE} \
        pack-mc-libexec \
        pack-mc-binaries \
        pack-mc-lib \
        pack-mc-config \
        pack-mc-config-server \
        pack-mc-config-client \
        pack-mc-launchd \
        pack-mc-mcollectived \
        pack-mc-preflight-all

    # Variable Declarations:  
    #    Any variable can be set from the command line by doing this:
    #    "make pkg PACKAGE_VERSION=1.0.0"
    PACKAGE_VERSION=1.0.0
    PACKAGE_MAJOR_VERSION=`echo ${PACKAGE_VERSION} | awk -F '.' '{print $$1}'`
    PACKAGE_MINOR_VERSION=`echo ${PACKAGE_VERSION} | awk -F '.' '{print $$2$$3}'`
    MCFILE=mcollective-${PACKAGE_VERSION}
    MCURL=http://puppetlabs.com/downloads/mcollective/${MCFILE}.tgz

    # Package Creation Limiters:
    #    These if-statements will check for one of three values for the TYPE variable:
    #    "COMMON, CLIENT, or BASE"  If either of these values are present (CASE SENSITIVE)
    #    the PAYLOAD variable will be changed to limit what is installed into the package.

    # COMMON Package:
    #    This package includes the Ruby libraries and MCollective plugins with nothing else.
    ifeq (${TYPE},COMMON)
      PAYLOAD=\
          unpack-mc-${MCFILE} \
          pack-mc-libexec \
          pack-mc-lib \
          pack-mc-preflight-common
      TITLE=MCollective_Installer_Common
    endif

    # CLIENT Package:
    #    This package includes the MCollective Binaries and the configuration file for
    #    MCollective's client binaries.  
    ifeq (${TYPE},CLIENT)
      PAYLOAD=\
          unpack-mc-${MCFILE} \
          pack-mc-config \
          pack-mc-binaries \
          pack-mc-config-client \
          pack-mc-preflight-client
      TITLE=MCollective_Installer_Client
    endif

    # BASE Package:
    #    This package includes the mcollectived daemon, Ruby Libraries, a launchd plist
    #    to call mcollectived, and the configuration files for the MCollective server.
    ifeq (${TYPE},BASE)
      PAYLOAD=\
          unpack-mc-${MCFILE} \
          pack-mc-config \
          pack-mc-config-server \
          pack-mc-launchd \
          pack-mc-mcollectived \
          pack-mc-lib \
          pack-mc-preflight-base
      TITLE=MCollective_Installer_Base
    endif

    # This rule will curl the selected version of MCollective and untar it into the directory
    #    in which the Makefile resides.
    unpack-mc-${MCFILE}:
      curl ${MCURL} -o ${MCFILE}.tgz
      @sudo ${TAR} xzf ${MCFILE}.tgz

    # This rule will install MCollective's plugin files to /usr/libexec/mcollective