In light of some discussions we've been having internally, and with various community members, it has been proposed that as of our next BoxGrinder release, we shall no longer build images with ephemeral devices[1] pre-attached (for EBS), or pre-mounted in any fashion[2] by default (for EBS or S3)[3].

Instead we will emit a message at build-time about device attachment and mounting, advising users to install a script such as cloud-init to auto-mount ephemeral devices, as well as a pointer to a boxgrinder.org resource page explaining how best to bake in an appropriate cloud configuration file (more on that soon).

Those that want device mappings baked into their image will continue to be able to do so by defining their desired mappings via configuration. Note that block storage mappings (including ephemeral, or otherwise), can be configured or reconfigured at runtime instead of, or in addition to, build time.

Providing block device mappings

For EBS AMIs no ephemeral devices are attached by default, whereas with S3 AMIs there is a default layout (although you can modify it as you see fit).

At run-time

All of the standard methods of providing (or modifying) a block device mapping at run-time still apply. For example:

ec2-run-instances ami-12345678 -b /dev/xvdb=ephemeral0

At build time

You can speculatively map devices that may not be present in every instance size at build time.

# -d ami or -d ebs
boxgrinder-build my.appl -p ec2 -d ebs --delivery-config \ 
  block_device_mapping:'/dev/xvdb=ephemeral0:/dev/xvdc=ephemeral1'

Why the changes?

The reasons for this change are multitudinous, but the foremost are:

• Ephemeral device mappings vary according to instance size: We cannot make any easy assumptions about which ephemeral disks will be present, as it varies depending on which instance size is selected, and is subject to arbitrary change by Amazon. For instance, recent problems with BoxGrinder S3 AMIs on m1.small instances were caused by BoxGrinder assuming a particular ephemeral device would be present; an assumption that fell over when AWS introduced a new device mapping layout[4].

• We do not want to maintain a script that maps and mounts which devices are provided by which instance sizes, as it will duplicate existing well-established efforts (e.g. cloud-init), in addition to being difficult to maintain, inferior in functionality, and surprising to the user.

• Confusion: Ephemeral device mappings do not show on Amazon API queries, or their web UI (!!). Users do not realise a device is attached until they attempt to attach their own device at the same location (e.g. /dev/xvdb is an ephemeral device mapping, but it does not show in the web UI; if the user attempts to attach a device at xvdb it will fail).

• Inconsistency: BoxGrinder can build a few different OSes, and some of the best cloud initialisation projects are not necessarily available on all of the OSes default repositories. We would rather not force disparate solutions onto our users, as consistency is one of our primary goals.

Therefore we concluded that rather than a prescriptive or inconsistent solution, we shall provide extremely simple default behaviour, and enable the user to choose an approach that is optimal for them.

Thoughts?

If you have any objections, comments or suggestions, leave them in the comments, or send them to via any of our community channels.

---

[1] Ephemeral disks are transient/instance storage devices that are available 'free' with most instance sizes. The overall capacity included, and the number of devices the space is subdivided into varies by instance size. For example, a m1.large instance has 2x420GiB instance storage.

[2] There is an important distinction between attaching and mounting devices on AWS. Attaching is akin to physically plugging a disk into a machine. Mounting is the usual process of making a device available to the machine's file system.

[3] For the sake of clarity, it is worth noting that S3 Backed AMIs always have a pre-defined set of ephemeral device mappings provided by EC2, but with EBS by default there are none.

[4] In this particular case we were expecting /dev/xvdb to exist, but for m1.small the ephemeral device we wanted was mapped to /dev/xvda2.

The long-awaited BoxGrinder Build 0.10.1 bugfix release is now available; with a variety of irritation eliminating alterations behaviour should be more consistent, and no longer prone to permissions errors.

Permission denied, log shifting errors

If you have seen any errors akin to:

FATAL -- : Logger::ShiftingError: Shifting failed. Permission denied - log/boxgrinder.log.2 or log/boxgrinder.log.3

The problem was caused when BoxGrinder switched to a local user from root, but the log file could still be owned by root. The issue was only apparent on certain systems, and even then often only occasionally.

Ruby 1.9

We've made some changes to ensure BoxGrinder runs correctly under Ruby 1.9, with a particular eye towards the forthcoming release of Fedora 17. You can see our earlier musings on the changes required.

Scientific Linux EBS AMIs

OS constraints on the EBS plugin have been removed, so you can now create a Scientific Linux EBS AMIs. As the limitation is generally eliminated, any community OS plugin is also be able to use the EBS plugin.

Bash tab completions

Basic bash tab completions have been sneaked into this release. Give it a try:

    [root@localhost ~]# boxgrinder-build my.appl --
    --backtrace        --delivery-config  --os-config        --plugins
    --debug            --force            --platform         --trace
    --delivery         --help             --platform-config  --version

Other

Snapshots with the S3 plugin are working correctly again, and some simple testing issues were fixed.

Release notes

Bug

• [BGBUILD-337] - In SL if default repos are disabled, /etc/yum.repos.d folder is not created
• [BGBUILD-338] - Weed out non-deterministic tests
• [BGBUILD-344] - Builds on some platforms impossible due to log (and/or other) files still being owned by root after boxgrinder switches to user
• [BGBUILD-351] - s3 plugin attempts to create bucket with whole pathname during snapshot

Enhancement

• [BGBUILD-349] - Use RbConfig instead of obsolete and deprecated Config deprecation warning with Ruby 1.9.3

Task

• [BGBUILD-332] - Add support for bash completion
• [BGBUILD-346] - Confirm Ruby 1.9.3 support

Sub-task

• [BGBUILD-348] - Simplecov coverage testing for Ruby >=1.9

Looking to the (beefy) future

Those of you who keep an eye on happenings in Fedora-land will undoubtedly be aware that Fedora 17 is due out in the near future. From BoxGrinder's perspective one of the more important changes is the move to Ruby 1.9.x from 1.8.

There are some syntactic changes and a few subtle semantic differences between versions, so it is important for us to ensure equivalent runtime behaviour in both flavours. Fedora 15 and 16 will both remain on 1.8.7, so we must straddle the fence.

Mercifully, the changes required were fairly minor; but for those of a more inquisitive persuasion, let's look at a few examples of alterations that were required.

Coverage Testing

We needed to provide code coverage analysis with both Rubies, under 1.9 via simplecov and RCov when using 1.8. We only want the relevant tool to be loaded and run for the appropriate version of Ruby.

Our tests are run using Rake, and as Rake tasks run in a new process, a bit of ingenuity is required to ensure the code you write is affecting the correct process.

In this instance the simplest solution was to create a couple of helper files that are run in the new process before the specs begin, ensuring everything required is kicked into action.

---

Below is a snippet from our Rakefile. Under 1.9, we set an environment variable to indicate to spec_helper that simplecov should be run.

When running with 1.8 a bit of path juggling ensures rcov_helper is run before RCov starts. As RCov is initialised before RSpec, we must ensure that some basic dependencies are met.

Rakefile:

RSpec::Core::RakeTask.new('spec:coverage') do |t|
  t.ruby_opts = "-I ../boxgrinder-core/lib"
  t.pattern = "spec/**/*-spec.rb"
  t.rspec_opts = ['-r spec_helper', '-r boxgrinder-core', 
          '-r rubygems', <snip>]
  t.verbose = true

  if RUBY_VERSION =~ /^1.8/
    t.rcov = true
    t.rcov_opts = ["-Ispec:lib spec/rcov_helper.rb", <snip>]
  else
    ENV['COVERAGE'] = 'true'
  end
end

spec_helper.rb:

if ENV['COVERAGE']
  require 'simplecov'

  FILTER_DIRS = ['spec']

  SimpleCov.start do
    FILTER_DIRS.each{ |f| add_filter f }
  end
end

This might seem fairly circuitous, but if we attempted to start code coverage in the Rakefile itself, we'd simply be analysing Rake, not our code!

Sycked up

module BoxGrinder
  # Avoid Psych::SyntaxError (<unknown>): couldn't parse YAML in 1.9
  if RUBY_VERSION.split('.')[1] == '9'
    require 'yaml'
    YAML::ENGINE.yamler = 'syck'
  end
end

Psych, the default cRuby 1.9.3 YAML parser, causes problems with our YAML parsing and validation (through Kwalify), but fortunately the only change required was to set the parser back to Syck.

Syntactical slips

-    when :ec2:
+    when :ec2
       disk_format = :ami
       container_format = :ami
-    when :vmware:
+    when :vmware
       disk_format = :vmdk

This is a single example of a few slightly unusual case (switch) syntaxes which had crept into the codebase, and due to 1.9's new hash syntax, something like :ec2: appears to be a mangled hash key.

String it out

-    repoquery_output.each do |line|
+    repoquery_output.each_line do |line|

String#each is no longer an alias to #each_line, which splits a string into an array with newline as the separator. The change is probably rather sensible, given that the behaviour is surprising at first encounter.

Regex

-    vmdk_image.scan(/^createType="(.*)"\s?$/).to_s.should == "vmfs"
+    vmdk_image.match(/^createType="(.*)"\s?$/)[1].should == "vmfs"

[1] pry(main)> 'createType="BG"'.scan(/^createType="(.*)"\s?$/).to_s
=> "example" # Ruby 1.8.7

[1] pry(main)> 'createType="BG"'.scan(/^createType="(.*)"\s?$/).to_s
=> "[[\"example\"]]" # Ruby 1.9.3

Numerous examples of slightly dodgy regex matching that relied upon to_s in our tests were eliminated from the codebase.

Other bits

Amongst other changes that bit us was our reliance upon quirky behaviour in Ruby 1.8 bindings (albeit our usage is slightly dubious anyway), and a couple of situations where implicit arrays were assumed.

RSpec 2, when dependencies strike

A few of our newer RSpec tests only work properly with rspec-expectations >= 2.7.0, which is available only on Fedora 17 and above.

- it "should add the path to the path_set" do
+ <snip> :if => RSpec::Expectations::Version::STRING >= '2.7.0' do
    expect{ simple_update }.to change(subject, :path_set).
      from(empty_set).to(mkset('/the/great/escape'))
  end

By using RSpec 2 filters we avoid running tests known to fail spuriously. This is a useful technique if you temporarily need to straddle multiple versions, and refactoring isn't desirable.

Onwards

Thankfully the process was rather easy, with all but a couple of issues being caught by our tests. It would seem that all of the cases were circumstances where we should have been using better approaches anyway, so the outcome was certainly positive.

We are really pleased to announce the next major version of BoxGrinder Build. Version 0.10 includes many exciting new features.

New plugin: libvirt delivery plugin

The most time-consuming feature was the libvirt delivery plugin, enabling appliances to be delivered and registered on the range of platforms that libvirt supports. These include KVM, Xen, VirtualBox and VMWare to name but a few, and whilst the current release of the plugin is a preview, we hope that your feedback will enable us to continue improving it, so please let us know your thoughts!

There are a large number of features and configurable options available, so we shall limit ourselves to some simple examples. We encourage you to consult the documentation for detailed information.

Examples

Deliver the appliance to a qemu hypervisor on the local machine, placing the appliance in the /var/lib/libvirt/images directory, and register the image:

boxgrinder-build definition.appl -d libvirt --delivery-config connection_uri:qemu:///system,image_delivery_uri:/var/lib/libvirt/images  

The plugin will attempt to determine optimal settings by probing the libvirt daemon for its capabilities. If, for instance, the libvirtd residing at the connection_uri machine advertises that it supports kvm, this will be preferred over a slower qemu domain. You can, of course, override these settings manually.

Connect via SSH to a remote vbox (VirtualBox) hypervisor. Deliver the appliance via SFTP to the remote machine and register:

[...] -d libvirt --delivery-config connection_uri:vbox+ssh://root@example.org/session,image_delivery_uri:sftp://root@example.org/var/lib/libvirt/images

In the above example, we assume that you have set up your ssh-agent, thus enabling the plugin to seamlessly use key authentication for both libvirt and SFTP. Password authentication works too, but it requires you to enter a password each time the plugin needs to connect, which is rather inconvenient.

There are a plethora of further features to allow arbitrary modification of domain definitions. Futhermore, we hope to make the plugin more user-friendly for those who require complex configurations, and these are topics we shall blog on in the future.

New plugin: OpenStack delivery plugin

It's a pleasure to submit to testing our OpenStack plugin. The OpenStack plugin allows you to deliver your appliance (in different formats) to a Glance server using its REST API.

Usage of this plugin is very simple. If you want to submit a KVM image to OpenStack:

boxgrinder-build definition.appl -d openstack

...and if you want to have an EC2 image registered:

boxgrinder-build definition.appl -p ec2 -d openstack

Please refer to plugin documentation for detailed usage instructions.

This plugin is in a tech-preview state. This means that we haven't tested it intensively, please help us with some testing. We really appreciate any comments and pull requests for this plugin.

New plugin: VirtualPC platform plugin

If you need the appliance in VHD format which is required by Virtual PC or Citrix XenServer - there you have it. Usage is very simple:

boxgrinder-build definition.appl -p virtualpc

This is an early stage of the plugin as we want to add more functionality in the future like thin/thick disk support.

Kickstart support removal

As of now support for using Kickstart files as input files for BoxGrinder Build is removed. Kickstart files support was the cause of confusion by many BoxGrinder users. They were expecting that the full plugin chain could be executed, but this wasn't ever true. Only operating system plugin execution was tested, everything else was in the hand of gods.

We're now clear about the support: there is no support for kickstart files.

Virtual machines disk alingment

This is not a direct feature of BoxGrinder, but there was a fix commited to the upstream appliance-tools which enables disk alignment by default. From now on, all appliances made by BoxGrinder are aligned.

New supported EC2 regions: sa-east-1 and us-west-2

You're now free to create your AMI's in new regions: sa-east-1 and us-west-2. More info about selecting the regions you can find in S3 plugin documentation.

Enhanced support for variables (parameters) in any string value field of an appliance definition

As requested by the community, it is now possible to use variables in any value field of an appliance:

name: boxgrinder-#OS_NAME#
os:
  name: fedora
  version: 16

Furthermore, you may now inject custom variables from your environment into your appliances simply by defining them.

$ export MY_ENV="I-LOVE-BoxGrinder"

[...]
  base:
    - "mkdir #MY_ENV#" # This becomes "mkdir I-LOVE-BoxGrinder"

There is a hierarchy in which a variable defined in the variables section of an appliance takes precedence over one defined in the environment. See our mailing list post for more detail.

As an added bonus it is now also possible to define nested variables. BoxGrinder can now resolve the following variable #A# successfully as BoxGrinder Rocks!:

#A#: #B# #C#!
#B#: BoxGrinder
#C#: Rocks

You could define these variables either in the variables section of your appliance, or in the environment. The variables section is better for portability, whereas the environment is useful for baking in one-shot customisations such as fixed MAC addresses.

Full release notes can be found below. If you have any comments - find us or our community.

Release Notes

Bug

• [BGBUILD-152] - Clarification of Post section inheritance behaviour
• [BGBUILD-242] - Additional EBS overwrite edge cases
• [BGBUILD-243] - Misleading error messages when YUM mirrors/mirror-lists are unreachable.
• [BGBUILD-308] - bg-build 0.9.6 complains about file validation based upon file name
• [BGBUILD-313] - boxgrinder build fails to build ec2 image if ec2-user already exists
• [BGBUILD-323] - Invalid kernel version recognition makes recreating initrd impossible
• [BGBUILD-326] - BoxGrinder can fail when build run from /

Enhancement

• [BGBUILD-304] - Standarize plugin callbacks
• [BGBUILD-312] - Only use root privileges when necessary
• [BGBUILD-318] - Add support for us-west-2 region
• [BGBUILD-320] - support #ARCH# and/or #BASE_ARCH# replacement anywhere in the appliance definition file
• [BGBUILD-327] - Resolve appliance definition variables in ENV if they are not defined.

Feature Request

• [BGBUILD-157] - Add Alignment options for virtual appliances
• [BGBUILD-195] - Add support for OpenStack
• [BGBUILD-211] - Support for registering appliances with libvirt (KVM/XEN)
• [BGBUILD-234] - Host contributed appliance definition files
• [BGBUILD-267] - Add CentOS 6 support
• [BGBUILD-302] - Add support for VirtualPC platform
• [BGBUILD-322] - Allow to specify kernel and ramdisk for ec2 plugin
• [BGBUILD-331] - Add support for sa-east-1 EC2 region

Task

• [BGBUILD-315] - Distribute qcow2 and thin vmdk disks for meta appliance
• [BGBUILD-325] - Remove kickstart support
• [BGBUILD-328] - Document libvirt plugin usage
• [BGBUILD-329] - Document openstack plugin usage
• [BGBUILD-330] - Document virtualpc plugin usage

It has been a long time we met last time! There are a few reasons. We both (Marc and me) were travelling a bit spreading the BoxGrinder love around the world. Marc is still working on a big BoxGrinder improvement - libvirt plugin. Additionally I was pulled into another task - making JBoss AS 7 available in Fedora.

But hey, we found some time to release 0.9.8 :) Although we planned to make 0.9.7 the last release of 0.9.x series - we were forced to do another one. The main reason was that appliance-tools version shipped in Fedora 16 wasn't able to build appliances where GRUB2 is the default boot loader. I took care of this and pushed new appliance-tools version over the weekend, but it required some changes to how we use appliance-creator internally. So, there we have 0.9.8.

CentOS 6 support

As a side effect of fixing appliance-creator - I added the ability to create CentOS 6 appliances. You'll even be able to create CentOS 6 AMIs now! Go, try it and let us know how it went!

Preserving your environment, and limiting time as root

There are some use cases where people get confused about BoxGrinder not using their own environment variables when executing BoxGrinder Build using sudo. Thanks to Marc this is now over, yay! Even more - we'll make sure that the created artifacts have the expected owner, and drop down from root user to standard user (where applicable) as soon as possible. This change also ensures that the user's agents such as ssh-agent are available to BoxGrinder, for instance for the SSH plugin or the upcoming libVirt plugin will seamlessly use your ssh-agent when required. Small things, but makes life easier.

To utilise it at its best, you can simply run boxgrinder-build without sudo. Although even under sudo and su, we now try to behave in a less surprising way.

The release is immediately available in Fedora 15/16 updates-testing repository.

Full release notes you can find below. If you have any comments - find us or our community.

Release Notes

Bug

• [BGBUILD-310] - BoxGrinder doesnt build appliances when Fedora 16 is the host
• [BGBUILD-321] - For EBS AMIs use the filesystem type specified for root partition

Enhancement

• [BGBUILD-312] - Only use root privileges when necessary

Feature Request

• [BGBUILD-157] - Add Alignment options for virtual appliances
• [BGBUILD-267] - Add CentOS 6 support