nanoc › Articles about v2.1

nanoc 2.1 released (August 17th, 2008)

nanoc 2.1 is here! Update your nanoc version the usual way:

% gem update nanoc

There is no comprehensive changelog, but the "What’s New in nanoc 2.1" series on this news archive should give you a good idea of what is new (hence the name of the series). Here are the relevant posts:

improved CLI tool
an open framework
improved data sources
routers
assets
miscellaneous

nanoc 2.1 is entirely backwards compatible with nanoc 2.0. This means that a site that can be compiled with nanoc 2.0 can be compiled without issues using nanoc 2.1.

A lot has changed between 2.0 and 2.1, so it is possible that you’ll run into bugs. If something odd occurs, please do let me know either on the mailinglist, on Trac or on Bitbucket. I’ll look into the issue as soon as possible and have it fixed in no time. Thanks for reporting!

WNin #6: Miscellaneous (August 11th, 2008)

Welcome to the final installment of What’s New in nanoc 2.1. In this article I’ll show you several miscellaneous improvements and features I couldn’t fit elsewhere.

The autocompiler now uses Rack, so this is good news if you don’t like WEBrick and would like to use something else instead. Thin, Mongrel, Ebb, … you name it.

Layout processors are gone entirely. Instead, use filters. Layouts consist of metafiles and content files now, and the meta files determine the filter to be run. This means less code duplication, as there’s no need to have both e.g. a Haml filter and a Haml layout processor.

A layout only has one attribute, filter, which determines the filter to be run. Here’s what the meta file of the nanoc site’s default layout contains, to show you how this works:

filter: 'erb'

Layouts can be stored hierarchically. This is useful in case you want to e.g. create a "partials" directory in your layouts directory, in which you’d store partials you want to render. Or, you could have a directory named "sidebars" in which you store sidebars, in case you want to render different sidebars for different sections on the site.

Here’s an example that shows how you’d render the "header" layout, located inside the "partials" directory inside the "layouts" directory:

<%= render 'partials/header' %>

Nothing really important, but the nanoc_require function is gone—simply use the good old require function now.

nanoc is now bundled with three new filters: rdiscount (a fast implementation of Markdown), maruku (an implementation of a Markdown superset) and erubis (a fast implementation of eRuby). Because the filter used for processing layouts is no longer determined by the file extension, you can actually use Erubis instead of ERB for layouts now.

A lot of helpers have been bundled with nanoc. There’s helpers for blogging, tagging, capturing, HTML-escaping, rendering, building XML sitemaps and linking. All of these helpers are quite thoroughly documented so using them should be a breeze. These helpers will first need to be activated before they can be used (except the Render and HTMLEscape ones), which works like this:

include Nanoc::Helpers::LinkTo

These helpers were previously available as plugins on the wiki, but now they’re actually bundled so you don’t have to download them separately. More helpers (and filters) will be added to the standard nanoc distribution later on; contributions are always welcome.

I forgot to mention it before, but nanoc 2.1 is entirely backwards compatible with nanoc 2.0. This means that a site that can be compiled with nanoc 2.0 (and doesn’t show any deprecation warnings) should compile flawlessly using nanoc 2.1.

So, this is the end of the very last installment of What’s New in nanoc 2.1. I hope you enjoyed it! nanoc 2.1 itself is nearly ready, but still needs a tiny bit of work. With a bit of luck, there’ll be a release at the end of the week. Stay tuned!

WNin #5: Assets (August 9th, 2008)

Welcome to installment #5 of What’s New in nanoc 2.1. This article will show you two of the biggest new features of nanoc 2.1: asset compilation and multiple representations.

The new asset compilation feature allows a site to have assets, which can be run through filters. Assets can be anything—Sass stylesheets, images, MP3s, videos, and so on. Filters can then compile Sass, rescale images, make images grayscale, change the volume of a MP3 file, play a video backwards (so you get to see the end first), etc. The only limit is your imagination.

nanoc supports two different kinds of assets: textual and binary assets. Textual assets will be compiled by textual filters, which are the filters you already know. An example of a textual asset would be a Sass stylesheet. Binary assets are all assets that aren’t text. Examples of binary assets include images, sound and music files, videos, etc.

Binary assets require a new kind of filters called binary filters. Binary filters, unlike textual filters, do not load assets into memory. This means binary assets can be arbitrarily big.

So how does nanoc handle assets? If you are using the filesystem data source, you can create a directory aptly named assets in your site directory, in which you put your assets. Assets are directories containing the actual asset file and a meta file, exactly like pages.

For example, the nanoc site has an asset named "style", whose directory contains "style.css" and "style.yaml". The attributes in the meta file look like this:

binary:  false
filters: []

There are only two built-in asset attributes. binary indicates whether the asset is binary or not (in this case it is a textual asset, thus false) and filters is the list of filters to run (empty, in this case).

If the stylesheet had been a Sass one, the attributes would have looked like this:

binary:  false
filters: [ 'sass' ]

… which should not be a big surprise.

Textual filters are exactly the same as the filters in nanoc 2.0, so there’s probably not much point in showing one, but I’ll just do it anyway:

class Sass < Nanoc::Filter

  identifier :sass

  def run(content)
    require 'sass'

    ::Sass::Engine.new(content).render
  end

end

Binary assets are very similar to textual assets. Simply set binary to true in the meta file and make sure that the list of filters contains only binary filters.

Filters for binary assets are slightly different from textual filters. Firstly, binary filters inherit from Nanoc::BinaryFilter instead of Nanoc::Filter. Secondly, their run method takes a File object and not a string containing the asset content. Additionally, binary filters should return a File object representing the processed file. The original File argument should not be modified (otherwise you’d lose the original asset).

For example, a binary filter for generating a thumbnail of an image (using ImageScience) could look like this (the code is non-functional, because the filter has a lot of irrelevant implementation details):

class ImageScienceThumbnail < Nanoc::BinaryFilter

  identifier :image_science_thumbnail

  def run(file)
    require 'image_science'

    # Get temporary path
    tmp_path = …

    # Create thumbnail and write it to the temp path
    …

    # Return thumbnail file
    File.open(tmp_path)
  end

end

The default router (see the article on routers) saves compiled assets in an assets directory inside the output directory. It’s possible to set assets_prefix in the config file to something else such as "/media", or even "" to save assets directly in the output directory.

Now you know how to compile assets.

There is one small issue, though. Generating thumbnails of images is fine, but what if you want to keep the full-size version? There’s not much point in only having a thumbnail, and not being able to show the full-size image.

This is where multiple representations come into play. A single asset can have different representations (or "reps"): a thumbnail rep, a grayscale rep, a full-size rep, … Even pages can have multiple reps.

Giving an asset multiple representations is quite easy. In the asset’s meta file, fill up the list of reps like in this example:

reps:
  default:
    filters:        []
  thumbnail:
    filters:        [ 'image_science_thumbnail' ]
    thumbnail_size: 100

The asset in this example has two representations: a default one (which is unprocessed) and a thumbnail one (which is run through the image_science_thumbnail filter).

Each representation has its own output file. The name of the output file is determined by the router. The default router appends the representation name to the filename.

Last time I wasn’t able to show you what a real router looks like (because I didn’t explain multiple representations before), but now I can. Let me just show you the implementation for path_for_asset_rep real quick:

def path_for_asset_rep(asset_rep)
  # Get data we need
  extension     = asset_rep.attribute_named(:extension)
  modified_path = asset_rep.asset.path[0..-2]
  version       = asset_rep.attribute_named(:version)

  # Initialize path
  assets_prefix = @site.config[:assets_prefix] || '/assets'
  path = assets_prefix + modified_path

  # Add rep name if necessary
  unless asset_rep.name == :default
    path += '-' + asset_rep.name.to_s
  end

  # Add extension
  path += '.' + extension

  # Done
  path
end

The implementation of path_for_page_rep is very similar, so I’m not showing it here.

The code should be fairly self-explanatory (the [0..-2] is used to strip off the trailing slash). The most important point is that routers determine paths of asset/page representations instead of paths of assets/pages.

So, that’s what asset compilation and multiple representations look like. The end of the What’s New In nanoc 2.1 series is coming closer, but there is still one more installment to go. The last article in the series will discuss several minor new features and changes. Stay tuned!

WNin #4: routers (July 23rd, 2008)

Welcome to the 4th installment of What’s New in nanoc 2.1. This article introduces a new feature in nanoc 2.1 named Routers.

First, a word about paths. Every page has a path starting and ending with a slash. The root and about page’s paths could be / and /about/, for example. A page’s path depends on the way it is stored; when using the filesystem or filesystem_combined data sources, the path is determined by the directory hierarchy in the content directory.

Then there’s a page’s output path or disk path, which includes the terminating "index.html". It is the path where the compiled page will be written to. The web path is the disk path without the trailing "index.html", for pretty URLs. Unless you’re using custom_path to set a custom output path, the web path is the same as the page’s path.

Speaking of pretty URLs: it is now possible to tell nanoc about the list of index filenames you want to have stripped of paths. This is quite useful when you want to generate "default.htm" files instead of "index.html" because you’re running some weird web server. In the site’s configuration file, simply set index_filenames to the list of index filenames to strip. For example:

index_filenames:
  - 'index.htm'
  - 'index.html'
  - 'default.htm'

If you want to generate "default.htm" instead of "index.html" files by default, simply set the filename and extension in the global metadata file. You could already do this with nanoc 2.0, but I’ll repeat it because it’s relevant. Here’s an example of what the global meta.yaml could look like:

# Built-in
filename:  'default'
extension: 'htm'
…

# Custom
…

With the list of index filenames including "default.htm", a page with path /foo/ will now have a web path /foo/ instead of /foo/default.htm. Pretty.

So… routers. Before 2.1, a page’s web path was always the same as the page path. A page with path /about/ would be written to /about/index.html, for example. Routers allow paths to be rewritten; the hierarchy of pages in output can, in fact, be entirely different.

I’d like to show you what a router looks like, but unfortunately that would require me to explain a new feature of nanoc 2.1 called multiple representations, which I’m going to discuss in a future installment. So, what I’m going to do is show fake, non-functional code which should never, ever be used. For illustrative purposes only, mkay?

The default router could, hypothetically, look like this:

class Default < Nanoc::Router

  identifier :default

  def path_for_page(page)
    # Get data we need
    filename   = page.attribute_named(:filename)
    extension  = page.attribute_named(:extension)
    path       = page.path

    # Build path
    path + filename + '.' + extension
  end

end

What this router does, is quite straightforward. When asking the router for the (disk) path for a given page, it simply returns the path with the filename and extension appended to it. Nothing special. (But remember that it’s a fake router and won’t work!)

At this point, routers may not seem very useful yet, but they play a larger role with multiple representations. You’ll see soon. There are a few useful things you can do with routers already, though:

Write a router that changes the paths of journal articles to something like /blog/year/month/day/title, without having to store pages this way (because storing pages hierarchically like this will require a lot of directories, which isn’t too pretty). Simply store articles like /blog/year/title and have a router clean up those paths.
When building a site in multiple languages, it’s best to keep a page’s different languages close together, e.g. /foo/bar/en/ and /foo/bar/fr/. When changing a page in one language, it’s easy to change the page’s other languages because they’re close. With routers it’s easy to rewrite the paths so the language appears at the beginning, e.g. /en/foo/bar/ and /fr/foo/bar/, respectively.

Again, I do realise that not being able to show what a router really looks like (with example code) is annoying, but that’ll all be fixed once I start talking about multiple representations. Don’t worry.

The next installment will discuss perhaps the two biggest new features new in 2.1, asset compilation and multiple representations. Stay tuned!

WNin #3: improved data sources (July 21st, 2008)

This is the third installment of What’s New in nanoc 2.1. This article discusses the new data sources, which have been improved a lot in nanoc 2.1.

To recap: data sources are classes that tell nanoc where to find site content. Even though you’ll usually stick to storing your pages as files on the filesystem, you may want to store the in a database instead, or maybe simply store them in a different way on the filesystem.

For example, one big difference between nanoc and one of its competitors, webby, is that the latter stores its pages as single files, and not directories. Webby stores its metadata embedded inside pages. nanoc 2.1 comes with a new data source named filesystem_combined which, similar to webby, stores content and attributes in a single file. This can clean up the content directory quite a bit.

Here’s what the default nanoc site looks like with a filesystem_combined data source:

-----
# Built-in

# Custom
title: Home
-----

<h1>A Brand New nanoc Site</h1>

… rest of the content goes here …

Creating a new site with this new filesystem_combined data source is easy. Just do this (the -d switch specifies the name of the data source):

% nanoc create_site -d filesystem_combined my_new_site

If you want to switch your existing site over to filesystem_combined, you can do that with a brand new switch command, which you invoke like this:

% nanoc switch -d filesystem_combined

If your site is stored in a VCS repository such as Subversion, git, Mercurial or Bazaar, you can tell nanoc to use this VCS' commands using the -c switch. At the moment, nanoc supports svn, git, hg and bzr, but adding support for new VCSes is trivial. As an example, here’s how you’d switch to switch to new data source using git:

% nanoc switch -d filesystem_combined -c git

If you want to get a list of supported VCSes, use the new info command (which also gives information about available filters, data sources, and more), which outputs something like this:

% nanoc info

VCSes:

  builtin:
    bazaar, bzr         (Nanoc::Extra::VCSes::Bazaar)
    dummy               (Nanoc::Extra::VCSes::Dummy)
    git                 (Nanoc::Extra::VCSes::Git)
    mercurial, hg       (Nanoc::Extra::VCSes::Mercurial)
    subversion, svn     (Nanoc::Extra::VCSes::Subversion)
  custom:
    (none)

… more stuff here …

The original filesystem data source you’re all used to (which is simply called filesystem) has been updated a bit. Layouts are now stored as directories instead of files, and in the new meta file you can specify the name of the filter to use when rendering this layout. The filter used for compiling layouts is no longer dependent on the file extension.

There’s a new update command to make updating the site content (e.g. updating the way layouts are stored) easier. Here’s how you’d use it for a site versioned using Mercurial:

% nanoc update -c hg

The data sources now also allow creating, updating and deleting stored data. This is probably not very useful for the commandline interface—after all, why have delete_page and move_page when there’s the standard unix commands rm and mv, respectively—but for new frontends (GUI frontend, web-based frontend, …) this may become quite useful later on.

A last important new feature is data sources now provide modification times for the data they read, which makes it possible for nanoc to only compile outdated pages, which creates a nice speed improvement.

So, that’s what data sources in nanoc 2.1 look like. Stay tuned for the third installment of What’s New in nanoc 2.1!

WNin #2: an open framework (July 20th, 2008)

Welcome to installment two of What’s New in nanoc 2.1! This installment discusses the technical, code-wise differences between this new version and the old 2.0 release.

The first big improvement is that nanoc is a lot more modular now. The commandline interface (which was discussed before) is now very loosely coupled with the backend. This means that it should be fairly easy to rip out the commandline frontend and replace it with something else—a web frontend (based off Rails or something similar) jumps to mind.

Also important is that nanoc has a public API now, which makes manipulating sites a lot easier. For example, it’s now possible to programmatically generate pages. If you have a blog where all articles are tagged, you can generate tag index pages easily using something like this:

# Load nanoc site
site = Nanoc::Site.new(YAML.load_file('config.yaml'))
site.load_data
pages = site.pages

# Get all tags used on the site
all_tags = pages.map { |p| p.attribute_named(:tags) }.flatten.compact.uniq

# Get tags for which an index page exists
tags_with_pages = pages.map { |p| p.attribute_named(:tag) }

# Get tags for which no index page exists
tags_without_pages = all_tags - tags_with_pages

# Build pages for each tag
tags_without_pages.each do |t|
  # Buid page
  page = Nanoc::Page.new("Tag: #{tag}", { :tag => t }, "/blog/tags/#{t}")
  page.site = site

  # Store page
  page.save
end

If that example above confused you, don’t worry, because…

The nanoc source code is now really, really well-documented. Every class and every method now has thorough documentation. Check out the source documentation over here. It should now be a lot easier to tweak nanoc to your own liking (but be sure to submit patches!).

At least as important as the documentation is the greatly expanded test suite, which now features 319 unit tests and over 1200 assertions. Test coverage is about 85%. There are still a few tests missing, but once they have been implemented, the test suite will be really strong. The goal is to make sure nanoc is virtually bug free, and to make regression bugs a thing of the past.

Just because I like showing off, here’s what the test suite run looks like at the moment:

.......................................................................
.......................................................................
.......................................................................
.......................................................................
...................................
Finished in 9.640952 seconds.

319 tests, 1222 assertions, 0 failures, 0 errors

Sweet, is it not?

In unrelated (but still useful) news, the nanoc repository is now mirrored over at Bitbucket, which is github-like hosting for Mercurial instead of git. There’s also an issue tracker over there, which you can use in case you dislike Trac.

That’s it for installment two. The next installment will briefly describe the changes to the data source API. Stay tuned!

WNin #1: improved CLI tool (July 12th, 2008)

Welcome to the first installment of What’s New in nanoc 2.1! This installment introduces the first new important feature of nanoc 2.1: the improved commandline interface, which is not only a lot prettier but offers a few new features as well.

The first of these new features includes built-in commandline help for every command. For example, the compile command has a help page that looks like this:

nanoc compile [options] [path]

compile pages and assets of this site

    Compile all pages and all assets of the current site. If a path is given,
    only the page or asset with the given path will be compiled. Additionally,
    only pages and assets that are outdated will be compiled, unless specified
    otherwise with the -a option.

options:

    -a --all        compile all pages and assets, even those that aren’t
                    outdated

As you can see, the compile command has grown up a bit and it’s now possible to tell nanoc only to compile a single page instead of the entire site. Note that nanoc also no longer compiles pages unless they’re considered "outdated"—a real time saver.

Also neat is that command abbreviations are detected automatically. The co command, for example, will be expanded to compile because there’s no other command starting with co. Additionally, if you type something ambiguous, you’ll be notified:

% nanoc c
nanoc: 'c' is ambiguous:
  create_template create_site create_page create_layout compile

Error reporting has also been improved. If a compilation error occurs at some point, the stack of what was being compiled will be printed. The following example shows that something went wrong when compiling the "meta" layout, which was rendered from inside the "journal_archive" layout, which in turn was rendered from inside the "journal/2008" page:

Compilation stack:
  - [layout] /meta/
  - [layout] /journal_archive/
  - [page]   /journal/2008/ (rep default)

Of course, the actual error message and the Ruby stack trace will be printed as well, which should make debugging any accidental errors on your site quite a bit easier. (Ignore the "rep" thing for now. We’ll talk about that later.)

When passing the --verbose switch to the compile command, a pretty table containing some profiling information will be printed. Very useful if you want to optimize some of your custom filters. For example, this is the output for my personal web site:

                        | count    min    avg    max     tot
------------------------+-----------------------------------
                    erb |    95  0.00s  0.01s  0.23s   0.74s
image_science_thumbnail |    69  0.22s  0.27s  0.35s  18.51s
              bluecloth |     2  0.06s  0.10s  0.15s   0.21s
      website_thumbnail |    12  0.25s  0.48s  0.97s   5.80s
            compile_css |     2  0.00s  0.00s  0.00s   0.00s
              rubypants |    53  0.01s  0.03s  0.09s   1.39s
              rdiscount |    11  0.00s  0.00s  0.01s   0.02s

This shows that the "compile_css" as well as "rdiscount" filters are extremely fast, while the "image_science_thumbnail" and especially the "website_thumbnail" filters are really slow. (Those thumbnail filters are filters that apply to image assets, not to pages—asset compilation is something that’ll be discussed later).

Last but not least: it’s now possible to write your own commands. Even though rake is sufficient for performing simple tasks, sometimes you need a more advanced tool that allows passing commandline arguments. Now you can write custom commands. Here’s a "Hello World" example:

require 'nanoc/cli'

class HelloWorldCommand < Nanoc::CLI::Command

  def name
    'hello_world'
  end

  def aliases
    %w( hw )
  end

  def short_desc
    'print a simple greeting'
  end

  def long_desc
    'Print the "hello world" greeting on the screen, in either ' +
    'lowercase or uppercase. Can also print a custom text.'
  end

  def usage
    "nanoc hello_world [options] [custom_text]"
  end

  def option_definitions
    [
      # --uppercase
      {
        :long => 'uppercase', :short => 'u', :argument => :forbidden,
        :desc => 'print the text in uppercase instead of lowercase'
      }
    ]
  end

  def run(options, arguments)
    # Get text
    text = (arguments[0] || 'hello world')

    # Print it
    puts (options.has_key?(:uppercase) ? text.upcase : text.downcase)
  end

end

Some example output:

% nanoc hello_world blah
blah
% nanoc hw -u blah
BLAH
% nanoc hello -u
HELLO WORLD
% nanoc hw
hello world

The auto-generated help page:

nanoc hello_world [options] [custom_text]

aliases: hw

print a simple greeting

    Print the "hello world" greeting on the screen, in either lowercase or
    uppercase. Can also print a custom text.

options:

    -u --uppercase  print the text in uppercase instead of lowercase

That’s it for now. Stay tuned for the second installment of What’s New in nanoc 2.1, which will discuss changes to the internals of nanoc and their implications.

nanoc 2.1: coming soon! (July 10th, 2008)

Good news! nanoc 2.1 is (almost) feature-complete. Testing and documentation still need a bit more work, but it’s well under way. My summer job is slowing down development a bit, so it’ll take a few weeks before nanoc 2.1 is released.

Don’t worry though! Today is the start of a series of articles describing nanoc 2.1 exciting new features. I’m calling this appetite-whetting series "What’s New in nanoc 2.1" (which is not a ripoff at all—honestly!). It’s going to be so cool that I simply had to craft a shiny logo:

Pretty. Maybe I should consider an art career. Anyway… the first installment is probably going to be published this weekend, depending on how much time I have. Stay tuned!