Metaprogramming is just programming (Ruby)

Metaprgramming in ruby is not something we can escape if we want to write a really powerful code. In fact in ruby metaprogramming is everywhere and sometimes we use it without even know it.

If we are a ruby web developer, ActiveRecord is a great implemetation of metaprogramming. As a good coder, we also need to write test, there too, metaprogramming plays an important role. When we need to dynamically create a method we also need metaprogramming. One more thing is block, we cannot create a concise code without using blocks. In this post I will try to show that in Ruby metaprogramming is omnipresents, one cannot become a expert in Ruby with knowing its metaprogramming.

1.What is “metaprogramming”?

Metaprogramming is best explained as programming of programming. Don’t let this abstract definition scare you away though, because Ruby makes metaprogramming as easy to understand as it is to work with.

Metaprogramming can be used a way to add, edit or modify the code of your program while it’s running. Using it, you can make new or delete existing methods on objects, reopen or modify existing classes, catch methods that don’t exist, and avoid repetitious coding to keep your program DRY.

2. Closure

Closure is something we encounter everyday but whose concepts are rarely clarified. There are 3 spots in ruby where scope will shift (these are properly dubbed Scope Gates):

• Class definitions
• Module definitions
• Methods

That makes something like this impossible:

my_var = "Success"
class MyClass
    # We want to print my_var here...

    def my_method
        # ..and here
    end
end

But with metaprogramming, we can bend scope to our will and make this happen. Before we do that though, we need to discuss the two ways that you can define a class in ruby.

Defining a Class – Statically

This is the way that we’re all familiar with:

the normal way

class Book
    def title
        "Ruby Metaprogramming"
    end
end

puts Book.new.title # => Ruby Metaprogramming

There’s nothing new going on here. But did you know we can also define a class at runtime? If you’re unsure what I mean, check out this next example.

Defining a Class – Dynamically

Book = Class.new do
    def foo
        "foo!"
    end

    #Both method declaration types work

    define_method('title') do
        "Ruby Metaprogramming"
    end
end
puts Book.new.foo # => foo!
puts Book.new.title # => Ruby Metaprogramming

This is an alternative way to define a class in Ruby. If you remember the object model we discussed in the previous post, you’ll recall that a class in ruby is also an object, and it has a class of Class. It was confusing to think about then, but here is an example of that. If we flip that phrase around, then that means if we instantiate an object out of class Class, then that object is also a class. That’s exactly what we’re doing here! We’re creating a class by calling Class.new – all at runtime!

This allows us to pass scope gates and access my_var inside of the class declaration. From there, you have two ways to define methods. You can define methods the classical way such as how we defined the foo method – but that’s still a scope gate. You can’t access my_var inside that method. If you want to access my_var inside of your method, you’ll need to dynamically define a method – just like how we defined the title method.

You can see a full example of this as we return to our previous discussion on scope:

my_var = "Success"

MyClass = Class.new do
    "#{my_var} in the class definition"

    # Have to use dynamic method creation to access my_var
    define_method :my_method do
        "#{my_var} in the method"
    end
end

puts MyClass.new.my_method # => Success in the method

This seemingly “scopeless” process is called a Flat Scope. Whether you use this concept or not is up to you, but ruby provides you with the tools to let you make that choice.

3.Lambdas, Blocks

What follows next are procs and lambdas, something that often aren’t fully understood. Most things in Ruby are objects. Blocks are not. To pass them around, you use the & operator.

def my_method(coding)
    "#{coding} in #{yield}!"
end

my_proc = proc { "Ruby" }
puts my_method("Coding", &my_proc) # => Coding in Ruby

I defined a block in one scope, and passed it to a method using the & operator – where it was then yielded. Let’s move on now to procs and lambdas.

Lambdas throw an ArgumentError if the argument count doesn’t match when you call them. Procs do not. Lambdas return in a local scope, whereas Procs return from the scope in which they were called. Let’s do some examples to illustrate exactly how these two block types work.

Starting off with a lambda example:

def lambda_example
  l      = lambda {|a,b| return (a + b) }
  result = l.call(2, 5) * 10
  return result
end

puts lambda_example # => 70

This executes about how we would expect. In the lambda_example method, we define a lambda block which just accepts two arguments and multiplies them together.

If we called the lambda with any more or less arguments than two, it would fail to execute and give us an ArgumentError.

Let’s move on to procs and its difference from lambda.

def proc_example
    p      = proc {|a,b| return (a+b) }
  result = p.call(2, 5) * 10
  return result
end

puts proc_example # => 7

Yes, it’s 8, and the reason why is because of how procs return in the scope they were called in. Whenever you return inside of a proc, the return statement doesn’t execute in the scope inside of that block – it executes where you initially call the proc, which in this example is on line 3. Therefore, whenever we call the proc and return the value 7, the proc forces the entire method to return with that value instead of continuing on with the rest of the code. In fact, line 4 never even gets called because the method has already returned at that point because of the proc. Now let’s move on to Closure.

4. ActiveRecord

Why would you want to dynamically define methods? Maybe to reduce code duplication, or to add cool functionality. ActiveRecord (the default ORM tool for Rails projects) uses it heavily. Check this example out.

class Book < ActiveRecord::Base
end

b = Book.new
b.title

If you’re familiar with ActiveRecord, then this looks like nothing out of the ordinary. Even though we don’t define the title attribute in the Book class, we assume that Book is an ORM wrapper around a Book database table, and that title is an attribute in that table. Thus, we return the title column for that particular database row that b represents.

Normally, calling title on this class should error with a NoMethodError – but ActiveRecord dynamically adds methods just like we’re about to do. The ActiveRecord code base is a prime example of how you can use metaprogramming to the max.

Let’s try this out and create our own methods:

Dynamically defining the methods

def foo
    puts "foo was called"
end

def baz
    puts "baz was called"
end

def bar
    puts "bar was called"
end

foo # => foo was called
baz # => baz was called
bar # => bar was called

See the duplication? Let’s fix that with metaprogramming.

%w(foo baz bar).each do |s|
    define_method(s) do
        puts "#{s} was called"
    end
end

foo # => foo was called
baz # => baz was called
bar # => bar was called

What we’re doing here is dynamically defining the methods foo, baz, and bar, and then we can call them

Dynamically Calling Methods

Dynamically calling methods or attributes is a form of reflection, and is something many languages can do. Here’s an example of how to call a method by either the string or symbol name of that method in ruby:

%w(book1 book2 book3).each do |s|
    define_method(s) do
        puts "#{s} was read"
    end
end

(1..3).each { |n| send("book#{n}") }

# => book1 was read
# => book2 was read
# => book3 was read

Because every object in Ruby inherits from Object, you can also call send as a method on any object to access one of its other methods or attributes – like this:

Ghost Methods

What happens if we try to execute this code?

class Book
end

b = Book.new
b.read

We would get a NoMethodError, because Book doesn’t know how to handle the method read. But it doesn’t have to be that way. Let’s explore method_missing.

class Book
    def method_missing(method, *args, &block)
        puts "You called: #{method}(#{args.join(', ')})"
        puts "(You also passed it a block)" if block_given?
    end
end

b = Book.new

b.read
b.read('a', 'b') { "foo" }

# => You called: read()
# => You called read(a, b)
# => (You also passed it a block)

BasicObject#method_missing provides you an option to build a handler that will automatically get called in the event of a NoMethodError – but before that error ever happens. You are then given as parameters the method name that you tried to call, its arguments, and its block. From there, you can do anything you want.

5.Evals

Putting metaprogramming to work with instance_eval and class_eval Having singleton classes is all good and well, but to truly work with objects dynamically you need to be able to re-open them at runtime within other functions. Unfortunately, Ruby does not syntactically allow you to have any class statements within a function. This is where instance_eval comes into the picture.

instance_eva

The instance_eval method is defined in Ruby’s standard Kernel module and allows you to add instance methods to an object just like our singleton class syntax.

foo = "bar"
foo.instance_eval do
  def hi
    "you smell"
  end
end

foo.hi # => "you smell"

The instance_eval method can take a block (which has self set to that of the object you’re operating on), or a string of code to be evaluated. Inside the block, you can define new methods as if you were writing a class, and these will be added to the singleton class of the object.

Methods defined by instance_eval will be instance methods. It’s important to note that the scope is that of instance methods, because it means you can’t do things like attr_accessor as a result. If you find yourself needing this, you’ll want to operate on the Class of the object instead using class_eval:

class_eval

bar = "foo"
bar.class.class_eval do
  def hello
    "i can smell you from here"
  end
end

bar.hello # => "i can smell you from here"

As you can see, instance_eval and class_eval are very similar, but their scope and application differs ever so slightly. You can remember what to use in each situation by remembering that you use instance_eval to make instance methods, and use class_eval to make class methods.

eval

We can now move on to the final eval function – just plain eval. This function is drop-dead simple to understand, but it’s extremely powerful – and very dangerous. All eval does is accept a single argument – a string – and run it as ruby code at the same point in runtime as when the eval method is called. Here’s a very basic example where we just use eval to append a value to an array:

array   = [10, 20]
element = 30

eval("array << element")
puts array # => [10, 20, 30]

We never actually run the ruby code ourselves to append the element variable to array, we just tell eval to do it by passing it the ruby code as a string. We don’t gain any benefit here by using eval, but take a look at this deeper example to begin to see powerful it can be:

klass = "Book"
instance_var = "title"

eval <<-CODE # This is just a multi-line string
    class #{klass}
        attr_accessor :#{instance_var}

        def initialize(x)
            self.#{instance_var} = x
        end
    end
CODE

b = Book.new("Ruby Metaprogramming")
puts b.title # => Ruby Metaprogramming

If you’re unfamiliar with the syntax after the eval method call, that’s just a multiline string in ruby. In this example, we have 2 local variables in scope when we call eval, and we are using those variables to open up a class, create an attr_accessor, and write a constructor. But we’re doing it all by embedding variables into our multiline string. This executes as valid ruby at runtime, but this would never, ever be valid ruby code that we could write without the use of eval.

Good. Now we can talk about how dangerous eval is.

Here’s the code:

def explore_string(method)
    code = "'abc'.#{method}"
    puts "Evaluating: #{code}"
    puts eval code
end

explore_string("capitalize") # => Abc

Nothing bad happened, in fact, it returned exactly what I wanted it to – the index of value c, which is 2. But watch what happens if we call explore_string with a different argument:

explore_string("object_id; Dir.glob('*')")

#=> code_literally.rb
#=> fibonacci.rb
#=> fibonacci_bottom_up
...
#=>lis_memoied.py

Woah, what is all that? Yup, by looking at the code, you guessed it. That’s a listing of all the files and subdirectories inside of the main directory that’s running this app. That’s bad – really bad. Using eval often times makes you susceptible to code injection, which is why you have to absolutely be sure you know what you’re doing when you use it.

6. Singleton Classes

Ruby gives you the full power of Object Oriented programming and allows you to create objects that inherit from other classes and call their methods; but what if only a single object requires an addition, alteration, or deletion?

The “singleton class” (sometimes known as the “Eigenclass”), is designed exactly for this and allows you to do all that and more. A simple example is in order:

greeting = "Hello World"

def greeting.greet
  self
end

greeting.greet # => "Hello World"

Let’s digest what’s just happened here line by line. On the first line, we create a new variable called greeting that represents a simple String value. On the second line, we create a new method called greeting.greet, and give it a very simple content. Ruby allows you to choose what object to attach a method definition to by using the format some_object.method_name, which you may recognize as the same syntax for adding class methods to classes (ie. def self.something).

As you’re about to see, adding methods using some_object.method_name is not always the best way to do such tasks. That’s why Ruby provides another far more useful way of working with objects and their methods in a dynamic way. Meet the Singleton class:

greeting = "i like metaprogramming"

class << greeting
  def greet
    "hello! " + self
  end
end

greeting.greet # => "hello! i like metaprogramming"

This syntax also allows you to add anything you would normally add during the declaration of a class, including attr_writer, attr_reader, and attr_accessor methods.

How does it work?

So how does it actually work? The name “singleton class” might have given this away slightly, but Ruby is sneaky and adds another class into our inheritance chain. When you try to operate on the singleton class, Ruby needs a way to add methods to the object we’re adding to, something that the language does not allow. To get around this it creates a secret new class, which we call the “singleton class”. This class is given the methods and changes instead, is made the parent of the object we’re working on. This singleton class is also made an instance of the previous parent of our object so that the inheritance chain remains mostly unchanged:

Ask the object if its singleton class can respond to the method, calling it if found. Ask the object’s parent class if it can respond to the method, calling it if found. Ask the next parent class up if it can respond to the method and call it if found, continuing this step towards the top of the inheritance chain for as long as necessary. If nothing in the inheritance chain can respond to the method being called, the method does not exist and an Exception should be raised.

7. Testing

Mocking objects for testing Some of the most useful features of Ruby’s metaprogramming have been shown off countless times in the vast array of testing frameworks available. Whilst a lot of these frameworks use various forms of metaprogramming to create their APIs, we’re going to explore one of the most important uses of metaprogramming used in Ruby test frameworks: metaprogramming for mocking and stubbing.

You can probably start to draw conclusions about how this might work given what you now know about metaprogramming. Let’s get started with a nice, simple example:

class Item
end

class Shop
end

Here we havebthe Item class, and the Shop class.

class Shop
  attr_accessor :item

  def initialize
    @items = []
  end
end

class Shop
  # ...

  def items_in_stock
    @items.length
  end

  # ...
end

Finally, We want to check if there is enough items in stocks, so that we can import more

class Borrower
  # ...

  def import_more?
    @items.length < 10
  end

  # ...
end

We want to write some test to make sure our code is correct. You could achieve this by adding some ites to your shop before you test the actual method, but this way of testing becomes painstakingly verbose and will fail if there are any problems with the Book class when creating these instances. Here’s where metaprogramming comes to the rescue:

describe Shop do
  before :each do
    @shop = Shop.new
  end

  describe "import_more? performs correctly" do
    it "returns false if equal to or over the limit" do
      @shop.items.instance_eval do
        def length
          20
        end
      end
      @shop.import_more?.should == false
    end

    it "returns true if under the limit" do
      @shop.items.instance_eval do
        def length
          5
        end
      end
      @shop.import_more?.should == true
    end
  end
end

You can begin to imagine the power of this methodology when you realise that you can stub dates and times, IO activity, database records, external API calls; the list goes on.

So the next time you find yourself creating a lot of external dependencies or relying on additional classes in your tests, consider adding a sprinkle of metaprogramming into the mix.

By now you’ve probably realised the power of what we’ve created. To make these methods by hand would be either extremely laborious or difficult to maintain, or impossible. we’ve created an extremely expressive and beautiful API that is DRY and easy to maintain.

8.Conclusion

In order to get the most out of Ruby we have to know its metaprogramming. It is not something obscure and used only in some special place, but instead used everywhere and everyday.