Mixing Our Way Out Of Instance Eval?

October 6th 17:36
by why

The lynchpin of Ruby’s pidgins and so-called DSLs (Douchebaggery as a Second Language) is the method known as instance_eval.

From an article titled Ruby DSL Blocks:

 def self.order(&block)
   order = Order.new
   order.instance_eval(&block)
   return order.drinks
 end

And another one Implementing an internal DSL in Ruby:

 def Expectations(&block)
   Expectations::Suite.instance.instance_eval(&block)
 end

From Creating DSLs with Ruby:

 class MyDSL
   def define_parameters
     yield self
   end

   def self.load(filename)
     dsl = new
     dsl.instance_eval(File.read(filename), filename)
     dsl
   end
 end#class MyDSL

So far, so good? Most often instance_eval is used, but you’ll see module_eval, too.


Now, the reason for this.

Jim Weirich: Within the builder code blocks, any method call with an implicit object target needs to be sent to our builder. To achieve this, the code blocks are evaluated with instance_eval which changes the value of self to be the builder.

He goes on to say why this could be troubling.

This is OK until you want to call a method in the calling object. Since self is no longer the calling object, you have to explicitly provide the caller.

In Weirich’s Builder, he prefers to use plain blocks and hand you the variable.

 builder { |xm|
   xm.em("emphasized")
   xm.em { xm.b("emp & bold") }
   xm.a("A Link", "href" => "http://onestepback.org")
   xm.div { xm.br }
   xm.target("name" => "compile", "option" => "fast")
   xm.instruct!
   xm.html {
     xm.head {
       xm.title("History")
     }
     xm.body {
       xm.comment! "HI"
       xm.h1("Header")
       xm.p("paragraph")
     }
   }
 }

With instance_eval, you’d end up with:

 builder {
   em("emphasized")
   em { b("emp & bold") }
   a("A Link", "href" => "http://onestepback.org")
   div { br }
   target("name" => "compile", "option" => "fast")
   instruct!
   html {
     head {
       title("History")
     }
     body {
       comment! "HI"
       h1("Header")
       p("paragraph")
     }
   }
 }

Builder once did use instance_eval, but now offers this explanation in the docs:

The instance_eval implementation which forces self to refer to the message receiver as self is now obsolete. We now use normal block calls to execute the markup block. This means that all markup methods must now be explicitly send to the xml builder.

Rails’ routing stuff also prefers to go without instance_eval:

 ActionController::Routing::Routes.draw do |map|
   map.with_options :controller => 'blog' do |blog|
     blog.show '',  :action => 'list'
   end
   map.connect ':controller/:action/:view'
 end

By now, it probably seems like instance_eval has been driven out of play by the mature libs and now only lingers in the dabbling blog posts.


I don’t care about the best way to do this. The fact is: even “normal blocks” are prone to bugs. I had to fix up the Builder example above because it’s wrong in the docs.

 xm.em("emphasized")             # => emphasized
 xm.em { xmm.b("emp & bold") }   # => emph & bold
 xm.a("A Link", "href"=>"http://onestepback.org")
                                 # => A Link
 xm.div { br }                    # => 

There are two errors in the above example. The date on the RDoc is Sun Feb 05 23:49:01 EST 2006.


Recently I discovered another option to all of this, thanks to the work of Guy Decoux.

While investigating a library Guy never released (called prop,) I realized that perhaps he was on to something while fooling with mixins and the inheritance chain.

Think about it: instance_eval changes self, intercepts method calls and alters instance variables. Really, all we really want to do is dispatch method calls. As Jim Weirich says, changing self is the troubling side effect here.

This is a pertinent topic. Even today, ruby-core discusses a with operator and Paul Brannan chimes in:

Instance_eval for initialization has surprising behavior for instance variables (e.g. as in Ruby/Tk).

A method that affects only method calls and not instance variables would make this idiom more viable.

I don’t know whether that is a good thing or a bad thing.

Again with the good and bad.

What if there was a way to temporarily add methods for the duration of a block?

 def to_html
    Builder.capture do
      html do
       head do
         title self.friendly_title
       end
       body do
         comment! "HI"
         h1("Header")
         p("paragraph")
       end
     end
   end
 end

The essence of Builder.capture is to mixin a bunch of Builder methods, inject them into the block’s binding. This adds the html, head, comment! methods into the calling self.

Using a very small extension called mixico:

 def Builder.capture &blk
   mix_eval(self, &blk)
 end

This extension enables and disables mixins atomically. It is a single, quick operation to add and remove a module from the inheritance chain. (See the mixico README for more on this technique.)


The mix_eval method code looks like this:

 class Module
   def mix_eval mod, &blk
     blk.mixin mod
     blk.call
     blk.mixout mod
   end
 end

The mixin and mixout methods enable and disable the Module in the block’s binding.

While you might be inclined to dismiss this on the grounds of it being mixin magic, I’m starting to believe that this is a notable omission from Ruby. It’s very quick and efficient to disable and enable mixins and could prove to be a very handy technique.

Like open classes, however, I’m afraid the timidity of the business community might label it as taboo, despite it offering great flexibility to you — all of my fine, able-minded friends out there.

Now begin the comments …

25 comments

lemon

said on October 6th 12:59

mix_eval is the kind of thing I’d use all the time were it part of ruby core.

(by the way, is markaby is getting a makeover any time soon?)

]\)et(,host

said on October 6th 13:17

Looks really neat, this could help clear up some of the weirdness you get with DSLs. Thankya!

Austin

said on October 6th 13:44

This seems like it could also be an answer to the much-asked-for selector namespaces.

Jim

said on October 6th 13:54

How does this relate: http://www.somethingnimble.com/bliki/mixology

_why

said on October 6th 14:18

Jim: It looks like mixico’s disable_mixin is identical to mixology’s unmix in the C extension. It just reverses what’s done in Ruby’s rb_include_module.

nakajima

said on October 6th 14:42

How wonderful! I had been fooling around with this sort of approach, except I was calling remove_method for each of a module’s methods. This is much cooler.

Jim Weirich

said on October 6th 16:11

How well does this work in the presence of threads?

Jim Weirich

said on October 6th 16:15

Hmmm … Actually, I’m thinking that this won’t work with builder because the builder object needs to remove methods (in order for method_missing to kick in) rather than add methods via a mixin. However, it might be useful in other contexts.

Jim Weirich

said on October 6th 16:18

I also experimented with another solution to this problem. Instance eval the block in a director object that will try the method on both the builder and the containing object. Hopefully the director object can choose the right object (probably by giving proirity to one or the other). I decided against this because the solution seemed unnecessarily complex.

_why

said on October 6th 16:31

Jim Weirich: I actually think that further work on this should focus on masking methods so that the mixin could capture everything except the stuff that’s in BlankSlate or 1.9’s BasicObject.

This exercise may not have any answers in this incarnation. The specific idea is more about investigating the greys between the two extremes of instance_eval and regular blocks, in the hopes of leading us to something genuinely useful SLASH futuristic!

charly

said on October 6th 17:29

hi,
i totally agree, mixins have i great unexplored potential, we just should think more architecture than magic. I ran in this “unconscious self” problem as well and used a similar “uninclude” lib in : Refactoring rails exercise

Zing

said on October 6th 19:15

If you get an exception in blk.call, it appears that you need to ensure the mixout is performed…

Dan42

said on October 6th 19:25

I am so going to start using mixico in my projects. I think this will become very important with ruby1.9, because now instance_eval impacts the lookup of constants in addition to instance variables.

Martin Dürst

said on October 6th 20:56

I have been (and still am) heavily using instance_eval in SVuGy. I have been trying to understand where exactly the problems are, and why e.g. Jim Weirich has abandoned it in Builder. The explanation provided (“This is OK until you want to call a method in the calling object. Since self is no longer the calling object, you have to explicitly provide the caller.”) didn’t really cut it for me.

All the examples use tons of methods on the Builder objects, and
virtually none on the calling object. Actually, as long as you already have a local variable outside the Builder blocks, things should be fine. If you have to create a local variable because all you have is an instance variable, it’s more of a hassle, but I currently still think that it’s preferable to sprikling around tons of xm..

But having a mixin on/off system gives you the best of two worlds, so why not? Actually, there’s one more advantage: You don’t really want your users to access the instance variables of your implementation internals. And making the client’s instance variables available will hide the implementation-internal instance variables, if I understand correctly.

This is too late for 1.9, but I hope it will go into 2.0, in one shape or another.

Robert Klemme

said on October 7th 02:22

IMHO the method has two issues:

  1. return value is lost
  2. no exception safety

So, I’d rather do

 class Module
   def mix_eval mod, &blk
     blk.mixin mod
     begin
       blk.call
     ensure
       blk.mixout mod
     end
   end
 end

I also agree with Jim’s concern about thread safety.

_why

said on October 7th 09:32

Robert Klemme: Hey, alright. Thank you for that fix, it is applied.

As for thread-safety, tell me your concerns. Mixin just uses extend and mixout is a single pointer assignment, totally atomic.

Pit Capitain

said on October 7th 10:07

While I would love to be able to uninclude modules or otherwise change the inheritance chain, I don’t think this feature is necessary for the examples shown. Here’s a simple way to implement the Builder example (note the local variable "myself")

  def to_html
    myself = self
    Builder.capture do
      html do
        head do
          title myself.friendly_title
        end
        body do
          comment! "HI"
          h1("Header")
          p("paragraph")
        end
      end
    end
  end

And Builder.capture can be almost as simple as

 def Builder.capture &blk
   instance_eval(&blk)
 end

No need to guess who is self, no need to change the inheritance chain, no problems with accidentally overriding methods, …

Tammer Saleh

said on October 7th 10:13

_why: I think the worries with thread safety are about a thread seeing a class with extended behavior because it interrupted another thread that temporarily extended that class.

_why

said on October 7th 10:31

Tammer Saleh: Look at mix_eval again. In this case, we’re mixing into objects rather than classes (using extend.) This could still be cause for concern about thread-safety (eventually it might be nice to mixin for only a given scope,) but the concern should be greatly diminished since we’re not effecting the global set of classes, only a single object.

Ola Bini

said on October 8th 01:54

_why: of course, pointer assignment is only atomic on a 32bit machine… =)

raggi

said on October 8th 07:31

This looks like a good deal of fun :-)

Maybe I’ll try my hand at some meta-fu, if you’ll have me for just a moment…


def MethodsOverride(obj_or_methods, &global_method)
  case obj_or_methods.size
  when 1
    meths = obj.instance_methods
  else
    meths = obj_or_methods
  end

  Module.new do
    define_method(:__global_method, &global_method)
    instance_methods.each { |m| undef m unless m =~ /^__/ }

    meths.each do |m|
      define_method(m) do |*args|
        __global_method(*args)
      end
    end
  end
end

Now inject that in your metaclass! :O

Hopefully I’ll have some time later to play with more of an example…

raggi

said on October 8th 07:34

oh drat, well I did forget the odd * and refactor, but you should get the idea, outside the busted arguments!

dok_pen

said on October 9th 08:09

myself, self = self, myself

lol

MenTaLguY

said on November 5th 10:47

Also, atomicity is only the thin end of the pencil when it comes to thread safety. You basically have to completely lock a shared object upon which mixico is in use, because otherwise other threads will be able to see the mixicoed-in methods. Also there are difficulties with frozen objects…

Which is a shame, because the thing that mixico is here being invoked to address really has more to do with method lookup in local execution contexts, something which shouldn’t have any thread issues at all.

banisterfiend

said on November 14th 22:08

@ MenTalGuy: here’s one way to make mixico thread-safe http://coderrr.wordpress.com/2008/11/14/making-mixico-thread-safe/

also it’s possible to make mix_eval a mix_exec (analogous to instance_exec) by calling/yielding to the block with parameters:

class Module
def mix_exec mod, *args, &blk
blk.mixin mod
begin
yield *args
ensure
blk.mixout mod
end
end

alias_method :mix_eval, :mix_exec

end

Comments are closed for this entry.