Boston, San Francisco and SDD

My colleague Jim Lindley and I will be going to Boston for a few days for our employer. We do TDD/BDD with RSpec and Rails currently, and the folks in the Boston office are at some point along a similar continuum. We'll show them what we do, they'll show us what they do, we'll learn and grow and laugh and share and love, etc. Then I'm off to San Francisco for some interviews related to my planned relocation out there.

Yesterday, I came across a blog post by Paul Barry about SDD with Rails. The two upcoming trips already had me thinking about my workflow, and Jim and I had been talking about trying move another abstraction step up from our current speccing practice, and SDD seems like a good match for our next project. It's a rewrite of an existing app that is currently in a different language - our job would be to match it feature-for-feature and also to make some necessary changes and additions.

SDD seems like a good way to keep the QA folks who already know how the app should work more involved in the the speccing process in a way that actually has impact. That's the beauty of having the human-readable descriptions be executable.

Paul also links to Bryan Helmkamp's talk on SDD at GoRuCo2008, which is very good. I would encourage anyone to read Paul's post and watch Bryan's talk.

Celebrity sighting(?)

My last post was about Perl and my experiences working with it and its fans. I'm not a huge fan of the language - I'm planning another post that details what I like and don't like about it. I think it has few characteristics that are fundamentally flawed, for example.

However, I'm pretty sure I saw Larry Wall in the Detroit airport a few weekends ago when I was flying to Seattle for a friend's wedding. The excitement that I heard in my own voice when I pointed him out to my wife Jenn made me realize that I think Perl's good points have been a positive influence on programming culture.

So thanks, Larry.

Perl and is_valid_aref()

This post is about Perl, the politics of interacting with fellow developers, and Duck Typing (although we didn't really know it at the time). It's also fairly stream-of-consciousness.

I used to work at a Perl shop, and in a big rewrite we did, we made heavy use of array references. A dynamic lanugage like Perl (or Ruby, or whatever) will encourage conscientious developers to have some tests and/or error checking, so we often wanted to ensure that our input parameters to some specific subroutines that expected array refs were actually valid arefs. How should one do this checking in Perl?

Here's one obvious approach.

sub is_valid_aref_mere_mortals {
  ref shift eq 'ARRAY';
}

Here we have a straightforward subroutine. We use Perl's shift to pull the first parameter off the stack, get the type to which it refers with ref, and check with the stringy equality operator whether or not the referred type is 'ARRAY'.

This wasn't good enough for an ex co-worker of mine, who fancied himself a Perl Wizard (with some justification, in fairness to him).

Some idiot could bless a non-ARRAY behaving scalar as a ref to an ARRAY. My solution in that case is to berate the offender in public for having done something so horrible. Such a blessing is a direct violation of the duck typing idea that you don't have a datum pretend to be a specific type (however your language defines type) unless it can implement all the pertinent behavior expected of that type in that context. With great power comes great responsibility.

We also had no instances of such blessing in our app. All of our args were all either just simple arefs, or errors (usually an undef value). So in practice, I think a simple ref check would work fine. However, if you're concerned about the blessed as aref issue, the solution below has some advantages.

sub is_valid_aref {
  my( $arg ) = @_; 
  $arg and eval { @{ $arg } or 1 };
}

It's a bit tricky, so I'll explain it. You extract the first parameter in the stack as $arg. The and keyword assures that $arg is truthy before we proceed to the eval. The eval attempts to dereference $arg into a regular (non-reference) array. Rubyists can think of this use of eval in Perl as akin to a begin rescue end block.

If the dereferencing results in an error, it will stop evaluation of the eval block, resulting in an undef value, meaning that the expression returned by the function is false.

Why is there an or 1 at the end of the eval block? That's for situation in which your $arg is an empty array ref. An array of zero length evaluated in truthy scalar context in Perl is false. However, we want such a data structure to produce a truthy return value from this function. Therefore, when a valid but empty array ref is dereferenced into an actual array, it is false but does not break out of the eval block. The expression with the eval block then continues with the or 1, which ends up being truthy, ensuring that a valid empty array ref is considered valid by this function.

I lie when I say this was my ex co-worker's solution. He strongly objected to putting this test inside a function, because developers should just know Perl idioms, and the function call would add too much overhead. I thought instead that the value of naming this obscure bit of code with what its purpose is would be worthwhile. He disagreed. So he would have tests like the line of code below copied and pasted with no explanatory comments wherever we needed to check aref validity:

fail if not $arg and eval { @{ $arg } or 1 };

I saw that for the first time and thought WTF? I then added a comment explaining what that particular monstrosity was supposed to do, something like 'checks if a valid aref'. He objected strongly (again), saying that coders should just know Perl idioms.

We finally got him to accept the presence of the comment after about 30 minutes of arguing.

Try them out for yourself, if you like. I use $proc as a generic name for a procedure/function/subroutine/code reference. Some other people prefer $cref for code ref or $sref for subroutine ref.

my( %proc_of ) = ( 
  'mere mortals' => \&is_valid_aref_mere_mortals,
  'eval version' => \&is_valid_aref
);

sub report {
  my( $label, $candidate, $proc_of_href ) = @_; 
  for $proc_name ( keys %{ $proc_of_href } ) { 
    my( $proc ) = $proc_of_href->{$proc_name};
    print $proc->( $candidate ) 
      ? "$label is a valid aref according to $proc_name\n"
      : "$label is not a valid aref according to $proc_name\n";
  }
}

report( '[]', [], \%proc_of );
report( '()', (), \%proc_of );
report( '{}', {}, \%proc_of );
report( '(undef)', (undef), \%proc_of );

Results in this output. Note that I haven't included the deceptive blessing situation, on which the two subs would differ:

[] is a valid aref according to mere mortals
[] is a valid aref according to eval version
{} is not a valid aref according to mere mortals
{} is not a valid aref according to eval version
(undef) is not a valid aref according to mere mortals
(undef) is not a valid aref according to eval version

It seems to me that the clarity improvement from having the name is_valid_aref for that code is worth the relatively small overhead of the function call. I also think that it serves a pedagogical use - a new developer who isn't familiar with the blessing concern could in fact be educated about it by seeing this code under such an explanatory label. But I'd be interested in any other reasons to avoid putting this test inside a named subroutine. If I'm being stubborn for insufficient reason, I'd like to be illuminated and corrected.

Mea Culpa re: Typing

I was going through some Rails code today, making sure that I had good coverage and writing or modifying whatever specs as needed. I found this code:

module DisplayHelper
  # snip ...
  EMPTY_STRING = %q[]
  # snip ...
  def should_see_edit_link?(options)
    return EMPTY_STRING if options[:new_record]
    return false unless (memoized_user = options[:user])
    # more stuff with memoized_user ...
  end
end

Do you see the problem? The first line within should_see_edit_link? should be

    return false if options[:new_record]

My initial version was not a proper predicate, in that it did not restrict its returned values to only true or false. Bad coder. For what it's worth, the methods that use should_see_edit_link? themselves return EMPTY_STRING if should_see_edit_link? is truthy. Alas, not a good enough excuse.

This is exactly the sort of error that advocates of static typing mention as support for their preferred type system. The most experience I have with bondage & discipline typing is with Haskell, where I find it takes its proper place as part of a cohesive whole for understanding Haskell's approach to currying, implicit typing, the pointfree style, and so on. All of that is great, I just haven't found that static typing reduces the error rate in my code. In fact, it gets in my way more often than anything else. Maybe that's why I'm getting more into Erlang rather than Haskell. All of this is probably inseparable from the fact that I got into coding with dynamic languages.

The type error that prompted this blog post is atypical for me. I'm not claiming to write completely bug-free code, it's just that my errors tend not be the sort that can be caught by a static type checker. In fact, the only errors of this sort that I can remember making are this individual error and some others in Perl related to composite data structures being coerced into scalar context. The latter were more indicative of Perl's idiosyncratic (and in my opinion, failed) experiement with context than with dynamic typing as it's generally implemented outside of Perl. So I'm not sure that they count.

How about anyone else? Has static typing ever really saved your butt dramatically?

Caveat: Please don't interpret this post as going anywhere near addressing static typing as it relates to hardware optimizations during compile time. I'm just taking about human errors resembling the example.

Mr. Neighborly's Ruby and Rails Bibliography

Jeremy McAnally recommended my book in his list of good Ruby books partitioned by skill level and specific interest. He placed mine in the sublist appropriate for "moderate programming experience". Seems appropriate.

Jeremy gave a very good presentation called Deep Ruby at MtnWest RubyConf2008 in Salt Lake City. He also has some of his own books in the same list where he graciously put mine. He's a good speaker who knows a lot about the language. Check him out.

I would add SICP to the list for becoming a more well-rounded programmer.

Wither Comments?

Jeff Atwood has an excellent post at Coding Horror about comments.


http://www.codinghorror.com/blog/archives/001150.html.

Worth reading. I believe that function / method names should tell you what's happening (at a suitable level of abstraction), and any comments that do exist should explain the rare cases where your well-named entities could be misleading.

Avi Pilosof has an excellent related point about the kinds of comments that are often underused at http://blogs.msdn.com/avip/archive/2008/07/25/code-commenting-try-business-commenting.aspx.

First-Class Procedures, Part III: being returned as values of procedures

SICP demands that for functions to qualify as First Class Citizens,
they must satisfy several requirements. I've written about this before in reference to the first requirement: being able to assign names to procedures, and the second requirement: the ability to be passed as an argument to another function. Next on the list is being returned as a value from a procedure. Here we go with that.

Again: Ruby, SICP's original Scheme, Erlang, and Haskell, with JavaScript by my friend Aubrey Keus.

In Ruby:

def get_method_that_does_name(owner, name_as_sym)
  owner.method(name_as_sym)
end

def get_double_plus_one
  lambda { |x| (x*2) + 1 } 
end

Here we've defined a method called get_method_that_does_name. It expects a Symbol argument preceded by an owning object, and returns a Method object, created via the aptly-named method method. The owning object provides the context for the method, similar to a closure in a purely-functional language. Here's an example of how get_method_that_does_name might be used in irb:

>> add1 = get_method_that_does_name(1, :+)
=> #<Method: Fixnum#+>
>> add1.call(2)
=> 3
>> greet = get_method_that_does_name('hello, ', :+)
=> #<Method: String#+>
>> greet.call('world')
=> "hello, world"

The method identified by + means something different when called on 1 than when called on hello, , as shown in the example.

The other example uses the more-familiar lambda keyword to generate a Proc.

>> dp1 = get_double_plus_one
=> #<Proc:0xb7cb2458@./first_class_functions_in_ruby.rb:39>
>> dp1.call(4)

In either case, the returned value from the method (whether
get_method_that_does_name or get_double_plus_one) is a procedure-like object callable with the call method.

In MIT Scheme:

$ mit-scheme

1 ]=> (define get-adder (lambda (x) (lambda (y) (+ x y))))
;Value: get-adder
1 ]=> (get-adder 2)
;Value 11: #[compound-procedure 11]
1 ]=> ((get-adder 2) 1)
;Value: 3

Thi sample code demonstrates a typical Scheme example of a procedure that returns another procedure. This particular syntax uses two lambdas, but note that the first line could have been expressed alternately in this manner:

1 ]=> (define (get-adder x) (lambda (y) (+ x y)))
;Value: get-adder

Usage in the 2nd and 3rd lines remain the same in either case. Line 2 shows us that calling (get-adder 2) returns a compound-procedure value, and line 3 shows us that that returned procedure takes an argument (y in our definition code), and adds it to the value of x that was used in the get-adder call that created the returned procedure in the first place.

One could argue that Scheme provides the most canonical syntax for defining a procedure that returns another procedure. This is unsurprising, given the roots of the language, especially in education.

In Erlang:

$ erl 
Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.5.5  (abort with ^G)
1> GetAdder = fun(X) -> (fun(Y) -> X + Y end) end.
#Fun<erl_eval.6.49591080>
2> GetAdder(2).
#Fun<erl_eval.6.49591080>
3> Add1 = GetAdder(1).
#Fun<erl_eval.6.49591080>
4> Add1(2).
3

This Erlang example differs slightly, of course. Erlang's variables are uppercase, and it uses the fun keyword rather than lambda. This particular example also happens to assign the value of GetAdder(1) into a variable called Add1, which erl dutifully reports back to us is of the type Fun. Calling Add1(2) then produces the expected result.

In Haskell:

Last time on this topic I used hugs, a REPL shell. This time I'll use GHCi.

$ ghci
GHCi, version 6.8.2: http://www.haskell.org/ghc/  :? for help
Loading package base ... linking ... done.
Prelude> let getAdder x = \y -> (y + x)
Prelude> let add1 = getAdder 1
Prelude> add1 2
3 
Prelude> :quit

This actually uses Haskell's equivalent of lambda in the \, apparently chosen for its visual resemblance to the letter λ without a lower-left leg. If we wanted to use the pointfree style mentioned in the hugs portion of my previous post on this topic, we could replace the first line with

Prelude> let getAdder = \y -> (y +)

Notice how it leaves the x variable out entirely.

We can use Haskell's type system to reveal some other things of interest.

Prelude> :t getAdder
getAdder :: Integer -> Integer -> Integer
Prelude> :t (getAdder 1)
(getAdder 1) :: Integer -> Integer

The :t command in GHCi reports the type of its argument. getAdder has the type Integer -> Integer -> Integer. This means that it can either take two Integer arguments and return a single Integer, or it can take one Integer and return a function of type Integer -> Integer. This means that the returned function takes one Integer and returns one Integer. When we check the type of the expression (getAdder 1), it reports, as expected, that the expression takes a single Integer, returning another. This type system is crucial to understanding Hashkell's handling of currying, which I hope to get into in greater detail in a later post.

If we want to eliminate middle variables entirely, we can simply feed two Integers directly to getAdder, as in

Prelude> getAdder 1 2
3

In JavaScript:

function call_proc_arg_on_2(first_class_proc) {
   return first_class_proc(2);
};

Number.prototype.call_proc_arg_on_self = function(first_class_proc) {
   return first_class_proc(this);
};

call_proc_arg_on_2(add1);
Number(100).call_proc_arg_on_self(add1);

Aubrey's JavaScript code demonstrates both the initial example common across all the languages, as well as the equivalent of my second Ruby example, in which the desired behavior is attached to existing number objects within your language's existing workspace.

So that's it. I think all of these languages satisfy the requirement of returning procedures (however defined) as values from other procedures, at least from a practical perspective. The next (and final) post in this series will be about incorporating procedures into data structures. I'll get to the post when I can.