Tuesday, September 30, 2008

Hidden Functions

Let's talk about hidden functions. What do I mean by that? When I refer to a hidden function, I mean one that exists in some non-public context, and is not directly accessible from the outside. There are several ways one can realize this goal, and we'll talk our usual stroll through some appropriate languages and how they approach his problem.


Let's start with Scheme. Assuming the follow content in a file called hidden.scm:

(define (obvious)
(define (hidden) `hidden)

We can then try it out using guile -l hidden.scm:

guile> (hidden)

In standard input:
1: 0* (hidden)

standard input:1:1: In expression (hidden):
standard input:1:1: Unbound variable: hidden
ABORT: (unbound-variable)

We see that (hidden) is not directly accessible from the outside, however...

guile> (obvious)

It is still defined and reachable through the intermediary function (obvious). This is great for all the expected black box reasons:

  1. Users/developers can be presented with a consistent interface

  2. Purpose can be separated from implementation details, which means that

  3. Testing is easier to do, which means that

  4. Refactoring is easier to do, whether literally just an improvement to internal clarity and readability, or wherein the inputs and outputs remain the same expressions, but other improvements occur, such as faster speed of execution, reduced memory footprint, or both


I've written about Perl a bit more than I expected to lately. Let's continue with that, and see an example in Perl.

#!/usr/bin/env perl
# hidden.pl

use constant HIDDEN => 'hidden';

sub obvious {
sub hidden { HIDDEN }

sub obvious2 {
my $hidden = sub { HIDDEN };

print obvious() . "\n";
print obvious2() . "\n";
print hidden() . " is really not that hidden\n";
print $hidden->() . "\n";

Executing this program at the command line with perl hidden.pl produces the following output:

hidden is really not that hidden
Undefined subroutine &main:: called at hidden.pl line 19.

I like constants for values that don't change and are reused - call me crazy. The examples obvious() and obvious2() are very similar. The only difference is that the function which we wish to hide is declared either as a traditional subroutine or as a subroutine reference. Why the distinction?

The reason is revealed when we make a direct call to hidden(). Perl does not scope the declaration of hidden() within obvious() as Scheme does. hidden() is accessible from outside obvious() where it was declared. However, attempting to call $hidden->() at the top level fails, because Perl does scope $hidden inside obvious2() where it was declared.

I'm not a real Perl guru. I get by in the language, but I'm sure someone who frequents Perl Monks or the like could explore this idea further.


Despite Ruby's comprehensive support for programming in the functional paradigm, it is commonly thought of as an object oriented language, not without cause. Many people are most familiar with the notion of hiding executable code through the process of private or protected access control common in OO languages.

I won't give an example of traditional OO access control in Ruby here. Rather, I'll explore ideas more akin to the preceding Scheme and Perl examples that use nested function declarations. Some of this code is intentionally broken and/or strange-looking.

class Owner

HIDDEN_PROC = lambda { %q[hidden value from Proc] }

def obvious
def hidden; %q[hidden value]; end
hidden.gsub(%r[hidden], %q[obvious])
def obvious2; hidden + %q[ from obvious2]; end
def obvious3; HIDDEN_PROC.call; end


if (__FILE__ == $0)
o = Owner.new
puts o.obvious
puts o.obvious2
puts o.obvious3
puts o.obvious.hidden
puts %q[Could not access o.obvious.hidden]
puts o::HIDDEN_PROC.call
puts %q[Could not access o::HIDDEN_PROC.call]
puts o.hidden + %q[ - not really all that hidden]

Let's take it for a spin with ruby hidden.rb. (Note that the boolean expression (__FILE__ == $0) is only truthy when the file is executed directly like this, and not when the file is required in the manner of a library file.)

obvious value
hidden value from obvious2
hidden value from Proc
Could not access o.obvious.hidden
Could not access o::HIDDEN_PROC.call
hidden value - not really all that hidden

hidden() is callable from within obvious(), and we can even do a regular expression substitution on its value. Fine and dandy. Its hidden status is called into question by it being accessible from within obvious2(), even though it was not declared there. Let's ponder why that is for a moment.

If a function in Ruby were genuinely hideable, as in Scheme, it would not be so available. However, one must keep in mind that there are technically no functions in Ruby at all: they're methods, and that distinction makes all the difference. A method is a function that is attached to an object (or class), and what would it imply if a method were not accessible by its owning object? It wouldn't be much of a method, and that's where Ruby's OO nature wins out.

In order to restrict access to some executable code within an object, we can either comply with OO practice and use traditional access control, or do something akin to obvious3(), in which we define a callable HIDDEN_PROC that is not publicly accessible, but is callable from within obvious3() (for example). If we wanted even more separation, we could define HIDDEN_PROC as a lexical variable within the obvious3() method, rather than as a class Constant. Such an approach would be very similar to the Perl example's definition of $hidden.


On to Erlang, which has some interesting features regarding exporting of functions. Here's our file hidden.erl.

-author("Kevin C. Baird").
-purpose("Demonstrate hidden functions in Erlang").

obvious() -> hidden().
hidden() -> 'value from hidden'.

Here's our erl session that uses it:

$ erl
Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.5.5 (abort with ^G)
1> c(hidden).
2> hidden:obvious().
'value from hidden'
3> hidden:hidden().

=ERROR REPORT==== 30-Sep-2008::10:05:09 ===
Error in process <0.31.0> with exit value: {undef,[{hidden,hidden,[]},{erl_eval,do_apply,5},{shell,exprs,6},{shell,eval_loop,3}]}

** exited: {undef,[{hidden,hidden,[]},
{shell,eval_loop,3}]} **

Because only obvious is exported, it is the only function publicly available. We could have used as complex a basis as we wanted to determine its eventual value, all of which could be safely hidden within our hidden module. That's what encapsulation's all about.


Another purely functional language I've discussed is Haskell. In fact, its strongest advocates would probably deny Erlang the label purely functional, as it does not segregate side effects as fully as Haskell does.

In any case, we can have a quite small Hidden.hs file:

-- Hidden.hs
-- Kevin C. Baird
-- Demonstrate hidden functions in Haskell

obvious = hidden
where hidden = "value from hidden"

That executes as you'd expect by this point, as seen in hugs:

$ hugs Hidden.hs
__ __ __ __ ____ ___ _________________________________________
|| || || || || || ||__ Hugs 98: Based on the Haskell 98 standard
||___|| ||__|| ||__|| __|| Copyright (c) 1994-2005
||---|| ___|| World Wide Web: http://haskell.org/hugs
|| || Bugs: http://hackage.haskell.org/trac/hugs
|| || Version: September 2006 _________________________________________

Haskell 98 mode: Restart with command line option -98 to enable extensions

Type :? for help
Main> obvious
"value from hidden"
Main> hidden
ERROR - Undefined variable "hidden"

Wednesday, September 17, 2008

Boston, San Francisco and SDD

My colleague Jim Lindley and I will be going to Boston for a few days for our employer. We do TDD/BDD with RSpec and Rails currently, and the folks in the Boston office are at some point along a similar continuum. We'll show them what we do, they'll show us what they do, we'll learn and grow and laugh and share and love, etc. Then I'm off to San Francisco for some interviews related to my planned relocation out there.

Yesterday, I came across a blog post by Paul Barry about SDD with Rails. The two upcoming trips already had me thinking about my workflow, and Jim and I had been talking about trying move another abstraction step up from our current speccing practice, and SDD seems like a good match for our next project. It's a rewrite of an existing app that is currently in a different language - our job would be to match it feature-for-feature and also to make some necessary changes and additions.

SDD seems like a good way to keep the QA folks who already know how the app should work more involved in the the speccing process in a way that actually has impact. That's the beauty of having the human-readable descriptions be executable.

Paul also links to Bryan Helmkamp's talk on SDD at GoRuCo2008, which is very good. I would encourage anyone to read Paul's post and watch Bryan's talk.

Thursday, August 28, 2008

Celebrity sighting(?)

My last post was about Perl and my experiences working with it and its fans. I'm not a huge fan of the language - I'm planning another post that details what I like and don't like about it. I think it has few characteristics that are fundamentally flawed, for example.

However, I'm pretty sure I saw Larry Wall in the Detroit airport a few weekends ago when I was flying to Seattle for a friend's wedding. The excitement that I heard in my own voice when I pointed him out to my wife Jenn made me realize that I think Perl's good points have been a positive influence on programming culture.

So thanks, Larry.

Monday, August 4, 2008

Perl and is_valid_aref()

This post is about Perl, the politics of interacting with fellow developers, and Duck Typing (although we didn't really know it at the time). It's also fairly stream-of-consciousness.

I used to work at a Perl shop, and in a big rewrite we did, we made heavy use of array references. A dynamic lanugage like Perl (or Ruby, or whatever) will encourage conscientious developers to have some tests and/or error checking, so we often wanted to ensure that our input parameters to some specific subroutines that expected array refs were actually valid arefs. How should one do this checking in Perl?

Here's one obvious approach.

sub is_valid_aref_mere_mortals {
ref shift eq 'ARRAY';

Here we have a straightforward subroutine. We use Perl's shift to pull the first parameter off the stack, get the type to which it refers with ref, and check with the stringy equality operator whether or not the referred type is 'ARRAY'.

This wasn't good enough for an ex co-worker of mine, who fancied himself a Perl Wizard (with some justification, in fairness to him).

Some idiot could bless a non-ARRAY behaving scalar as a ref to an ARRAY. My solution in that case is to berate the offender in public for having done something so horrible. Such a blessing is a direct violation of the duck typing idea that you don't have a datum pretend to be a specific type (however your language defines type) unless it can implement all the pertinent behavior expected of that type in that context. With great power comes great responsibility.

We also had no instances of such blessing in our app. All of our args were all either just simple arefs, or errors (usually an undef value). So in practice, I think a simple ref check would work fine. However, if you're concerned about the blessed as aref issue, the solution below has some advantages.

sub is_valid_aref {
my( $arg ) = @_;
$arg and eval { @{ $arg } or 1 };

It's a bit tricky, so I'll explain it. You extract the first parameter in the stack as $arg. The and keyword assures that $arg is truthy before we proceed to the eval. The eval attempts to dereference $arg into a regular (non-reference) array. Rubyists can think of this use of eval in Perl as akin to a begin rescue end block.

If the dereferencing results in an error, it will stop evaluation of the eval block, resulting in an undef value, meaning that the expression returned by the function is false.

Why is there an or 1 at the end of the eval block? That's for situation in which your $arg is an empty array ref. An array of zero length evaluated in truthy scalar context in Perl is false. However, we want such a data structure to produce a truthy return value from this function. Therefore, when a valid but empty array ref is dereferenced into an actual array, it is false but does not break out of the eval block. The expression with the eval block then continues with the or 1, which ends up being truthy, ensuring that a valid empty array ref is considered valid by this function.

I lie when I say this was my ex co-worker's solution. He strongly objected to putting this test inside a function, because developers should just know Perl idioms, and the function call would add too much overhead. I thought instead that the value of naming this obscure bit of code with what its purpose is would be worthwhile. He disagreed. So he would have tests like the line of code below copied and pasted with no explanatory comments wherever we needed to check aref validity:

fail if not $arg and eval { @{ $arg } or 1 };

I saw that for the first time and thought WTF? I then added a comment explaining what that particular monstrosity was supposed to do, something like 'checks if a valid aref'. He objected strongly (again), saying that coders should just know Perl idioms.

We finally got him to accept the presence of the comment after about 30 minutes of arguing.

Try them out for yourself, if you like. I use $proc as a generic name for a procedure/function/subroutine/code reference. Some other people prefer $cref for code ref or $sref for subroutine ref.

my( %proc_of ) = (
'mere mortals' => \&is_valid_aref_mere_mortals,
'eval version' => \&is_valid_aref

sub report {
my( $label, $candidate, $proc_of_href ) = @_;
for $proc_name ( keys %{ $proc_of_href } ) {
my( $proc ) = $proc_of_href->{$proc_name};
print $proc->( $candidate )
? "$label is a valid aref according to $proc_name\n"
: "$label is not a valid aref according to $proc_name\n";

report( '[]', [], \%proc_of );
report( '()', (), \%proc_of );
report( '{}', {}, \%proc_of );
report( '(undef)', (undef), \%proc_of );

Results in this output. Note that I haven't included the deceptive blessing situation, on which the two subs would differ:

[] is a valid aref according to mere mortals
[] is a valid aref according to eval version
{} is not a valid aref according to mere mortals
{} is not a valid aref according to eval version
(undef) is not a valid aref according to mere mortals
(undef) is not a valid aref according to eval version

It seems to me that the clarity improvement from having the name is_valid_aref for that code is worth the relatively small overhead of the function call. I also think that it serves a pedagogical use - a new developer who isn't familiar with the blessing concern could in fact be educated about it by seeing this code under such an explanatory label. But I'd be interested in any other reasons to avoid putting this test inside a named subroutine. If I'm being stubborn for insufficient reason, I'd like to be illuminated and corrected.

Monday, July 28, 2008

Mea Culpa re: Typing

I was going through some Rails code today, making sure that I had good coverage and writing or modifying whatever specs as needed. I found this code:

module DisplayHelper
# snip ...
# snip ...
def should_see_edit_link?(options)
return EMPTY_STRING if options[:new_record]
return false unless (memoized_user = options[:user])
# more stuff with memoized_user ...

Do you see the problem? The first line within should_see_edit_link? should be

return false if options[:new_record]

My initial version was not a proper predicate, in that it did not restrict its returned values to only true or false. Bad coder. For what it's worth, the methods that use should_see_edit_link? themselves return EMPTY_STRING if should_see_edit_link? is truthy. Alas, not a good enough excuse.

This is exactly the sort of error that advocates of static typing mention as support for their preferred type system. The most experience I have with bondage & discipline typing is with Haskell, where I find it takes its proper place as part of a cohesive whole for understanding Haskell's approach to currying, implicit typing, the pointfree style, and so on. All of that is great, I just haven't found that static typing reduces the error rate in my code. In fact, it gets in my way more often than anything else. Maybe that's why I'm getting more into Erlang rather than Haskell. All of this is probably inseparable from the fact that I got into coding with dynamic languages.

The type error that prompted this blog post is atypical for me. I'm not claiming to write completely bug-free code, it's just that my errors tend not be the sort that can be caught by a static type checker. In fact, the only errors of this sort that I can remember making are this individual error and some others in Perl related to composite data structures being coerced into scalar context. The latter were more indicative of Perl's idiosyncratic (and in my opinion, failed) experiement with context than with dynamic typing as it's generally implemented outside of Perl. So I'm not sure that they count.

How about anyone else? Has static typing ever really saved your butt dramatically?

Caveat: Please don't interpret this post as going anywhere near addressing static typing as it relates to hardware optimizations during compile time. I'm just taking about human errors resembling the example.

Saturday, July 26, 2008

Mr. Neighborly's Ruby and Rails Bibliography

Jeremy McAnally recommended my book in his list of good Ruby books partitioned by skill level and specific interest. He placed mine in the sublist appropriate for "moderate programming experience". Seems appropriate.

Jeremy gave a very good presentation called Deep Ruby at MtnWest RubyConf2008 in Salt Lake City. He also has some of his own books in the same list where he graciously put mine. He's a good speaker who knows a lot about the language. Check him out.

I would add SICP to the list for becoming a more well-rounded programmer.

Friday, July 25, 2008

Wither Comments?

Jeff Atwood has an excellent post at Coding Horror about comments.


Worth reading. I believe that function / method names should tell you what's happening (at a suitable level of abstraction), and any comments that do exist should explain the rare cases where your well-named entities could be misleading.

Avi Pilosof has an excellent related point about the kinds of comments that are often underused at http://blogs.msdn.com/avip/archive/2008/07/25/code-commenting-try-business-commenting.aspx.

Monday, July 21, 2008

First-Class Procedures, Part III: being returned as values of procedures

SICP demands that for functions to qualify as First Class Citizens,
they must satisfy several requirements. I've written about this before in reference to the first requirement: being able to assign names to procedures, and the second requirement: the ability to be passed as an argument to another function. Next on the list is being returned as a value from a procedure. Here we go with that.

Again: Ruby, SICP's original Scheme, Erlang, and Haskell, with JavaScript by my friend Aubrey Keus.

In Ruby:

def get_method_that_does_name(owner, name_as_sym)

def get_double_plus_one
lambda { |x| (x*2) + 1 }

Here we've defined a method called get_method_that_does_name. It expects a Symbol argument preceded by an owning object, and returns a Method object, created via the aptly-named method method. The owning object provides the context for the method, similar to a closure in a purely-functional language. Here's an example of how get_method_that_does_name might be used in irb:

>> add1 = get_method_that_does_name(1, :+)
=> #<Method: Fixnum#+>
>> add1.call(2)
=> 3
>> greet = get_method_that_does_name('hello, ', :+)
=> #<Method: String#+>
>> greet.call('world')
=> "hello, world"

The method identified by + means something different when called on 1 than when called on hello, , as shown in the example.

The other example uses the more-familiar lambda keyword to generate a Proc.

>> dp1 = get_double_plus_one
=> #<Proc:0xb7cb2458@./first_class_functions_in_ruby.rb:39>
>> dp1.call(4)

In either case, the returned value from the method (whether
get_method_that_does_name or get_double_plus_one) is a procedure-like object callable with the call method.

In MIT Scheme:

$ mit-scheme

1 ]=> (define get-adder (lambda (x) (lambda (y) (+ x y))))
;Value: get-adder
1 ]=> (get-adder 2)
;Value 11: #[compound-procedure 11]
1 ]=> ((get-adder 2) 1)
;Value: 3

Thi sample code demonstrates a typical Scheme example of a procedure that returns another procedure. This particular syntax uses two lambdas, but note that the first line could have been expressed alternately in this manner:

1 ]=> (define (get-adder x) (lambda (y) (+ x y)))
;Value: get-adder

Usage in the 2nd and 3rd lines remain the same in either case. Line 2 shows us that calling (get-adder 2) returns a compound-procedure value, and line 3 shows us that that returned procedure takes an argument (y in our definition code), and adds it to the value of x that was used in the get-adder call that created the returned procedure in the first place.

One could argue that Scheme provides the most canonical syntax for defining a procedure that returns another procedure. This is unsurprising, given the roots of the language, especially in education.

In Erlang:

$ erl
Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.5.5 (abort with ^G)
1> GetAdder = fun(X) -> (fun(Y) -> X + Y end) end.
2> GetAdder(2).
3> Add1 = GetAdder(1).
4> Add1(2).

This Erlang example differs slightly, of course. Erlang's variables are uppercase, and it uses the fun keyword rather than lambda. This particular example also happens to assign the value of GetAdder(1) into a variable called Add1, which erl dutifully reports back to us is of the type Fun. Calling Add1(2) then produces the expected result.

In Haskell:

Last time on this topic I used hugs, a REPL shell. This time I'll use GHCi.

$ ghci
GHCi, version 6.8.2: http://www.haskell.org/ghc/ :? for help
Loading package base ... linking ... done.
Prelude> let getAdder x = \y -> (y + x)
Prelude> let add1 = getAdder 1
Prelude> add1 2
Prelude> :quit

This actually uses Haskell's equivalent of lambda in the \, apparently chosen for its visual resemblance to the letter λ without a lower-left leg. If we wanted to use the pointfree style mentioned in the hugs portion of my previous post on this topic, we could replace the first line with

Prelude> let getAdder = \y -> (y +)

Notice how it leaves the x variable out entirely.

We can use Haskell's type system to reveal some other things of interest.

Prelude> :t getAdder
getAdder :: Integer -> Integer -> Integer
Prelude> :t (getAdder 1)
(getAdder 1) :: Integer -> Integer

The :t command in GHCi reports the type of its argument. getAdder has the type Integer -> Integer -> Integer. This means that it can either take two Integer arguments and return a single Integer, or it can take one Integer and return a function of type Integer -> Integer. This means that the returned function takes one Integer and returns one Integer. When we check the type of the expression (getAdder 1), it reports, as expected, that the expression takes a single Integer, returning another. This type system is crucial to understanding Hashkell's handling of currying, which I hope to get into in greater detail in a later post.

If we want to eliminate middle variables entirely, we can simply feed two Integers directly to getAdder, as in

Prelude> getAdder 1 2

In JavaScript:

function call_proc_arg_on_2(first_class_proc) {
return first_class_proc(2);

Number.prototype.call_proc_arg_on_self = function(first_class_proc) {
return first_class_proc(this);


Aubrey's JavaScript code demonstrates both the initial example common across all the languages, as well as the equivalent of my second Ruby example, in which the desired behavior is attached to existing number objects within your language's existing workspace.

So that's it. I think all of these languages satisfy the requirement of returning procedures (however defined) as values from other procedures, at least from a practical perspective. The next (and final) post in this series will be about incorporating procedures into data structures. I'll get to the post when I can.

Friday, June 27, 2008

What are Rails Helpers for?

I was reading Dan Mange's Smart Model, Dumb Controller post, which has a very interesting comment from Greg Willits in which he proposes an additional logic layer between Controllers and Models. He and Dan go back a forth a bit on this - check it out.

This prompted me to think about what I use Helpers for. The exchange above brought up Helpers in their canonical sense, as being a sort of glue between the Controller and the View. My colleague Jim Lindley and I certainly use Helpers in that fashion, witness

module PreceptorsHelper
def self.select_list
Preceptor.find(:all).map { |p| [p.dropdown_name, p.id] }

Pretty typical. We have a Preceptor model, and our PreceptorsHelper has a select_list method that DRYs up how all of our Preceptor-related views get their select lists for dropdown menus.

Additionally, we have a dropdown_name, which differs from the plain old name field of the Preceptor, but neither the PreceptorsHelper nor the views know about that - it's properly encapsulated within the Preceptor model itself. This is basic OO design that provides tools for managing multiple levels of complexity and abstractions to handle that complexity. No surprises so far.

However, this all led me to reflect on my own use of Helpers that may not be textbook Controller<->View glue activity. I also use Helpers for logic that pertains to a given model (or more likely a collection of those models) without being a characteristic of any one of those model instances. This sounds a bit like Greg's additional controllers idea. Here's an example from another of our helpers:

module EnrollmentsHelper
def self.filter_by_readiness(enrollments, params)
ready_for_review = params[:ready_for_review]
return enrollments unless ready_for_review
enrollments.select { |e| e.ready_for_review? }

Something akin to this could also be accomplished via named_scope nowadays, I realize. However, let's abstract a bit. This is an operation on a set of things - Enrollments in this case. We certainly don't want this in the EnrollmentsController. We could make the case that it should be a class method of Enrollment. But making it a function that takes the Enrollments as a parameter makes this much easier to test. It's also more of a functional style, which I freely cop to preferring.

I guess I don't even think about Helpers as necessarily needing to provide utility for views so much as providing utility that is "about" a given topic without being a characteristic of any one of those instances.

Note that the topic doesn't even need to be a model. Our current app has several Models presenting different types of humans, and some of our clients care about gender. Presto:

module GenderHelper

:allow_blank => true,
:allow_nil => true,
:message => MessageHelper::MESSAGE[:invalid_gender]


def self.distill_gender!(gender)
gender = %q[M] if MALE_VARIANTS.include?(gender)
gender = %q[F] if FEMALE_VARIANTS.include?(gender)


(Note also the similar delegation to MessageHelper). This allows us to DRYly do something like this in our Student and similar models:

validates_inclusion_of :gender, GenderHelper::VALID_OPTIONS

Nothing view-related there.

Much of this coding style sprang from a desire to simplify controllers. After doing this, I noted many comparatively large model files. One of them, Lottery, had a lot of activity that dealt with processing collections of Enrollments, which I've blogged about before. Why not move this into something more directly related to Enrollments?, I thought.

I like the results.

What do other people think about Helpers used in this fashion? Should there be two different ypes of them: view-related and non-view-related? More? I'm still sorting out my thoughts on this, and welcome new ideas to think about (or old ideas to consider anew).

Tuesday, June 24, 2008

First-Class Procedures, Part II: passing as an argument

SICP demands that for functions to qualify as First Class Citizens,
they must satisfy several requirements. I've written about this before in reference to the first requirement: being able to assign names to procedures. The second requirements is the ability to be passed as an argument to another function, which is the topic of this post.

I'll just dive in, again showing multiples languages: Ruby, SICP's original Scheme, Erlang, and Haskell. Again, my friend Aubrey Keus has provided some JavaScript, as well.

This post is fairly Ruby-heavy. A lot of them are likely to be, as it's the language I know best among those being discussed, as well as the the one I use to earn my Yankee Dollars.

In Ruby:

$ irb

irb(main):001:0> def call_proc_arg_on_2(first_class_proc)
irb(main):002:1> first_class_proc.call 2
irb(main):003:1> end
=> nil
irb(main):004:0> add1 = lambda { |x| 1 + x }
=> #
irb(main):005:0> call_proc_arg_on_2(add1)
=> 3

Here we've defined a method that expects an argument called first_class_proc, which is (as you might expect) a Proc. add1 is such a Proc which adds 1 to its argument - hence the name. When we pass that in as the argument for call_proc_arg_on_2, we get the expected value of 3.

Proc & block syntax

Ruby has a curious subtlety in its treatment of Procs. Notice that we need to use the call method in order to make the Proc do its thing. Not true for some other languages below. Why would anyone do this when designing a language?

Below we see a more typical way of passing a procedure as an argument to a higher-order method.

irb(main):006:0> def call_block_on_2()
irb(main):007:1> yield 2
irb(main):008:1> end
=> nil
irb(main):009:0> call_block_on_2 { |x| 1 + x }
=> 3

The main differences between this and the previous example are as follows:

  1. The name of the method reflects the differences in expected arguments: a Proc vs. a block

  2. call_block_on_2 does not have a parameter in its declaration, just the empty parentheses

  3. When we call call_block_on_2, we don't pass a variable with a name (like add1), instead we just give it a block, which is the business between the {} braces. Note that this block is what we give to the lambda method to create a full-fledged Proc in the earlier example.

  4. The yield method is roughly akin to the call method, except that it automatically knows that it should operate on whatever block was given, without having to refer to it by name.

So what's the deal with this? Standard Ruby style is to call methods with blocks for operations like iteration (with each), list transformation (map) and filtering (select, partition), and so on. This is so common that the syntactic sugar of being able to easily do this outweighed the consistency of being able to call function arguments without needing the call method.

In MIT Scheme:

$ mit-scheme

1 ]=> (define call-on-2 (lambda (x) (x 2)))

;Value: call-on-2

1 ]=> (define add1 (lambda (x) (+ x 1)))

;Value: add1

1 ]=> (call-on-2 add1)

;Value: 3

Check it out. Simple procedure declarations, and procedures that can be passed as args with no special syntax. Some would say no syntax at all, both among the pro- and anti-Lisp communities. Not wishing to add to an old flame war, I'll move on.

In Erlang:

$ erl

Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.5.5 (abort with ^G)
1> Add1 = fun(X) -> 1 + X end.
2> CallOn2 = fun(F) -> F(2) end.
3> CallOn2(Add1).

I'm becoming a pretty big Erlang fan. Note that the variable Add1 is in all caps - this serves a semantic purpose in Erlang. User-defined entities in Erlang that are in lower-case are generally either straightforward functions or atoms, which are essentially just values. They're very similar to Ruby's Symbols, for example.

This Erlang example lets the coder define a function as a variable which is then usable as an argument to another function without any special syntax tricks. CallOn2 just applies the function arg F to the literal number 2 and returns the result, which is just what we want in all these examples.

In Haskell:

(I haven't written about Haskell before now. It's purely functional, strongly typed with inferencing, and offers lazy (non-strict/non-eager) evaluation. Check it out. It has several implementations - I'll show it in hugs, a REPL shell.

Assuming this file CallOnTwo.hs:

add1 = (+) 1
callOnTwo f = f 2

Executing hugs as follows:

$ hugs CallOnTwo.hs
__ __ __ __ ____ ___ _________________________________________
|| || || || || || ||__ Hugs 98: Based on the Haskell 98 standard
||___|| ||__|| ||__|| __|| Copyright (c) 1994-2005
||---|| ___|| World Wide Web: http://haskell.org/hugs
|| || Bugs: http://hackage.haskell.org/trac/hugs
|| || Version: September 2006 _________________________________________

Haskell 98 mode: Restart with command line option -98 to enable extensions

Type :? for help
Main> callOnTwo add1

What's going on here? The actual use of callOnTwo in the REPL example should be readable enough - it's quite similar to all of the other examples.

The lib file CallOnTwo.hs is a bit more interesting. Note the definition of add1 as (+) 1 - it leaves off the argument to add1. Rather than saying the equivalent of add1 of x is x plus one, it says (in Haskell) add1 is the function that adds one to whatever it gets.

This is what Haskell calls pointfree style. In the pointfree style, a coder leaves off terms from a definition when able. The resulting code is generally thought to be cleaner and more compact. It also lends itself well to groking function composition.

One further complication is that the + is in parentheses. This is because + is most commonly used as an infix operator, such that its arguments appear on either side, as opposed to Scheme's (+ x 1), where the function appears first (as is usual in Scheme), and the arguments follow. The wrapping in parentheses converts the traditionally infix + into a prefix function.

One could argue that this slight syntax massaging is roughly similar to Ruby's need for call, thereby angering fans of Ruby, Haskell, and Scheme simultaneously. Probably others too. Maybe F# fans. Who knows?

More Ruby:

In Ruby again, following the Object-Oriented nature of the language, we can do similar operations in which the functions are explicitly understood to be methods attached to specific objects. In this case, the Integer 1.

Assuming the existence of this file func_as_arg.rb:

class Integer
def call_symbol_arg_on_2(first_class_proc_as_sym)
2.send(first_class_proc_as_sym, self)

We can then do operations in irb again.

$ irb -r func_as_arg.rb
irb(main):001:0> 1.call_symbol_arg_on_2(:+)
=> 3
irb(main):002:0> 1.call_symbol_arg_on_2(:-)
=> 1

With the results that you'd expect.

In JavaScript:

function call_proc_arg_on_2(first_class_proc) {
return first_class_proc(2);

Number.prototype.call_proc_arg_on_2 = function(first_class_proc) {
return first_class_proc(this);


Aubrey's JavaScript code demonstrates both the initial example common across all the languages, as well as the equivalent of my second Ruby example, in which the desired behavior is attached to existing number objects within your language's existing workspace.

I split infinitives. If such things get you riled up, I suggest you read about scientific linguistics a bit more. A good starting point is Language Myths, edited by Laurie Bauer. In this particular case, the splitting was to clarify that easily modifies the use the use of the more-common block syntax in Ruby, rather than suggesting that easily modified the strength of the outweighing. Embedding the adverb within the verb is excellent for disamgiuation of this sort. Try it - you'll like it. Trust me.

Wednesday, June 11, 2008

(Road to) Partition

I'm a big fan of Ruby's Enumerable#partition method. It's very functional, and comes in quite handy. Here's the basic deal: it takes a block and returns two Arrays. The first of which contains all elements for which the block is truthy, and the second of which contains all elements for which the block is false. Here's some code that more-or-less duplicates how it works:

module Enumerable

def my_partition(&block)
return select(&block), reject(&block)


It returns a comma-separated list, so you can pull the returned values out into two separate Arrays, use the * operator to cram them together, what have you.


Here's some Rails code in its initial quick 'n dirty state. One of the models in the app is called Enrollment, and restore_previous_state! is basically an undo method for some operations on enrollments. Methods run earlier in this app either set the ready_for_review boolean flag to true, or autovivified some enrollments entirely. So this takes in some enrollments, destroys them if they were autovivified, and just marks those that it didn't create itself as unready.

I also wanted to keep track of how many changes of each type were done, for reporting purposes. Hence the += 1 counter operations. Then I spit out the results Hash at the end to know what I did. I think this version is fairly procedural for Ruby code, at least for my usual style. That's not necessarily bad, although I do prefer the changes below.

def restore_previous_state!(enrollments_to_modify)
results = { :reset => 0, :destroyed => 0 }
enrollments_to_modify.each do |e|
if e.autovivified?
results[:destroyed] += 1
e.ready_for_review = false
results[:reset] += 1


Here's the same method after some functionally-oriented refactorings that use the partition method.

def restore_previous_state!(enrollments_to_modify)
to_destroy, to_reset = enrollments_to_modify.partition(&:autovivified?)
to_reset.map { |e| e.ready_for_review = false; e.save }
{ :reset => to_reset.size, :destroyed => to_destroy.size }

In the new version, I partition the enrollments based on whether or not they were autovivified, because (as we already know) my method has to behave quite differently in each case. Then I just map the destroy operation onto all the enrollments that I should destroy, and a combined unready & save for the others. Since to_destroy still exists in memory after the destroy operations (it just had its database records deleted), I can still get its size. And of course to_reset is still around, just with some readiness changes.

Presto, spit out an anonymous Hash reporting how many changes of each type, and the external behavior of the method stays entirely the same, which is as it should be for a refactoring. The new version trims some lines (14 -> 6), and I also find it more readable.

Elsewhere, the app goes even further with partition. I won't explain this in as much detail, because if you followed the stuff above, this should be fairly clear as an additional example.

One curious thing about this is that it uses a state-changing operation (save) as the partitioning condition, so it becomes a try to save, and separate based on whether or not it worked composite operation. Also, since I only need to keep counts, I immediately map the size method onto each sub-Array that comes out of partition.

My co-worker Jim calls this sort of code dense. I prefer compact. He assures me that he was just teasing me.

def save_cleanup_and_report!(post_delayed_rules_to_save, rejected_potentials)
ready_to_save, invalid_for_saving = post_delayed_rules_to_save.partition(&:valid?)
actually_saved_count, failed_save_count = ready_to_save.partition(&:save).map(&:size)
rejected_potentials.map { |e| e.ready_for_review = false; e.save! }
[actually_saved_count, (rejected_potentials.size + failed_save_count)]

I apologize for the pun in the title, of course.

Monday, June 9, 2008

I am Jesus of Borg

I don't want to pick on anyone, and I realize this doesn't speak for all of the world's approximately Billion self-identified Christians, but does anyone else find this a bit disturbing?

Tuesday, June 3, 2008

Notes from RailsConf, sort of

I just got back from RailsConf2008 in Portland, and I'm still a little jet-lagged. Lots of good stuff, especially about scaling, the deployment ecosystem, and distributed / "cloud" computing. Everyone and their brother will be talking about the presentations, so I'm going to relate a couple of anecdotes.

Predicate Truth vs. Resource Protection

When is a vegan not a vegetarian?

I'm a vegan (just reporting data - eat what you think best). They had food for us at the conference, as well as for other dietary needs. I had a nice but brief chat with a guy waiting for the Kosher food, for example. So I would go up to the little cart labelled vegan with the intention of getting food. Nothing shocking there.

The guy serving the food would then ask people who approached Are you a vegetarian?. Since the category vegetarian is a subset of the category vegan, I answered yes. He then directed me to the standard food tables, since people who are ovo-lacto vegetarians had plenty of options among the standard fare. I then pointed out that I'm also a vegan, and got some food.

This raises some interesting points - interesting to me, anyway. I told the guy that among a group of programmers, we're culturally very inclined to answer predicate questions with the literal truth: some_vegan.vegetarian? => true. He then explained that his primary goal in asking the question was not to enquire about someone else's eating habits, but to make sure they don't run out of the vegan options too early. He was basically acting as a return guard, making sure that I had enough to eat.

Another interpretation is that his question was based more on the weirdness factor. Vegetarians are weirder than omnivores, but to a lesser degree than vegans. So a vegan in that scenario would reject the boolean type signature implicit in Are you a vegetarian? and offer more information: I'm not just a vegetarian, I'm a vegan.

Who Pays for Dinner?

Sushi as a load balancing problem

When out eating dinner with my colleagues, we discussed the issue of payment. Initially, we decided on a simple hocket: one guy pays, and then the next night a different guy pays, and so on. My fellow nerds will recognize that as a Round Robin load balancer approach.

Things were fine until the fourth night. They were only three of us, so this was the first instance of a potential repeat. I had paid for very good (and very expensive) sushi meal two nights earlier, and my colleague Jim had splurged for Quizno's the previous night. Jim volunteered to shift to Fair load balancing, using how much some one had paid so far in the trip as the fairness criterion - which of course stuck him with paying for the final meal.

Monday, May 19, 2008

Wigs for Kids

I promise both my readers that I'll get back to first-class procedures soon. I'd like to talk about something else before that post will be ready.

Hippy / Alfalfa

I periodically donate my hair to an organization called Wigs for Kids. It was time for me to do it again this past weekend. I do it every 2 years or so. The basic idea is that you donate a 12 inch or longer ponytail of hair, and you help out some kid by giving them a wig if they have no hair, such as from the results of chemotherapy.

Hippy Side / Mullet

Chemotherapy was sadly appropriate to this particular donation. As my wife Jenn and I were getting ready to leave that morning, we learned that our good friend Travis Nixon (RedToenail blog) had died the night before. He had had cancer for years, and had recently taken a downturn, but it was still a shock. The fact that this occurred the evening before we had already planned to donate hair to some unknown person, who (like Travis) has been diagnosed with cancer far too early in life, is the sort of thing that supernaturally-minded people often seem to see as especially significant in some way that's never been exactly clear to me (as a secular person). That's not a dig against religious people -- my mind is just focused on these sorts of topics at the moment.


In any case, I'm grateful to our friend Kevin Farrell for doing the actual cutting. I think he did quite a good job. He refused to take any money, which speaks well of his character, but we hid some cash at the register while he was sweeping.

The Kevins

If you're able to grow your hair to the appropriate length, you might want to look into doing something similar. It's not addressing the root of the problem in any medical way, but I figure if we can do something that provides some comfort to a sick person with so little effort, it's a worthwhile thing to do.


He was a good friend, and I miss him.


Tuesday, April 22, 2008

5 Rails Tips

If you're not already watching Ryan Bates' screencasts at Railscasts.com, I strongly recommend it. They're really quite well done and interesting. He's running a contest, in which participants suggest 5 Rails tips. For my examples, I've decided to focus on areas that are either informed by ideas from functional programming (which this blog is ostensibly about), or areas in which Ryan and I have different approaches (or both). Here are my tips:

Use Module methods

Here's where I disagree with something Ryan said in his 101st Railscast, in which he suggests using Class (or instance) methods with variables over using Module methods. I prefer to use Module methods. Here's a typical (truncated) example:

module DisplayHelper

def self.get_stylesheets_by_request(request)
user_agent = request ? request.user_agent : nil

def self.get_stylesheets_by_ua(some_user_agent)
return STYLESHEETS_FOR[:palm] if some_user_agent =~ %r[Palm]


Then, in the RSpec file:

describe %q[get_stylesheets_by_request] do
mobile = %w[mobile]
it %Q(should add the stylesheet #{mobile} for Palm browsers) do
user_agent = %q[Some Palm User Agent]
request = mock_model(Object, :user_agent => user_agent)
DisplayHelper.get_stylesheets_by_request(request).should == mobile
normal_browser = %w[normal_browser]
it %Q(should add the stylesheet #{normal_browser} for all other browsers) do
user_agent = %q[Some User Agent]
request = mock_model(Object, :user_agent => user_agent)
DisplayHelper.get_stylesheets_by_request(request).should == normal_browser

As I said above, Ryan and I have different preferences regarding Class vs. Module. I like having helper methods that are testable in a more "purely functional" paradigm, wherein we mock a request, but don't need to mess around within the base controller object or the like. We just pass our mock request model into the get_stylesheets_by_request method and take it from there. Either approach works, you may find that one or the other matches your own thought process more closely.

Use Constants

In the code example above, notice the use of the STYLESHEETS_FOR Constant, which we can define as follows:

:normal => %w[normal_browser],
:palm => %w[mobile]

It gives us access to either normal_browser.css or mobile.css, as appropriate. This is a somewhat contrived example, but you could use whatever list of stylesheets you want for the values in each pair, expanding to customize for Safari, Epiphany, Opera, etc.

This technique could be used for any data that is unlikely to change in the course of an app running, but is not tied specifically enough to a given model to be stored in the DB. This allows you to avoid continually reconstructing never-changing local variables inside a method call. Some more realistic examples appear again in the MessageHelper tip below.

Return guards can simplify flow control

I loath if - elseif - else - end flow control, and strongly prefer return guards. It's a personal bigotry that I freely admit, because I think return guards make code more readable, shorter, and more in line with how I think. Here's a rewrite of Ryan's star_type method from Railscast #101 that uses return guards rather than an if block.

def star_type(value)
return 'full' if value > 1
return 'half' if value == 1

You save 4 lines, and also comply with my enraged, semi-coherent rantings. You can also get some additional shrinkage with a ternary. Here's star_type again, with a ternary:

def star_type(value)
return 'full' if value > 1
value == 1 ? 'half' : 'empty'

You save a line. I find it more readable than the if block. That may be because I like reading Haskell code. Who knows. Again, your flow control preference mileage (or kilometrage) may vary.

DRY up your messages in a MessageHelper

I like to create a MessageHelper.rb file that contains my messages. I usually have several Constant Hashes that vary depending on the purposes of the message.

module MessageHelper

:name_of_resource => %q[My description...]

:not_empowered_to_delete => lambda { |type| %Q[You are not empowered to delete a #{type}.] }

:overlapping_blocks => %q[Overlapping blocks of time]

:confirm_short => %q[Are you sure?],
:confirm_long => lambda { |x| %Q[Are you sure you want to destroy this #{x}?] },
:logout_successful => %q[You have been logged out.]

:new => lambda { |x| %Q[Make a new #{x}] },
:owner => lambda { |x| %Q[Owner of #{x}] },
:show => lambda { |x| %Q[Show #{x}] },


These are obviously truncated. I use DESCRIPTION_OF_RESOURCE and EXPLAIN for hover explanations and the like. Notice also that the EXPLAIN and MESSAGE[:confirm_long] values are Procs, allowing you to reuse them abstractly. The purpose of the ERROR, MESSAGE and TITLE hashes should be obvious from their names.1 The main benefit of this practice is that you can conform to DRY principles, while also separating your messages logically according to what they'll be needed for.

Of course, you can wrap the use of each of these hashes inside helper methods, like explain, message, or what have you.

1(Maybe the purpose of TITLE isn't obvious: It's to facilitate link differentiation with <a> titles as per http://www.w3.org/TR/WCAG10-HTML-TECHS/#link-text.)

Use parallel (pattern-matching) assignments

Whenever assigning into multiple variables, it can often be helpful to do simultaneous parallel assignments using pattern matching. Here's an example from an RSpec file. I have a resource called Preceptor which has many Rotations, and a helper method called create_preceptor_and_rotations that does what you (hopefully) expect:

@preceptor, @preceptor_rotations = create_preceptor_and_rotations
@preceptor, discard__rotations = create_preceptor_and_rotations

In the first case, I want to use both @preceptor and @preceptor_rotations for some purpose. In the 2nd case, I want to keep the @preceptor, but the name of the rotations variable informs anyone reading the code that I don't care about the rotations in this particular spec instance. In both cases, I've done assignments into two variables simultaneously.

This sort of pattern is common in situations like this:

def some_method_that_takes_a_list(*args)
head, *tail = *args
# do some operations, maybe something recursive
return [head, tail]

Which returns as follows:

some_method_that_takes_a_list(7) -> [7, []]
some_method_that_takes_a_list(7, 8) -> [7, [8]]
some_method_that_takes_a_list(7, 8, 9) -> [7, [8, 9]]
some_method_that_takes_a_list([7, 8, 9]) -> [[7, 8, 9], []]
some_method_that_takes_a_list(*[7, 8, 9]) -> [7, [8, 9]]

So there's my list. These are all obviously fairly subjective points, but I find that these approaches work well for me, and my co-workers seem pretty agreeable to them. Maybe they'll work for you, too.

First-Class Procedures, Part I: naming by a variable

In section 1.3.4 of SICP, the authors discuss (after Christopher Strachey) the rights and privileges of first-class citizens of a programming language, justifiably praising Lisp for awarding first-class status to procedures.

The first few real posts here will explore this topic as it relates to some other languages. Are procedures first-class citizens of Ruby? How about Erlang, or even JavaScript?


I'll start with Ruby, where the topic seems most controversial. I'll also start with the first condition that SICP suggests first-class citizens must satisfy: being nameable by a variable. Here's a sample irb session:

$ irb
irb(main):001:0> add1a = 1.method(:+)
=> #<Method: Fixnum#+>

On line 1, we identify a given procedure by the name add1a by using the method method to extract the action that is referred to when sending the :+ symbol as a message to the Fixnum 1. Ruby defines this operation as simple numeric addition, more or less, and the fact that we're grabbing this method from 1, as opposed to 42 or -6 acts as a closure, meaning that the extracted method remembers that the item to which its arguments should be added is 1, rather than 42 or any of the other instances from which we could have extracted our method. (Had we extracted the :+ method from a different instance, perhaps hello, world!, the operation wouldn't even have been defined as simple numeric addition, but that's another story).

irb(main):002:0> add1b = lambda { |x| 1 + x }
=> #<Proc:0xb7c8a638@(irb):2>

On line 2, we're identifying another procedure by the name add1b. This time, we're using lambda to construct the procedure, rather than extract a pre-existing method from an instance.

irb(main):003:0> add1c = Proc.new { |x| 1 + x }
=> #<Proc:0xb7c823c0@(irb):3>

On line 3, we're identifying yet another procedure by the name add1c. This time, we're using Proc.new rather than lambda, both of which create Proc objects.

None of these assignments of procedures into names would be terribly useful if they weren't callable.

irb(main):004:0> add1a.call(2)
=> 3
irb(main):005:0> add1b.call(2)
=> 3
irb(main):006:0> add1c.call(2)
=> 3

We see that all 3 of these named variables perform the expected operation when called. How does this compare with some other languages that are more definitively functional?


In deference to SICP, let's see how this operates in MIT Scheme:

$ mit-scheme
1 ]=> (define add1 (lambda (x) (+ x 1)))

;Value: add1

1 ]=> add1

;Value 11: #[compound-procedure 11 add1]

1 ]=> (add1 2)

;Value: 3

The theory is the same: we identify a procedure by the name (add1 in this case), and then call it (apply it to its arguments). Note, however, that calling the add1 procedure is slightly more straightforward in Scheme. We simply construct a list within parentheses such that the procedure to be called happens to be the first item, and the arguments to that procedure call are the remaining elements of the list. There is no need for a separate .call syntax. Ruby Procs can also be called without a .call method if one puts the arguments within [], rather than parentheses, but the main difference still stands, which is that the syntax of calling such a procedure differs (however slightly) from calling the built-in methods. Lisp fans who deride Ruby fans' claims of Ruby having first-class procedures point to this, not entirely without merit.


Another language that fits very well within the functional paradigm is Erlang:

$ erl
Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.5.5 (abort with ^G)
1> Add1 = fun(X) -> 1 + X end.
2> Add1(5).

here, we define a named variable Add1 that holds the procedure functionally equivalent to all of our examples so far, the action of adding 1 to the argument. Here, the action of calling our new named variable as a function is more straightforward, reminiscent of Scheme. We simply provide the argument within parentheses, as one calls any function in Erlang. There is a slight difference, in that variables in Erlang are capitalized, while built-in functions are lowercase. Still, functions are clearly nameable in Erlang.


What about JavaScript? JavaScript is a language that has gotten a largely undeserved bad rap, largely due to some bad implementations and what is perhaps the worst and most misleading name in programming language history. None of this detracts from the language itself, which my good friend Aubrey Keus calls Scheme with syntax. JavaScript reminds me a great deal of Ruby, in that it makes heavy use of both object-oriented and functional paradigms. Let's take a look at some JavaScript code that Aubrey provided:

add1 = function(x) { return x + 1; }

Look at how trivial that is to do. All one has to do is call add1(2) and 3 is returned, as one would expect. Again, the act of calling such a function is closer to (indistinguishable from) calling a built-in function.

I would argue that in all of these languages, procedures (whether implemented under the hood as functions or methods) satisfy this first criterion of being a first-class citizen. You may disagree, thinking that the additional hoops to be jumped through in the Ruby calls are too egregious. I find that from a pragmatic perspective, it's not that big a deal for me.

I also think Hashes are more like functions than Arrays, but that's another story.

So what's with the name?

I'm about to start writing a series of short posts, each about the question of whether procedures are first-class citizens in Ruby, as well as some other languages. Look for those starting in a few days. However, some folks might not be aware of the pun that inspired the name of this blog.

There may or may not be a legendary organization called The Knights of the λ Calculus. I wouldn't claim to call myself a wizard, nor do I primarily work in Lisp or Scheme, so I'm clearly not a member of such an illustrious group. However, I'll make frequent references to topics discussed in SICP, the Wizard Book, so I equally clearly have certain aspirations in this direction. Hence the blog name.