Is a better Elixir REPL possible?

From the perspective of being an Elixir developer, one of the things I miss about Clojure is it’s REPL and the ability to eval code directly from the editor to the REPL session. For example, when working on a function, I can send the current code to the REPL to redefine the function and then play with it in the REPL. Elixir has a REPL, but you can’t do this.

Clojure has the concept of a “current namespace” (namespace is the nearest Clojure equivalent to Elixirs module). Anything evaluated in the REPL is done so in the context of this namespace. You can switch namespaces using the in-ns function. Elixir isn’t this loose, you can re-evaluate a function you need to specify it’s module, there’s no implicit module context.

def foo(), do: true
> ** (ArgumentError) cannot invoke def/2 outside module

defmodule X
  defn foo(), do: true
end
> X.foo() => true

I wonder if it would be possible to implement the concept of a “current module” and then automatically wrap any use of def with defmodule <current_module> do; …code…; end?

Elixir Protocols vs Clojure Multimethods

I am coming to appreciate José Valim’s creation, Elixir, very much. It is fair to say that Rich Hickey set a very high bar with Clojure and Elixir for the most part saunters over it. I think that is a reflection of two very thoughtful language designers at work. But there is some chafing when moving from Clojure to Elixir.

I wrote previously about how Clojure’s metadata feature lets one subtly piggyback data in a way that doesn’t require intermediate code to deal with it, or even know it exists. It creates a ‘functional’ backchannel. Elixir has no equivalent feature.

If your data is a map you can use “special” keys for your metadata. Elixir does this itself with the __struct__ key that it injects into the maps it uses as the implementation of structs. You mostly don’t know it’s there but would have to implement a special case if you ever treated the struct as a map.

However, if the value you want to attach metadata to is, say, an anonymous function then you’re out of luck. In that case, you have to convert your function to a map containing the function and metadata and then change your entire implementation. That could be a chore, or worse.

Today I hit the problem of wanting to define a function with multiple implementations depending on its arguments. Within a module, this is not hard to do as Elixir functions allow for multiple heads using pattern matching. It’s one of the beautiful things about writing Elixir functions. So:

defmodule NextWords do
  def next-word("hello"), do: "world"
  def next-word("foo"), do: "bar"
end

Works exactly as you would expect. Of course, the patterns can be considerably more complex than this and allow you to, for example, match values inside maps. So you could write:

def doodah(%{key: "foo"}), do: "dah"
def doodah(%{key: "dah"}), do: "doo"

And that would work just as you expect too! Fantastic!

But what about if you want the definitions of the function spread across different modules?

defmodule NextWords1 do
  def next-word("hello"), do: "world"
end

defmodule NextWords2 do
  def next-word("foo"), do: "bar"
end

This does not work because while NextWords1.next-word and NextWords2.next-word share the same name they are in all other respects unrelated functions. In Elixir a function is specified as a tuple of {module, name, arity} so functions in different modules are totally separate regardless of name & arity.

In Clojure, when I need to do something like this I would reach for a multi. A Clojure multi uses a function defined over its arguments to determine which implementation of the multi to use.

(defmulti foo [x y z] (fn [x y z] … turn x, y, z into a despatch value, e.g. :bar-1, :bar-2 or what have you))

(ns 'bar-1)
(defmethod foo :bar-1 [x y z] … implementation)

(ns 'bar-2)
(defmethod foo :bar-2 [x y z] … another implementation)

The dispatch function returns a dispatch value and the methods are parameterised on dispatch value. Different method implementations can live in different namespaces and a call with the right arguments will always resolve to the right implementation, regardless of where it was defined.

Now Elixir has an equivalent to multi, the Protocol. We can define the foo protocol:

defprotocol Fooish do
  def foo(x)
end

Now in any module, we can define an implementation for our Fooish protocol. But, and here’s the rub, an implementation looks like:

defmodule Foo-1 do
  defimpl Fooish, for: String
    def foo(s), do: …impl…
  end
end

So an Elixir protocol can only be parameterised on the type of its first argument! This means that there’s no way to dispatch protocol implementations based on lovely pattern matching. Disappointing.

It may even be that a smarter Elixir programmer than I could implement Clojure style multi-methods in Elixir. For now, I can find a work-around and I’m still digging Elixir a lot.

Value metadata is a subtle but useful language feature

Since I am using it to create AgendaScope, i’ve written quite a lot of Elixir code to get myself up to speed. Most recenty a parser combinator library, Ergo. And it’s in building Ergo that I’ve realised I really miss a feature from Clojure, metadata.

In short Clojure allows you to attach arbitrary key-value metadata to any value. While there is syntax sugar it boils down to:

meta(obj) to get the map of metadata key-value pairs for the value obj.

vary-meta(obj, f, args) to modify the meta-data associated with the value obj.

If you make a copy of obj the metadata is carried with it. In all other respects obj is unaffected. Why is this so useful?

Well I first used it to manage whether a map, representing a SQL row, comes from the database or not as well as associated data about the record. I could have put that in the map using “special” keys but this pollutes the map with stuff not strictly related to its purpose and more complex code with special cases to handle those keys. But what if obj wasn’t a map to which I could add special keys?

Clojure makes heavy use of it for self-documenting, tracking attributes, and tagging of all different kinds of structures.

What about Ergo? Well being a parser combinator library Ergo is about building functions, usually returning them as anonymous functions. When debugging a parser you’d really like to know more about it & how it was constructed but what you have is a function f.

In Clojure I’d just put all that in metadata of f and it would be no problem later to get at that metadata for debugging purposes by calling meta(f). The code of the parser itslef needing to know nothing about the metadata at all. But in Elixir… I am not sure what to do.

Elixir functions can have multiple heads so it occurs to me that I could return my parsers like:

fn
  %{Context} = ctx -> do parsing stuff here
  :info -> return some metadata instead
end

And this could work but also feels brittle to me. If a user defines a parser and doesn’t add an :info head then my debugging code that invokes f.(:info) is going to generate an error. It doesn’t smell right. Is this the best I can do?

I being to wish Elixir had an equivalent meta functionality to Clojure.