While I was working on a complex bit of packing and unpacking cryptographic tokens, a vexing problem started a journey:

How to test private functions (defp) in Elixir



A brief search yielded three answers:

  1. Not possible
  2. You should not test private functions
  3. @compile :export_all

I was stuck at 1 already. As for 2 I think testing a module internal is a good way of ensuring intermediate states are as expected and are a kind of active documentation. Both of these points are generally true about testing, why should they not apply to defp functions?

How to use @compile :export_all

I haven't needed to mess with compiler flags, so this was the first I'd seen of the @compile attribute. I gave it a try using this recommended syntax:

 @compile if Mix.env == :test, do: :export_all

The best part is that it works perfectly! For testing, any defp function can be called.

What is @compile :export_all

In this case, the question is really "why does it work?" The Elixir module documentation about @compile makes no direct mention of :export_all. In the compile options section there is a link to the Erlang compile page. I missed that link at first and followed my curiosity through the Elixir source code and found nothing explicit about :export_all. In elixir_module.erl, in the function compile a variable ModuleMap is built that includes the key compile_opts and that seems to be the place where the @compile attributes end up. I'm not familiar with the compiler internals and I may be missing some subtlety. From that point, things dive into the compiler.

At this point I took a step back and considered what happens from an Erlang standpoint. To do that I came up with a little test program:

defmodule P do
  @compile :export_all
  defp a, do: 15
  defp c, do: 18
  def b, do: a()
end

And compiled it directly:

elixirc p.ex

The first time I did this I just declared defp a and def b, but b did not call a which results in an error similar to what c now creates:

warning: function c/0 is unused
  p.ex:4

The translation of this message is: "the module doesn't call a private function c/0 so I (the compiler) am going to omit it from the beam file altogether." So my first attempt failed for reasons having nothing to do with :export_all

Now we have a file Elixir.P.beam. At this point, both Erlang and Elixir are dealing with the same result, a beam file. So now lets take a look at what we have:

$ erl
Erlang/OTP 22 [erts-10.4.4] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [hipe]

Eshell V10.4.4  (abort with ^G)
1> 'Elixir.P':module_info().
[{module,'Elixir.P'},
 {exports,[{'__info__',1},
           {a,0},
           {b,0},
           {module_info,0},
           {module_info,1}]},
 {attributes,[{vsn,[327438995637687391950271710388398484745]}]},
 {compile,[{version,"7.4.4"},
           {options,[no_spawn_compiler_process,from_core,
                     no_auto_import,export_all]},
           {source,"/home/sample_api/p.ex"}]},
 {native,false},
 {md5,<<"öVtè\ròiÞí]fQè¾m\t">>}]

In case you were not aware, the compiler adds module_info to all modules at compile time. The interesting bit is the compile keyword, the subkey options. This is what a keyword list looks like in Erlang. See the export_all atom, there it is. Also notice that the functions a/0 and b/0 are in the exports keyword. Even though a is a defp the :export_all has overridden that. For comfort, this is what it looks like in iex:

Erlang/OTP 22 [erts-10.4.4] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [hipe]

Interactive Elixir (1.9.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> P.module_info()
[
  module: P,
  exports: [__info__: 1, a: 0, b: 0, module_info: 0, module_info: 1],
  attributes: [vsn: [327438995637687391950271710388398484745]],
  compile: [
    version: '7.4.4',
    options: [:no_spawn_compiler_process, :from_core, :no_auto_import,
     :export_all],
    source: '/home/sample_api/p.ex'
  ],
  native: false,
  md5: <<246, 86, 116, 232, 13, 242, 105, 222, 237, 93, 102, 81, 232, 190, 109,
    9>>
]

It turns out that searching for export_all in the Erlang documentation doesn't get anywhere. But I knew it had to be in the compiler section, so I followed links in and around the compiler, finally getting to the options section of the compile page (the one linked to from the Elixir Module page noted above), to find:

export_all
Causes all functions in the module to be exported.

A few things snap together right here. First, a defp in Elixir seems to be nothing more than a non-exported function in Erlang terms. Conversely a def results in the function being put in a -export() in Erlang syntax. Second, a scan through the Erlang source tree finds many instances of -compile(export_all), the way to pass such a compiler flag in Erlang.