Calling C and Fortran Code - BenoitKnecht/julia GitHub Wiki

Though most code can be written in Julia, there are many high-quality, mature libraries for numerical computing already written in C and Fortran. To allow easy use of this existing code, Julia makes it simple and efficient to call C and Fortran functions. Julia has a "no boilerplate" philosophy: functions can be called directly from Julia without any "glue" code, code generation, or compilation — even from the interactive prompt. This is accomplished in three steps:

Load a shared library and create a handle to it.
Lookup a library function by name, getting a handle to it.
Call the library function using the built-in ccall function.

The code to be called must be available as a shared library. Most C and Fortran libraries ship compiled as shared libraries already, but if you are compiling the code yourself using GCC (or Clang), you will need to use the -shared and -fPIC options. The machine instructions generated by Julia's JIT are the same as a native C call would be, so the resulting overhead is the same as calling a library function from C code. (Non-library function calls in both C and Julia can be inlined and thus may have even less overhead than calls to shared library functions. When both libraries and executables are generated by LLVM, it is possible to perform whole-program optimizations that can even optimize across this boundary, but Julia does not yet support that. In the future, however, it may do so, yielding even greater performance gains.)

Shared libraries are loaded with dlopen function, which provides access to the functionality of the POSIX dlopen(3) call: it locates a shared library binary and loads it into the process' memory allowing the program to access functions and variables contained in the library. The following call loads the standard C library, and stores the resulting handle in a Julia variable called libc:

libc = dlopen("libc")

Once a library has been loaded, functions can be looked up by name using the dlsym function, which exposes the functionality of the POSIX dlsym(3) call. This returns a handle to the clock function from the standard C library:

libc_clock = dlsym(libc, :clock)

Finally, you can use ccall to actually generate a call to the library function. Inputs to ccall are as follows:

Function reference from dlsym — a value of type Ptr{Void}.
Return type, which may be any bits type, including Int32, Int64, Float64, or Ptr{T} for any type parameter T, indicating a pointer to values of type T, or just Ptr for void* "untyped pointer" values.
A tuple of input types, like those allowed for the return type.
The following arguments, if any, are the actual argument values passed to the function.

As a complete but simple example, the following calls the clock function from the standard C library:

julia> t = ccall(dlsym(libc, :clock), Int32, ())
5380445

julia> typeof(ans)
Int32

clock takes no arguments and returns an Int32. One common gotcha is that a 1-tuple must be written with with a trailing comma. For example, to call the getenv function to get a pointer to the value of an environment variable, one makes a call like this:

julia> path = ccall(dlsym(libc, :getenv), Ptr{Uint8}, (Ptr{Uint8},), "SHELL")
Ptr{Uint8} @0x00007fff5fbfd670

julia> cstring(path)
"/bin/zsh"

Note that the argument type tuple must be written as (Ptr{Uint8},), rather than just (Ptr{Uint8}). This is because (Ptr{Uint8}) is just Ptr{Uint8}, rather than a 1-tuple containing Ptr{Uint8}:

julia> (Ptr{Uint8})
Ptr{Uint8}

julia> (Ptr{Uint8},)
(Ptr{Uint8},)

In practice, especially when providing reusable functionality, one generally wraps ccall usages in Julia functions that set up arguments and then check for errors in whatever manner the C or Fortran function indicates them, propagating to the Julia caller as exceptions. This is especially important since C and Fortran APIs are notoriously inconsistent about how they indicate error conditions. For example, the getenv C library function is wrapped in the following Julia function in env.j:

function getenv(var::String)
  val = ccall(dlsym(libc, :getenv),
              Ptr{Uint8}, (Ptr{Uint8},), cstring(var))
  if val == C_NULL
    error("getenv: undefined variable: ", var)
  end
  cstring(val)
end

The C getenv function indicates an error by returning NULL, but other standard C functions indicate errors in various different ways, including by returning -1, 0, 1 and other special values. This wrapper throws an exception clearly indicating the problem if the caller tries to get a non-existent environment variable:

julia> getenv("SHELL")
"/bin/zsh"

julia> getenv("FOOBAR")
getenv: undefined variable: FOOBAR

Here is a slightly more complex example that discovers the local machine's hostname:

function gethostname()
  hostname = Array(Uint8, 128)
  ccall(dlsym(libc, :gethostname), Int32, 
        (Ptr{Uint8}, Ulong),
        hostname.data, ulong(length(hostname)))
  return cstring(convert(Ptr{Uint8}, hostname))
end

This example first allocates an array of bytes, then calls the C library function gethostname to fill the array in with the hostname, takes a pointer to the hostname buffer, and converts the pointer to a Julia string, assuming that it is a NUL-terminated C string. It is common for C libraries to use this pattern of requiring the caller to allocate memory to be passed to the callee and filled in. Allocation of memory from Julia like this is generally accomplished by creating an uninitialized array and passing a pointer to its data to the C function.

In case of a Fortran function, all inputs must be passed by reference. The following example computes a dot product using a BLAS function.

libBLAS = dlopen("libLAPACK")

function compute_dot(DX::Vector, DY::Vector)
  assert(length(DX) == length(DY))
  n = length(DX)
  incx = incy = 1
  product = ccall(dlsym(libBLAS, :ddot_),
                  Float64,
                  (Ptr{Int32}, Ptr{Float64}, Ptr{Int32}, Ptr{Float64}, Ptr{Int32}),
                  int32(n), DX, int32(incx), DY, int32(incy))
  return product
end

Note that no C header files are used anywhere in the process. Currently, it is not possible to pass structs and other non-primitive types from Julia to C libraries. However, C functions that generate and use opaque structs types by passing around pointers to them can return such values to Julia as Ptr{Void}, which can then be passed to other C functions as Ptr{Void}. Memory allocation and deallocation of such objects must be handled by calls to the appropriate cleanup routines in the libraries being used, just like in any C program.

Previous: Parallel Computing — Next: Standard Library Reference