Interfacing between Ruby and Fortran (Via C)
(Spring - 2014)
Introduction
You might be thinking that Ruby and Fortran is an odd mixture... and you're certainly correct. But in addition to being an exercise in hooking things together, I feel that the languages complement each other:
Ruby | Fortran |
---|---|
Elegant, expressive, & maleable... at the expense of speed and native support for matrix operations. | Awesome for numerical computing... at the expense of being rigid, unelegant and antiquated. |
Has an extensive and vibrant community. (Modern codebase) | Historically established (Some things are only in Fortran) and rich (There are tons of Fortran programs sitting around on the web from years past.) (Large historical codebase) |
Overview:
- Fortran & C -- Calling a Fortran function from C
- Compilation Flags and Automation -- Creating a build script
- C and Ruby -- Calling a C function from Ruby
- Fortran from Ruby -- Putting it all together with the Sieve of Eratosthenes
- Benchmarks & Conclusion
I don't know if there's a reason or way to use this in production... but lets figure it out anyway!
If you're interested in the subject and you have time, you might want to check out the reference reading as well.
Section 1: Calling Fortran from C.
Lets first look at calling a Fortran function. This function will simply add to its integer parameter.
:::fortran ! test1.f08 function test_function(test_input) bind(c, name="test_function") result(test_output) use, intrinsic :: iso_c_binding implicit none integer(kind=c_int), intent(in) :: test_input integer(kind=c_int) :: test_output test_output = test_input + 10 end function test_function
Notable features:
- According to the GFortran spec, the bind statement is required. In practice, it allows us to specify the exported symbol name which by default has an extra underscore suffix.
- The iso_c_binding module is used to get interoperable datatypes. We make Fortran play by C's rules. Although I only used integers, there is a whole set of equivalents.
And then we're going to need the C program which calls it:
:::C // test1.c #include <stdio.h> int test_function(int *); // Equivalent to int test_function_(int*); int main() { int i = 10; int return_value; return_value = test_function(&i); // NOT equivalent to test_function_(&i) unless we omit bind(c, "test_function") printf("Return value was: %d\n", return_value); return 0; }
Notes:
- We forward-declare our imported function. If we omit bind(c), it doesn't seem to matter if this declaration has the underscore suffix.
- We pass the function parameter by reference... C can handle either method, but Fortran only handles by references.
- We can see that C recieves two different symbol names for our function depending on the bind statement.
Finally, we can compile, link and execute to prove our method works.
Section 2: Creating a Build Script
In order to compile our C files with Ruby support, we need to include the Ruby headers. This is most easily accomplished with rbconfig. Lets automate the process using a Rakefile:
:::ruby # Rakefile require 'rake/clean' require 'rbconfig' # Add our output files to clobber list so we can call 'rake clobber' CLOBBER.include('*.o', '*.so') # Use RbConfig to get the Ruby library locations for inclusion into our C programs. CFLAGS = RbConfig::CONFIG["CFLAGS"] HDRDIR = RbConfig::CONFIG["rubyhdrdir"] ARCHHDRDIR = RbConfig::CONFIG["rubyarchhdrdir"] # Task for test 1 (C and Fortran interop). # If you've never used Rake (or Make) before, our output files depend on the input files, # and the task being completed depends on us having those output files. task :test1 => ["test1_f.o", "test1_c.o"] file "test1_f.o" => "test1.f08" do |t| compile(t.prerequisites.first, t.name) end file "test1_c.o" => "test1.c" do |t| compile(t.prerequisites.first, t.name) end # Task for test 2 (Ruby and C interop) task :test2 => "test2.o" file "test2.o" => "test2.c" do |t| compile_c_for_ruby(t.prerequisites.first, t.name) end # Task for test 3 (Calling Fortran from Ruby through C) task :test3 => "sievemodule.so" task :default => :test3 file "sievemodule.so" => ["driver.o", "provider.o"] do |t| sh "gcc -shared -o sievemodule.so driver.o provider.o -lgfortran -lm -lruby" end file "driver.o" => "driver.c" do |t| compile_c_for_ruby(t.prerequisites.first, t.name) end file "provider.o" => "provider.f08" do |t| compile(t.prerequisites.first, t.name) end # Generic compilation function which works for both C and F08 files without special inclusions. def compile(src, target) sh "gcc -c -fPIC -o #{target} #{src}" end # Specific compilation function to include Ruby headers for C. def compile_c_for_ruby(src, target) sh "gcc -c -I #{ARCHHDRDIR} -I#{HDRDIR} #{CFLAGS} -o #{target} #{src}" end
Some features of our build script:
- We set up three commands: 'rake test1', 'rake test2' and 'rake test3' for compilation (and in the third case, linking together) and 'rake clobber' to remove our generated files.
- RbConfig provides the Ruby header locations and C compilation flags.
- Both the Fortran and C files are compiled with fPIC. (It's included in CFLAGS.)
- In the third case, the C and Fortran object code is linked into our sievemodule.so, linked against the GFortran, math and Ruby libraries.
With this in hand, we can proceed into the realm of C/Ruby integration...
Section 3: Calling C from Ruby
Calling C from Ruby is more complicated... Not only do we have to write our functional C code, but also have to create a ruby module & method using the tools provided with the Ruby header, and handle data types between the two languages. I strongly suggest looking over the Ruby/C Interop references to understand what's going on. This time, lets just print out a string.
:::C // test2.c #include "ruby.h" #include "stdio.h" VALUE TestModule = Qnil; VALUE method_test_function(VALUE self) { printf("Hello, Ruby. (From C)\n"); return 0; } void Init_testmodule() { TestModule = rb_define_module("TestModule"); rb_define_method(TestModule, "test_function", method_test_function, 0); }
Lets see what's going on here:
- We need to include the Ruby header, which requires an extra statement during compilation.
- C uses the VALUE type to represent Ruby data. Everything is an object, so everything in our interface code is a VALUE!
- We initialize our module, TestModule, to the Ruby nil datatype, Qnil.
- We define a method with the prefix 'method_'. Also, Ruby functions written in C need the 'VALUE self' argument even when our actual parameter count is zero.
- Every module calls its 'Init_' function when loaded, which we use to build our Ruby module object.
- We also define the method as belonging to the module. That last argument corresponds to parameter count.
The ruby script that consumes this, however, is blissfully simple:
:::Ruby # test2.rb require './testmodule.so' include TestModule test_function()
All we need to do is invoke our Rakefile in order to compile and then run the ruby driver.
rake test2 gcc -shared -o testmodule.so test2.o -lruby ruby test2.rb --> Hello, Ruby. (From C)
Section 4: Putting it Together: Calling Fortran from Ruby
Now we can put everything together. Lets so something slightly more interesting and calculate primes using the Sieve of Eratosthenes algorithm. First we need a Fortran subroutine to call:
subroutine sieve_of_eratosthenes(max_value, number_list) bind(c, name="sieve_of_eratosthenes") use, intrinsic :: iso_c_binding implicit none integer(kind=c_int), intent(in) :: max_value integer(kind=c_int), dimension(max_value), intent(out) :: number_list(max_value) integer(kind=c_int) :: outer_high_bound, inner_high_bound, i number_list = 1 number_list(1) = 0 outer_high_bound = int (sqrt (real (max_value))) inner_high_bound = max_value do i = 2, outer_high_bound if (number_list(i) == 1) number_list(i*i : max_value : i) = 0 end do end subroutine sieve_of_eratosthenes
Notes:
- Apparently a Fortran function won't let us pass an array as a parameter, but a subroutine will.
- We create an array to hold the value in C and pass the reference in to be filled by Fortran.
The C file serving as a middleman:
#include#include "ruby.h" int sieve_of_eratosthenes(int *, int *); VALUE SieveModule = Qnil; VALUE method_invoke_sieve(VALUE self, VALUE iterations) { int output [iterations], ii; VALUE result; int c_iterations = NUM2INT(iterations); if (c_iterations) { sieve_of_eratosthenes(&c_iterations, output); } result = rb_ary_new(); for(ii=0; ii<c_iterations; ii++) { int prime = ii + 1; rb_ary_push(result, INT2FIX(output[prime])); } return result; } void Init_sievemodule() { SieveModule = rb_define_module("SieveModule"); rb_define_method(SieveModule, "invoke_sieve", method_invoke_sieve, 1); }
Notes:
- We do a forward-declaration for the sieve subroutine and initialize our module variable to Ruby-nil.
- We build a ruby method which we package into a module and expose to Ruby below.
- There's a little bit of data-type-dancing going on... we accept a 'number' iterations from Ruby which needs to be converted into a C-int. Then when packing our array to return to Ruby, we want to convert integers into fixed numbers.
Now we can put the final piece in place: A very simple Ruby driver:
require './sievemodule.so' include SieveModule iterations = 1000 result = invoke_sieve(iterations) puts "Primes under #{iterations}:" result.each_with_index do |item, ii|; if result[ii] == 1 # Then value is a prime; Print it out. print ii + 2 print " " else # Value is composite. end end puts "\n"
Notes:
- The array we actually returned from the Sieve subroutine was a list of integers with '1' if that value is prime and '0' if not.
Now we just have to compile, link, and try it out:
Benchmarks and Conclusions
Methodology:
I tested three different implementations of the Sieve of Eratosthenes: My own Ruby/C/Fortran stack (shown above,) a pure Fortran implementation using the same core algorithm, and a pure Ruby implementation. (Codes for which have been lost but were completely basic implementations.)
Each program was tested against 10,000 iterations and 1,000,000 each. In addition, each run was repeated without I/O processing (printing of the prime numbers and determining which ones are prime.) For each circumstance, run time was measured with the Unix utility time 10 times and averaged (arithmetic mean.)
Implementation | Run time (10,000) | Run time (1,000,000) | Description |
---|---|---|---|
Pure Ruby | 0.1566 (seconds) | 1.7979 s | I/O included |
Pure Ruby | 0.0267 s | 0.6064 s | Computation only |
Pure Fortran | 0.0138 s | 2.0481 s | I/O included |
Pure Fortran | 0.003 s | 0.0192 s | Computation only |
Full stack | 0.0393 s | 1.404 s | I/O included |
Full stack | 0.0221 s | 0.0592 s | Computation only |
Conclusion:
Here we see that I/O is a significant factor, with Fortran actually coming in as the slowest when I/O is included. Naturally, it is lightning-fast again sans output. Note that this particular implementation of the Sieve requires the I/O routine to check which data are primes, increasing the load on that part of the full task while lessening the work done by the "computation only" circumstances.
Pure Ruby is on par with its C/Fortran extension given a small task with such an efficient algorithm. But after boosting the work up to 1 million integers, pure Ruby is left in the dust.
I conclude that a binary extension is definitely worth the time spent for Ruby scripts that need to do any significant calculation. We all already knew that, of course. I'm not sure that writing any new functions in Fortran is a better idea than writing them in C, because every new datatype is another torrent of complexity when you have to convert it back and forth.
Further investigation is needed to determine whether Fortran extensions would be even remotely robust. In addition, I intend to explore interfacing with existing legacy code, which is probably a much more viable use of Ruby/Fortran.
Thanks for reading! Hope you're inspired to play further with Fortran or Ruby.
References:
- [1] Pickaxe: C Extensions
- The quinessential 'learning Ruby' book, available online, discussing C extensions.
- [2] Eqqon & Matz: C Extension Libraries
- An annotated version of an explanatory text that Matz, creator of Ruby, checked somewhere into the source.
- [3] GFortran: Mixed Language Programming
- GCC Fortran compiler documentation on C/Fortran interoperation and extensions.
- [4] UCLA: C/Fortran Interoperation
- Example code and compilation options for C calling Fortran code, as well as Fortran calling C code.
- [5] GFortran: ISO_C_BINDING Intrinsic Module
- Documentation for the Fortran module intended for C interoperability. Good datatype reference.
- [5] Wikipedia: Sieve of Eratosthenes
- Wiki explanation of Sieve of Eratosthenes history and algorithm