Wrong x86 instruction generated for function call via raw pointer

[spersson]

See example test case below, with two crates, one dynamically loading the other and trying to call it.

The wrong "callq" instruction is generated for x86-64, by rustc 0.12.0-dev, for the call site where the "main" crate calls a function from the "app" crate using the function pointer returned from DynamicLibrary.symbol().

app.rs:

#![crate_type="dylib"]

#[no_mangle]
pub extern fn application_loop() {
    println!("Hello from lib!");
}

main.rs:

#![crate_type="bin"]

use std::dynamic_lib::DynamicLibrary;
use std::path::Path;

fn test1() {
    println!("Hello from test1!");
}

fn main() {
// note that the following does not work, still need to run with $LD_LIBRARY_PATH 
// set to "." for the lib to be found
    let l_local_dir_path = std::os::make_absolute(&Path::new("."));
    DynamicLibrary::prepend_search_path(&l_local_dir_path);

    let l_library = match DynamicLibrary::open(Some("libapp.so")) {
        Err(msg) => fail!("Could not load the library: {}", msg),
        Ok(lib) => lib,
    };

    let l_app_loop: *mut extern fn() = unsafe {
        match l_library.symbol("application_loop") {
            Err(error) => fail!("Could not load function application_loop: {}", error),
            Ok(sym) => sym
        }
    };

     let f = test1;
// this call site works
    f();

    unsafe {
        println!("address is: {}", l_app_loop);
// this call site gets wrong addressing
        (*l_app_loop)();
    }
}

This gets generated:

0000000000008ffb <_ZN4main20hc17179379a8277f8UaaE+0x7bb>:
    8ffb:       48 8b 85 90 fd ff ff    mov    -0x270(%rbp),%rax
    9002:       ff 10                   callq  *(%rax)

The correct instruction would be this:

0000000000008ffb <_ZN4main20hc17179379a8277f8UaaE+0x7bb>:
    8ffb:       48 8b 85 90 fd ff ff    mov    -0x270(%rbp),%rax
    9002:       ff d0                   callq  *%rax

In other words, the address of the function to call ends up in register %rax, but instead of jumping to that address, the generated instruction will read the first 8 bytes of that function and interpret them as the address to jump to. Segfault.

Would have liked to help by doing more than reporting but this really seems like a problem for gurus, not for me... at least within reasonable timeframe...:)

[spersson]

turns out the best way to find a bug in your code is to report a bug in someone else's.
I found this now which has the solution:
http://www.reddit.com/r/rust/comments/2depfr/dynamiclibrary_usage/