How to make Miri see symbols in a rust-based cdylib?

[HKalbasi]

I have a binary crate with this main.rs:

extern "C" fn foo1() {
    dbg!(1);
}

extern "C" {
    fn foo2();
    fn foo4();
}

fn main() {
    foo1();
    unsafe {
        foo2();
        foo4();
    }
}

foo2 is defined in a normal cargo dependency, but foo4 is defined in a file gav.rs that I build by invoking rustc directly in the build script:

use std::process::Command;

fn main() {
    let out_dir = build::out_dir();
    let rustc = build::rustc();
    build_print::note!("{rustc:?}");
    let mut cmd = Command::new(rustc);
    cmd.arg("--crate-name=gav")
        .arg("--crate-type=cdylib")
        .arg("--emit=link")
        .arg("--out-dir")
        .arg(&out_dir)
        .arg("gav.rs");
    cmd.status().expect("failed to invoke rustc");
    build::rustc_link_search(&out_dir);
    build::rustc_link_lib("gav");
}

Miri can find and execute the foo2, but not foo4. I believe if I call miri sufficiently close to how cargo calls it, it will work. What should I do to make it working?

[bjorn3]

All dependencies need to be compiled with MIR for all functions available in the crate metadata for miri to work. The cdylib crate type however doesn't generate any crate metadata at all (even --emit metadata will produce an empty file). It doesn't need to anyway as it doesn't expose any Rust ABI by definition.

[HKalbasi]

I tried to detect if miri is present and change cdylib to rlib, but it didn't work.

[RalfJung]

A Rust-based cdylib is indistinguishable from a C-based native library, so... no this does not work, unfortunately. You can try playing around with Miri's native FFI support (but note that sharing memory with the native library can miss UB and not all cases are supported). Making that more automatic is loosely tracked in #4077, though there are no concrete plans or even ideas at the moment.

[HKalbasi]

A Rust-based cdylib is indistinguishable from a C-based native library

But Miri can see the definition of foo4 with some effort (for example building with rlib instead of cdylib) in the build script I guess. The problem is to find a way to trick Miri to keep the symbol in its final execution session.

[RalfJung]

Miri can see everything in the current crate graph, that's why it can see foo2. That's all directly accessible inside rustc. But there's a hard boundary at the edge of the crate graph -- if you do linking on the level of C libraries rather than Rust crates, there's nothing we can do.

I don't know if linking this as an rlib will suffice -- we do a whole lot of magic in cargo miri to make everything work. I have no clue what it would take to do the same from a build script or whether that would even work, sorry.

[HKalbasi]

Miri can see everything in the current crate graph,

That would be hopeless. It seems there is no way to add a rlib to the crate graph in a build script rust-lang/cargo#14065.

What about dylib? Is it as bad as cdylib or is it imaginable for it to contain MIR in future (currently it doesn't work)? It seems cargo can add a dylib similar to a cdylib in a build script.

[RalfJung]

Miri needs the MIR of that crate, loaded into the current rustc session. I don't think there is any way to do that without --extern; that's literally what that flag is for.