Skip to content

Redox Cross Compilation #38401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Dec 23, 2016
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
341d2d1
Add redox target
jackpot51 Dec 13, 2016
c7aa284
Fix typo
jackpot51 Dec 13, 2016
a621d12
Fix issue with setting cfg(unix)
jackpot51 Dec 13, 2016
d2707aa
Use panic abort by default
jackpot51 Dec 13, 2016
ece703a
Add Redox make config
jackpot51 Dec 13, 2016
f86e014
Use alloc_system as default allocation crate
jackpot51 Dec 14, 2016
3e7543a
WIP: Cross-compilation for Redox target
jackpot51 Dec 15, 2016
773a0a2
Add start functions, switch allocation crate to ralloc
jackpot51 Dec 15, 2016
07e313d
Add openlibm to redox
jackpot51 Dec 15, 2016
6d7c2ec
Revert libstd/Cargo.toml to master
jackpot51 Dec 15, 2016
57bc1a9
Add arm syscalls
jackpot51 Dec 20, 2016
86f85c1
Move start functions into libstd/rt
jackpot51 Dec 20, 2016
01157e6
Link openlibm only in libstd
jackpot51 Dec 20, 2016
e55596f
Move rt into sys::rt, fix tidy
jackpot51 Dec 20, 2016
65eecf8
Readd statvfs
jackpot51 Dec 20, 2016
fd4bc88
Fix building without backtrace
jackpot51 Dec 21, 2016
7697c72
Static link openlibm
jackpot51 Dec 21, 2016
2ca1f0b
Switch back to alloc_system
jackpot51 Dec 21, 2016
bf50acb
Fix tidy
jackpot51 Dec 21, 2016
e909e43
Update liblibc, go back to lazy linking openlibm
jackpot51 Dec 21, 2016
92c8e0f
Merge branch 'redox_cross' of https://github.com/redox-os/rust into r…
jackpot51 Dec 21, 2016
7d3ae87
Add RawFd traits for net
jackpot51 Dec 22, 2016
e7b006d
In order to successfully build, go back to ralloc
jackpot51 Dec 22, 2016
1eb6c44
Remove start functions, use newlib instead of openlibm + ralloc
jackpot51 Dec 22, 2016
2ddd117
Revert rt.rs
jackpot51 Dec 22, 2016
474eb62
Do not build emutls on Redox
jackpot51 Dec 23, 2016
c59bb49
Correct target_family mess
jackpot51 Dec 23, 2016
4dcb867
Convert fam to Symbol
jackpot51 Dec 23, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions mk/cfg/x86_64-unknown-redox.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# rustbuild-only target
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old build system is complicated to use, so there is no support for using it.

3 changes: 2 additions & 1 deletion src/liballoc_jemalloc/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ fn main() {
// targets, which means we have to build the alloc_jemalloc crate
// for targets like emscripten, even if we don't use it.
if target.contains("rumprun") || target.contains("bitrig") || target.contains("openbsd") ||
target.contains("msvc") || target.contains("emscripten") || target.contains("fuchsia") {
target.contains("msvc") || target.contains("emscripten") || target.contains("fuchsia") ||
target.contains("redox") {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not support jemalloc, so we will build with the dummy.

println!("cargo:rustc-cfg=dummy_jemalloc");
return;
}
Expand Down
45 changes: 44 additions & 1 deletion src/liballoc_system/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
issue = "27783")]
#![feature(allocator)]
#![feature(staged_api)]
#![cfg_attr(target_os = "redox", feature(libc))]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that ralloc is used as the default allocator in librust_back. What we have done here is to ensure that liballoc_system builds, as it is a required dependency of libstd. We never link to it, however.

#![cfg_attr(unix, feature(libc))]

// The minimum alignment guaranteed by the architecture. This value is used to
Expand Down Expand Up @@ -71,7 +72,49 @@ pub extern "C" fn __rust_usable_size(size: usize, align: usize) -> usize {
imp::usable_size(size, align)
}

#[cfg(unix)]
#[cfg(target_os = "redox")]
mod imp {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just so we have a stub for potentially using a libc allocator, we implement alloc_system similarly to how the unix implementation works. If desired, these two imp blocks could be combined.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're the one defining the abi of the system allocator, so this is largely up to you. If you follow unix's malloc/free conventions then I'd recommend unifying with the above block for Unix. That handles bits and pieces such as alignment for more-aligned types (e.g. simd types).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will probably change this to be different from Unix so that it defines externs that ralloc will fulfill.

extern crate libc;

use core::cmp;
use core::ptr;
use MIN_ALIGN;

pub unsafe fn allocate(size: usize, _align: usize) -> *mut u8 {
libc::malloc(size as libc::size_t) as *mut u8
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memalign is not used like in the unix implementation, perhaps this is a problem

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I this is one case where I think you'll want to align with the implementation above. This really ends up just needing an allocator interface which supports alignment as an argument, and that's what memalign is targeted at.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, thanks

}

pub unsafe fn reallocate(ptr: *mut u8, old_size: usize, size: usize, align: usize) -> *mut u8 {
if align <= MIN_ALIGN {
libc::realloc(ptr as *mut libc::c_void, size as libc::size_t) as *mut u8
} else {
let new_ptr = allocate(size, align);
if !new_ptr.is_null() {
ptr::copy(ptr, new_ptr, cmp::min(size, old_size));
deallocate(ptr, old_size, align);
}
new_ptr
}
}

pub unsafe fn reallocate_inplace(_ptr: *mut u8,
old_size: usize,
_size: usize,
_align: usize)
-> usize {
old_size
}

pub unsafe fn deallocate(ptr: *mut u8, _old_size: usize, _align: usize) {
libc::free(ptr as *mut libc::c_void)
}

pub fn usable_size(size: usize, _align: usize) -> usize {
size
}
}

#[cfg(any(unix))]
mod imp {
extern crate libc;

Expand Down
3 changes: 3 additions & 0 deletions src/librustc_back/target/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ mod windows_base;
mod windows_msvc_base;
mod thumb_base;
mod fuchsia_base;
mod redox_base;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We must add a new OS base for Redox


pub type TargetResult = Result<Target, String>;

Expand Down Expand Up @@ -184,6 +185,8 @@ supported_targets! {
("aarch64-unknown-fuchsia", aarch64_unknown_fuchsia),
("x86_64-unknown-fuchsia", x86_64_unknown_fuchsia),

("x86_64-unknown-redox", x86_64_unknown_redox),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We support a single architecture for Redox, x86_64


("i386-apple-ios", i386_apple_ios),
("x86_64-apple-ios", x86_64_apple_ios),
("aarch64-apple-ios", aarch64_apple_ios),
Expand Down
53 changes: 53 additions & 0 deletions src/librustc_back/target/redox_base.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
// Copyright 2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

use PanicStrategy;
use target::TargetOptions;
use std::default::Default;

pub fn opts() -> TargetOptions {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This target is a bit different from others. Redox does not link to a libc, and does not have a rational C cross compiler. For this reason, some things have to be added. I will explain later.

TargetOptions {
pre_link_args: vec![
// We want to be able to strip as much executable code as possible
// from the linker command line, and this flag indicates to the
// linker that it can avoid linking in dynamic libraries that don't
// actually satisfy any symbols up to that point (as with many other
// resolutions the linker does). This option only applies to all
// following libraries so we're sure to pass it as one of the first
// arguments.
"-Wl,--as-needed".to_string(),

// Always enable NX protection when it is available
"-Wl,-z,noexecstack".to_string(),

// Do not link libc
"-nostdlib".to_string(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-nostdlib must be added to prevent gcc from attempting to link the host's libc.


// Static link
"-static".to_string()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redox does not support dynamic linking, so all files should be linked statically.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we've recently added support for cfg(crt-static = "..."), so could you be sure to set that flag in this file as well?

],
late_link_args: vec![
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We late link libc and libm so that they can be provided when binaries are linked, rather than using the compiler defaults.

// Link to openlibm for math functions
"-lopenlibm".to_string()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only C library we utilize is openlibm for math functions, we must specify it explicitly. It is compiled out of tree from here: https://github.com/redox-os/openlibm

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general the arguments passed in this file, src/librustc_back/target/*.rs are linker arguments required for all binaries generated for the target. Presumably, though, openlibm isn't required for all binaries?

What I'd recommend here is moving openlibm linkage to the standard library's build script, as this is essentially a dependency of the standard library, not all executables.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, it is only a dependency of libstd, not of a #[no_std] binary. I will move this.

],
executables: true,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are able to generate executables. This is done rather unhygienically in libstd with a _start function.

relocation_model: "static".to_string(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no support for relocation in Redox.

disable_redzone: true,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Red zones are not currently supported by Redox. This may be enabled in the future after testing.

eliminate_frame_pointer: false,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not eliminate the frame pointer so that stack frames are easily traversable.

target_family: Some("redox".to_string()),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a controversial change. With the way things are currently handled, setting this to None implies Some("unix".to_string()), which enables a lot of code that is not compatible with Redox.

linker_is_gnu: true,
no_default_libraries: true,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No default libraries must be linked, as they would come from the host.

lib_allocation_crate: "ralloc".to_string(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ralloc is used as the default allocation crate. It is meant to be compiled out of tree from this: https://github.com/redox-os/ralloc/tree/redox_cross

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may have discussed this a bit on IRC, but I think this'll be best as alloc_system as these values are really intended for crates which are shipped as part of the main distribution. If this is a crate which is supplied externally it could cause lots of problems down the road.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will set it back to alloc_system and have alloc_system use externs that ralloc will fill in

exe_allocation_crate: "ralloc".to_string(),
has_elf_tls: true,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do support TLS, so we enable it.

panic_strategy: PanicStrategy::Abort,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not support unwinding, so we panic using abort

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unwinding is not supported at the moment, so we abort.

.. Default::default()
}
}
30 changes: 30 additions & 0 deletions src/librustc_back/target/x86_64_unknown_redox.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
// Copyright 2016 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

use target::{Target, TargetResult};

pub fn target() -> TargetResult {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These settings are very similar to those in other targets, and are nothing special.

let mut base = super::redox_base::opts();
base.cpu = "x86-64".to_string();
base.max_atomic_width = Some(64);
base.pre_link_args.push("-m64".to_string());

Ok(Target {
llvm_target: "x86_64-unknown-redox".to_string(),
target_endian: "little".to_string(),
target_pointer_width: "64".to_string(),
data_layout: "e-m:e-i64:64-f80:128-n8:16:32:64-S128".to_string(),
arch: "x86_64".to_string(),
target_os: "redox".to_string(),
target_env: "".to_string(),
target_vendor: "unknown".to_string(),
options: base,
})
}
2 changes: 1 addition & 1 deletion src/libstd/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ fn main() {
let target = env::var("TARGET").expect("TARGET was not set");
let host = env::var("HOST").expect("HOST was not set");
if cfg!(feature = "backtrace") && !target.contains("apple") && !target.contains("msvc") &&
!target.contains("emscripten") && !target.contains("fuchsia") {
!target.contains("emscripten") && !target.contains("fuchsia") && !target.contains("redox") {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We disable the build of libbacktrace, as it will not currently compile for Redox

build_libbacktrace(&host, &target);
}

Expand Down
6 changes: 6 additions & 0 deletions src/libstd/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,7 @@
#![feature(unwind_attributes)]
#![feature(vec_push_all)]
#![feature(zero_one)]
#![cfg_attr(target_os = "redox", feature(naked_functions))]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hygiene error! In order to generate executables, we have to have a naked _start function at the libstd root.

#![cfg_attr(test, feature(update_panic_count))]

// Explicitly import the prelude. The compiler uses this same unstable attribute
Expand Down Expand Up @@ -422,6 +423,11 @@ pub use core_collections::vec;
#[stable(feature = "rust1", since = "1.0.0")]
pub use std_unicode::char;

// Reexport the start module on platforms that provide it
#[unstable(feature = "start_fn", issue="0")]
#[cfg(target_os = "redox")]
pub use sys::start::*;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hygiene error! In order to generate executables, we have to have a naked _start function at the libstd root.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually currently have src/libstd/rt.rs which is full of hacks like this, could you move this over there?

Copy link
Contributor Author

@jackpot51 jackpot51 Dec 20, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is that it must be exposed at the crate root (they are no_mangle functions for the linker to find). I could have pub use sys::start; in libstd::rt, but then rt::start::* would still have to be pub used in lib.rs

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe you just need the symbols exported, right? In that case they just need to be reachable publicly, not literally at the crate root. That's what the rt module suffices to do (it's entirely unstable)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes.


pub mod f32;
pub mod f64;

Expand Down
4 changes: 2 additions & 2 deletions src/libstd/sys/redox/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@

#![allow(dead_code, missing_docs, bad_style)]

pub extern crate syscall;

use io::{self, ErrorKind};
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have moved the syscall crate in-tree to the syscall module.


pub mod args;
Expand All @@ -33,7 +31,9 @@ pub mod process;
pub mod rand;
pub mod rwlock;
pub mod stack_overflow;
pub mod start;
pub mod stdio;
pub mod syscall;
pub mod thread;
pub mod thread_local;
pub mod time;
Expand Down
3 changes: 2 additions & 1 deletion src/libstd/sys/redox/net/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,11 @@ use vec::{IntoIter, Vec};

use self::dns::{Dns, DnsQuery};

pub extern crate libc as netc;
pub use self::tcp::{TcpStream, TcpListener};
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We no longer use the libc crate for netc definitions. They are now inside the netc module.

pub use self::udp::UdpSocket;

pub mod netc;

mod dns;
mod tcp;
mod udp;
Expand Down
47 changes: 47 additions & 0 deletions src/libstd/sys/redox/net/netc.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
pub type in_addr_t = u32;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the definitions from libc that are relevant in std::net

pub type in_port_t = u16;

pub type socklen_t = u32;
pub type sa_family_t = u16;

pub const AF_INET: sa_family_t = 1;
pub const AF_INET6: sa_family_t = 2;

#[derive(Copy, Clone)]
#[repr(C)]
pub struct in_addr {
pub s_addr: in_addr_t,
}

#[derive(Copy, Clone)]
#[repr(C)]
pub struct in6_addr {
pub s6_addr: [u8; 16],
__align: [u32; 0],
}

#[derive(Copy, Clone)]
#[repr(C)]
pub struct sockaddr {
pub sa_family: sa_family_t,
pub sa_data: [u8; 14],
}

#[derive(Copy, Clone)]
#[repr(C)]
pub struct sockaddr_in {
pub sin_family: sa_family_t,
pub sin_port: in_port_t,
pub sin_addr: in_addr,
pub sin_zero: [u8; 8],
}

#[derive(Copy, Clone)]
#[repr(C)]
pub struct sockaddr_in6 {
pub sin6_family: sa_family_t,
pub sin6_port: in_port_t,
pub sin6_flowinfo: u32,
pub sin6_addr: in6_addr,
pub sin6_scope_id: u32,
}
25 changes: 21 additions & 4 deletions src/libstd/sys/redox/rand.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,19 @@
// except according to those terms.

use io;
use libc;
use rand::Rng;

pub struct OsRng;
// FIXME: Use rand:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to specify a simple OsRng algorithm until the rand: process is started. If you have more questions, please ask them here. In the future, rand: will be in the kernel to allow the init process to execute. (Rust currently uses OsRng in every process for the HashState)

pub struct OsRng {
state: [u64; 2]
}

impl OsRng {
/// Create a new `OsRng`.
pub fn new() -> io::Result<OsRng> {
Ok(OsRng)
Ok(OsRng {
state: [0xBADF00D1, 0xDEADBEEF]
})
}
}

Expand All @@ -26,7 +30,20 @@ impl Rng for OsRng {
self.next_u64() as u32
}
fn next_u64(&mut self) -> u64 {
unsafe { libc::random() }
// Store the first and second part.
let mut x = self.state[0];
let y = self.state[1];

// Put the second part into the first slot.
self.state[0] = y;
// Twist the first slot.
x ^= x << 23;
// Update the second slot.
self.state[1] = x ^ y ^ (x >> 17) ^ (y >> 26);

// Generate the final integer.
self.state[1].wrapping_add(y)

}
fn fill_bytes(&mut self, buf: &mut [u8]) {
for chunk in buf.chunks_mut(8) {
Expand Down
Loading