Skip to content

Struct with too many fields causes stack overflow in tests #2218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Bromeon opened this issue Jun 6, 2022 · 8 comments · Fixed by #2219
Closed

Struct with too many fields causes stack overflow in tests #2218

Bromeon opened this issue Jun 6, 2022 · 8 comments · Fixed by #2219

Comments

@Bromeon
Copy link

Bromeon commented Jun 6, 2022

The newly released bindgen 0.60.0 introduces a regression in generated tests. When using a struct with a large number of fields, the tests cause a stack overflow. This could also be an issue in rustc itself.

I discovered it in godot-rust/gdnative#897 (this CI run) and could reproduce it locally.

Input C/C++ Header

For me, the threshold was 694 fields (693 worked fine); this might differ on other machines depending on stack size. So consider increasing the number of fields if you can't reproduce it with below code.

Click to expand
typedef struct {
	int field_0;
	int field_1;
	int field_2;
	int field_3;
	int field_4;
	int field_5;
	int field_6;
	int field_7;
	int field_8;
	int field_9;
	int field_10;
	int field_11;
	int field_12;
	int field_13;
	int field_14;
	int field_15;
	int field_16;
	int field_17;
	int field_18;
	int field_19;
	int field_20;
	int field_21;
	int field_22;
	int field_23;
	int field_24;
	int field_25;
	int field_26;
	int field_27;
	int field_28;
	int field_29;
	int field_30;
	int field_31;
	int field_32;
	int field_33;
	int field_34;
	int field_35;
	int field_36;
	int field_37;
	int field_38;
	int field_39;
	int field_40;
	int field_41;
	int field_42;
	int field_43;
	int field_44;
	int field_45;
	int field_46;
	int field_47;
	int field_48;
	int field_49;
	int field_50;
	int field_51;
	int field_52;
	int field_53;
	int field_54;
	int field_55;
	int field_56;
	int field_57;
	int field_58;
	int field_59;
	int field_60;
	int field_61;
	int field_62;
	int field_63;
	int field_64;
	int field_65;
	int field_66;
	int field_67;
	int field_68;
	int field_69;
	int field_70;
	int field_71;
	int field_72;
	int field_73;
	int field_74;
	int field_75;
	int field_76;
	int field_77;
	int field_78;
	int field_79;
	int field_80;
	int field_81;
	int field_82;
	int field_83;
	int field_84;
	int field_85;
	int field_86;
	int field_87;
	int field_88;
	int field_89;
	int field_90;
	int field_91;
	int field_92;
	int field_93;
	int field_94;
	int field_95;
	int field_96;
	int field_97;
	int field_98;
	int field_99;
	int field_100;
	int field_101;
	int field_102;
	int field_103;
	int field_104;
	int field_105;
	int field_106;
	int field_107;
	int field_108;
	int field_109;
	int field_110;
	int field_111;
	int field_112;
	int field_113;
	int field_114;
	int field_115;
	int field_116;
	int field_117;
	int field_118;
	int field_119;
	int field_120;
	int field_121;
	int field_122;
	int field_123;
	int field_124;
	int field_125;
	int field_126;
	int field_127;
	int field_128;
	int field_129;
	int field_130;
	int field_131;
	int field_132;
	int field_133;
	int field_134;
	int field_135;
	int field_136;
	int field_137;
	int field_138;
	int field_139;
	int field_140;
	int field_141;
	int field_142;
	int field_143;
	int field_144;
	int field_145;
	int field_146;
	int field_147;
	int field_148;
	int field_149;
	int field_150;
	int field_151;
	int field_152;
	int field_153;
	int field_154;
	int field_155;
	int field_156;
	int field_157;
	int field_158;
	int field_159;
	int field_160;
	int field_161;
	int field_162;
	int field_163;
	int field_164;
	int field_165;
	int field_166;
	int field_167;
	int field_168;
	int field_169;
	int field_170;
	int field_171;
	int field_172;
	int field_173;
	int field_174;
	int field_175;
	int field_176;
	int field_177;
	int field_178;
	int field_179;
	int field_180;
	int field_181;
	int field_182;
	int field_183;
	int field_184;
	int field_185;
	int field_186;
	int field_187;
	int field_188;
	int field_189;
	int field_190;
	int field_191;
	int field_192;
	int field_193;
	int field_194;
	int field_195;
	int field_196;
	int field_197;
	int field_198;
	int field_199;
	int field_200;
	int field_201;
	int field_202;
	int field_203;
	int field_204;
	int field_205;
	int field_206;
	int field_207;
	int field_208;
	int field_209;
	int field_210;
	int field_211;
	int field_212;
	int field_213;
	int field_214;
	int field_215;
	int field_216;
	int field_217;
	int field_218;
	int field_219;
	int field_220;
	int field_221;
	int field_222;
	int field_223;
	int field_224;
	int field_225;
	int field_226;
	int field_227;
	int field_228;
	int field_229;
	int field_230;
	int field_231;
	int field_232;
	int field_233;
	int field_234;
	int field_235;
	int field_236;
	int field_237;
	int field_238;
	int field_239;
	int field_240;
	int field_241;
	int field_242;
	int field_243;
	int field_244;
	int field_245;
	int field_246;
	int field_247;
	int field_248;
	int field_249;
	int field_250;
	int field_251;
	int field_252;
	int field_253;
	int field_254;
	int field_255;
	int field_256;
	int field_257;
	int field_258;
	int field_259;
	int field_260;
	int field_261;
	int field_262;
	int field_263;
	int field_264;
	int field_265;
	int field_266;
	int field_267;
	int field_268;
	int field_269;
	int field_270;
	int field_271;
	int field_272;
	int field_273;
	int field_274;
	int field_275;
	int field_276;
	int field_277;
	int field_278;
	int field_279;
	int field_280;
	int field_281;
	int field_282;
	int field_283;
	int field_284;
	int field_285;
	int field_286;
	int field_287;
	int field_288;
	int field_289;
	int field_290;
	int field_291;
	int field_292;
	int field_293;
	int field_294;
	int field_295;
	int field_296;
	int field_297;
	int field_298;
	int field_299;
	int field_300;
	int field_301;
	int field_302;
	int field_303;
	int field_304;
	int field_305;
	int field_306;
	int field_307;
	int field_308;
	int field_309;
	int field_310;
	int field_311;
	int field_312;
	int field_313;
	int field_314;
	int field_315;
	int field_316;
	int field_317;
	int field_318;
	int field_319;
	int field_320;
	int field_321;
	int field_322;
	int field_323;
	int field_324;
	int field_325;
	int field_326;
	int field_327;
	int field_328;
	int field_329;
	int field_330;
	int field_331;
	int field_332;
	int field_333;
	int field_334;
	int field_335;
	int field_336;
	int field_337;
	int field_338;
	int field_339;
	int field_340;
	int field_341;
	int field_342;
	int field_343;
	int field_344;
	int field_345;
	int field_346;
	int field_347;
	int field_348;
	int field_349;
	int field_350;
	int field_351;
	int field_352;
	int field_353;
	int field_354;
	int field_355;
	int field_356;
	int field_357;
	int field_358;
	int field_359;
	int field_360;
	int field_361;
	int field_362;
	int field_363;
	int field_364;
	int field_365;
	int field_366;
	int field_367;
	int field_368;
	int field_369;
	int field_370;
	int field_371;
	int field_372;
	int field_373;
	int field_374;
	int field_375;
	int field_376;
	int field_377;
	int field_378;
	int field_379;
	int field_380;
	int field_381;
	int field_382;
	int field_383;
	int field_384;
	int field_385;
	int field_386;
	int field_387;
	int field_388;
	int field_389;
	int field_390;
	int field_391;
	int field_392;
	int field_393;
	int field_394;
	int field_395;
	int field_396;
	int field_397;
	int field_398;
	int field_399;
	int field_400;
	int field_401;
	int field_402;
	int field_403;
	int field_404;
	int field_405;
	int field_406;
	int field_407;
	int field_408;
	int field_409;
	int field_410;
	int field_411;
	int field_412;
	int field_413;
	int field_414;
	int field_415;
	int field_416;
	int field_417;
	int field_418;
	int field_419;
	int field_420;
	int field_421;
	int field_422;
	int field_423;
	int field_424;
	int field_425;
	int field_426;
	int field_427;
	int field_428;
	int field_429;
	int field_430;
	int field_431;
	int field_432;
	int field_433;
	int field_434;
	int field_435;
	int field_436;
	int field_437;
	int field_438;
	int field_439;
	int field_440;
	int field_441;
	int field_442;
	int field_443;
	int field_444;
	int field_445;
	int field_446;
	int field_447;
	int field_448;
	int field_449;
	int field_450;
	int field_451;
	int field_452;
	int field_453;
	int field_454;
	int field_455;
	int field_456;
	int field_457;
	int field_458;
	int field_459;
	int field_460;
	int field_461;
	int field_462;
	int field_463;
	int field_464;
	int field_465;
	int field_466;
	int field_467;
	int field_468;
	int field_469;
	int field_470;
	int field_471;
	int field_472;
	int field_473;
	int field_474;
	int field_475;
	int field_476;
	int field_477;
	int field_478;
	int field_479;
	int field_480;
	int field_481;
	int field_482;
	int field_483;
	int field_484;
	int field_485;
	int field_486;
	int field_487;
	int field_488;
	int field_489;
	int field_490;
	int field_491;
	int field_492;
	int field_493;
	int field_494;
	int field_495;
	int field_496;
	int field_497;
	int field_498;
	int field_499;
	int field_500;
	int field_501;
	int field_502;
	int field_503;
	int field_504;
	int field_505;
	int field_506;
	int field_507;
	int field_508;
	int field_509;
	int field_510;
	int field_511;
	int field_512;
	int field_513;
	int field_514;
	int field_515;
	int field_516;
	int field_517;
	int field_518;
	int field_519;
	int field_520;
	int field_521;
	int field_522;
	int field_523;
	int field_524;
	int field_525;
	int field_526;
	int field_527;
	int field_528;
	int field_529;
	int field_530;
	int field_531;
	int field_532;
	int field_533;
	int field_534;
	int field_535;
	int field_536;
	int field_537;
	int field_538;
	int field_539;
	int field_540;
	int field_541;
	int field_542;
	int field_543;
	int field_544;
	int field_545;
	int field_546;
	int field_547;
	int field_548;
	int field_549;
	int field_550;
	int field_551;
	int field_552;
	int field_553;
	int field_554;
	int field_555;
	int field_556;
	int field_557;
	int field_558;
	int field_559;
	int field_560;
	int field_561;
	int field_562;
	int field_563;
	int field_564;
	int field_565;
	int field_566;
	int field_567;
	int field_568;
	int field_569;
	int field_570;
	int field_571;
	int field_572;
	int field_573;
	int field_574;
	int field_575;
	int field_576;
	int field_577;
	int field_578;
	int field_579;
	int field_580;
	int field_581;
	int field_582;
	int field_583;
	int field_584;
	int field_585;
	int field_586;
	int field_587;
	int field_588;
	int field_589;
	int field_590;
	int field_591;
	int field_592;
	int field_593;
	int field_594;
	int field_595;
	int field_596;
	int field_597;
	int field_598;
	int field_599;
	int field_600;
	int field_601;
	int field_602;
	int field_603;
	int field_604;
	int field_605;
	int field_606;
	int field_607;
	int field_608;
	int field_609;
	int field_610;
	int field_611;
	int field_612;
	int field_613;
	int field_614;
	int field_615;
	int field_616;
	int field_617;
	int field_618;
	int field_619;
	int field_620;
	int field_621;
	int field_622;
	int field_623;
	int field_624;
	int field_625;
	int field_626;
	int field_627;
	int field_628;
	int field_629;
	int field_630;
	int field_631;
	int field_632;
	int field_633;
	int field_634;
	int field_635;
	int field_636;
	int field_637;
	int field_638;
	int field_639;
	int field_640;
	int field_641;
	int field_642;
	int field_643;
	int field_644;
	int field_645;
	int field_646;
	int field_647;
	int field_648;
	int field_649;
	int field_650;
	int field_651;
	int field_652;
	int field_653;
	int field_654;
	int field_655;
	int field_656;
	int field_657;
	int field_658;
	int field_659;
	int field_660;
	int field_661;
	int field_662;
	int field_663;
	int field_664;
	int field_665;
	int field_666;
	int field_667;
	int field_668;
	int field_669;
	int field_670;
	int field_671;
	int field_672;
	int field_673;
	int field_674;
	int field_675;
	int field_676;
	int field_677;
	int field_678;
	int field_679;
	int field_680;
	int field_681;
	int field_682;
	int field_683;
	int field_684;
	int field_685;
	int field_686;
	int field_687;
	int field_688;
	int field_689;
	int field_690;
	int field_691;
	int field_692;
	int field_693;
} MyStruct;

Bindgen Invocation

No special flags.

bindgen::Builder::default()
    .header("header.h")
    .generate()
    .expect("generate bindings");

Actual Results

Codegen works fine, running tests causes runtime crash.

thread 'gen::bindings::bindgen_test_layout_MyStruct' has overflowed its stack
error: test failed, to rerun pass '--bin rust-sandbox'

Caused by:
  process didn't exit successfully: `<path>\rust-sandbox\target\debug\deps\rust_sandbox-50e2500daf4dd049.exe --format=json -Z unstable-options --show-output` (exit code: 0xc00000fd, STATUS_STACK_OVERFLOW)

Process finished with exit code -1073741571 (0xC00000FD)

Expected Results

Well... tests don't crash 🙂

@emilio
Copy link
Contributor

emilio commented Jun 6, 2022

This is a regression from #2203, which creates a MaybeUninit<Type> on the stack to compute the field offsets.

The tests look like:

    assert_eq!(
        unsafe {
            let uninit = ::std::mem::MaybeUninit::<MyStruct>::uninit();
            let ptr = uninit.as_ptr();
            ::std::ptr::addr_of!((*ptr).field_694) as usize - ptr as usize
        },
        5552usize,
        concat!(
            "Offset of field: ",
            stringify!(MyStruct),
            "::",
            stringify!(field_694)
        )
    );
    assert_eq!(
        unsafe {
            let uninit = ::std::mem::MaybeUninit::<MyStruct>::uninit();
            let ptr = uninit.as_ptr();
            ::std::ptr::addr_of!((*ptr).field_695) as usize - ptr as usize
        },
        5560usize,
        concat!(
            "Offset of field: ",
            stringify!(MyStruct),
            "::",
            stringify!(field_695)
        )
    );

It seems rustc with -C opt-level=0 doesn't manage to reuse the stack space for all those assertions, but it doesn't create a stack overflow with -C opt-level=2, can you confirm that?

@Bromeon
Copy link
Author

Bromeon commented Jun 6, 2022

Indeed, RUSTFLAGS=-C opt-level=2 makes the issue disappear.

I also copy-pasted the assertions to main(), same behavior (stack overflow with opt-level=0, works with opt-level=2).
So it seems really related to rustc and not the test runner.

Should I file an issue with rustc?

@emilio
Copy link
Contributor

emilio commented Jun 6, 2022

Well, sorta, we can work around it. But yeah it'd be nice if rustc didn't use so much stack space.

emilio added a commit that referenced this issue Jun 6, 2022
So that rustc doesn't take too much stack space without optimizations.

Fixes #2218
@emilio
Copy link
Contributor

emilio commented Jun 6, 2022

#2219 has a fix / workaround.

@Bromeon
Copy link
Author

Bromeon commented Jun 6, 2022

That was fast! I also tried to surround each assert_eq! with a block scope { }, but it didn't fix the issue.

I tested godot-rust locally with the revision from your PR:

bindgen = { git = "https://github.com/rust-lang/rust-bindgen.git", rev = "7fe8e55847af1f9c26f1ad5b5e78f178a2d53f7f", default-features = false, features = ["runtime"] }

And the tests pass now!

@emilio
Copy link
Contributor

emilio commented Jun 6, 2022

That was fast! I also tried to surround each assert_eq! with a block scope { }, but it didn't fix the issue.

Yeah, I had tried that locally as well, hoping that'd work too but... :)

I tested godot-rust locally with the revision from your PR:

bindgen = { git = "https://github.com/rust-lang/rust-bindgen.git", rev = "7fe8e55847af1f9c26f1ad5b5e78f178a2d53f7f", default-features = false, features = ["runtime"] }

And the tests pass now!

Awesome, thanks for confirming.

emilio added a commit that referenced this issue Jun 6, 2022
So that rustc doesn't take too much stack space without optimizations.

Fixes #2218
@emilio
Copy link
Contributor

emilio commented Jun 6, 2022

Published 0.60.1 with that workaround.

@Bromeon
Copy link
Author

Bromeon commented Jun 6, 2022

Thanks a lot for the quick help with this!
I applied the fix downstream and reported the issue in the rust repo.

aatifsyed pushed a commit to aatifsyed/rust-bindgen that referenced this issue Sep 22, 2022
So that rustc doesn't take too much stack space without optimizations.

Fixes rust-lang#2218
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants