Skip to content

Build suddenly failing (new GCC issue) #3185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
molpopgen opened this issue Oct 24, 2017 · 15 comments
Closed

Build suddenly failing (new GCC issue) #3185

molpopgen opened this issue Oct 24, 2017 · 15 comments
Labels
Needed: more information A reply from issue author is required Support Support question

Comments

@molpopgen
Copy link

Details

Expected Result

The Python extension module should build, and then the manual should build.

Actual Result

GCC fails with an internal compiler error. This is new behavior. Also, attempting to install GCC via conda results in failure to link errors.
@RichardLitt RichardLitt added the Bug A bug label Oct 25, 2017
@agjohnson agjohnson added Support Support question and removed Bug A bug labels Oct 25, 2017
@agjohnson
Copy link
Contributor

It looks like gcc is being killed:

gcc: internal compiler error: Killed (program cc1plus)

I'm not sure under what causes this -- it could be part of our build limits like the memory limit, but that usually appears in a more obvious way.

Can you branch off of a working commit (last commit is 9acb311a) and see if the build works? If it doesn't, it might be a matter of our last image deploy -- though timeframe doesn't quite line up there.

@molpopgen
Copy link
Author

molpopgen commented Oct 25, 2017 via email

@agjohnson
Copy link
Contributor

RTD should pick up on the new branch if you have a webhook set up. You can check this in your project admin dashboard, under integrations.

If it does have a webhook setup, there might be something else happening

@molpopgen
Copy link
Author

I do have the webhook set up. I just deleted and set up a new one.

My point is that the webhooks work, but the process is very slow. I often have to come back the next day if I'm trying to test building a new branch. This slowness has been going on for a really long time. I figured this was expected.

@molpopgen
Copy link
Author

Here is the output from the webhook from pushing a new copy of a branch based on the last commit to work on RTD.

There is no indication of error, but the new branches do not show up under "Versions", but I expect they will in several hours, or possibly as late as tomorrow.

The link is https://readthedocs.org/dashboard/fwdpy11/integrations/16694/exchange/8527d289-d1d1-4cf1-a4a9-201244aae761/

The output is:

Request

Sslclientcipher: SSL_ECDHE_RSA_WITH_AES_256_GCM_SHA384, version=TLSv1.2, bits=256 Sslclientcertstatus: NoClientCert X-Github-Event: push X-Github-Delivery: eba0a130-b9b3-11e7-8759-4eb82e9ada71 Connection: close Accept: */* User-Agent: GitHub-Hookshot/95a0199 X-Cluster-Client-Ip: 192.30.252.34 Sslsessionid: 42213D858286792127D892F31EB8B5092EECD555931A4EABA8CF7CC79FAB20C3 Host: readthedocs.org Content-Type: application/json

And

{
  "after": "9acb311afb192eafd1251fef8f052277eaf3c495", 
  "base_ref": "refs/heads/last_working_RTD_commit", 
  "before": "0000000000000000000000000000000000000000", 
  "commits": [], 
  "compare": "https://github.com/molpopgen/fwdpy11/compare/last_working_RTD_commit_v2", 
  "created": true, 
  "deleted": false, 
  "forced": false, 
  "head_commit": {
    "added": [], 
    "author": {
      "email": "[email protected]", 
      "name": "molpopgen", 
      "username": "molpopgen"
    }, 
    "committer": {
      "email": "[email protected]", 
      "name": "molpopgen", 
      "username": "molpopgen"
    }, 
    "distinct": true, 
    "id": "9acb311afb192eafd1251fef8f052277eaf3c495", 
    "message": "update change log", 
    "modified": [
      "doc/pages/changelog.rst"
    ], 
    "removed": [], 
    "timestamp": "2017-10-17T22:09:37-07:00", 
    "tree_id": "3ca10d8c9e5d44943c5c0b67c68065ca77676dbd", 
    "url": "https://github.com/molpopgen/fwdpy11/commit/9acb311afb192eafd1251fef8f052277eaf3c495"
  }, 
  "pusher": {
    "email": "[email protected]", 
    "name": "molpopgen"
  }, 
  "ref": "refs/heads/last_working_RTD_commit_v2", 
  "repository": {
    "archive_url": "https://api.github.com/repos/molpopgen/fwdpy11/{archive_format}{/ref}", 
    "archived": false, 
    "assignees_url": "https://api.github.com/repos/molpopgen/fwdpy11/assignees{/user}", 
    "blobs_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/blobs{/sha}", 
    "branches_url": "https://api.github.com/repos/molpopgen/fwdpy11/branches{/branch}", 
    "clone_url": "https://github.com/molpopgen/fwdpy11.git", 
    "collaborators_url": "https://api.github.com/repos/molpopgen/fwdpy11/collaborators{/collaborator}", 
    "comments_url": "https://api.github.com/repos/molpopgen/fwdpy11/comments{/number}", 
    "commits_url": "https://api.github.com/repos/molpopgen/fwdpy11/commits{/sha}", 
    "compare_url": "https://api.github.com/repos/molpopgen/fwdpy11/compare/{base}...{head}", 
    "contents_url": "https://api.github.com/repos/molpopgen/fwdpy11/contents/{+path}", 
    "contributors_url": "https://api.github.com/repos/molpopgen/fwdpy11/contributors", 
    "created_at": 1490029317, 
    "default_branch": "master", 
    "deployments_url": "https://api.github.com/repos/molpopgen/fwdpy11/deployments", 
    "description": "Forward-time simulation in Python using fwdpp", 
    "downloads_url": "https://api.github.com/repos/molpopgen/fwdpy11/downloads", 
    "events_url": "https://api.github.com/repos/molpopgen/fwdpy11/events", 
    "fork": false, 
    "forks": 2, 
    "forks_count": 2, 
    "forks_url": "https://api.github.com/repos/molpopgen/fwdpy11/forks", 
    "full_name": "molpopgen/fwdpy11", 
    "git_commits_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/commits{/sha}", 
    "git_refs_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/refs{/sha}", 
    "git_tags_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/tags{/sha}", 
    "git_url": "git://github.com/molpopgen/fwdpy11.git", 
    "has_downloads": true, 
    "has_issues": true, 
    "has_pages": true, 
    "has_projects": true, 
    "has_wiki": true, 
    "homepage": null, 
    "hooks_url": "https://api.github.com/repos/molpopgen/fwdpy11/hooks", 
    "html_url": "https://github.com/molpopgen/fwdpy11", 
    "id": 85603555, 
    "issue_comment_url": "https://api.github.com/repos/molpopgen/fwdpy11/issues/comments{/number}", 
    "issue_events_url": "https://api.github.com/repos/molpopgen/fwdpy11/issues/events{/number}", 
    "issues_url": "https://api.github.com/repos/molpopgen/fwdpy11/issues{/number}", 
    "keys_url": "https://api.github.com/repos/molpopgen/fwdpy11/keys{/key_id}", 
    "labels_url": "https://api.github.com/repos/molpopgen/fwdpy11/labels{/name}", 
    "language": "C++", 
    "languages_url": "https://api.github.com/repos/molpopgen/fwdpy11/languages", 
    "master_branch": "master", 
    "merges_url": "https://api.github.com/repos/molpopgen/fwdpy11/merges", 
    "milestones_url": "https://api.github.com/repos/molpopgen/fwdpy11/milestones{/number}", 
    "mirror_url": null, 
    "name": "fwdpy11", 
    "notifications_url": "https://api.github.com/repos/molpopgen/fwdpy11/notifications{?since,all,participating}", 
    "open_issues": 4, 
    "open_issues_count": 4, 
    "owner": {
      "avatar_url": "https://avatars0.githubusercontent.com/u/6416159?v=4", 
      "email": "[email protected]", 
      "events_url": "https://api.github.com/users/molpopgen/events{/privacy}", 
      "followers_url": "https://api.github.com/users/molpopgen/followers", 
      "following_url": "https://api.github.com/users/molpopgen/following{/other_user}", 
      "gists_url": "https://api.github.com/users/molpopgen/gists{/gist_id}", 
      "gravatar_id": "", 
      "html_url": "https://github.com/molpopgen", 
      "id": 6416159, 
      "login": "molpopgen", 
      "name": "molpopgen", 
      "organizations_url": "https://api.github.com/users/molpopgen/orgs", 
      "received_events_url": "https://api.github.com/users/molpopgen/received_events", 
      "repos_url": "https://api.github.com/users/molpopgen/repos", 
      "site_admin": false, 
      "starred_url": "https://api.github.com/users/molpopgen/starred{/owner}{/repo}", 
      "subscriptions_url": "https://api.github.com/users/molpopgen/subscriptions", 
      "type": "User", 
      "url": "https://api.github.com/users/molpopgen"
    }, 
    "private": false, 
    "pulls_url": "https://api.github.com/repos/molpopgen/fwdpy11/pulls{/number}", 
    "pushed_at": 1508956805, 
    "releases_url": "https://api.github.com/repos/molpopgen/fwdpy11/releases{/id}", 
    "size": 680, 
    "ssh_url": "[email protected]:molpopgen/fwdpy11.git", 
    "stargazers": 3, 
    "stargazers_count": 3, 
    "stargazers_url": "https://api.github.com/repos/molpopgen/fwdpy11/stargazers", 
    "statuses_url": "https://api.github.com/repos/molpopgen/fwdpy11/statuses/{sha}", 
    "subscribers_url": "https://api.github.com/repos/molpopgen/fwdpy11/subscribers", 
    "subscription_url": "https://api.github.com/repos/molpopgen/fwdpy11/subscription", 
    "svn_url": "https://github.com/molpopgen/fwdpy11", 
    "tags_url": "https://api.github.com/repos/molpopgen/fwdpy11/tags", 
    "teams_url": "https://api.github.com/repos/molpopgen/fwdpy11/teams", 
    "trees_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/trees{/sha}", 
    "updated_at": "2017-10-24T01:47:50Z", 
    "url": "https://github.com/molpopgen/fwdpy11", 
    "watchers": 3, 
    "watchers_count": 3
  }, 
  "sender": {
    "avatar_url": "https://avatars0.githubusercontent.com/u/6416159?v=4", 
    "events_url": "https://api.github.com/users/molpopgen/events{/privacy}", 
    "followers_url": "https://api.github.com/users/molpopgen/followers", 
    "following_url": "https://api.github.com/users/molpopgen/following{/other_user}", 
    "gists_url": "https://api.github.com/users/molpopgen/gists{/gist_id}", 
    "gravatar_id": "", 
    "html_url": "https://github.com/molpopgen", 
    "id": 6416159, 
    "login": "molpopgen", 
    "organizations_url": "https://api.github.com/users/molpopgen/orgs", 
    "received_events_url": "https://api.github.com/users/molpopgen/received_events", 
    "repos_url": "https://api.github.com/users/molpopgen/repos", 
    "site_admin": false, 
    "starred_url": "https://api.github.com/users/molpopgen/starred{/owner}{/repo}", 
    "subscriptions_url": "https://api.github.com/users/molpopgen/subscriptions", 
    "type": "User", 
    "url": "https://api.github.com/users/molpopgen"
  }
}

@agjohnson
Copy link
Contributor

Nope, expected behavior should be that the branches appear once we receive the webhook. A delay either indicates a malfunctioning/missing webhook, or a bug on our version creation code. I seem to recall another report of this which was tied to the cloned repository on our build server. I'll open a separate issue for this.

@molpopgen
Copy link
Author

This should perhaps be a separate issue, but the docs for webhooks could use some updating. For example this text appears to be incorrect:

It might be necessary to re-establish a webhook if you are noticing problems. To resync a webhook from Read the Docs, visit the integration detail page and follow the directions for re-syncing your repository webhook.

No such directions exist.

Nope, expected behavior should be that the branches appear once we receive the webhook. A delay either indicates a malfunctioning/missing webhook, or a bug on our version creation code. I seem to recall another report of this which was tied to the cloned repository on our build server. I'll open a separate issue for this.

@agjohnson agjohnson mentioned this issue Oct 25, 2017
Closed
@agjohnson
Copy link
Contributor

Regarding the original bug report, does building the previous commit work as expected?

@molpopgen
Copy link
Author

molpopgen commented Oct 28, 2017 via email

@molpopgen
Copy link
Author

Regarding the original bug report, does building the previous commit work as expected?

One thing that would be helpful is if RTD provided an example of a Python extension requiring compilation and linking to other C/C++ libraries installed via Conda. I have brought this up before in other issues, but that request has never been addressed. Many of the existing docs are quite vague in how to set up such projects for RTD integration, and I only got things running via help from another user who had figured it out. For examples, #2707 and #2958 were never addressed, and the latter is a somewhat serious problem, IMO. Further, I know of other packages where the maintainer gave up on trying to build on RTD and instead mocked all the docs, due to lack of documentation in how to set things up.

The project of mine is pretty thoroughly tested on Travis. All the builds that are failing on RTD pass on Travis, using both Anaconda GCC and several combos of Ubuntu GCC 5 and 6 with various Python versions (https://travis-ci.org/molpopgen/fwdpy11/builds/291754232).

@molpopgen
Copy link
Author

OK, I was able to get a branch built based on the last commit successfully built on RTD. It did indeed pass:

https://readthedocs.org/projects/fwdpy11/builds/6244258/

I guess I should look at what the next commit did...

@molpopgen
Copy link
Author

What GCC version is RTD using? I notice that the failing builds are compiling with C++14, implying that a GCC >= 5.something is being used. Further, when I attempt to install GCC from conda (which is 4.8.5), builds fail with no meaningful error. For this build, the only change was to add GCC as dependency in the environment file for RTD:

https://readthedocs.org/projects/fwdpy11/builds/6246474/

@molpopgen
Copy link
Author

molpopgen commented Nov 9, 2017 •
edited
Loading

For reasons I do not understand, build using conda GCC suddenly started to pass within the last hour.

This commit was all it took: molpopgen/fwdpy11@39ab23b

Results are here: https://readthedocs.org/projects/fwdpy11/builds/6246693/

The change uses conda-forge gcc, which is 4.8.5. The implication is that there is a config issue with the RTD GCC that is interacting badly with recent versions of my project. Importantly, this interaction cannot be attributed to a problem with the code in my project--all builds pass on conda GCC, GCC5, and GCC6 on Travis/Linux as well as several other systems where we use and test the code.

Edit:

For example, here's the recent build results on Travis: https://travis-ci.org/molpopgen/fwdpy11. That's presumably spanning whatever RTD is using?

@stsewd
Copy link
Member

stsewd commented Jun 25, 2018

RTD uses this docker images to build your docs https://github.com/rtfd/readthedocs-docker-images/tree/releases/3.x. Anyway, looks like your problem is solved, right?

@stsewd stsewd added the Needed: more information A reply from issue author is required label Aug 31, 2018
@stsewd
Copy link
Member

stsewd commented Aug 31, 2018

Closing this as the user hasn't responded or given more information, feel free to reopen with more information if you are still having this problem :).

@stsewd stsewd closed this as completed Aug 31, 2018
@cmacmackin cmacmackin mentioned this issue Mar 4, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Needed: more information A reply from issue author is required Support Support question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@RichardLitt @agjohnson @stsewd @molpopgen