Skip to content

Cython 3.0.1 seems to break our compile process #54756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
phofl opened this issue Aug 25, 2023 · 4 comments · Fixed by #54784
Closed

Cython 3.0.1 seems to break our compile process #54756

phofl opened this issue Aug 25, 2023 · 4 comments · Fixed by #54784
Labels
CI Continuous Integration Dependencies Required and optional dependencies Upstream issue Issue related to pandas dependency

Comments

@phofl
Copy link
Member

phofl commented Aug 25, 2023

https://app.circleci.com/pipelines/github/pandas-dev/pandas/35409/workflows/a4c9fd69-5f8b-462d-a3ef-af13db8ead32/jobs/103623

cc @lithomas1 maybe

@phofl phofl mentioned this issue Aug 25, 2023
5 tasks
@lithomas1
Copy link
Member

Looks like Cython is crashing. We should be getting some kind of error message.
For reference,

cython -M --fast-fail -3 --include-dir /home/circleci/project/build/cp310/pandas/_libs/tslibs '-X always_allow_keywords=true' /home/circleci/project/pandas/_libs/tslibs/fields.pyx -o pandas/_libs/tslibs/fields.cpython-310-aarch64-linux-gnu.so.p/pandas/_libs/tslibs/fields.pyx.c
  Traceback (most recent call last):
    File "/home/circleci/miniconda3/envs/pandas-dev/bin/cython", line 11, in <module>
      sys.exit(setuptools_main())
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 757, in setuptools_main
      return main(command_line = 1)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 785, in main
      result = compile(sources, options)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 676, in compile
      return compile_multiple(source, options)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 650, in compile_multiple
      result = run_pipeline(source, options,
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 542, in run_pipeline
      err, enddata = Pipeline.run_pipeline(pipeline, source)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Pipeline.py", line 398, in run_pipeline
      data = run(phase, data)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Pipeline.py", line 375, in run
      return phase(data)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Pipeline.py", line 52, in generate_pyx_code_stage
      module_node.process_implementation(options, result)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ModuleNode.py", line 222, in process_implementation
      self.generate_c_code(env, options, result)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ModuleNode.py", line 516, in generate_c_code
      self.body.generate_function_definitions(env, code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 404, in generate_function_definitions
      stat.generate_function_definitions(env, code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 404, in generate_function_definitions
      stat.generate_function_definitions(env, code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 349, in generate_function_definitions
      self.body.generate_function_definitions(env, code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 404, in generate_function_definitions
      stat.generate_function_definitions(env, code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 3499, in generate_function_definitions
      self.defaults_getter.generate_function_definitions(env.global_scope(), code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 3506, in generate_function_definitions
      FuncDefNode.generate_function_definitions(self, env, code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 2200, in generate_function_definitions
      self.generate_function_body(env, code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 1953, in generate_function_body
      self.body.generate_execution_code(code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/Nodes.py", line 6856, in generate_execution_code
      value.generate_evaluation_code(code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ExprNodes.py", line 831, in generate_evaluation_code
      self.generate_subexpr_evaluation_code(code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ExprNodes.py", line 846, in generate_subexpr_evaluation_code
      node.generate_evaluation_code(code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ExprNodes.py", line 837, in generate_evaluation_code
      self.generate_result_code(code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ExprNodes.py", line 8017, in generate_result_code
      self.generate_operation_code(code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ExprNodes.py", line 8576, in generate_operation_code
      self.generate_sequence_packing_code(code)
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ExprNodes.py", line 8084, in generate_sequence_packing_code
      code.put_incref(arg.result(), arg.ctype())
    File "/home/circleci/miniconda3/envs/pandas-dev/lib/python3.10/site-packages/Cython/Compiler/ExprNodes.py", line 10113, in result
      self.defaults_struct.lookup(self.arg.name).cname)
  AttributeError: 'NoneType' object has no attribute 'cname'

is the error.

https://github.com/pandas-dev/pandas/blob/main/pandas/_libs/tslibs/fields.pyx is the failing file for reference.

cc @scoder @da-woods

@lithomas1 lithomas1 added CI Continuous Integration Dependencies Required and optional dependencies Upstream issue Issue related to pandas dependency labels Aug 25, 2023
@scoder
Copy link

scoder commented Aug 25, 2023

From the stack trace, it seems that we're generating a tuple of default argument values here but somehow an argument name isn't found in the defaults struct. Might be related to the change in cython/cython#5554
As a wild guess, we might have (accidentally?) turned a literal default value into a non-literal (generated) one but haven't set it up properly to actually be generated. But that's just guessing. Would be good to know where in the source file it's failing (e.g. print(self.pos) in the line where it's crashing).

@da-woods
Copy link

I'm away from a computer for the next few days but will have a look on Monday if the bug still exists then.

@scoder
Copy link

scoder commented Aug 27, 2023

Fixed in Cython 3.0.2, just released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Dependencies Required and optional dependencies Upstream issue Issue related to pandas dependency
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants