Skip to content

ENH: Use docker compose #46532

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
WillAyd opened this issue Mar 27, 2022 · 0 comments
Closed

ENH: Use docker compose #46532

WillAyd opened this issue Mar 27, 2022 · 0 comments
Labels
CI Continuous Integration Enhancement

Comments

@WillAyd
Copy link
Member

WillAyd commented Mar 27, 2022

The arrow project uses docker compose to make it easier to reproduce environments for development and testing. While we don't have as complicated of an environment requirement, there are some places where this could help:

  1. SQL test cases would become more reproducible for developers

  2. We can more clearly delimitate what is required for particular development and testing workflows

Particularly for point 2 above, we right now have a rather monolithic environment.yml file that could be broken up into different docker compose services.

A sample file might look something like this:

  services:

  development:
    image: python:3.10
    build:
      context: .
      dockerfile: ci/docker/pandas-dev.dockerfile
    volumes:
      - .:/pandas

  lint:
    image: python:3.10
    build:
      context: .
      dockerfile: <ci/docker/pandas-lint.dockerfile
    volumes:
      - .:/pandas

  testing:
    image: python:3.10
    build:
      context: .
      dockerfile: <ci/docker/pandas-testing.dockerfile
    volumes:
      - .:/pandas

  sql-testing:
    image: python:3.10
    build:
      context: .
      dockerfile: <ci/docker/pandas-sql-testing.dockerfile
    volumes:
      - .:/pandas

This is just a rough sketch; We can get more advanced and parametrize the Python versions to match what we support

@WillAyd WillAyd added Enhancement Needs Triage Issue that has not been reviewed by a pandas team member CI Continuous Integration and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Mar 27, 2022
@WillAyd WillAyd mentioned this issue Mar 30, 2022
1 task
@WillAyd WillAyd closed this as completed Dec 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant