Skip to content

chore(generative_ai): unmark test as fixture #13397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

chore(generative_ai): unmark test as fixture #13397

wants to merge 4 commits into from

Conversation

glasnt
Copy link
Contributor

@glasnt glasnt commented May 30, 2025

Description

Addresess issues identified in #13393, #13396

The tests in #13388 all passed originally, but it appears the batch test was marked as a fixture, and was not being executed, thus the changes appeared to succeed.

From https://btx.cloud.google.com/invocations/c57aafde-585e-490e-a9e2-384a1d47d7a1/log

============================= test session starts ==============================
platform linux -- Python 3.12.7, pytest-8.2.0, pluggy-1.6.0 -- /workspace/generative_ai/embeddings/.nox/py-3-12/bin/python
cachedir: .pytest_cache
rootdir: /workspace
configfile: pytest.ini
plugins: asyncio-0.23.6, anyio-4.9.0
asyncio: mode=Mode.STRICT
collecting ... collected 7 items

test_embeddings_examples.py::test_multimodal_embedding_image_video_text PASSED [ 14%]
test_embeddings_examples.py::test_multimodal_embedding_video PASSED      [ 28%]
test_embeddings_examples.py::test_multimodal_embedding_image PASSED      [ 42%]
test_embeddings_examples.py::test_generate_embeddings_with_lower_dimension PASSED [ 57%]
test_embeddings_examples.py::test_text_embed_text PASSED                 [ 71%]
test_embeddings_examples.py::test_code_embed_text PASSED                 [ 85%]
test_embeddings_examples.py::test_tune_embedding_model PASSED            [100%]
  • Please merge this PR for me once it is approved

@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label May 30, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @glasnt, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello team, Gemini here to provide a quick summary of this pull request. This PR, authored by glasnt, aims to fix an issue in our CI where a specific test, test_embed_text_batch, was not being executed. The description indicates that this test was incorrectly marked as a pytest.fixture, which prevented pytest from discovering and running it as a standard test. This oversight led to issues #13393 and #13396, and explains why changes in a previous PR (#13388) appeared to pass when they should have failed. The core change in this PR is the removal of the @pytest.fixture decorator from the affected test function, ensuring it runs as intended.

Highlights

Changelog

  • generative_ai/embeddings/test_embeddings_examples.py
    • Removed the @pytest.fixture(scope="session") decorator from the test_embed_text_batch function (line 38), allowing it to be discovered and run as a standard test.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.


A test marked fixture,
Does not run, a strange mixture.
Remove the decorator,
Now the test is a runner.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is an excellent and important fix! You've correctly identified that the test_embed_text_batch function was mistakenly decorated as a @pytest.fixture(scope="session"), which prevented it from being executed as a regular test case. Removing this decorator is the right solution and will ensure that this test is now properly discovered and run by pytest. This change is crucial for maintaining the integrity of the CI process and ensuring all intended tests contribute to the quality assessment. Well done for spotting and addressing this!

Summary of Findings

  • Test Execution Correction: This pull request successfully resolves a significant issue in the test suite. The test_embed_text_batch function was incorrectly marked with @pytest.fixture, causing it to be skipped during test execution. The removal of this decorator is the correct action and ensures the test will now run as intended, thereby improving the overall reliability and coverage of the automated tests.

Merge Readiness

The change in this pull request is clear, concise, and directly addresses a critical oversight in the test configuration. By ensuring test_embed_text_batch runs, this PR strengthens the test suite. From a code review perspective, these changes are sound and significantly improve test reliability. I recommend merging this PR once CI has successfully completed, as the CI results will now reflect the execution of this previously skipped test. As an AI, I am not authorized to approve pull requests, so please ensure it undergoes any further necessary reviews and approvals according to your team's policies.

@glasnt
Copy link
Contributor Author

glasnt commented May 30, 2025

As at 243596e, Do not support publisher model gemini-embedding-001 successfully errors.

Copy link
Contributor

Thanks for the update, @glasnt!

It's great to hear that the test is now correctly erroring as expected at 243596e. This confirms that the changes in this PR, which unmark the batch test as a fixture, are successfully allowing the test to execute and reveal the underlying issue with the gemini-embedding-001 publisher model.

This is exactly the intended outcome of the pull request, ensuring our tests accurately reflect the current state and catch potential issues. Great work!

@glasnt
Copy link
Contributor Author

glasnt commented May 30, 2025

Rebasing to main to introduce the changes in 13396 should resolve the tests

@glasnt glasnt marked this pull request as ready for review May 30, 2025 02:23
@glasnt glasnt requested review from a team as code owners May 30, 2025 02:23
@glasnt glasnt changed the title chore(ci): unmark test as fixture chore(generative_ai): unmark test as fixture May 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
samples Issues that are directly related to samples.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants