Skip to content

fix: Reproducible Builds not working when using modular jar #205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 2, 2022

Conversation

jorsol
Copy link
Contributor

@jorsol jorsol commented Mar 11, 2022

Fixes #164

@jorsol jorsol marked this pull request as ready for review March 15, 2022 09:35
@jorsol jorsol force-pushed the 164-fix-modular-jar-time branch from 03cb92e to 9ec768e Compare March 24, 2022 10:32
@jorsol jorsol force-pushed the 164-fix-modular-jar-time branch 5 times, most recently from dc382e3 to 2f298e0 Compare April 1, 2022 11:23
@jorsol
Copy link
Contributor Author

jorsol commented Apr 1, 2022

Anyone that have some time to review this?

@plamentotev
Copy link
Member

Hi,

Thanks for picking this up and the great work. I took a quick look at the changes and I have a couple of questions to makes sure I got everything right.

The first thing is that it looks like there are multiple changes in this MR:

  • refactoring of existing code that is not related to the change, but appear to be near to it
  • the jar tools handles timestamps differently. There is a change that makes Plexus Archiver output to match the jar tools output but it is not strictly required for the build to be reproducible.
  • the change to make the build reproducible.

Is my understanding correct? Also if those changes are in different commits (if that makes sense) would ease the review (at least for me).

@jorsol
Copy link
Contributor Author

jorsol commented Apr 2, 2022

Thank you for your review @plamentotev, your understanding is correct, just a couple of comments:

refactoring of existing code that is not related to the change, but appear to be near to it

I will try to reduce the refactoring to the minimum to ease the review.

the jar tools handles timestamps differently. There is a change that makes Plexus Archiver output to match the jar tools output but it is not strictly required for the build to be reproducible.

Actually it's required for the build to be reproducible, it's "interconnected" with the jar tool, essentially the jar tool will have a new --date option that it's used to make the build reproducible, so if the jar tool produces a jar with a timestamp and Plexus Archiver produces a jar with a different timestamp, the build will not be reproducible.

the change to make the build reproducible.

As mentioned, the change to make the build reproducible is done by the new --date option of the jar tool, to better explain how it works:

Actual scenario:

  1. Plexus Archiver creates a JAR file as usual with timestamps rounded up.
  2. Plexus Archiver calls the jar tool and "updates" the module declaration with the version and other verification's, this step also change the timestamp of module-info.class and possibly also MANIFEST.MF set by the previous step making the build non reproducible.

New scenario:

  1. Plexus Archiver creates a JAR file as usual with timestamps rounded down (default behavior).
  2. Plexus Archiver calls the jar tool and "updates" the module declaration with the version and other verification's with two paths:
    1. If the jar tool has the new option --date it will use that option to update the timestamp of all entries to make the build reproducible in the same step.
    2. If the jar tool doesn't have the new option --date, Plexus Archiver will "fallback" to rewrite the ZipFile after the jar run and set all the entries time to make the build reproducible.

When I said it's interconnected is because the output should be reproducible on both cases, assume that you run a build with JDK 17.0.2, Plexus Archiver will "fallback" (ii) to rewrite the JAR file with the correct timestamp of all entries (doesn't have the --date option), and then later you run the exact build changing the JDK to 17.0.3 (with --date option), now the timestamp is written by the jar tool --date option (i) not Plexus Archiver, if the behavior of both is different the build will not be reproducible.

You may ask why not leave only the fallback mechanism that should work in any case? the reason is that there is already a post create archive step that already run the jar command, the fallback is like a post-post create archive step making the build unnecessarily slower, so if we are using a JDK with the provided option we can use that instead.

Making different commits could be possible but this change is logically more consistent in a single commit (due to the interconnected nature).

@jorsol jorsol force-pushed the 164-fix-modular-jar-time branch from 2f298e0 to 7a6b0f0 Compare April 2, 2022 11:08
@jorsol jorsol force-pushed the 164-fix-modular-jar-time branch from 7a6b0f0 to 3d0bebf Compare April 2, 2022 11:30
@hboutemy
Copy link
Member

hboutemy commented Apr 2, 2022

wow, what a great job in code and in explanations: thanks a lot

@hboutemy hboutemy merged commit ae29b75 into codehaus-plexus:master Apr 2, 2022
@jorsol jorsol deleted the 164-fix-modular-jar-time branch April 2, 2022 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reproducible Builds not working when using modular jar
3 participants