-
Notifications
You must be signed in to change notification settings - Fork 325
Add follow-up blog about the sbt community repository #1503
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
julienrf
merged 3 commits into
scala:main
from
scalacenter:sbt-community-repo-follow-up
May 15, 2023
Merged
Changes from 1 commit
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
38 changes: 38 additions & 0 deletions
38
blog/_posts/2023-05-20-sbt-community-repository-follow-up.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
--- | ||
layout: blog-detail | ||
post-type: blog | ||
by: Julien Richard-Foy, Scala Center | ||
title: "sbt Community Repository (Follow-Up)" | ||
--- | ||
|
||
[Last month]({% link blog/_posts/2023-04-20-sbt-plugins-community-repository.md %}), I published a blog article where I described the sbt community repository. In this follow-up article, I present our plan to strengthen the infrastructure that supports the sbt ecosystem. | ||
|
||
This blog article is also an initial response to the related Advisory Board proposal [SCP-029 sbt community repository](https://github.com/scalacenter/advisoryboard/blob/main/proposals/029-sbt-community-repository.md), which has been approved by the Advisory Board during the meeting of the [27th of April](https://github.com/scala/scala.epfl.ch/blob/main/minutes/_posts/2023-04-27-april-27-2023.md). | ||
|
||
## Current State | ||
|
||
The sbt community repository was set up a couple of years ago with two primary purposes: | ||
1. host “old” versions of sbt plugins so that “old” Scala projects continue to work today and in the future, | ||
julienrf marked this conversation as resolved.
Show resolved
Hide resolved
|
||
2. give some time to the ecosystem of sbt plugins to migrate to Maven Central. | ||
julienrf marked this conversation as resolved.
Show resolved
Hide resolved
julienrf marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
On average, the community repository serves 6 TB of data per month and 700 requests per second. These numbers are not huge, but I think they are higher than they should be, especially when we take into account the fact that the content delivered by the community repository is usually downloaded once per client and then cached forever. I would like to thank again JFrog for their continued support of the sbt project and the Scala ecosystem. | ||
julienrf marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The reason why these numbers are so high is that the migration to Maven Central did not completely happen and many sbt plugins still have their latest stable version on the community repository. | ||
As a consequence, any new developer joining a Scala project or any continuous integration system building a Scala project with a fresh cache will probably download artifacts from the community repository, even if the project has up-to-date dependencies. | ||
|
||
The event of the [7th of April]({% link blog/_posts/2023-04-20-sbt-plugins-community-repository.md %}) has shown that the sbt community repository plays a critical role in the ecosystem. Therefore, we need to increase our safety margins. | ||
|
||
## Action Plan | ||
|
||
Our goal is to ensure all the Scala projects can be built in the future regardless of which versions of sbt plugins they use. | ||
|
||
To achieve this, we will: | ||
|
||
1. Accelerate the migration of the actively used sbt plugins to Maven Central. This will reduce the load on the community repository and make it easier to host mirrors of it. We will need the help of the plugin authors to make progress on this item. | ||
julienrf marked this conversation as resolved.
Show resolved
Hide resolved
|
||
2. Host the Linux packages of sbt releases in a separate place. This will avoid the need to create regular backups of the community repository since all its content will be read-only. | ||
|
||
Concretely, we have already collected a backup of the content of the community repository, and set up a backup server hosting a mirror of the repository. We are currently investigating which plugins are actively used and should be migrated to Maven Central. The next step we will be to explore alternative solutions to host the Linux packages of sbt releases. | ||
julienrf marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Conclusion | ||
|
||
The sbt community repository is currently hosted by the company JFrog, which generously provides an Artifactory instance. We are taking steps to reduce the load on their infrastructure in order to both make sure our partnership with JFrog will continue, and to reduce the costs for making the repository redundant. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.