Skip to content

Fix shipping static resources for static site #12043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/reference/experimental/erased-defs.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ object Machine:
// State must be Off
```

Note that in [Inline](./inline.md) we discussed `erasedValue` and inline
Note that in [Inline](../metaprogramming/inline.md) we discussed `erasedValue` and inline
matches. `erasedValue` is implemented with `erased`, so the state machine above
can be encoded as follows:

Expand Down
8 changes: 4 additions & 4 deletions docs/docs/reference/features-classification.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,12 @@ These constructs replace existing constructs with the aim of making the language
- [Export clauses](other-new-features/export.md)
provide a simple and general way to express aggregation, which can replace the
previous facade pattern of package objects inheriting from classes.
- [Vararg patterns](changed-features/vararg-patterns.md) now use the form `: _*` instead of `@ _*`, mirroring vararg expressions,
- [Vararg splices](changed-features/vararg-splices.md) now use the form `*` instead of `@ _*`, mirroring vararg expressions,
- [Creator applications](other-new-features/creator-applications.md) allow using simple function call syntax
instead of `new` expressions. `new` expressions stay around as a fallback for
the cases where creator applications cannot be used.

With the exception of early initializers and old-style vararg patterns, all superseded constructs continue to be available in Scala 3.0. The plan is to deprecate and phase them out later.
With the exception of early initializers and old-style vararg splices, all superseded constructs continue to be available in Scala 3.0. The plan is to deprecate and phase them out later.

Value classes (superseded by opaque type aliases) are a special case. There are currently no deprecation plans for value classes, since we might bring them back in a more general form if they are supported natively by the JVM as is planned by project Valhalla.

Expand All @@ -70,7 +70,7 @@ For the next several versions, old features will remain available and deprecatio
These constructs are restricted to make the language safer.

- [Implicit Conversions](contextual/conversions.md): there is only one way to define implicit conversions instead of many, and potentially surprising implicit conversions require a language import.
- [Given Imports](contextual/import-delegate.md): implicits now require a special form of import, to make the import clearly visible.
- [Given Imports](contextual/given-imports.md): implicits now require a special form of import, to make the import clearly visible.
- [Type Projection](dropped-features/type-projection.md): only classes can be used as prefix `C` of a type projection `C#A`. Type projection on abstract types is no longer supported since it is unsound.
- [Multiversal equality](contextual/multiversal-equality.md) implements an "opt-in" scheme to rule out nonsensical comparisons with `==` and `!=`.
- [infix](https://github.com/lampepfl/dotty/pull/5975)
Expand Down Expand Up @@ -173,7 +173,7 @@ To enable porting most uses of macros, we are experimenting with the advanced la
by itself a straightforward implementation of some simple macros and is at the same time an essential building block for the implementation of complex macros.
- [Quotes and splices](metaprogramming/macros.md) provide a principled way to express macros and staging with a unified set of abstractions.
- [Type class derivation](contextual/derivation.md) provides an in-language implementation of the `Gen` macro in Shapeless and other foundational libraries. The new implementation is more robust, efficient and easier to use than the macro.
- [Implicit by-name parameters](contextual/implicit-by-name-parameters.md) provide a more robust in-language implementation of the `Lazy` macro in Shapeless.
- [Implicit by-name parameters](contextual/by-name-context-parameters.md) provide a more robust in-language implementation of the `Lazy` macro in Shapeless.

**Status: not yet settled**

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/reference/other-new-features/explicit-nulls.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ When explicit nulls are enabled, the type hierarchy changes so that `Null` is on

This is the new type hierarchy:

!["Type Hierarchy for Explicit Nulls"](../../../images/explicit-nulls/explicit-nulls-type-hierarchy.png)
!["Type Hierarchy for Explicit Nulls"](/images/explicit-nulls/explicit-nulls-type-hierarchy.png)

After erasure, `Null` remains a subtype of all reference types (as forced by the JVM).

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/release-notes/0.1.2.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ This release ships with the following features:
[4]: ../reference/new-types/union-types.html
[5]: ../reference/enums/adts.html
[6]: ../reference/enums/desugarEnums.html
[7]: ../reference/other-new-features/implicit-by-name-parameters.html
[7]: ../reference/contextual/by-name-context-parameters.html
[8]: https://infoscience.epfl.ch/record/228518
[9]: http://docs.scala-lang.org/sips/pending/static-members.html
[10]: http://docs.scala-lang.org/sips/pending/improved-lazy-val-initialization.html
Expand All @@ -103,7 +103,7 @@ This release ships with the following features:
[27]: https://github.com/lampepfl/dotty/pull/2513
[28]: https://github.com/lampepfl/dotty/pull/2361
[29]: https://github.com/lampepfl/dotty/pull/1453
[30]: ../reference/contextual/implicit-function-types.html
[30]: ../reference/contextual/context-functions.html
[31]: https://github.com/lampepfl/dotty/pull/2136
[32]: https://github.com/lampepfl/dotty/pull/1758
[33]: ../reference/metaprogramming/inline.html
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/usage/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Start using the Dotty IDE in any Dotty project by following the
### Standalone installation
Releases are available for download on the [Releases Section](https://github.com/lampepfl/dotty/releases)
of the Dotty repository. Releases include three executables: `scalac` the Dotty compiler,
`scaladoc` the [Scaladoc](scaladoc.md) and `scala` the Dotty REPL.
`scaladoc` the [Scaladoc](./scaladoc/index.md) and `scala` the Dotty REPL.

```
.
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/usage/worksheet-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,18 +29,18 @@ evaluate the code of the worksheet. Each line of output is printed on the right
of the expression that produced it. The worksheets run with the classes of your
project and its dependencies on their classpath.

![](../../images/worksheets/worksheet-run.png "Run worksheet")
![](images/worksheets/worksheet-run.png "Run worksheet")

By default, the worksheets are also run when the file is saved. This can be
configured in VSCode preferences:

![](../../images/worksheets/config-autorun.png "Configure run on save")
![](images/worksheets/config-autorun.png "Configure run on save")

Note that the worksheet are fully integrated with the rest of Dotty IDE: While
typing, errors are shown, completions are suggested, and you can use all the
other features of Dotty IDE such as go to definition, find all references, etc.

![](../../images/worksheets/worksheet-help.png "IDE features in the worksheet")
![](images/worksheets/worksheet-help.png "IDE features in the worksheet")

Implementation details
======================
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,16 +102,19 @@ class HtmlRenderer(rootPackage: Member, val members: Map[DRI, Member])(using ctx
def siteRoot = staticSite.get.root.toPath
def pathToResource(p: String) = Resource.File(p, siteRoot.resolve(p))

val siteImages = staticSite.toSeq.flatMap { _ =>
val siteImgPath = siteRoot.resolve("images")
def harvestResources(path: String) =
val siteImgPath = siteRoot.resolve(path)
if !Files.exists(siteImgPath) then Nil
else
val allPaths = Files.walk(siteImgPath, FileVisitOption.FOLLOW_LINKS)
val files = allPaths.filter(Files.isRegularFile(_)).iterator().asScala
files.map(p => siteRoot.relativize(p).toString).toList

val staticResources = staticSite.toSeq.flatMap { _ =>
harvestResources("images") ++ harvestResources("resources")
}

val siteResourcesPaths = allPages.toSet.flatMap(specificResources) ++ siteImages
val siteResourcesPaths = allPages.toSet.flatMap(specificResources) ++ staticResources

val resources = siteResourcesPaths.toSeq.map(pathToResource) ++ allResources(allPages)
resources.flatMap(renderResource)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,4 +81,4 @@ trait Locations(using ctx: DocContext):
case Nil => ""
case seq => seq.mkString("", "/", "/")

def driExisits(dri: DRI) = true // TODO implement checks!
def driExists(dri: DRI) = effectiveMembers.get(dri).isDefined || dri.isStaticFile
36 changes: 24 additions & 12 deletions scaladoc/src/dotty/tools/scaladoc/renderers/SiteRenderer.scala
Original file line number Diff line number Diff line change
Expand Up @@ -28,27 +28,39 @@ trait SiteRenderer(using DocContext) extends Locations:

def siteContent(pageDri: DRI, content: ResolvedTemplate): AppliedTag =
import content.ctx
def tryAsDri(str: String) =
def tryAsDri(str: String): Option[String] =
val (path, prefix) = str match
case HashRegex(path, prefix) => (path, prefix)
case _ => (str, "")

val res = ctx.driForLink(content.template.templateFile, path).filter(driExisits)
if res.isEmpty then report.warn(s"Unable to resolve link '$str'", content.template.file)
res.headOption.fold(str)(pathToPage(pageDri, _) + prefix)
val res = ctx.driForLink(content.template.templateFile, path).filter(driExists)
res.headOption.map(pathToPage(pageDri, _) + prefix)

def processLocalLink(str: String): String =
if str.startsWith("#") || str.isEmpty then str
else Try(URL(str)).map(_ => str).getOrElse(tryAsDri(str))
Try(URL(str)).map(_ => str).toOption.orElse {
tryAsDri(str)
}.orElse {
Option.when(
Files.exists(Paths.get(content.ctx.root.toPath.toAbsolutePath.toString, str))
)(
resolveLink(pageDri, str.stripPrefix("/"))
)
}.getOrElse {
report.warn(s"Unable to resolve link '$str'", content.template.file)
str
}

def processLocalLinkWithGuard(str: String): String =
if str.startsWith("#") || str.isEmpty then
str
else
processLocalLink(str)

val document = Jsoup.parse(content.resolved.code)
document.select("a").forEach(element =>
element.attr("href", processLocalLink(element.attr("href")))
element.attr("href", processLocalLinkWithGuard(element.attr("href")))
)
document.select("img").forEach { element =>
val link = element.attr("src")
Try(new URL(link)).getOrElse {
if(link.startsWith("/")) element.attr("src", resolveLink(pageDri, link.drop(1)))
}
}// foreach does not work here
element.attr("src", processLocalLink(element.attr("src")))
} // foreach does not work here. Why?
raw(document.outerHtml())
18 changes: 9 additions & 9 deletions scaladoc/src/dotty/tools/scaladoc/site/StaticSiteContext.scala
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,12 @@ class StaticSiteContext(
file.getName.endsWith(".html")


private def loadTemplate(from: File, isBlog: Boolean = false): Option[LoadedTemplate] =
private def loadTemplate(from: File, isBlog: Boolean): Option[LoadedTemplate] =
if (!isValidTemplate(from)) None else
try
val topLevelFiles = if isBlog then Seq(from, new File(from, "_posts")) else Seq(from)
val allFiles = topLevelFiles.filter(_.isDirectory).flatMap(_.listFiles())
val (indexes, children) = allFiles.flatMap(loadTemplate(_)).partition(_.templateFile.isIndexPage())
val (indexes, children) = allFiles.flatMap(loadTemplate(_, isBlog)).partition(_.templateFile.isIndexPage())

def loadIndexPage(): TemplateFile =
val indexFiles = from.listFiles { file => file.getName == "index.md" || file.getName == "index.html" }
Expand Down Expand Up @@ -135,13 +135,13 @@ class StaticSiteContext(
if link.startsWith("/") then root.toPath.resolve(link.drop(1))
else template.file.toPath.getParent().resolve(link).normalize()

val baseFileName = baseFile.getFileName.toString
val mdFile = baseFile.resolveSibling(baseFileName.stripSuffix(".html") + ".md")
def trySuffix(pref: String) =
if baseFileName == pref then Seq(baseFile.getParent) else Nil
val strippedIndexes = trySuffix("index.html") ++ trySuffix("index.md")

(Seq(baseFile, mdFile) ++ strippedIndexes).filter(Files.exists(_)).map(driFor)
baseFile.getFileName.toString.split("\\.").headOption.toSeq.flatMap { baseFileName =>
Seq(
Some(baseFile.resolveSibling(baseFileName + ".html")),
Some(baseFile.resolveSibling(baseFileName + ".md")),
Option.when(baseFileName == "index")(baseFile.getParent)
).flatten.filter(Files.exists(_)).map(driFor)
}
}.toOption.filter(_.nonEmpty)
pathsDri.getOrElse(memberLinkResolver(link).toList)

Expand Down
2 changes: 1 addition & 1 deletion scaladoc/src/dotty/tools/scaladoc/site/common.scala
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ val apiPageDRI: DRI = DRI(location = "api")

val defaultMarkdownOptions: DataHolder =
new MutableDataSet()
.setFrom(ParserEmulationProfile.KRAMDOWN.getOptions)
.setFrom(ParserEmulationProfile.COMMONMARK.getOptions)
.set(AnchorLinkExtension.ANCHORLINKS_WRAP_TEXT, false)
.set(AnchorLinkExtension.ANCHORLINKS_ANCHOR_CLASS, "anchor")
.set(EmojiExtension.ROOT_IMAGE_PATH, "https://github.global.ssl.fastly.net/images/icons/emoji/")
Expand Down
1 change: 0 additions & 1 deletion scaladoc/src/dotty/tools/scaladoc/site/templates.scala
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ case class TemplateFile(
val code = if (isHtml || layoutTemplate.exists(!_.isHtml)) rendered else
val parser: Parser = Parser.builder(defaultMarkdownOptions).build()
HtmlRenderer.builder(defaultMarkdownOptions).build().render(parser.parse(rendered))

layoutTemplate match
case None => ResolvedPage(code, resources ++ ctx.resources)
case Some(layoutTemplate) =>
Expand Down