Skip to content

Improve modeling of Child in CodeGeneration #1914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 19, 2023
Merged

Improve modeling of Child in CodeGeneration #1914

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
689efdf
Make `Child.varName` a `TokenSyntax`
ahoppen Jul 16, 2023
9adf2df
Change type of `Child.documentation` to `Trivia`
ahoppen Jul 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 30 additions & 13 deletions CodeGeneration/Sources/SyntaxSupport/Child.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
//
//===----------------------------------------------------------------------===//

import SwiftSyntax

/// The kind of token a node can contain. Either a token of a specific kind or a
/// keyword with the given text.
public enum TokenChoice: Equatable {
Expand Down Expand Up @@ -56,13 +58,33 @@ public enum ChildKind {
/// A child of a node, that may be declared optional or a token with a
/// restricted subset of acceptable kinds or texts.
public class Child {
/// The name of the child.
///
/// The first character of the name is always uppercase.
public let name: String

/// If the child has been renamed, its old, now deprecated, name.
///
/// This is used to generate deprecated compatibility layers.
public let deprecatedName: String?

/// The kind of the child (node, token, collection, ...)
public let kind: ChildKind
public let nameForDiagnostics: String?
public let documentation: String?

/// Whether this child is optional and can be `nil`.
public let isOptional: Bool

/// A name of this child that can be shown in diagnostics.
///
/// This is used to e.g. describe the child if all of its tokens are missing in the source file.
public let nameForDiagnostics: String?

/// A doc comment describing the child.
public let documentation: SwiftSyntax.Trivia

/// The first line of the child's documentation
public let documentationAbstract: String
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been thinking if this also should be using a Trivia type but in the end a String type makes more sense. Changing it would mean that in methods like this: func generateInitializerDocComment() -> SwiftSyntax.Trivia we would have to use something like TriviaPiece.commentValue property proposed in #1890.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the problem is that we use this string for the parameter documentation of the node’s initializers. And for those, we don’t want it prefixed with ///


public var syntaxNodeKind: SyntaxNodeKind {
switch kind {
case .node(kind: let kind):
Expand All @@ -77,16 +99,16 @@ public class Child {
}

/// A name of this child that's suitable to be used for variable or enum case names.
public var varName: String {
return lowercaseFirstWord(name: name)
public var varOrCaseName: TokenSyntax {
return .identifier(lowercaseFirstWord(name: name))
}

/// The deprecated name of this child that's suitable to be used for variable or enum case names.
public var deprecatedVarName: String? {
public var deprecatedVarName: TokenSyntax? {
guard let deprecatedName = deprecatedName else {
return nil
}
return lowercaseFirstWord(name: deprecatedName)
return .identifier(lowercaseFirstWord(name: deprecatedName))
}

/// If the child ends with "token" in the kind, it's considered a token node.
Expand Down Expand Up @@ -146,12 +168,6 @@ public class Child {
}
}

/// Returns `true` if this child's type is one of the base syntax kinds and
/// it's optional.
public var hasOptionalBaseType: Bool {
return hasBaseType && isOptional
}

/// If a classification is passed, it specifies the color identifiers in
/// that subtree should inherit for syntax coloring. Must be a member of
/// ``SyntaxClassification``.
Expand All @@ -172,7 +188,8 @@ public class Child {
self.deprecatedName = deprecatedName
self.kind = kind
self.nameForDiagnostics = nameForDiagnostics
self.documentation = documentation
self.documentation = docCommentTrivia(from: documentation)
self.documentationAbstract = String(documentation?.split(whereSeparator: \.isNewline).first ?? "")
self.isOptional = isOptional
}
}
9 changes: 8 additions & 1 deletion CodeGeneration/Sources/SyntaxSupport/CommonNodes.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ public let COMMON_NODES: [Node] = [
kind: .token(choices: [.token(tokenKind: "IdentifierToken")], requiresLeadingSpace: false, requiresTrailingSpace: false),
documentation: """
A placeholder, i.e. `<#decl#>`, that can be inserted into the source code to represent the missing declaration.

This token should always have `presence = .missing`.
"""
),
Expand All @@ -201,6 +202,7 @@ public let COMMON_NODES: [Node] = [
kind: .token(choices: [.token(tokenKind: "IdentifierToken")], requiresLeadingSpace: false, requiresTrailingSpace: false),
documentation: """
A placeholder, i.e. `<#expression#>`, that can be inserted into the source code to represent the missing expression.

This token should always have `presence = .missing`.
"""
)
Expand All @@ -221,6 +223,7 @@ public let COMMON_NODES: [Node] = [
kind: .token(choices: [.token(tokenKind: "IdentifierToken")], requiresLeadingSpace: false, requiresTrailingSpace: false),
documentation: """
A placeholder, i.e. `<#pattern#>`, that can be inserted into the source code to represent the missing pattern.

This token should always have `presence = .missing`.
"""
)
Expand All @@ -241,6 +244,7 @@ public let COMMON_NODES: [Node] = [
kind: .token(choices: [.token(tokenKind: "IdentifierToken")], requiresLeadingSpace: false, requiresTrailingSpace: false),
documentation: """
A placeholder, i.e. `<#statement#>`, that can be inserted into the source code to represent the missing pattern.

This token should always have `presence = .missing`.
"""
)
Expand All @@ -261,6 +265,7 @@ public let COMMON_NODES: [Node] = [
kind: .token(choices: [.token(tokenKind: "IdentifierToken")], requiresLeadingSpace: false, requiresTrailingSpace: false),
documentation: """
A placeholder, i.e. `<#syntax#>`, that can be inserted into the source code to represent the missing pattern.

This token should always have `presence = .missing`
"""
)
Expand All @@ -280,7 +285,9 @@ public let COMMON_NODES: [Node] = [
name: "Placeholder",
kind: .token(choices: [.token(tokenKind: "IdentifierToken")], requiresLeadingSpace: false, requiresTrailingSpace: false),
documentation: """
A placeholder, i.e. `<#type#>`, that can be inserted into the source code to represent the missing type. This token should always have `presence = .missing`.
A placeholder, i.e. `<#type#>`, that can be inserted into the source code to represent the missing type.

This token should always have `presence = .missing`.
"""
)
]
Expand Down
2 changes: 1 addition & 1 deletion CodeGeneration/Sources/SyntaxSupport/GrammarGenerator.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ struct GrammarGenerator {
return
children
.filter { !$0.isUnexpectedNodes }
.map { " - `\($0.varName)`: \(generator.grammar(for: $0))" }
.map { " - `\($0.varOrCaseName)`: \(generator.grammar(for: $0))" }
.joined(separator: "\n")
}
}
12 changes: 12 additions & 0 deletions CodeGeneration/Sources/SyntaxSupport/Utils.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,15 @@ public extension Collection {
}
}
}

public extension TokenSyntax {
var backtickedIfNeeded: TokenSyntax {
if KEYWORDS.contains(where: {
$0.name == self.description && $0.isLexerClassified
}) {
return "`\(self)`"
} else {
return self
}
}
}
39 changes: 21 additions & 18 deletions CodeGeneration/Sources/generate-swiftsyntax/LayoutNode+Extensions.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,18 +39,18 @@ extension LayoutNode {
}
}

let parameterName: String
let parameterName: TokenSyntax

if useDeprecatedChildName, let deprecatedVarName = child.deprecatedVarName {
parameterName = deprecatedVarName
} else {
parameterName = child.varName
parameterName = child.varOrCaseName
}

return FunctionParameterSyntax(
leadingTrivia: .newline,
firstName: child.isUnexpectedNodes ? .wildcardToken(trailingTrivia: .space) : .identifier(parameterName),
secondName: child.isUnexpectedNodes ? .identifier(parameterName) : nil,
firstName: child.isUnexpectedNodes ? .wildcardToken(trailingTrivia: .space) : parameterName,
secondName: child.isUnexpectedNodes ? parameterName : nil,
colon: .colonToken(),
type: paramType,
defaultArgument: child.defaultInitialization
Expand All @@ -77,13 +77,10 @@ extension LayoutNode {

func generateInitializerDocComment() -> SwiftSyntax.Trivia {
func generateParamDocComment(for child: Child) -> String? {
guard let documentation = child.documentation,
let firstLine = documentation.split(whereSeparator: \.isNewline).first
else {
if child.documentationAbstract.isEmpty {
return nil
}

return " - \(child.varName): \(firstLine)"
return " - \(child.varOrCaseName): \(child.documentationAbstract)"
}

let formattedParams = """
Expand Down Expand Up @@ -114,12 +111,12 @@ extension LayoutNode {
for child in children {
/// The expression that is used to call the default initializer defined above.
let produceExpr: ExprSyntax
let childName: String
let childName: TokenSyntax

if useDeprecatedChildName, let deprecatedVarName = child.deprecatedVarName {
childName = deprecatedVarName
} else {
childName = child.varName
childName = child.varOrCaseName
}

if child.type.isBuilderInitializable {
Expand All @@ -129,12 +126,12 @@ extension LayoutNode {
if child.type.builderInitializableType != child.type {
let param = Node.from(type: child.type).layoutNode!.singleNonDefaultedChild
if child.isOptional {
produceExpr = ExprSyntax("\(raw: childName)Builder().map { \(raw: child.type.syntaxBaseName)(\(raw: param.varName): $0) }")
produceExpr = ExprSyntax("\(childName)Builder().map { \(raw: child.type.syntaxBaseName)(\(param.varOrCaseName): $0) }")
} else {
produceExpr = ExprSyntax("\(raw: child.type.syntaxBaseName)(\(raw: param.varName): \(raw: childName)Builder())")
produceExpr = ExprSyntax("\(raw: child.type.syntaxBaseName)(\(param.varOrCaseName): \(childName)Builder())")
}
} else {
produceExpr = ExprSyntax("\(raw: childName)Builder()")
produceExpr = ExprSyntax("\(childName)Builder()")
}
builderParameters.append(
FunctionParameterSyntax(
Expand All @@ -145,14 +142,20 @@ extension LayoutNode {
produceExpr = convertFromSyntaxProtocolToSyntaxType(child: child, useDeprecatedChildName: useDeprecatedChildName)
normalParameters.append(
FunctionParameterSyntax(
firstName: .identifier(childName),
firstName: childName,
colon: .colonToken(),
type: child.parameterType,
defaultArgument: child.defaultInitialization
)
)
}
delegatedInitArgs.append(TupleExprElementSyntax(label: child.isUnexpectedNodes ? nil : child.varName, expression: produceExpr))
delegatedInitArgs.append(
TupleExprElementSyntax(
label: child.isUnexpectedNodes ? nil : child.varOrCaseName,
colon: child.isUnexpectedNodes ? nil : .colonToken(),
expression: produceExpr
)
)
}

guard shouldCreateInitializer else {
Expand Down Expand Up @@ -185,11 +188,11 @@ extension LayoutNode {
}

fileprivate func convertFromSyntaxProtocolToSyntaxType(child: Child, useDeprecatedChildName: Bool = false) -> ExprSyntax {
let childName: String
let childName: TokenSyntax
if useDeprecatedChildName, let deprecatedVarName = child.deprecatedVarName {
childName = deprecatedVarName
} else {
childName = child.varName
childName = child.varOrCaseName
}

if child.type.isBaseType && !child.kind.isNodeChoices {
Expand Down
2 changes: 1 addition & 1 deletion ...s/generate-swiftsyntax/templates/swiftparserdiagnostics/ChildNameForDiagnosticsFile.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ let childNameForDiagnosticFile = SourceFileSyntax(leadingTrivia: copyrightHeader
for node in NON_BASE_SYNTAX_NODES.compactMap(\.layoutNode) {
for child in node.children {
if let nameForDiagnostics = child.nameForDiagnostics {
SwitchCaseSyntax("case \\\(raw: node.type.syntaxBaseName).\(raw: child.varName):") {
SwitchCaseSyntax("case \\\(raw: node.type.syntaxBaseName).\(child.varOrCaseName):") {
StmtSyntax(#"return "\#(raw: nameForDiagnostics)""#)
}
}
Expand Down
4 changes: 2 additions & 2 deletions ...neration/Sources/generate-swiftsyntax/templates/swiftsyntax/ChildNameForKeyPathFile.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ let childNameForKeyPathFile = SourceFileSyntax(leadingTrivia: copyrightHeader) {
for child in node.children {
SwitchCaseSyntax(
"""
case \\\(raw: node.type.syntaxBaseName).\(raw: child.varName):
return \(literal: child.varName)
case \\\(raw: node.type.syntaxBaseName).\(child.varOrCaseName):
return \(literal: child.varOrCaseName.description)
"""
)
}
Expand Down
10 changes: 5 additions & 5 deletions CodeGeneration/Sources/generate-swiftsyntax/templates/swiftsyntax/RawSyntaxNodesFile.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ fileprivate extension Node {
return node.children.compactMap { child -> (name: TokenSyntax, choices: [(caseName: TokenSyntax, kind: SyntaxNodeKind)])? in
switch child.kind {
case .nodeChoices(let choices):
return (.identifier(child.name), choices.map { (.identifier($0.varName), $0.syntaxNodeKind) })
return (.identifier(child.name), choices.map { ($0.varOrCaseName, $0.syntaxNodeKind) })
default:
return nil
}
Expand Down Expand Up @@ -202,8 +202,8 @@ let rawSyntaxNodesFile = SourceFileSyntax(leadingTrivia: copyrightHeader) {
let params = FunctionParameterListSyntax {
for child in node.children {
FunctionParameterSyntax(
firstName: child.isUnexpectedNodes ? .wildcardToken(trailingTrivia: .space) : .identifier(child.varName),
secondName: child.isUnexpectedNodes ? .identifier(child.varName) : nil,
firstName: child.isUnexpectedNodes ? .wildcardToken(trailingTrivia: .space) : child.varOrCaseName,
secondName: child.isUnexpectedNodes ? child.varOrCaseName : nil,
colon: .colonToken(),
type: child.rawParameterType,
defaultArgument: child.isUnexpectedNodes ? child.defaultInitialization : nil
Expand All @@ -218,7 +218,7 @@ let rawSyntaxNodesFile = SourceFileSyntax(leadingTrivia: copyrightHeader) {
ExprSyntax("layout.initialize(repeating: nil)")
for (index, child) in node.children.enumerated() {
let optionalMark = child.isOptional ? "?" : ""
ExprSyntax("layout[\(raw: index)] = \(raw: child.varName.backtickedIfNeeded)\(raw: optionalMark).raw")
ExprSyntax("layout[\(raw: index)] = \(child.varOrCaseName.backtickedIfNeeded)\(raw: optionalMark).raw")
.with(\.leadingTrivia, .newline)
}
}
Expand All @@ -238,7 +238,7 @@ let rawSyntaxNodesFile = SourceFileSyntax(leadingTrivia: copyrightHeader) {
}

for (index, child) in node.children.enumerated() {
try VariableDeclSyntax("public var \(raw: child.varName.backtickedIfNeeded): Raw\(raw: child.type.buildable)") {
try VariableDeclSyntax("public var \(child.varOrCaseName.backtickedIfNeeded): Raw\(raw: child.type.buildable)") {
let iuoMark = child.isOptional ? "" : "!"

if child.syntaxNodeKind == .syntax {
Expand Down
16 changes: 10 additions & 6 deletions ...Sources/generate-swiftsyntax/templates/swiftsyntax/RenamedChildrenCompatibilityFile.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,13 @@ let renamedChildrenCompatibilityFile = try! SourceFileSyntax(leadingTrivia: copy

DeclSyntax(
"""
@available(*, deprecated, renamed: "\(raw: child.varName)")
@available(*, deprecated, renamed: "\(child.varOrCaseName)")
public var \(raw: deprecatedVarName): \(raw: type) {
get {
return \(raw: child.varName.backtickedIfNeeded)
return \(child.varOrCaseName.backtickedIfNeeded)
}
set {
\(raw: child.varName.backtickedIfNeeded) = newValue
\(child.varOrCaseName.backtickedIfNeeded) = newValue
}
}
"""
Expand All @@ -57,7 +57,7 @@ let renamedChildrenCompatibilityFile = try! SourceFileSyntax(leadingTrivia: copy

let deprecatedNames = layoutNode.children
.filter { !$0.isUnexpectedNodes && $0.deprecatedName != nil }
.map { $0.varName }
.map { $0.varOrCaseName.description }
.joined(separator: ", ")

try! InitializerDeclSyntax(
Expand All @@ -71,9 +71,13 @@ let renamedChildrenCompatibilityFile = try! SourceFileSyntax(leadingTrivia: copy
TupleExprElementSyntax(label: "leadingTrivia", expression: ExprSyntax("leadingTrivia"))
for child in layoutNode.children {
if child.isUnexpectedNodes {
TupleExprElementSyntax(expression: ExprSyntax("\(raw: child.deprecatedVarName ?? child.varName)"))
TupleExprElementSyntax(expression: ExprSyntax("\(raw: child.deprecatedVarName ?? child.varOrCaseName)"))
} else {
TupleExprElementSyntax(label: "\(child.varName)", expression: ExprSyntax("\(raw: child.deprecatedVarName ?? child.varName)"))
TupleExprElementSyntax(
label: child.varOrCaseName,
colon: .colonToken(),
expression: IdentifierExprSyntax(identifier: child.deprecatedVarName ?? child.varOrCaseName)
)
}
}
TupleExprElementSyntax(label: "trailingTrivia", expression: ExprSyntax("trailingTrivia"))
Expand Down
Loading