migrate content to issue

Author: alamb

README.md

@@ -105,11 +105,11 @@ issues and distinguish different syntaxes in the AST.
 This crate allows recovering source locations from AST nodes via the [Spanned]
 trait, which can be used for advanced diagnostics tooling. Note that this
 feature is a work in progress and many nodes report missing or inaccurate spans.
-Please see [this document] for information on how to contribute missing
+Please see [this ticket] for information on how to contribute missing
 improvements.
 
 [Spanned]: https://docs.rs/sqlparser/latest/sqlparser/ast/trait.Spanned.html
-[this document]: ./docs/source_spans.md#source-span-contributing-guidelines
+[this ticket]: https://github.com/apache/datafusion-sqlparser-rs/issues/1548
 
 ```rust
 // Parse SQL

docs/source_spans.md

@@ -78,14 +78,13 @@ use sqlparser_derive::{Visit, VisitMut};
 /// assert_eq!(AttachedToken(tok1), AttachedToken(tok2)); // attached tokens are
 /// ```
 /// // period @ line 2, column 20
-
-/// ```
 #[derive(Clone)]
 #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
 #[cfg_attr(feature = "visitor", derive(Visit, VisitMut))]
 pub struct AttachedToken(pub TokenWithLocation);
 
 impl AttachedToken {
+    /// Return a new Empty AttachedToken
     pub fn empty() -> Self {
         AttachedToken(TokenWithLocation::wrap(Token::EOF))
     }
@@ -130,3 +129,9 @@ impl From<TokenWithLocation> for AttachedToken {
         AttachedToken(value)
     }
 }
+
+impl From<AttachedToken> for TokenWithLocation {
+    fn from(value: AttachedToken) -> Self {
+        value.0
+    }
+}

src/ast/helpers/attached_token.rs

@@ -25,7 +25,7 @@
 //! 1. [`Parser::parse_sql`] and [`Parser::new`] for the Parsing API
 //! 2. [`ast`] for the AST structure
 //! 3. [`Dialect`] for supported SQL dialects
-//! 4. [`Spanned`] for source text locations
+//! 4. [`Spanned`] for source text locations (see "Source Spans" below for details)
 //!
 //! [`Spanned`]: ast::Spanned
 //!
@@ -64,13 +64,62 @@
 //! // The original SQL text can be generated from the AST
 //! assert_eq!(ast[0].to_string(), sql);
 //! ```
-//!
 //! [sqlparser crates.io page]: https://crates.io/crates/sqlparser
 //! [`Parser::parse_sql`]: crate::parser::Parser::parse_sql
 //! [`Parser::new`]: crate::parser::Parser::new
 //! [`AST`]: crate::ast
 //! [`ast`]: crate::ast
 //! [`Dialect`]: crate::dialect::Dialect
+//!
+//! # Source Spans
+//!
+//! Starting with version  `0.53.0` sqlparser is introducing source spans to the
+//! AST. This feature is intended to provide more accurate source locations for
+//! syntax errors and to enable better error messages. See [issue #1548] for more
+//! information.
+//!
+//! [issue #1548]: https://github.com/apache/datafusion-sqlparser-rs/issues/1548
+//!
+//! ## Migration Guide
+//!
+//! For the next few releases, we will be incrementally adding source spans to the
+//! AST, trying to minimizes the impact on existing users, though some breaking
+//! changes are inevitable. The following is a summary of the changes:
+//!
+//! #### New fields for spans (must be added to any existing pattern matches)
+//!
+//! The primary change is that new fields will be added to AST nodes to store the source `Span` or `TokenWithLocation`.
+//!
+//! This will require
+//! 1. Adding new fields to existing pattern matches.
+//! 2. Filling in the proper span information when constructing AST nodes.
+//!
+//! For example, since `Ident` now stores a `Span`, so to construct an `Ident` you
+//! must provide a `Span` when constructing one:
+//!
+//! Previously:
+//! ```rust
+//! # use sqlparser::ast::Ident;
+//! Ident {
+//!     value: "name".into(),
+//!     quote_style: None,
+//! }
+//! ```
+//! Now
+//! ```rust
+//! # use sqlparser::ast::Ident;
+//! # use sqlparser::ast::Span;
+//! Ident {
+//!     value: "name".into(),
+//!     quote_style: None,
+//!     span: Span::empty(),
+//! }
+//! ```
+//! #### Misc.
+//! - [`TokenWithLocation`] stores a full `Span`, rather than just a source location.
+//!   Users relying on `token.location` should use `token.location.start` instead.
+//!
+//![`TokenWithLocation`]: tokenizer::TokenWithLocation
 
 #![cfg_attr(not(feature = "std"), no_std)]
 #![allow(clippy::upper_case_acronyms)]