[discussion] Scope reduction possibilities

[tmcw]

I'm considering removing less popular or vital features from documentation.js to keep the project tight and fast.

• Remove polyglot mode #850 ✅ Polyglot support (ability to parse C++): downsides of this right now is that it makes file extension handling more complex, the upstream library we use for polyglot comment extraction is buggy and unmaintained. This was great for node-mapnik but I don't think it has enough users to produce any feature owners.
• HTML output. We could rely on gatsby instead or another framework, and try to only output JSON. This gets tricky with remark and interlinking though: lots of implicit links, like linked types, need to be resolved to HTML anchor links or page links. This makes documentation generation much more complex than blog generation.
• Dependency following. Downside of this right now is that module-deps is a major performance hog, and doesn't work with some new types of imports, like import type for importing Flow types.

[ianwremmel]

removing html output is an interesting idea: i was digging through the issue queue to try to see if a bug i'm seeing was known and it seems like most of the bugs you're dealing with are theme related. I know there are some parser issues that could use some love and offloading the theme stuff seems like a great idea. that said, I think some drop-in means for htmloutput without requiring someone to roll their own theme seems very important.

I'm curious what you had in mind with gatsby. I'm only loosely familiar with it, but it may align with an idea i had for building a react-based theme with (a) a component corresponding to each jsdoc primitive and (b) something that bundles up storybook to make custom themes super easy to get started.

[tmcw]

I'd love to decouple theme generation - my main qualm is noted above, specifically that existing theming systems are very comfortable with 'unstructured input' like Markdown and images, whereas the ability of documentation.js to interlink makes its output far from unstructured: if you mention something in your documentation with, for instance, {@link foo}, resolving foo to #foo if it's on the page, or foo.html if you're generating a page per method, or linking to MDN if it's a built-in - that makes the generation of text output pretty tricky, and, currently, somewhat intertwined with the documentation generation. I'd love to find a way to avoid that paradox.

[ianwremmel]

Interesting. So, does that mean the JSON output has those kinda of hints in it?

[tmcw]

Yep, it does! Here's an example of JSON output and its input: https://gist.github.com/tmcw/87f3ad764851a03e679cf814c1d8a95a

Note on line 30, the {@link Foo} gets parsed into a link object with "jsdoc": true to differentiate it from a link to a URL 'Foo' - instead, this is linking to the namespace 'Foo'.

[ianwremmel]

so jsdoc:true implies it's a link to a documented entity, whereas jsdoc:true implies it's simply a url? I think I'm still a little fuzzy on why that's an issue. (but, obviously, you've been doing this way more than I have 😀).

[ianwremmel]

Oh, is the concern more about links in markdown and determining if they're pointed at documented entities or just plain urls?

[tmcw]

Oh, is the concern more about links in markdown and determining if they're pointed at documented entities or just plain urls?

Yep, that's it. Markdown has no facility for indicating this distinction. We could invent some pattern but it's unlikely that any markdown-based generator will work out-of-the-box.

[MichelSimonot]

RE: HTML output.

Any further thoughts on this? As someone trying to decide whether to create my own theme or customize the default_theme (it's a bit inflexible for my needs), knowing the direction that it's going could be important.

Having decoupled theme generation would be great. Could you expand on what you're thinking this may look like? (eg. would this be moving it back out of core, creating something new, ... ?)

To provide a bit of feedback to the [discussion], I'm not a big fan of the current HTML output system. It works great, as long as you don't need to customize anything with the HTML. When playing around with it, I found a small problem with it and so needed to modify the theme to render part of the JSON output that was being missed. Trying to understand how the template works (both on its own and with the JSON structure) wasn't very newbie friendly. Hence why I'm deciding between the long-term "create my own theme" so it's both easier for my team to maintain and customized to our needs, or the short-term "fork the default theme" so we don't stray too far from master. Is it possible that it is a "less popular" feature because of this type of issue?

As a passer-by, I think it makes sense for docjs to focus on the JSON output. Both since that's the "hard part" and people are much more likely to want to do their own HTML thing. But it's still important to have a quick-n-easy HTML output system for new users to test out and play around with. For my scenario, it would be very convenient if the default theme was split out from the docjs repo (much like the light and serif themes), so that I could work with a fork of it without needing to worry about keeping the rest of the project in-sync. But I understand that probably doesn't go far enough in terms of "removing" the feature.

[ianwremmel]

@tmcw, I see you're committing on https://github.com/gatsbyjs/gatsby-starter-documentation. Is gatsby going to be the way forward for documentation.js? (I'm just starting to play with gatsby and I'm really excited by that possibility).

[tmcw]

It's a big part of the plan for the output stage. For dynamic output, it's pretty good and I think will let us simplify documentation.js.

The crux (for my work / interest) lies more in the format & expectations around it, which I've been trying to form in dx-spec, and that includes more... aggressive changes, like discontinuing CommonJS support and moving toward something different / simpler / more explicitly specified than JSDoc.