Hierarchical dependencies #3425
Conversation
Also, ensure that legacy libraries have a global name.
Version warnings currently only implemented in the case of a direct import.
The tooltips will sometimes get this wrong, unfortunately, as they do not have enough context.
|
I realise my library import mechanism is kind of terrible. I am not accounting for preprocessor conditionals, for one thing. Also, I have just become aware of the existing thread "build dependency improvement" from this May, the 5 previous pull requests, and issue #236. Apologies for not doing sufficient research before. However, I was really just going for a proof of concept here. I wanted to explore some of the wider issues around hierarchical dependencies, particularly the problems of accurate library identification and user experience. I don't know whether these have been addressed before. I am willing to do more work to improve my implementation and/or retrofit some of my ideas into a previous pull request, but I would like to get some feedback from the community first. |
|
hi martino, I have a test case, i write two lib, the first may use or may not use the second lib. |
|
hello @martinjos and sorry for the late reply. your PR was a bit too stuffed and hard to test. smaller PRs are easier to validate and merge. Anyway, we implemented hierarchical dependencies since 9fadb54 which is available with hourly builds http://www.arduino.cc/en/Main/Software#hourly |
|
I believe that the global naming scheme isn't actually useful by itself, but mostly serve to allow the dependency specs to work in a meaningful way (since they now have somewhat unique names to refer to). @ffissore, are you considering adding these dependency specs as well? Or do you have another idea for them? Perhaps, instead of dependency specs, you could include the library name in the include line? e.g. What you say about forking and changing the unique name, perhaps the library metadata could specify multiple names for a single library? E.g. one primary name (which is changed whenever a library is forked) and zero or more secondary names (which can be used to store the name of the library forked from). When resolving library includes, one could look at primary names first, and only if a primary name isn't available, look at secondary names. Or, perhaps the user could be involved in making choices here, but then it also helps to know that a forked library is a candidate when the original library is being included. |
|
Uhm seems complicated, but I'll leave this call to @cmaglie |
This branch achieves 4 things:
The general idea of this is to permit the development of mid-level libraries that encapsulate the details of the low-level libraries that they depend on, providing a simpler, more accessible, and perhaps more featureful interface to the same functionality, without bloating the low-level libraries; and to allow common parts of sketches to be factored out. Specifically, it should be possible to do these things without the user having to worry about additional headers to include because of recursive dependencies.
More details follow:
Libraries now have global names. The ordinary name of a library cannot be relied upon to be unique. Nor can documentation URLs (which are not, in any case, intended for identification purposes). This problem is especially exacerbated by forking. A hash, meanwhile, is too specific - you may want to work with different versions of the same library, possibly tweaked by the user immediately prior to compilation.
The solution is to require each library to have a globally unique name, which should be based on a reverse domain name in the manner of Java or Android package names. For instance, if your project is stored at https://github.com/myaccount/myproject, the global name may be something like io.github.myaccount.myproject (note the use of GitHub's user-only namespace - this is probably best practice even if the user domain is not in use). If myproject contains several subprojects, then they may have global names like "io.github.myaccount.myproject.subproject".
When the project is forked by otheraccount, the global name of "io.github.myaccount.myproject" should be changed to "io.github.otheraccount.myproject", etc. (unless it is to be considered the same library - e.g. when there is a change of maintainer).
There is code to deal with legacy projects and projects predating the introduction of the global name, generally by trying to infer the repository/project URL and convert it to a suitable form, and, failing that, falling back on the simple name, combined with the author if present. For this proof-of-concept, the code only looks at the url, name, and author fields of the library.properties file, but other things to try include the repository information in the library.json file if present, repository metadata (.git, .svn...), etc. However, this should probably only ever be a stopgap solution before introducing a proper global name.
Dependencies are found by dependency spec (where present), rather than by header name alone. Next to each #include statement, there can be a comment consisting of //!Lib followed by a dependency spec. A dependency spec consists of a global name optionally followed by : and a version spec. A version spec may be a version number, or a partial version number followed by *, or a version number followed by +, or two version numbers separated by -. These are interpreted in the obvious way. A simple dependency spec is generated for each new #include line when the IDE is used to import a library.
The global name part alone is used for deciding between alternative libraries, while the version is used to report an incorrect version. (At the moment, this reporting is only partially implemented.)
Dependencies are now found hierarchically (i.e. recursively). All source files within each imported library are examined for #include statements (which may also have dependency specs - see note 2), and additional libraries are added to the project accordingly. These additional imported libraries are also examined for further dependencies, and so on. Naturally, this is implemented such that it is able to deal with dependency loops. The information is also cached - although the cache is updated whenever a source file has changed.
In order to keep users in-the-loop about what libraries they are importing, when a #include statement is hovered over, a tooltip is displayed showing details of all libraries drawn in by that statement, starting with the one directly imported by the #include itself. Friendly name, global name, version and directory are shown for each library.
The tooltips will always be correct about indirect dependencies (i.e. libraries imported by a library you import). However, they sometimes currently don't have enough context to know that a first-level dependency is affected by previous first-level dependencies (although this shouldn't be that difficult to fix). However, first-level dependency choices are still reported (correctly) with the compiler output, as before.
Further work needed: