Support for library projects

[jonbarril]

As you (jantje) suggested, I have started a new issue rather than commenting on an old issue.. I also included below what I have developed as a workaround for support of "library" projects, which was previously posted in the old issue.

I read the article http://eclipse.baeyens.it/how_to.shtml#/j that you suggested regarding the creation of user libraries as a solution to the problem I was trying to solve. I think your solution has a major deficiency when compared to mine: In mine, if I edit a source file in a library project, and then verify/upload the parent executable, the child project will be rebuilt automatically and the changes included in the executable. I believe in your solution the edited source files must be explicitly copied to the external library directory before rebuild, which is outside of the workspace and therefore not managed by and Eclipse.

I look forward to discussing this and hopefully in the process helping to improve the IDE you have developed.

Thanks
--jon

============================

Background:

Using the latest Eclipse and Sloeber Arduino plugin as of 2022-10-15, the plugin
provides Eclipse with an Arduino IDE. It has several serious limitations as
regards project development involving reusable "libaries":

It does NOT support project references (i.e. project executable A requires
library project B). In Eclipse, normally, a project can reference another by
right-clicking on a project and using the "Referenced Projects" tab to specify
dependent projects. This does NOT work with the Sloeber plugin.

Sloeber supports one and only one executable sketch per project. Upon a clean
build the plugin generates a "sloeber.ino.cpp" file, which includes a reference
to the project .ino executable file. Without the sloeber file the compile and
upload will fail. Note that even if a project is intended as a "library",
without an .ino executable, the plugin automatically generates a sloeber file
anyway.

Details:

The following procedures have been developed through trial and error. These are
not foolproof, especially since the Eclipse-Sloeber platform is brittle, easily
succumbing to missteps and synchronization problems.

For a given "parent" project, for each of its "dependent" projects...
1.1) Right click on the parent project.
1.2) Select Import->Arduino->"Import a folder containing source...."
1.3) Browse to and select the dependent project folder.

For a "library" project (i.e. with no executable .ino file)...
2.1) Rename the "sloeber.ino.cpp" file as "sloeber.ino.cpp.xxx" (for later
possible use, as in 4).
2.2) Delete the "Release" directory.
2.3) Verify the project by clicking on the "verify" button in the toolbar. This
builds the "Release" directory, including the projects linked via import (as
in 1). The project will not fully compile since it is missing the sloeber file.
This prevents the project from being executed, but does not affect its use as a
library.

For an "executable" project (i.e. with an executable .ino file)...
3.1) Assure there is a "sloeber.ino.cpp" in the project, with the "#includes"
line referencing the executable .ino file. All but the arduino.h include file
references are not needed.
3.2) Delete the "Release" directory.
3.3) Verify the project by clicking on the "verify" button in the toolbar. This
builds the "Release" directory, including any projects linked via import (as
in 1).
3.4) Upload the project by clicking on the "upload" button in the toolbar.
3.5) If the code is good but the upload fails the most likely problem is that
extra sloeber file .o and .d reference have crept into the imported library
projects. Rebuild all library projects as in 2).

For a library project that also contains an executable (i.e. used as a library
but also contains a .ino file, such as for testing)...
4.2) Treat it as in 3), an "executable" project.
4.3) The resulting "Release" directory will subsequently contain sloeber
file .o and .d references. This will spoil its use as a library.
4.4) Before using the project as a library treat it as in 2), a "library"
project. Failure to do so will cause sloeber file references to be included via
the "Release" directory, which will be pulled into the parent projects.

[jantje]

I believe in your solution the edited source files must be explicitly copied to the external library directory before rebuild,

Your believe is wrong.

which is outside of the workspace and therefore not managed by and Eclipse.

It does not have to be outside of the workspace. It can be; but I would advice against it as you may want to trigger version control from inside sloeber.

"Referenced Projects" tab to specify dependent projects. This does NOT work with the Sloeber plugin.

Sloeber does not do anything with "dependent projects" so Sloeber behaviour should be standard CDT behaviour.
IMHO a referenced project in CDT means that it will be build before your project when the referenced project is "dirty".
As there is no support for "archive libraries" in the Arduino framework (I'm working on adding it to Sloeber right now but it is very work intensive) and library source code is always build with the sketch there is no use to build the library in the other project and as such the referenced projects is useless in Arduino framework context.

Sloeber supports one and only one executable sketch per project.

One executable sketch per configuration.

Upon a clean build the plugin generates a "sloeber.ino.cpp" file

Sloeber only creates a sloeber.ino.cpp when your project contains an ino file. This has nothing to do with libraries.

Note that even if a project is intended as a "library", without an .ino executable, the plugin automatically generates a sloeber file anyway.

Sloeber does not support a "library project" Library projects should be standard eclipse projects.
When there is no ino file in the project (and there does not need to be one) there is no sloeber.ino.cpp file created.

You seem to have many views very different from me. You really should try the way as it has been designed.

[jonbarril]

Well, if you search the forum I first posted about this back in 2016. What I wanted wasn't supported by Sloeber then and you effectively confirmed that. I am now picking up 6 years later. Perhaps the trouble is in our use of the term "library". I am not talking about a formal C/C++ library. I am talking about a dependent project containing source code that gets included in the parent executable project's build.

Perhaps I am misunderstanding what you are recommending but for what I want to do, in 2016 as well as now, what I described seems to work just fine for me with minimal problems as long as I stick to the procedure I described.

BTW: I confirmed what you said that, without an .ino file, no sloeber file is generated. Thanks for the correction. Also, for projects containing multiple .ino files (e.g. unit tests), I simply edit the "#includes" reference in the one sloeber file.

Not sure if there is anything more to discuss. Thanks again.
--jon

[jantje]

I fully understand the Libraries confusion problem.
I had written a whole epistle about "binary libraries" and rereading your post I see you talk about a library type I had not yet identified. (I kept the epistle at the end of this post)

I am not talking about a formal C/C++ library. I am talking about a dependent project containing source code that gets included in the parent executable project's build.

I though that that the definition of a Arduino library is "source code from somewhere else that gets included in the project's build". I think your definition matches mine.
In Sloeber you can add a private library. A private library is a folder on your local system that contains source code (at a minimum a header file) which sloeber can add to your code based on the inclusion of the header (if header name=folder name -.h). As an example see this Arduino Lbrary https://github.com/jantje/ArduinoLibraries/tree/master/FastRunningMedian

I'm serious, I really do not see the difference between your library and an Arduino library.

What I wanted wasn't supported by Sloeber then and you effectively confirmed that.

I guess I was thinking about binary libraries then. But he; 6 years ago ... I don't remember and Sloeber came a long way since then. Thanks for being a long term user.

The original epistle is below

For me in a Sloeber/arduino context libraries are Arduino libraries and are source code level sharing. Simply because I'm not aware of any binary library support in the Arduino framework.

I currently do not see a good way to use "binary" libraries" in sloeber. Your way described above uses source code sharing. Look at your point 1.2
You use imports just like Sloeber does (import as a link). Sloeber also maintains include path and excludes example folders and so on.
So basically your way of working does exactly the same as what I referenced to, except for .. it is way more complicated.

If you really want to use binary libraries there is something new.
issue #1498 (is only available in the nighty) makes you can have multiple archives in the sloeber build.
It is still very and I know it won't work with all boards in all cases.
The major problem is the inclusion in the arduino framework provided recipe. The Sloeber/CDT make file generator does not allow to put the input files at different locations and some boards need the archives to be on a different spot than the objects.

#1498 (if compatible with the board/use case you are using) allows for including "binary libraries".
However Sloeber does not create archives from libraries so I do not know a good way to create the "binary library".
That is why I'm working on #1126.

in short
Without #1498 I would not know how to use binary libraries in a Sloeber project. (Well I do but not without some serious Sloeber knowledge)
Without #1126 I would not know how to create a binary library with Sloeber. (You can in CDT but then you do not need Sloeber)

With #1126 and #1498 and "CDT project dependencies" and "standard eclipse linking capabilities" it should be possible to use "binary libraries" in Sloeber
Still a long way to go though

And a final note on this. When using this new functionality you will have to take into account that the binary library is compiled with a toolchain for a specific board with a specific version. I don't know a easy way to pick the "right for your board binary library"

[jonbarril]

I am in no way advocating or attempting to use binary libraries -- way beyond my pay grade.

As for "importing" dependent projects, yes, what you mention is the same as what I am doing. To be clear, however, your writeup specifically mentioned the library directory being outside of the workspace. Mine are all inside the workspace, i.e. other projects.

As for the "complication" of my approach it may just look that way because I was trying to be thorough and helpful, especially for a newbie to Sloeber. Besides the project importing, the only real complication is in projects that also include executables, such as unit tests. In this case you have to make sure you have the proper sloeber file when using the project as an executable, and no sloeber references (.o, .d iin the Release directory) when using the project as a library -- the root executable must only have one and only one sloeber reference. If a project is just a library or just an executable then things are a lot easier.

Hmmm. It would be great if there were a way to designate a project as being an executable or a library project. The former would get a sloeber file, and the latter would not, even if the project contains .ino files (i.e. unit tests and such).

Also, I am able to include multiple .ino files in a given project, such as for unit testing, by editing the sloeber file accordingly. (I noticed that if I have multiple .ino files Sloeber will include all of them automatically, and I have to manually edit it to include only one .ino reference and one set of setup()/loop() calls. I also generally delete all includes except arduino.h). It would be great if, in the case of multiple .ino files in a project, Sloeber would ask which one will be the uploadable executable, if any (i.e. if used as a library project then it should have no executable and no sloeber references).

Thanks
--jon

[jantje]

however, your write up specifically mentioned the library directory being outside of the workspace.

I guess you are referring to

IMPORTANT Your library code is no longer in your workspace folder on disk!!

The reason I write is this way is because most people assume that when they copy a eclipse project they have all the source code. (That was how eclipse used to be and that was my assumpion and I was bitten by this so I want to warn people)
However they do not know Sloeber uses links to cores and libraries. As such the files are not in the in a subfolder of the eclipse project.

As in this example I wrote:

Outside of eclipse: create a subfolder with the library name in one of your private library folders
As I have C:\Users\jan\Documents\Arduino\libraries listed as a private Library; I created a folder in C:\Users\jan\Documents\Arduino\libraries with the name Libcodeclass. Add a file with extension .h in the folder (I used findMe.h)

The source code is in the C:\Users\jan\Documents\Arduino\libraries folder which is not in your project.
If you would have used a folder that is in a project in the workspace you currently use the code would be in your workspace.
In my workspaces I have a github project imported https://github.com/jantje/ArduinoLibraries that contains all my personal libraries. The

Outside of eclipse: create a subfolder with the library name

Then becomes in the project ArduinoLibraries in your workspace create a subfolder with the library name
Note that: as this is a github project the statement

IMPORTANT Your library code is no longer in your workspace folder on disk!!

Is still true as the code is in the git folder which is unlikely to be in the current workspace.

In other words: The source code is not in the workspace because in this example I specifically opted to not do so. This is not a limitation of the functionality but a consequence of the decisions made in this example.

I'm curious on how you do the unit tests. Are you using my hardware repository?

In this case you have to make sure you have the proper sloeber file when using the project as an executable, and no sloeber references (.o, .d iin the Release directory) when using the project as a library -- the root executable must only have one and only one sloeber reference.

I have reread this a couple of times but I can't make sense of it. Could we have a video chat so I have a better understanding?

Hmmm. It would be great if there were a way to designate a project as being an executable or a library project. The former would get a sloeber file, and the latter would not, even if the project contains .ino files (i.e. unit tests and such).

Please understand that Sloeber does not need .ino files. I started supporting .ino files when I started to add code to support library development. Because if you want to create examples you need .ino files.
The sloeber.ino.cpp file is only created when you have one or more .ino files and is a generated file you should not touch.
I currently assume "project as being an executable" is a sloeber project (not sure how your unit tests fit in here) so the designated project is a sloeber project.
IMHO "a library project" is just a bunch of code. I do not see any actions (apart from standard project actions such as version control actions) one would want to do on such a project that would require a specific project. Therefore I think a basic eclipse project is all that is needed.
As only a sloeber project converts .ino files in a sloeber.ino.cpp file at build time that meets your requirement "The former would get a sloeber file, and the latter would not,"
I'm still unsure how you fit in the unit tests and libraries and I guess that is causing the problems.
But I think a conf call or a video showing how you do things will explain more than any text can do.

[jantje]

It would be great if, in the case of multiple .ino files in a project, Sloeber would ask which one will be the uploadable executable,

The arduino framework specifies that when multiple .ino files are in the project all the ino files need to be processed with the .ino file with the same name as the folder first.
So that is the behaviour sloeber has ad will keep.
If you want to exclude .ino files from the build: have you tried "exclude from build"? and if multiple setups are needed: are you aware of configurations?

[jonbarril]

Sorry I haven't gotten back to you sooner. I've been sick this week. Hopefully I'll be better later this week and can catch up and schedule a call with you. Thanks so much for your offer.

--jon

[jonbarril]

I thought I had posted earlier. Oh well.....

I have some time in the next week or so if you want to pursue this issue I raised last year.

--jon

[jantje]

Can this be closed?

[jonbarril]

I've since moved on to other things and haven't done Arduino in quite a while. I may need to revisit old projects in the future. The ability to have sloeber projects reference other projects for whatever reason (libraries and/or executables), the same as other eclipse projects, I feel, is a good feature to have. However, if you would like to close it I understand.

[jantje]

The ability to have sloeber projects reference other projects for whatever reason (libraries and/or executables), the same as other eclipse projects, I feel, is a good feature to have.

I agree and that feature exists in several ways.
If you revisit Arduino and have issues please open a new issue.
Hope to see you.