Preprocessor failure on various prototypes.

[Chris--A]

1. C++11 trailing return types.

void setup(){
  func();
}

void loop() {}

auto func() -> void{

}

This problem isn't really a big issue, a user can simply add a prototype or define above the code where it is called from (preprocessor ignores completely). Its the next problems which cause quite a disaster.

2. Returning a function pointer.

void setup(){
  func()();
}

void loop(){}

void (*func())(){
  return setup;
}

This causes a complete failure as the prototype is generated wrong. As the generated prototype differs only by return type, the function is ambiguous and the compiler cannot continue.

#line 1
#line 1 "C:\\Users\\Chris\\AppData\\Local\\Temp\\arduino_cdba69ab332f1f472a84ede0e3d7b376\\sketch_oct08c.ino"
void setup();
void loop();
void func();
#line 1
void setup(){
  func()();
}

void loop(){}

void (*func())(){
  return setup;
}

3. Returning an array reference.

int array[5];

void setup(){
  func();
}

void loop(){}

int (&func())[5]{
  return array;
}

Just like problem 2, the function prototype is incorrectly generated.

#include <Arduino.h>
#line 1
#line 1 "C:\\Users\\Chris\\AppData\\Local\\Temp\\arduino_55ab6e1d0319f8e5a301e3286e28d726\\sketch_oct08d.ino"

int array[5];

void setup();
void loop();
int func();
#line 4
void setup(){
  func();
}

void loop(){}

int (&func())[5]{
  return array;
}

4. Combining 2 & 3

Here is a function which accepts and returns: A reference to an array of function pointers which take and int as a parameter and return nothing.

Bare with me, I know the example below is a little abnormal, its the preprocessor that is interesting.

void setup() {
  void (*arr[5])(int);
  func(arr);
}

void loop() {}

void (*(&func(void (*(&in)[5])(int)))[5])(int){
  return in;
}

And the resulting cpp:

#include <Arduino.h>
#line 1
#line 1 "C:\\Users\\Chris\\AppData\\Local\\Temp\\arduino_b86f1f9b94d497bb6d338315715b6c94\\sketch_oct08a.ino"
void setup();
void loop();
void func(void (*(&in)[5])(int));
#line 1
void setup() {
  void (*arr[5])(int);
  func(arr);
}

void loop() {}

void (*(&func(void (*(&in)[5])(int)))[5])(int){
  return in;
}

As you can see, the preprocessor has correctly copied the functions parameter, but completely removed the return type (apart from the void).

---

If you need help, point me in the right direction and I can 'go' have a bit of an investigation.
Or is this a problem within the ctags repo?

I'm just having fun breaking things, I'm sure I'll have a few more additions to my list in the future.

[ffissore]

Thank you! I'll take a look at it in the next weeks. Can you tell if adding the prototype yourself works around the issues?

[ffissore]

Also, what would be the correct prototypes?

[Chris--A]

Yes adding a prototype does prevent the incorrect generation. But without it, the errors are very confusing if you aren't expecting it.

Here is an example of the errors for 2 (simple function returning a function pointer)

sketch_oct08c:3: error: void value not ignored as it ought to be

   func()();

        ^

C:\Users\Chris\AppData\Local\Temp\arduino_cdba69ab332f1f472a84ede0e3d7b376\sketch_oct08c.ino: In function 'void (* func())()':

sketch_oct08c:8: error: new declaration 'void (* func())()'

 void (*func())(){

                ^

sketch_oct08c:4: error: ambiguates old declaration 'void func()'

 }

      ^

exit status 1
void value not ignored as it ought to be

[Chris--A]

The prototypes are an exact copy, the functions are correct as written, just without the { & }

[matthijskooijman]

Yes adding a prototype does prevent the incorrect generation. But without it, the errors are very confusing if you aren't expecting it.

I suspect that this is mostly because the line numbers are completely screwed up? I previously discussed an approach with Federico to generate a #line directive for every generated prototype, pointing to the original function definition, which should at least improve this.

[ffissore]

These are the outputs of ctags for 2, 3 and 4 https://gist.github.com/ffissore/625f6743e8f643311ad8
While it correctly identifies the function declaration, like in /^void (*func())(){$/;", it also tell that the function name is func. I use the function name for composing the prototype.
How much safe it is to say something like: if function declaration contains more than 1 nested parenthesis, use function declaration as prototype?

[ffissore]

note: func is not correct, it should be (*func())

[matthijskooijman]

it also tell that the function name is func

That is ok, the function name is func. However, the return type is wrong - it's not void. And even if the return type was correctly parsed by ctags, the current "return type" + "name" + "signature" way of generating a prototype is not sufficient for the complicated types shown here.

How much safe it is to say something like: if function declaration contains more than 1 nested parenthesis, use function declaration as prototype?

I would not go there - AFAICS the function "declaration" is just the entire line, which might even contain a full function body (like loop(){} in these examples) or other stuff. At best, I would try to detect if a function is too complex for generating a prototype and then skip it.

One generic way would be to see if the original declaration matches the generated prototype. If so, you're probably good, if not, skip generating the prototype?

[ffissore]

I would not go there - AFAICS the function "declaration" is just the
entire line, which might even contain a full function body (like
|loop(){}| in these examples) or other stuff.

yes, indeed. I think it's best when limitations are known and an easy
workaround is available. like "don't put your function body on the same
line" or "provide the prototype yourself"

One generic way would be to see if the original declaration matches the
generated prototype. If so, you're probably good, if not, skip
generating the prototype?

if the prototype ALWAYS equal to its related function?

[matthijskooijman]

if the prototype ALWAYS equal to its related function?

Note sure what you mean there?

What I mean is that for example, for:

setup ctags_target.cpp /^void setup(){$/;" kind:function line:2 signature:() returntype:void

the generated prototype (based on returntype, name and signature) is void setup();. If you take the actual function definition line (void setup(){) up to the first { (if any) and that matches (equals) the generated prototype, the prototype is likely ok and it should be output.

Looking at:

func ctags_target.cpp /^void (*func())(){$/;" kind:function line:8 signature:() returntype:void

the generated prototype is void func(), which does not match the actual line void (*func())(){, so no prototype should be output.

[ffissore]

This is what I came up with: https://github.com/arduino/arduino-builder/commits/functionpointer

[Chris--A]

Also, this next one relates to #27 / arduino/ctags#1

The space is removed between typename and the nested name specifier.

I have added a PR here: arduino/ctags#3

template< typename T >
  struct Foo{
    typedef T Bar;
};

void setup() {
  func();
}

void loop() {}

typename Foo<char>::Bar func(){

}

[Chris--A]

However the issue above is still not solved completely. The space should be fixed by the PR, however the template parameters are removed.

#include <Arduino.h>
#line 1
#line 1 "C:\\Users\\Chris\\AppData\\Local\\Temp\\arduino_38f0e5c3a68a41fc40766eb5fba8fc90\\sketch_oct08a.ino"

template< typename T >
  struct Foo{
    typedef T Bar;
};

void setup();
void loop();
typenameFoo::Bar func();
#line 7
void setup() {
  func();
}

void loop() {}

typename Foo<char>::Bar func(){

}

[matthijskooijman]

I found another case that is still broken with the functionpointer branch:

static void function () {}

Which generates:

sketch_oct30a:6: error: 'void function()' was declared 'extern' and later 'static' [-fpermissive]

Because the new code does a "substring" test, the prototype (void function ()) is deemed ok (it's a substring of the code line), and the (erronous) prototype is generated. I'll have a look at fixing this in the way I think it should be in a minute.

[matthijskooijman]

I added a commit to https://github.com/arduino/arduino-builder/commits/functionpointer that changes the matching to prefix matching instead of substring matching, which fixes the static function prototype. I tried doing full exact matching, by taking the code line up to the first {, if any, but that failed when a function definition was followed by a comment before the {. To prevent having to parse comments as well, I switched to prefix matching instead. Since, AFAICS, only function attribute can appear after a function's argument list (and it's not needed to specify those on both the definition and the prototype), I don't think this will generated any broken prototypes.

[ffissore]

Let's recap. @Chris--A all your cases are now handled and no wrong prototype is added. The IDE won't cast its magic on these sketches, not on all function at least, so the only way to compile them is to add prototypes yourself
As for static, it's been solved
Thank you all for your time and patience

[Chris--A]

Nice, will pull & rebuild.