Include application plugin in update site

[wimjongman]

The application plugin should be included in the update site. These are the advantages.

• we don't have to move the perspective
• people can run with -perspective id_of_perspective
• people can run with -application id_of_application

[wimjongman]

I will also add the update sites to the product.

• http://www.baeyens.it/eclipse/nightly

[jantje]

I don't want to disagree but I'd like to know what all the projects (to many already to my taste) are for.
At least to know what should go where.
So there is:
Eclipse update site: which is the content of the web site
it.baeyens.arduino.application: which is for... wel I don't know
it.baeyens.arduino.common: which is for very basic things and can be used by all plugins
it.baeyens.arduino.core: which is the core functionality of the arduino eclipse plugin.
it.baeyens.arduino.core.nl1: which has been made by the internationalisation wizard. (my bad)
it.baeyens.arduino.feature: defines the feature of the arduino eclipse plugin
it.baeyens.arduino.monitor: is the serial port monitor and the scope (bad name now :-()
it.baeyens.arduino.packaging.feature: I assume is the feature of the product
it.baeyens.arduino.parent: I don't know
it.baeyens.arduino.product: defines the product
it.baeyens.arduino.updatesite: which is defining the update site for the plugin version

[jantje]

I will also add the update sites to the product.
I'm not sure what you are saying. Is this so the product can be updated via the nightly update site?
I tried that and that didn't work. May be because it needed what you are planning to do?

[wimjongman]

to many already to my taste
We don't have many projects.

it.baeyens.arduino.application

Contains the initial startup code of the application. Shows the splash screen, creates the shell, the toobar etcetera.

it.baeyens.arduino.common

Which is for very basic things and can be used by all plugins. Should be i.b.arduino.core

it.baeyens.arduino.core

Is the core functionality of the arduino eclipse plugin. Should be called i.b.arduino.ui

it.baeyens.arduino.core.nl1

Which has been made by the internationalisation wizard. Which is good. At some point people will start translating the product and then it can go here.

it.baeyens.arduino.feature

Used to group multiple plug-ins. Defines the feature of the arduino eclipse plugin

it.baeyens.arduino.monitor

Is the serial port monitor and the scope.

it.baeyens.arduino.packaging.feature

Used for the: I assume is the feature of the product

it.baeyens.arduino.parent

A required artifact for the build. A maven thing.

it.baeyens.arduino.product

defines the product

it.baeyens.arduino.updatesite

which is defining the update site for the plugin version

[wimjongman]

I'm not sure what you are saying. Is this so the product can be updated via the nightly update site?

yes

[jantje]

Why do you say "Should be called i.b.arduino.ui"
I'm not fully agreeing common should be in core (or ui). This because the monitor does not use the core (or ui). In principle the monitor and the scope have nothing to do with the arduino stuff as they work on serial connections whatever source the connection makes.

[wimjongman]

What we normally do in Eclipse plug-in development is to define a bundle that hooks into Eclipse for views etcetera. This is commonly the "ui" bundle because it is depending on a framework.

Then there is a "core" bundle that could depend on some eclipse bundles (e.g. e.c.runtime) but does not pull in the complete IDE as a dependency. This "core" bundle should therefore be independent of the "ui" used, in this case Eclipse.

As it looks now, the "common" bundle fills in this role of "core". The "common" bundle does what I described earlier for "core".

Having a separate "monitor" bundle is excellent. In theory this should be split in "monitor.core" for the generic stuff and "monitor.ui" for the hook into Eclipe.

Please note that all this is not required, it is just the common practice in Eclipse. It only starts paying off if:

• you indeed need the "core" bundles outside Eclipse and
• the project gets very big

[jantje]

I was more referring to the "i.b." than to core versus ui.

[jantje]

I'm aware the current project cut does not follow MVC model and that is a really good practice I have been advocating for decades.
As an Architect I only have one excuse: This is my first eclipse project (as developer and architect) and first java project (as developer) and I have been focussing on getting things done, not on getting things done properly.
Now things work and other people are willing to share their opinion it may be the right time to do a re factoring to get the things more in line with "good practise".
My knowledge about eclipse -in this particular case the install/upgrade system- is to limited to get a view on the impact of these changes.
I expect the feature is there to handle "removed" and "new" plugins. But does that work fine?

[wimjongman]

I was more referring to the "i.b." than to core versus ui.

Ah! Shortcut for it.bayens ;)

[wimjongman]

As an Architect I only have one excuse:

"Premature optimization is the root of all evil (or at least most of it) in programming"
Donald Knuth

[jantje]

"Premature optimization is the root of all evil (or at least most of it) in programming"

I agree to that but I don't get the link.

[wimjongman]

Well, in my opinion it is like this. First make it work and then, if it is a success, re-engineer / refactor it to a final form.

[jantje]

ach you are referring to optimisation in "design". I mostly see optimisation in "speed".

[wimjongman]

Interesting that you look at it like that. Personally I never worry about program size, I hardly worry about speed but I always worry about design.

btw, did you re-spin the build?

[xaljox]

Normally if the design is „good’ the other things arent bad either.

Op 1 jul. 2014, om 13:52 heeft Wim Jongman [email protected] het volgende geschreven:

Interesting that you look at it like that. Personally I never worry about program size, I hardly worry about speed but I always worry about design.

btw, did you re-spin the build?

—
Reply to this email directly or view it on GitHub.

[jantje]

I probably answered to tearse again. I read "Premature optimization is the root of all evil (or at least most of it) in programming" as
"If you optimize your program for speed before you have speed issues you will get bad code."
When coding my first commandment is: make sure it is readable

I havn't respinned the build. I will after I chequed in a due issue also caused by #34

[xaljox]

On the other hand I have seen projects that were a complete disaster because they looked at speed too late. Al was wel functionally and then during tests with real world loads it game to a craw. Then it was really hard to get the speed in and to be honest they never did, it improved but still wasn’t great.
But this case was not really about bad design, but more about wrong tooling. It was a project with very large in core memory structures that was also very dynamic in behavior. Parts of the data in memory was replaced regularly with new information and during the day normally the whole set of data was altered once or more, but normally the same part not directly in succession.
Thus if you are familiar with how garbage collection works, you know that this is the worst case for it, all memory that can be regained with a garbage collection sweep is always second generation and thus can only be found with a full sweep.
When you allocate memory in a garbage collection environment the memory is labeled as level 0. During a sweep it investigates only memory of level 0, thus a limited set and what is no longer needed is marked as free and what is in use is marked as level 1. After around 10 rounds also level 1 memory is investigated and this is a more serious amount of work. What is in use after level 1 get marked as level 2 and after 10 rounds of level 1, also level 2 memory is investigated, this is a full sweep and is quite expensive.
Thus we were in a situation that the garbage collector can only find stuff al level 2, due to the usage update pattern of the updates.
Note that level 0 memory can more or less be compared with memory on the stack, since most often in a Garbage collector language there is no concept of stack variables own types can only be invoked with new. So after exiting a function, setting back the stack pointer will not regain the local variable, since they are not local, they are on the garbage collector heap. Those variables can easily be reclaimed with a level 0 sweep of the Garbage collector.
Further freed memory is not freed, they stay as gaps, thus memory allocation is very fast it is only putting foreword a pointer, no search in empty buckets etc. The down side is that once in a while the memory will be compacted. If the amount of memory that is free in the gaps is about 4 times the memory that is actually used, then the compaction kicks in. luckily this compaction process can be spread easily over multiple threads and thus cores, and the idea is that this happens at moments that the application is relatively quit.
But in our case if your memory footprint is say 1 or 2 GB byte, then compaction 5-10GB of memory is a serious thing from a performance point of view. Do not forget that data structures in a Garbage collector environment are bigger than in an environment without it.
Normally you will have a pointer of 4 or 8 bytes depending if 32 or 64 application. In a Garbage collector environment your pointer will be a 64bits unique object identifier that is translated via a hash table to the actual memory location. During compaction the memory is first copied to a new location and once its there the hash table entry is updated with the new location. Knowing this it is easy to understand why there is no issue to keep the program running while doing compaction.
Thus in core data structures are bigger in a Garbage collector environment depending on 32 or 64, the differences in memory needed for navigating between the structures is 2 to 4 times as much.
Thus in this case it would have been better to use another programming language to solve this particular problem. But by looking at performance to late and there was no way back due to time lines committed with the customer, they worked around it by making own free list of objects no longer used and try to use them first if memory needed etc. Thus essentially workaround Garbage collection, but in a much harder way as in an environment without it. So the argument that it is easier to work in those kinds environments went down the drain in this case.
But to be honest there were other projects were the garbage collector was no issue and were successful with the same tooling. Thus you have to be careful and know if you can expect problems and if you are in doubt try to do small experiments to get the uncertainties out. Normally the uncertainties are only in a small part and often you can model that part easily and do some experiments with it. With that knowledge if something is feasible and how it is feasible you go into the design fase. But with ignoring potential problems, hitting them in the end can be quite expensive.
As to show to colleagues how Garbage collection can impact performance I made a small program, it was just a spread sheet kind of thing, containing 4 classes, SpreadSheet, Row, Column and Cell. Because of that it was easily scale able, each Cell had a simple array, just to give it some memory footprint. In memory a Spreadsheet was instantiated of say 1000 x 1000. Next in a loop it goes over all rows and every 10th row was deleted, so 100 rows are deleted, after that 100 rows were added at the end. Next the same thing was done for the columns. The above deleting of rows and columns and creating them again was done in a loop of about 100000 iterations. Thus with very little code we had an example that shows what happens if semi random, a rather large part of in core memory structure is replaced with altered information. Although it contained only 4 classes and was not comparable with the real datamodel, that was a 100+ class thing, it could still mimic the memory usage pattern and show us the impact on performance of various approaches.
Originally the code (C++) was created with ClassBuilder a freeware code generation tool, I’m the original author of (can be found on SourceForge) it took me less then 15 minutes to make. Running this at a certain size of spreadsheet a certain amount of iterations took about 40 seconds. A colleague created an alternative C++ implementation using the STL library, that solution after some optimization, that took longer than my 15 minutes, took about 400 seconds to run. It shows effectively the difference between the normal link list kind approach with STL and the more efficient intrusive link list approach ClassBuilder is using. Next another colleague did the same exercise in .NET, he aborted the test after a few hours of run time. His solution was similar to the C++ STL solution, but now in a Garbage collector environment and a virtual machine, for executing some sort of intermediate code.
Al three designs are effectively identical, they use the same UML class diagram to describe them, just 4 classes and a few relations between them. The pseudo code of the actual actions performed is more or less also the same, but the performance is drastically different.
Note that this is not a crusade against Garbage collection, there are enough things can can be developed nicely with it, without any problem. But there are certain type of problems that are less suited for it, it is best to know those upfront instead of discovering it late and hard.
I’m using ClassBuilder also for my Arduino projects, it is very suited for it, because of its intrusive link list approach the memory footprint is as small as possible, plus since no seperate link list class is used like STL, static structures like menus for LCD can be invoked without the use of new/malloc and still be object oriented, with relations between objects that are easily navigateable. I only had to edit an include file, since it uses some sort of C++ construct with template classes that the compiler did not like. This had to do with some sort of Critical Section lock on an iterator, but since there is no multithreading on the arduino board I’m aware of, this was easily stripped and it all compiled and works nicely. Thus I’m using a smaller and edited set of include files as I’m normally using for code development, but I’m totally comfortable with a known tool and a good IDE for development.

Op 1 jul. 2014, om 14:15 heeft jantje [email protected] het volgende geschreven:

I probably answered to tearse again. I read "Premature optimization is the root of all evil (or at least most of it) in programming" as
"If you optimize your program for speed before you have speed issues you will get bad code."
When coding my first commandment is: make sure it is readable

I havn't respinned the build. I will after I chequed in a due issue also caused by #34

—
Reply to this email directly or view it on GitHub.

[jantje]

wow. This shows experience.

[jantje]

This issue Is fixed. If you disagree please reopen.