Maven build fails on the nightly testing (basically build fails) #1204

jantje · 2020-07-10T18:38:09Z

Since accepting pull request #1201 the nightly testing fails in maven.
When run from inside eclipse the testing is successful.
The maven fails on my windows system, on travis https://travis-ci.org/github/Sloeber/arduino-eclipse-plugin and on my jenkins build server (linux).
Looking -on my windows system- at the log of the workspace C:\Users\jan\git\arduino-eclipse-plugin\io.sloeber.tests\target\work\data.metadata.log (attached to this issue)
I see following errors

!ENTRY org.eclipse.cdt.core.native 4 0 2020-07-10 19:11:52.695
!MESSAGE no spawner in java.library.path

!ENTRY org.eclipse.cdt.core 1 0 2020-07-10 19:11:53.774
!MESSAGE Indexed '00001_Library_RTCZero_Epoch' (0 sources, 0 headers) in 0,032 sec: 0 declarations; 0 references; 0 unresolved inclusions; 0 syntax errors; 0 unresolved names (0%)

!ENTRY org.eclipse.osgi 4 0 2020-07-10 19:11:57.083
!MESSAGE An error occurred while automatically activating bundle org.eclipse.cdt.ui (23).
!STACK 0
org.osgi.framework.BundleException: Exception in org.eclipse.cdt.ui.CUIPlugin.start() of bundle org.eclipse.cdt.ui.
	at org.eclipse.osgi.internal.framework.BundleContextImpl.startActivator(BundleContextImpl.java:863)
....
Caused by: java.lang.IllegalStateException: Workbench has not been created yet.
	at org.eclipse.ui.PlatformUI.getWorkbench(PlatformUI.java:95)
	at org.eclipse.cdt.internal.ui.viewsupport.CDTContextActivator.install(CDTContextActivator.java:65)
	at org.eclipse.cdt.ui.CUIPlugin.start(CUIPlugin.java:596)
	at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:842)
	at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:1)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.eclipse.osgi.internal.framework.BundleContextImpl.startActivator(BundleContextImpl.java:834)
	... 108 more
Root exception:
java.lang.IllegalStateException: Workbench has not been created yet.
	at org.eclipse.ui.PlatformUI.getWorkbench(PlatformUI.java:95)

and then later

!ENTRY org.eclipse.equinox.registry 4 1 2020-07-10 19:11:57.088
!MESSAGE Plug-in org.eclipse.cdt.ui was unable to load class org.eclipse.cdt.internal.ui.buildconsole.CBuildConsole.
!STACK 0
java.lang.ClassNotFoundException: An error occurred while automatically activating bundle org.eclipse.cdt.ui (23).
	at org.eclipse.osgi.internal.hooks.EclipseLazyStarter.postFindLocalClass(EclipseLazyStarter.java:126)

So it looks as if org.eclipse.cdt.ui can not be loaded because org.eclipse.cdt.internal.ui.buildconsole.CBuildConsole can not be loaded.
org.eclipse.cdt.internal.ui.buildconsole.CBuildConsole is part of the package org.eclipse.cdt.internal.ui.buildconsole of the plugin org.eclipse.cdt.ui
org.eclipse.cdt.ui has always been a required plugin for io.sloeber.core. I added org.eclipse.cdt.internal.ui.buildconsole to the imported packages but to no avail.
I fail to find clues in the content of the pull request.
I have no clue how to further investigate/progress this issue

.log

The text was updated successfully, but these errors were encountered:

jantje · 2020-08-01T10:07:19Z

@wimjongman I still havn't found out what is the problem. Can you take a look please :-)

wimjongman · 2020-08-06T15:05:22Z

Sure, let me take a look.

wimjongman · 2020-08-06T15:44:36Z

The build is fine but the test is failing. Only issue555 test is success.

Tests run: 6, Failures: 5, Errors: 0, Skipped: 0, Time elapsed: 214.246 s <<< FAILURE! - in io.sloeber.core.RegressionTest

is_extern_C_taken_into_account(io.sloeber.core.RegressionTest) Time elapsed: 39.945 s <<< FAILURE!
java.lang.AssertionError: Failed to compile the project:defines_and_includes defines have not been taken into account properly.
at io.sloeber.core.RegressionTest.is_extern_C_taken_into_account(RegressionTest.java:287)

are_defines_before_includes_taken_into_account(io.sloeber.core.RegressionTest) Time elapsed: 4.65 s <<< FAILURE!
java.lang.AssertionError: Failed to compile the project:externc extern "C" has not been taken into account properly.
at io.sloeber.core.RegressionTest.are_defines_before_includes_taken_into_account(RegressionTest.java:251)

issue1047_Board_Names_Can_Be_used_as_Strings(io.sloeber.core.RegressionTest) Time elapsed: 32.339 s <<< FAILURE!
java.lang.AssertionError: Failed to compile the project:issue1047_Board_Names_Can_Be_used_as_Strings issue1047 is not fixed
at io.sloeber.core.RegressionTest.issue1047_Board_Names_Can_Be_used_as_Strings(RegressionTest.java:169)

issue555(io.sloeber.core.RegressionTest) Time elapsed: 0.001 s

issue687(io.sloeber.core.RegressionTest) Time elapsed: 3.771 s <<< FAILURE!
java.lang.AssertionError: Failed to compile the project:issue687 issue687 is not fixed
at io.sloeber.core.RegressionTest.issue687(RegressionTest.java:141)

are_[secure]s_options_taken_into_account(io.sloeber.core.RegressionTest) Time elapsed: 4.94 s <<< FAILURE!
java.lang.AssertionError: Failed to compile the project:are_defines_found The defines have not been taken into account properly
at io.sloeber.core.RegressionTest.are_[secure]s_options_taken_into_account(RegressionTest.java:215)

Results:
Failures:
RegressionTest.are_defines_before_includes_taken_into_account:251 Failed to compile the project:externc extern "C" has not been taken into account properly.
RegressionTest.are_[secure]s_options_taken_into_account:215 Failed to compile the project:are_defines_found The defines have not been taken into account properly
RegressionTest.is_extern_C_taken_into_account:287 Failed to compile the project:defines_and_includes defines have not been taken into account properly.
RegressionTest.issue1047_Board_Names_Can_Be_used_as_Strings:169 Failed to compile the project:issue1047_Board_Names_Can_Be_used_as_Strings issue1047 is not fixed
RegressionTest.issue687:141 Failed to compile the project:issue687 issue687 is not fixed
Tests run: 6, Failures: 5, Errors: 0, Skipped: 0

jantje · 2020-08-06T17:14:52Z

Thanks @wimjongman
Issue 555 is the only one that does not fail as it is skipped because teensy is not installable via json file.
In other words issue555 is always skipped on travis

jantje · 2020-08-11T12:42:28Z

@wimjongman Any ideas what could be the root cause?

jantje · 2020-08-18T21:35:33Z

I had the idea to create a class that imports all the packages (needed to turn off organize imports) that had been removed and now the build fires :-)

package io.sloeber.core.tools;

import org.eclipse.cdt.core.language.settings.providers.ILanguageSettingsProvider;
import org.eclipse.cdt.core.language.settings.providers.ILanguageSettingsProvidersKeeper;
import org.eclipse.cdt.core.language.settings.providers.LanguageSettingsManager;
import org.eclipse.cdt.core.language.settings.providers.ScannerDiscoveryLegacySupport;
import org.eclipse.cdt.core.model.CoreModel;
import org.eclipse.cdt.core.settings.model.CSourceEntry;
import org.eclipse.cdt.core.settings.model.ICConfigurationDescription;
import org.eclipse.cdt.core.settings.model.ICExclusionPatternPathEntry;
import org.eclipse.cdt.core.settings.model.ICProjectDescription;
import org.eclipse.cdt.core.settings.model.ICProjectDescriptionManager;
import org.eclipse.cdt.core.settings.model.ICResourceDescription;
import org.eclipse.cdt.core.settings.model.ICSettingEntry;
import org.eclipse.cdt.core.settings.model.ICSourceEntry;
import org.eclipse.cdt.core.settings.model.extension.CConfigurationData;
import org.eclipse.cdt.managedbuilder.core.IBuilder;
import org.eclipse.cdt.managedbuilder.core.IConfiguration;
import org.eclipse.cdt.managedbuilder.core.IToolChain;
import org.eclipse.cdt.managedbuilder.core.ManagedBuildManager;
import org.eclipse.cdt.managedbuilder.internal.core.Configuration;
import org.eclipse.cdt.managedbuilder.internal.core.ManagedBuildInfo;
import org.eclipse.cdt.managedbuilder.internal.core.ManagedProject;



@SuppressWarnings({ "restriction", "unused" })
public class FixTestIssue {
 public static void useStuff(	
 org.eclipse.cdt.core.language.settings.providers.ILanguageSettingsProvider t1 ,

	 org.eclipse.cdt.core.language.settings.providers.ILanguageSettingsProvidersKeeper t2,
	 org.eclipse.cdt.core.language.settings.providers.LanguageSettingsManager t3,
	 org.eclipse.cdt.core.language.settings.providers.ScannerDiscoveryLegacySupport t4,

	 org.eclipse.cdt.core.model.CoreModel t5,
	 org.eclipse.cdt.core.settings.model.CSourceEntry  t6,
	 org.eclipse.cdt.core.settings.model.ICConfigurationDescription t7,
	 org.eclipse.cdt.core.settings.model.ICExclusionPatternPathEntry t8,
	 org.eclipse.cdt.core.settings.model.ICProjectDescription t9,
	 org.eclipse.cdt.core.settings.model.ICProjectDescriptionManager t10,
	 org.eclipse.cdt.core.settings.model.ICResourceDescription t11,
	 org.eclipse.cdt.core.settings.model.ICSettingEntry t12,
	 org.eclipse.cdt.core.settings.model.ICSourceEntry t13,
	 org.eclipse.cdt.core.settings.model.extension.CConfigurationData t14,
	 org.eclipse.cdt.managedbuilder.core.IBuilder t15,
	 org.eclipse.cdt.managedbuilder.core.IConfiguration t16,
	 org.eclipse.cdt.managedbuilder.core.IToolChain t17,
	 org.eclipse.cdt.managedbuilder.core.ManagedBuildManager t18,

	org.eclipse.cdt.managedbuilder.internal.core.Configuration t19,
	 org.eclipse.cdt.managedbuilder.internal.core.ManagedBuildInfo t20,
	 org.eclipse.cdt.managedbuilder.internal.core.ManagedProject t21)
	 {;}
}

jantje · 2020-08-18T23:23:52Z

I still have a problem that the indexer kicks in to soon and doesn't kick in anymore after this.
That way the projects needing libraries do not build as the libraries are not attached

jantje · 2020-08-19T14:16:29Z

I moved the registration of the listeners forwards and ow it seems the library is added but the indexer (or is it the build) crashes again due to missing CBuildConsole

I'm doing this because it seems like the listeners are not triggered when running in maven. Now they are but the indexer seems to fail on the second run

wimjongman · 2020-09-11T16:43:38Z

I have updated the pom to include the latest eclipse version. Also updated Tycho to 2.0.0. Everything builds fine on my machine. Travis is now building.

jantje · 2020-09-11T21:01:59Z

@wimjongman
Thanks for persisting.
I'm sorry but I do not understand your last comment.
It looks like it is fixed but travis states in #1213 that the build failed
I tried to switch to this branch (is a pull request a branch hummm....) and that failed.
I'll try to do the changes you did locally and see what that gives

FYI I have been doing contributions to CDT10 and I want to make a new release "soon" So I want to move to CDT10 and JAVA 11
CDT10 is not out yet but ...
The CDT p2 repo is https://download.eclipse.org/tools/cdt/builds/10.0/cdt-10.0.0-rc2
You know the ropes

Thanks again

jantje · 2020-09-12T01:14:31Z

I succeeded the build to be with Tycho 2.0, java 11 (source and target) and CDT 10
But I still have the same problem.
The maven build fails due to test failing (on windows) and the eclipse run of the jenkins test is ok
IMHO the test fails because the build fails (due to a missing arduino library) which leads to a null because "java.lang.IllegalStateException: Workbench has not been created yet."
I guess that if the build would not fail the build would not require the Workbench.

Because the indexer hits in to early (or any other reason) it can be that the first build fails. Then the indexer should find "unresolved headers" resulting in the Sloeber listener to add arduino libraries that satisfy the headers. Hopefully the next build done in the test is successful (max 3 tries now).
Apparently in maven the Sloeber index listener never kicks in resulting in missing Arduino libraries.

wimjongman · 2020-09-12T15:07:28Z

I never see this error "Workbench has not been created yet". Where do you see this?

In what bundle is the index listener?

wimjongman · 2020-09-12T15:10:35Z

I tried to switch to this branch (is a pull request a branch hummm....) and that failed.

The branch name is #1204

git checkout --track origin/#1204

jantje · 2020-09-12T15:20:17Z

I never see this error "Workbench has not been created yet". Where do you see this?

I see it in ...\git\arduino-eclipse-plugin\io.sloeber.tests\target\work\data.metadata.log

In what bundle is the index listener?

The listener gets registered in

arduino-eclipse-plugin/io.sloeber.core/src/io/sloeber/core/Activator.java

Line 253 in 584cd0d

registerListeners();

jantje · 2020-09-12T15:38:30Z

I tried to switch to this branch (is a pull request a branch hummm....) and that failed.

The branch name is #1204

git checkout --track origin/#1204

That is what I did. I guess I fell in a java1.! 11 mess as I saw the same things when I copied the changes which I successfully handled in my master.

wimjongman · 2020-09-12T18:08:05Z

That is what I did

I guess you have to pull first:
git pull --all
then
git branch -a

wimjongman · 2020-09-12T19:09:36Z

This error "workbench is not created yet" indicates that UI components are used. Specifically PlatformUI.getWorkbench(). UI components are not allowed in a headless JUnit test. The core Activator is an AbstractUIPlugin. It can only be loaded after the UI was created.

When I run the build on my machine, it ends without issues. I cannot get to the Travis log file because there is no filesystem in Travis. Do you know how to access it?

jantje · 2020-09-12T19:30:12Z

I get the "workbench is not created yet" in my windows maven build

wimjongman · 2020-09-13T21:19:39Z

I get the "workbench is not created yet" in my windows maven build

Please attach that log here.

jantje · 2020-09-14T10:55:32Z

There is a .log in the original post
Here is my .log though I have changed some stuff in my repository as I did your changes and moved to java11 and CDT 10
.log

wimjongman · 2020-09-14T12:22:57Z

I have filed a CDT issue:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=566953

However, we are not out of the woods here. Despite its name, io.sloeber.core is importing ui bundles. This is not correct. All classes that reference the UI should go to io.sloeber.ui.

Let's see what the CDT folks have to say and then we continue from there.

wimjongman · 2020-09-14T12:27:17Z

Where is that "Shared" class. Is this new stuff?

jantje · 2020-09-14T15:36:22Z

Where is that "Shared" class. Is this new stuff?

You mean this? https://github.com/Sloeber/arduino-eclipse-plugin/blob/master/io.sloeber.tests/src/io/sloeber/core/Shared.java

jantje · 2020-09-14T15:48:58Z

With the change as Jonah proposed I no longer have errors in the log file
I know the failed test are due to a missing library in the sketch/project
This means that the issue must be in
-The indexer not firing
-The indexer not finding the "unresolved header"
-The listener not firing (There are actually 2 listeners but they are in the same class)
-The listener not finding the library based on the "unresolved header"
-The listener not adding the library

jantje · 2020-09-25T16:46:49Z

I have removed the failing test from the nightly test.
I want to check in all the changes so we can work on the same code.
that will be java11, CDT10 and case sensitive in windows10 :-)
I'm rebased the windows case sensitivity and I'm currently doing the really long running tests.
The tests failed on linux because of a casing problem in the Release folder
I propose to put this one on wait untill things a stable again

jantje · 2020-09-27T14:16:19Z

Al seems fine now in git branch master (didn't test linux yet) but ....
I fail to get jenkins to work. (which means the update site is not updated)

In a nutshell:
Seems tycho 2.0 needs a recent maven and java 11
And Jenkins doesn't like to run in java11 :-( (fails on startup)
As a Jenkins maven project I fail to tell Jenkins to start maven with java 11 (setting JAVA_HOME should do the trick, but...)
As a standard jenkins project I Can start maven with java 11 and then ... I don't have make, so the tests fail

How come I didn't have the make issue with a jenkins maven project?
I'm going crazy here 😵

jantje · 2020-09-27T14:20:16Z

Before anyone tells me to install make ....
I can' do
sudo apt install make
as I'm running on a qnap nas

wimjongman · 2020-10-06T00:22:11Z

Jan, you have to install the java 11 in jenkins config and then select the java you want the job to run with.

In the job:

wimjongman · 2020-10-06T00:23:03Z

jdk location: https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.8%2B10/OpenJDK11U-jdk_x64_linux_hotspot_11.0.8_10.tar.gz

jantje · 2020-10-06T11:11:08Z

So I tried with the jdk you proposed.
I can see the jdk being downloaded in the log
Then a can't remove from the prebuild steps (because the previous build failed the files are not there)
and then the error

Started by user unknown or anonymous
Running as SYSTEM
Building in workspace /share/homes/admin/.jenkins/jobs/arduino_eclipse_plugin_build/workspace
Unpacking https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.8%2B10/OpenJDK11U-jdk_x64_linux_hotspot_11.0.8_10.tar.gz to /share/homes/admin/.jenkins/tools/hudson.model.JDK/jdk11_van_wim on Jenkins

The recommended git tool is: NONE
using credential 37aebe28-16bb-40e8-a8e3-dc998e2c865a
 > /usr/bin/git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > /usr/bin/git config remote.origin.url https://[email protected]/jantje/arduino-eclipse-plugin.git # timeout=10
Fetching upstream changes from https://[email protected]/jantje/arduino-eclipse-plugin.git
 > /usr/bin/git --version # timeout=10
 > git --version # 'git version 2.10.0'
using GIT_ASKPASS to set credentials github
 > /usr/bin/git fetch --tags --progress -- https://[email protected]/jantje/arduino-eclipse-plugin.git +refs/heads/*:refs/remotes/origin/* # timeout=10

 > /usr/bin/git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 6f966053741f3562ea2533289bd6d61a7ec6406e (refs/remotes/origin/master)
 > /usr/bin/git config core.sparsecheckout # timeout=10
 > /usr/bin/git checkout -f 6f966053741f3562ea2533289bd6d61a7ec6406e # timeout=10
Commit message: "#1163 indexlistener is buggy and unreliable"
 > /usr/bin/git rev-list --no-walk 6f966053741f3562ea2533289bd6d61a7ec6406e # timeout=10

Run condition [Execute Shell] enabling prebuild for step [[Send build artifacts over FTP]]
[workspace] $ /bin/ash /share/homes/admin/tmp/jenkins7703440894102025948.sh
rm: can't remove '*.tar.gz': No such file or directory
Parsing POMs

ERROR: Failed to parse POMs
org.apache.maven.project.ProjectBuildingException: Some problems were encountered while processing the POMs:
[ERROR] Invalid artifact repository: Unable to provision, see the following errors:

1) Error in custom provider, java.lang.TypeNotPresentException: Type org.eclipse.tycho.core.p2.P2ArtifactRepositoryLayout not present
  at ClassRealm[extension>org.eclipse.tycho:tycho-maven-plugin:2.0.0, parent: AntClassLoader[/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ftp-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-webdav-jackrabbit-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.connector-factory-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/commons-lang3-3.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.jsch-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-http-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-repository-metadata-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-transport-wagon-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-external-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.pageant-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-provider-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-reporting-api-3.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-utils-3.1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-interpolation-1.24.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-artifact-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven33-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/slf4j-api-1.7.25.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/cdi-api-1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.usocket-jna-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jcl-over-slf4j-1.7.25.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-spi-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-compat-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-plugin-api-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-sec-dispatcher-1.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-util-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-commons-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-component-annotations-1.7.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-settings-builder-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-interactivity-api-1.0-alpha-6.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/commons-cli-1.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsoup-1.11.3.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-classworlds-2.6.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven33-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-http-shared-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-core-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.core-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-impl-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven2.1-interceptor-1.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jna-platform-4.1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-common-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/lib-jenkins-maven-embedder-3.15.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven31-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven35-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/annotations-3.0.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/org.eclipse.sisu.inject-0.3.3.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-plugin.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-builder-support-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-file-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jackrabbit-webdav-2.14.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-model-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven35-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-settings-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-cipher-1.8.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.usocket-nc-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-api-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-connector-basic-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/doxia-sink-api-1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-shared-utils-3.2.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/commons-net-3.6.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/org.eclipse.sisu.plexus-0.3.3.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-model-builder-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/lib-jenkins-maven-artifact-manager-1.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/javax.inject-1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-embedder-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.sshagent-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsr250-api-1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-provider-api-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven31-agent-1.13.jar]] (via modules: org.eclipse.sisu.wire.WireModule -> org.eclipse.sisu.plexus.PlexusBindingModule)
  at ClassRealm[extension>org.eclipse.tycho:tycho-maven-plugin:2.0.0, parent: AntClassLoader[/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ftp-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-webdav-jackrabbit-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.connector-factory-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/commons-lang3-3.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.jsch-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-http-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-repository-metadata-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-transport-wagon-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-external-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.pageant-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-provider-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-reporting-api-3.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-utils-3.1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-interpolation-1.24.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-artifact-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven33-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/slf4j-api-1.7.25.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/cdi-api-1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.usocket-jna-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jcl-over-slf4j-1.7.25.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-spi-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-compat-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-plugin-api-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-sec-dispatcher-1.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-util-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-commons-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-component-annotations-1.7.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-settings-builder-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-interactivity-api-1.0-alpha-6.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/commons-cli-1.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsoup-1.11.3.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-classworlds-2.6.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven33-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-http-shared-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-core-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.core-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-impl-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven2.1-interceptor-1.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jna-platform-4.1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-common-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/lib-jenkins-maven-embedder-3.15.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven31-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven35-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/annotations-3.0.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/org.eclipse.sisu.inject-0.3.3.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-plugin.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-builder-support-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-file-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jackrabbit-webdav-2.14.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-model-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven35-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-settings-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-cipher-1.8.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.usocket-nc-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-interceptor-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-api-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-resolver-connector-basic-1.1.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/doxia-sink-api-1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-shared-utils-3.2.1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/commons-net-3.6.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/org.eclipse.sisu.plexus-0.3.3.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-model-builder-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/lib-jenkins-maven-artifact-manager-1.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.13.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/javax.inject-1.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven-embedder-3.5.4.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsch.agentproxy.sshagent-0.0.9.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/jsr250-api-1.0.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-provider-api-3.3.2.jar:/share/homes/admin/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven31-agent-1.13.jar]] (via modules: org.eclipse.sisu.wire.WireModule -> org.eclipse.sisu.plexus.PlexusBindingModule)
  while locating org.apache.maven.artifact.repository.layout.ArtifactRepositoryLayout annotated with @com.google.inject.name.Named(value=p2)

1 error @ 

	at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:383)
	at hudson.maven.MavenEmbedder.buildProjects(MavenEmbedder.java:370)
	at hudson.maven.MavenEmbedder.readProjects(MavenEmbedder.java:340)
	at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1329)
	at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1126)
	at hudson.FilePath.act(FilePath.java:1076)
	at hudson.FilePath.act(FilePath.java:1059)
	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:987)
	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
	at hudson.model.Run.execute(Run.java:1894)
	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)

[workspace] $ /bin/ash /share/homes/admin/tmp/jenkins8901936440324419164.sh

mv: can't rename '/share/homes/admin/.jenkins/jobs/arduino_eclipse_plugin_build/workspace/io.sloeber.product/target/products/sloeber-ide-sloeber_release-linux.gtk.x86_64.zip': No such file or directory
mv: can't rename '/share/homes/admin/.jenkins/jobs/arduino_eclipse_plugin_build/workspace/io.sloeber.product/target/products/sloeber-ide-sloeber_release-macosx.cocoa.x86_64.zip': No such file or directory

[workspace] $ /bin/bash /share/homes/admin/tmp/CommandInterpreterCondition7284858773862299867.sh

No change
Run condition [Execute Shell] preventing perform for step [[Send build artifacts over FTP]]
Sending e-mails to: [email protected]

Finished: FAILURE

jantje · 2020-10-06T11:13:29Z

Here is the content of my current config.xml (seems I can not attach files of type xml so I renamed to txt)

config.txt

wimjongman · 2020-10-06T12:06:32Z

Tycho 2.0.0 requires maven 3.6.3 or higher. Is that what you use?

jantje · 2020-10-06T12:12:49Z

maven 3.6.3 latest and greatest as to my config

wimjongman · 2020-10-06T13:51:52Z

I see that it fails on io.sloeber.core.NightlyJenkins

The test plugin depends on sloeber.ui and sloeber.ui depends on eclipse.ui.

The test is ran with the headless surefire application. This means that there is no ui available.

By the way, your jenkins build now runs (until it starts the unit tests).

jantje · 2020-10-06T14:06:03Z

Great
The tests run headless on my system as well. And they used to work. So that must be fixable.
What did you change?

wimjongman · 2020-10-06T14:08:30Z

I created a new project and then looked at what was different from yours. Your project had checked a bunch of maven option in the config. I unchecked them all and then it worked.

jantje · 2020-10-06T14:40:59Z

I feel stupid now :-(

jantje · 2020-10-08T19:24:34Z

Thanks @wimjongman for helping me out.
So things moved a long way now and this is my current understanding of what is going on.
Managers summary: The only remaining problem is getting make on my system.

What I think is the root cause of the misery here -apart of my entry level skills in this area- is that oracle java changed it's policy to having to sign papers which means the autoinstall for java 11 is not available in Jenkins
That combined with the trouble running Jenkins on Java 11 on my system; making I could not default to (oracle) java 11 on my build machine.
That combined with upgrading everything else in this area.

So I think I got lost in java providers combined with "incompatible system settings"
Though I installed oracle java 11.0.2 and tried to make jenkins (running in java8) to use this one for everything related to the build (FI by using JAVA_HOME and define the JAVA version in configuration) I never got it to work.
So I moved on to openJ9 11.08+10 (by installing it on the system and referencing it)

Then Wim proposed using openJ9 11.08+10 by letting Jenkins install it

I didn't get that to work and then Wim proposed to go and take a look on the server. (Thanks again Wiim)

It looked as if Wim got things going untill the nightly tests failing to create projects. but that was not really what was going on.

A build command had been added to the pre-build actions and that worked fine because it uses the JAVA_HOME environment var that was set to a java that worked. -I'm not saying wim didn't do anything, I'm just saying Wim got things back rolling but it wasn't rolling yet as wanted-

Because of "it for sure should work" extra energy boost I started searching again.

The first thing I found out is that the openJ9 11.08+10 installed by Jenkins results in the error

ERROR: Failed to parse POMs
java.io.IOException: Cannot run program "/share/homes/admin/.jenkins/tools/hudson.model.JDK/jdk11_van_wim/bin/java" (in directory "/share/homes/admin/.jenkins/jobs/arduino_eclipse_plugin_build/workspace"): error=2, No such file or directory

Basically because -as I found out during this writing- there is another subfolder between jdk11_van_wim and bin which should be added in the config :-(
So I switched back to the system installed openJ9 11.08+10.
This now worked untill project creation where java tries to load a UI plugin which fails. I traced this back to WorkspaceModifyOperation which is part of a UI plugin (why I don't know) which is the "eclipse advised" way. As ui stuff should not be in core I tried to remove all ui stuff from core (didn't get all of it) in 4d020f1

And with this change we get back to make not found.
Uptill now, the only reasonable explanation I can think of is that this worked and now no longer works is: one of the NAS upgrades removed make.
I tried lots of stuff to get make on my system but until now I didn't get make to work together with jenkins.
FI I installed a package manage and then installed make but then jenkins didn't want to access github any more.
I copied the make installled, removed the package manager and the jenkins was working again. The copied makefile acts like it is non-executable

But the jenkins install openJ9 11.08+10 works now

If anyone knows a zip to download make for linux, or other manual make installs, please please let me know.

jantje · 2020-10-10T11:10:02Z

Managers summary: I got make to work, I think qnap removed make

After trying to find a make (tried to compile make but didn't have gcc :-( ) I went back to entware (with is a package manager installable on qnap) reinstalled make and was back to the jenkins build fail issue
I got this error https://stackoverflow.com/questions/8329485/unable-to-find-remote-helper-for-https-during-git-clone
The qnap answer didn't work for me but I could use the git:// instead of https://
Then make still didn't work. That was because Jenkins runs as "system" and the system account doesn't have the patch changes made by entware.
So I added /opt/bin to the path in jenkins
Now make was found but 1 test (issue1047_Board_Names_Can_Be_used_as_Strings) still failed due to python3 not found. Using entware I installed python3 and now all tests currently in the test package are successful.
Given that both make and python "disappeared" from the jenkins build and given that QNAP has removed their own package manager recently; I think it is safe to say that a qnap upgrade removed make and python

jantje · 2020-10-10T11:11:33Z

During all the searching to get Jenkins back to work I noticed that the examples project creation do not add the library to the project where it did do so.
That may be the cause that the tests fail.
So I'll fix that one first

That is the creation of a project based on a library example should also link in the library

jantje · 2020-10-12T14:56:29Z

Now the libraries are correctly added to the project at creation time the test run successful in maven build.
This means that the indexer behaves differently in maven build and that this defect was actually a regression based on the fact libraries no longer wore added during project creation. When running in the gui the loop indexer->sloeber add libraries->indexer->...
resolved this issue but apparently this does not happen during maven build.

jantje added domain: releng Release Engineering importance: internal stuff OS: all labels Jul 10, 2020

wimjongman added a commit that referenced this issue Aug 6, 2020

Added travis logging #1204

36e5476

wimjongman added a commit that referenced this issue Aug 6, 2020

#1204 Build was ok. Adding regression test

17c3b73

jantje mentioned this issue Aug 11, 2020

No rule to make target Arduino.h #1198

Closed

jantje pushed a commit that referenced this issue Aug 28, 2020

#1204 move the registration of the listeners more forward

4d4bbe0

I'm doing this because it seems like the listeners are not triggered when running in maven. Now they are but the indexer seems to fail on the second run

jantje mentioned this issue Aug 31, 2020

Nested variables in platform.txt do not resolve #1184

Closed

wimjongman added a commit that referenced this issue Sep 12, 2020

#1204 added ui to test

d4b94ad

wimjongman added a commit that referenced this issue Sep 12, 2020

#1204 revert added ui to test

5b72462

wimjongman added a commit that referenced this issue Sep 14, 2020

#1204 Adding logging

27d8591

wimjongman added a commit that referenced this issue Sep 14, 2020

#1204 Adding logging

396a7d2

jantje pushed a commit that referenced this issue Sep 25, 2020

#1204 disable the failing test; this should be undone

356556d

jantje pushed a commit that referenced this issue Sep 25, 2020

#1204 avoid IllegalStateException: Workbench has not been created yet.

f2f438a

jantje mentioned this issue Oct 3, 2020

Indexer starts to soon after creation of project #1163

Closed

jantje pushed a commit that referenced this issue Oct 11, 2020

#1204 Add libraries at new library example crreation

e1ff929

That is the creation of a project based on a library example should also link in the library

jantje closed this as completed Oct 12, 2020

Maven build fails on the nightly testing (basically build fails) #1204

Maven build fails on the nightly testing (basically build fails) #1204

Comments

jantje commented Jul 10, 2020

jantje commented Aug 1, 2020

wimjongman commented Aug 6, 2020

wimjongman commented Aug 6, 2020

jantje commented Aug 6, 2020

jantje commented Aug 11, 2020

jantje commented Aug 18, 2020

jantje commented Aug 18, 2020

jantje commented Aug 19, 2020

wimjongman commented Sep 11, 2020

jantje commented Sep 11, 2020

jantje commented Sep 12, 2020 • edited Loading

wimjongman commented Sep 12, 2020

wimjongman commented Sep 12, 2020 • edited Loading

jantje commented Sep 12, 2020

jantje commented Sep 12, 2020

wimjongman commented Sep 12, 2020 • edited Loading

wimjongman commented Sep 12, 2020

jantje commented Sep 12, 2020

wimjongman commented Sep 13, 2020

jantje commented Sep 14, 2020

wimjongman commented Sep 14, 2020

wimjongman commented Sep 14, 2020

jantje commented Sep 14, 2020

jantje commented Sep 14, 2020

jantje commented Sep 25, 2020

jantje commented Sep 27, 2020

jantje commented Sep 27, 2020

wimjongman commented Oct 6, 2020

wimjongman commented Oct 6, 2020

jantje commented Oct 6, 2020

jantje commented Oct 6, 2020

wimjongman commented Oct 6, 2020 via email

jantje commented Oct 6, 2020

wimjongman commented Oct 6, 2020

jantje commented Oct 6, 2020

wimjongman commented Oct 6, 2020

jantje commented Oct 6, 2020

jantje commented Oct 8, 2020

jantje commented Oct 10, 2020

jantje commented Oct 10, 2020

jantje commented Oct 12, 2020

jantje commented Sep 12, 2020 •

edited

Loading

wimjongman commented Sep 12, 2020 •

edited

Loading

wimjongman commented Sep 12, 2020 •

edited

Loading