maven and tycho vs an Orion feature

EclipseCon 2011 has come and gone, and I was lucky enough to be there.  One of the nice things about going to speak at EclipseCon is the opportunity to see the other projects and technologies congregating around the Eclipse Foundation.  I'll blog later about the talks I gave, but one of the things I tried while down there was using Maven and Tycho to build a p2 repo.

I picked one of the Orion features.  My efforts can be seen in git://git.eclipse.org/gitroot/e4/org.eclipse.orion.server.git in the mavenExperiment branch.

There are some things maven needs to get started, which I put in a parent pom.  A tycho-version property (I needed the 0.11.0-SNAPSHOT so a javax.servlet import package could resolve) and a pluginRepository so I could load that version of tycho.  The repositories to build from: 3 p2 repos in my case.  Some build plugin configuration I copied from the tycho RCP example, including configuring the qualifier with <format>'v'yyyyMMdd-HHmm</format>

The plugin poms are straight forward, as I want manifest first builds.  You just need to make sure the versions and IDs match.  Same with feature poms.

To generate the p2 repo you need to specify 2 things.  An eclipse-repository pom (relatively straight forward, although I've copied in extra maven plugin statements). And a category.xml that lists the features you want in the repo.

I list all of my modules in my aggregator pom, and then run "mvn clean install".

Ta da: a p2 repo with my 2 features in it.

I'm still encountering some problem either with my configuration or with tycho itself.  I open bugs for the $HOME/.m2/repository interfering with a second build: https://issues.sonatype.org/browse/TYCHO-606 and trying to add extra categories to the p2 repo using category.xml results in the generated repo missing things: https://issues.sonatype.org/browse/TYCHO-605

I think there's some work to be done on the Execution Environment side as well, although JIRA appears to be down at the moment.

Once the build has been configured to run with maven, the next person just has to check out the code and "mvn clean install".  That looks promising.  I'm glad I had the opportunity to try out maven and tycho.

A working state from a p2 update

In one of our recent build/update cycles we made a change that was source compatible but not binary compatible.  In this case (there are no errors anywhere) it's easy to miss that you need to tag the consuming bundle as well.

The build ran fine, but the previous build's consuming bundle was used when creating the update site (since we didn't tag that bundle, the qualifier did not change).

When an unsuspecting co-worker updated to the latest build, his system would no longer start at all with a number of java.lang.NoSuchMethodErrors.

So how do you fix your install, given that you can't even bring up the workbench?  As it turns out, p2 can help you.  The SDK profile saves its state on each installation change.  You can usually find this information in:

bash-4.1$ ls eclipse/p2/org.eclipse.equinox.p2.engine/profileRegistry/SDKProfile.profile
1299467192263.profile.gz  1299499722731.profile.gz  1299761082054.profile.gz
1299467192395.profile.gz  1299499723473.profile.gz  1299761705917.profile.gz
1299467285942.profile.gz  1299499764937.profile.gz  state.properties
1299467287251.profile.gz  1299500329056.profile.gz

Then you just need to pick the last stable version of your installation, and you can revert your install and restart your system.

 eclipse/eclipse -noSplash \
-application org.eclipse.equinox.p2.director \
-repository file:$(pwd)/eclipse/p2/org.eclipse.equinox.p2.engine/profileRegistry/SDKProfile.profile \
-profile SDKProfile \
-revert 1299499722731

Obviously you'll need to use a different eclipse install and the necessary -destination arguments if the corrupt bundle is *in* the director part of your install's p2 framework :-)

The director won't run this command without specifying a -repository, but it's not important which one you use so I used the installations own SDKProfile repository.

For more director information see p2 director.

PDE build as workspace export

A question came up on the newgroups^H^H^H^Hforums recently about a headless way to build/export features and plugins from a workspace. It turns out it is possible, using the pluginPath property.

When you configure the PDE build build.properties:

1. Your provided topLevelElementId has to be your feature, or a "master" feature.
2. You must provide an elementPath property that's the absolute path to your master feature.
3. Your pluginPath can then point to your workspace.
4. You probably want to skipBase, skipMaps, and skipFetch
5. p2.gathering = true will generate the p2 metadata
6. p2.category.site=file://site.project/site.xml will include the site categories in the generated build repo

You also have to override "gatherLogs" in your customTargets.xml.  Your build repo is ready to install!

But I wouldn't recommend it.  Without digging into some arcane PDE build properties, it pollutes your workspace with build files, @dot directories, etc.

The simple RCP product build

A question came up the other day on the difference between building small RCP apps pre-3.5 with the delta pack, and what to do in the new p2 world. So I did what everybody wishes they could do ... I cornered Andrew Niefer and asked him a few questions.

I created the standard RCP mail app and then a feature to contain it. The I created a .product file that includes the RCP mail feature and org.eclipse.rcp (a few more plugins than the mail app technically needs, but a good starting point). The .product also had some branding information/splash information in it.

I want to use a product build, as I'm building a product. If you're going to be creating your runnable repo from p2 repos, you don't set baseLocation any more. You care about 2 different properties:

• repoBaseLocation - a directory where you can mirror your p2 repos, or at least the pieces you need.  You can create multiple repo directories under this location
• transformedRepoLocation - PDE Build will take your repoBaseLocation and convert it to a runnable repo (to be consumed by the build)

Out of the regular build properties, this looks like:

buildDirectory=${base}/buildDirectory
repoBaseLocation=${base}/repoBase
transformedRepoLocation=${base}/transformedRepos

You also need to set the location(s) of the repos you will consume, and provide a product file:

eclipsePlatformRepo=http://download.eclipse.org/eclipse/updates/3.6
product=${builder}/rcp_mail.product
runPackager=true
p2.gathering=true

There are other options that need to be filled in the build.properties file, but that's left as an exercise to the reader. The next step is to set up your customTargets.xml. Because I assume the defaults for most everything, I don't have to copy out the template any more. For the p2 part of this exercise, just provide a preProcessRepos target

<project name="RCP Mail customTargets overrides" >
        <import file="${eclipse.pdebuild.templates}/headless-build/customTargets.xml" />

        <target name="preProcessRepos">
                <p2.mirror source="${eclipsePlatformRepo}" destination="${repoBaseLocation}/launchers">
                        <iu id="org.eclipse.equinox.executable.feature.group" version=""/>
                        <iu id="org.eclipse.rcp.feature.group" version="" />
                </p2.mirror>
        </target>

        <target name="postFetch">
                <replace file="${buildDirectory}/pluginVersions.properties" token="HEAD" value="${timestamp}" />
        </target>
</project>

org.eclipse.rcp.feature.group will mirror everthing you need for the org.eclipse.rcp feature, including the different platform fragments. org.eclipse.equinox.executable.feature.group will mirror the features that a product build can use to create your specific executables (rebranded or renamed, etc). PDE build will run the p2.repo2runnable step for you automatically.

Now you just need to run it:

eclipse/eclipse -application org.eclipse.ant.core.antRunner \
 -buildfile eclipse/plugins/org.eclipse.pde.build_3.6.0.v20100603/scripts/productBuild/productBuild.xml \
 -Dbuilder=/opt/local/rcp/builder  \
 -Dbase=/opt/local/rcp/base \
 -Dtimestamp=20100924-1100

This generate 4 zips (the platforms from my configs= property) although the names were a little odd, they were complete product zips for each platform.

Thank you to Andrew for the examples and tips I used to set this up.

Import Projects from a Repository

I've been focused on e4/4.0 lately. A lot. Both on the modeled workbench infrastructure and on building the darn thing (I worship the ground Kim M walks on). But I discovered a feature in 3.6 that I find extremely useful. When importing plugins and fragments from any target platform, you now have the option to import them from (the CVS) repository.

Being able to import the source has always been useful if you want to see how something in JDT or PDE works, or if you want to tweak part of the behaviour of one of your plugins, or for the really brave, you want to see if you can fix a bug :-)

But while useful it has always been a problematic feature. Creating source bundles always lost the shape of the original project and its build information, since the source bundle is a representation of the binary bundle. The more complicated the layout or building of the original bundle, the further away the source bundle is from the actual plugin project.

And so "Import Project from Repository". Just grab the project. You have the option to take the tag that was used to build the bundle (the exact source in your target) or just grab it from HEAD (to get the latest). Fiddle with it. Reset it. Generate a patch for a fix that would actually apply to the bundle source! The committers will love you forever!!

Well, perhaps that's a little over the top, but I think it's a handy feature for plugin developers that need to poke around in someone else's bundles.

Maybe a comment policy

As much as I do enjoy the comments I get from such as "佩GailBohanan1蓉" I've decided to try comment moderation so I don't have to continually delete comments from my posts. So what's my moderation policy:

1) no adult site links
2) keep the language clean
3) stay more or less on topic

#1 is obvious

#2 well, this is a work related/technical blog. While sometimes eclipse might drive us to distraction, those moments can be captured using English (or the Language of your choice)

#3 this has a lot of flexibility (discussions will go wherever they want), but it is mostly in place to keep eclipse blogs from digressing into discussions about Chris Aniszczyk's Armani jorts... :-)

PDE will tell you

We're experimenting with a lot of new technology in e4 and with different development paradigms. It's all good. But development on 3.x and with PDE will continue for many years to come, and PDE has some often overlooked tools that can really help you develop your plugin and find your IDs.

1) Plug-in Selection Spy (ALT+SHIFT+F1) activate a part or dialog page and hit ALT+SHIFT+F1. Plug-in spy will open a popup and describe the contents (at least it will try). It will provide information about what ID and implementation class the focus part has, as well as which plugin contributed it, what are the active identifiers (menu, help, etc) and what type is the selection that part publishes.

2) Plug-in Menu Spy (ALT+SHIFT+F2). Hit ALT+SHIFT+F2 and then pick a menu item. The popup will provide information about where that item lives, action ID, command IDs if available, etc.

3) In the PDE Editor, the Browse... button. Many extensions need IDs provided in another extension. For example a menu contribution (org.eclipse.ui.menus) needs a commandId (org.eclipse.ui.commands). If you are asked to fill in an ID and there is a Browse... button, use it. It will give you a filterable list, and cuts down on cut&paste errors.

These 3 are examples of PDE tools that you probably don't use very often, but when you need them they're *really* helpful.

e4 and "early" compatibility

As the model for the e4 workbench stabilizes we're back working hard on the compatibility layer. Right now it consists of the gutted org.eclipse.ui.workbench plugin. The idea is to support the API we have in org.eclipse.workbench, but based on the e4 workbench model and e4 services, instead of the mass of internal code in parts, perspectives, and presentations.

We're taking a 2 pronged approach. Creating an e4 IDE application and slowly adding useful views and actions, seeing what is needed to bring them up. We want to support a useful number of views (like the Project Explorer and Problems view) sooner rather than later.

We're also running the org.eclipse.ui.tests.api.ApiTestSuite (after cleaning up internal references in the tests themselves with the aid of a tweaklet). ApiTestSuite covers the most common scenarios (opening and closing windows, perspectives, views, and editors), and supporting our API is a good way to help 3.x plugins run on e4 with the compatibility layer.

There's always a lot to do, so if you are interested please check out http://wiki.eclipse.org/E4/Compatibility/Running_the_compatibility_layer. You can also post to the e4-dev mailing list, or join us on irc at irc://freenode.net/#eclipse-e4

e4 and injectable command execution

I'm fond of most of the Command Framework in Eclipse 3.x (I have to maintain it after all :-) The notion of an abstract command (as opposed to the Command class) has 2 interesting responsibilities:

1. Am I enabled?
2. Do something.

In 3.x, commands delegate them to the command's currently active handler. Two aspects of the 3.x implementation aren't quite right in my opinion.

Responsibility #1 is implemented as state (isEnabled()) on the Command itself. Because a command is a global singleton (it's supposed to represent the operation) this has the unintended side effect that a command is enabled or disabled globally for the application. The active handler determines the state, but based on whatever information it wants (which may or may not be the same information it will execute against).

Responsibility #2 is implemented in the handler's execute method. It can then use the execution event to get the IEvaluationContext and extract information out of it. This mostly works well, and you end up treating the IEvaluationContext as a map.

In e4 we're looking IEclipseContext "Contexts: the service broker" which provides a localized view into the application state (i.e. what you can see from your view or editor makes sense). Information relevant to the state of the application is stored in your context in 2 forms. Some information is identified by class (ECommandService.class.getName(), EventAdmin.class.getName()) and some information is identified by name (selection, activePart).

Now the fun part. In e4 a handler for a command can define 2 methods:

• canExecute(*) - This will be called by the framework before trying to execute a command
• execute(*) - This will be called by the framework when appropriate (can you guess what it does? :-)

But in e4 you don't need to extract your variables from your context, you can ask for them directly:

public boolean canExecute(
    @Named("selection") IStructuredSelection selection);
public void execute(
    @Named("selection") IStructuredSelection selection, 
    Shell shell);

When the framework needs to call either method, it will inject the needed information from the appropriate IEclipseContext. It uses the @Named annotation if it is present, or the type of the parameter (for example, Shell.class.getName()) if @Named is not.

In e4 1.0 we're adopting the JSR 330 annotations for injection (plus a few extras that we need). This is still a work in progress, but it's moving right along. If you are interested, now is the time to get involved - http://wiki.eclipse.org/E4.

There's a lot more to this story, so stay tuned for the next update.

PW

Continuous integration of Platform UI tests

With the work I was doing for e4 builds I've had the opportunity to set up a PDE build and test environment. So I thought I would setup the Platform UI automated tests to run in a continuous build cycle on a machine in the lab.

I think it's interesting because:

1. The tests need to run against the latest I build for that week
2. I need to patch 2 features to install my plugins from HEAD. I just followed Andrew's instructions.
   
3. I wanted to use p2 to configure the tests, not the Automated Test Framework
   

As there's no real packaging or reuse involved, I just set up 3 features to drive the build. My 2 patch features, which include all of the Platform UI plugins that need to be replaced in org.eclipse.rcp and org.eclipse.platform. And my test feature, which includes the 2 patch feature, all of the Platform UI test plugins that contain my automated tests, and the automated test framework plugins:

• org.eclipse.core.tests.harness
• org.eclipse.test.performance
• org.eclipse.test
• org.eclipse.ant.optional.junit
  

Setting up PDE build involves copying the headless build templates (build.properties, customTargets.xml) and setting the parameters to match your build environment. Then creating a map project that's pointing to the correct features/plugins, and away you go.

For continuous builds you tend to want to build from HEAD. The best way (especially if you have a real map that's is regularly tagged for something else) is to set to properties:

   forceContextQualifier=v${buildId}
   fetchTag=HEAD

That way you don't end up with a lot of plugins that look like com.example.plugin_HEAD :-)

I also want to generate a p2 repo so I can consume my build easily. That's the standard set of p2 properties at the bottom of the build.properties file.

Once that's up (and a few tweaks to the customTargets.xml to use the latest I-build I will provide) and then you can build with the org.eclipse.ant.core.antRunner application.

Now I want to use p2 to run my automated tests. First I had to use XSLT to fix the content.xml and relax the feature patch version ranges (feature patches will target a very specific feature qualifier combination). There's a potential p2 fix in the works, but simple XSL will do for now.

I'm running the automated tests in my postBuild, with modified code I swiped from the Eclipse SDK automated test framework. The difference is that instead of having to generate a test.properties with the correct test plugin to test plugin version mapping and trying to unzip the build plugins into the eclipse test instance, the setup to run the tests now involve:

1. unzip the eclipse test instance
2. Call the director to install the plugin under test, the support plugins, and the patch features
3. call the plugin's test.xml
   

It manages all of the version numbers and dependencies for free, so your list of IUs looks like:

-installIUs ${testPlugin}, org.eclipse.test, org.eclipse.ant.optional.junit, org.eclipse.ui.test.platform.patch.feature.group, org.eclipse.ui.test.rcp.patch.feature.group

And now I have my headless automated test environment :-) The relevant build files and map files can all be found in CVS.

• :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse
• platform-incubator/ui/org.eclipse.ui.automated.build/org.eclipse.ui.releng
  

There are a lot of opportunities for improvement in this environment (continually updating the build target from http://download.eclipse.org/eclipse/updates/3.6-I-builds for example) but this is enough to get me started.

PW

Eclipse again

This last year has been very busy, working to finish Eclipse 3.5 and getting the Eclipse e4 Tech Preview into shape. Over the next few months, I hope to expound on some of the new techniques that can be used with Commands, Handlers, and Menu contributions as of 3.5

But really, I'll post something this time :-)

PW

Place command in menus in eclipse 3.2

In eclipse 3.2, there is good support for keybindings, commands, and handlers. There are some snippets available at Platform Command Framework.

But it's not obvious how to link your command+handler implementation to a menu or toolbar item. The <action/> elements have a definitionId attribute that can link the action to a command, but that allows the action's IActionDelegate to be linked to the keybinding (through the command). It turns the action's IActionDelegate into a handler for keybinding purposes through the use of an ActionHandler proxy class.

But what if you would prefer to implement your commands and handlers, and then just have the command called from your menus the same way it is through keybindings.

As it turns out, there is public API that will allow you to create a GenericCommandActionDelegate. I have an example available on the Platform Command Framework page.

The implementation takes advantage of the fact that the action element is created through IConfigurationElement#createExecutableExtension(*), which means that your action delegate can implement IExecutableExtension and receive extra configuration information from the plugin.xml.

It's not a perfect solution for 3.2 (there would need to be more work to support something like checked state or radio buttons), but it will allow you to focus on creating command and handlers, and then using the action based extensions to execute your commands.

The same strategy could be used for 3.1 as well, although some modifications would have to be made. In 3.1, you have to build your own ExecutionEvent and call the Command#execute*(*) method yourself.

PW

What can the Command framework do for you?

In 3.2 much more of the eclipse action infrastructure has moved onto the command framework. It is based on 4 extension points.

org.eclipse.ui.commands and the ICommandService create the abstract user command (like COPY). org.eclipse.ui.handlers and the IHandlerService allow plugins and interested parts to provide an implementation for the COPY command.

org.eclipse.ui.bindings can map key sequences to commands, and IBindingService can programmatically review those keybindings. org.eclipse.ui.contexts and the IContextService can activate contexts, which can be used to scope the keybindings.

The IBindingService can be used to programmatically add new keybindings ... but not easily. There was some example code and discussion on the newsgroup:

Re: Action accelerators

In 3.2 there is still no replacement for placing the command in a menu or toolbar. <actions/> (which should always be linked to commands, BTW) are still used, and the work is done in 4 extension points. org.eclipse.ui.actionSets, org.eclipse.ui.editorActions, org.eclipse.ui.viewActions, and org.eclipse.ui.popupMenus.

Providing this replacement will be part of the command framework work for 3.3.

I'm still wondering if this information should be captured on a wiki for updating (until it can make it into the 3.3 documentation), or just left floating around the newsgroups ...

Later,
PW

Eclipse, you say? Very interesting.

It's been just over a year since I joined the Eclipse team. I've been hearing about these blog postings that other Eclipse committers and Eclipse Foundation members maintain, and thought I would add my 2 cents into the pot.

But later, this first post was very tiring :-)

PW