Tuesday, July 24, 2012

CBI and the eclipse project build

The CBI initiative is working at providing some infrastructure and templates to make it easy to get a new eclipse project up and running. So with a little setup, a new project can take advantage of some of the technologies (maven/tycho) and services (hudson.eclipse.org, eclipse jar signing) that the Eclipse Foundation offers.

One of CBI's trial projects is the Eclipse Project build (now called Eclipse Classic SDK :-) It's like the poster child for build edge conditions.

Right now the Eclipse project is built with a rather large set of PDE build scripts. But as we recently went through a lot of work trying to get them to build on eclipse.org, I can attest to the fact that they're not terribly portable.

With the need for various organizations to be able to build eclipse in their own environments, the CBI initiative is taking advantage of the fact that the second person to build something with maven gets a (mostly :-) free ride.

After cloning a large git repo made up mostly of submodules, all you have to do is install 2 poms and you're off to the races:

bash$ git clone -b Juno_RC4_R4 --recursive  \
  ssh://git.eclipse.org/gitroot/cbi/platform-aggregator.git \
  R4_platform-aggregator

bash$ cd R4_platform-aggregator

bash$ mvn -f eclipse-parent/pom.xml \
  clean install \
  -Dmaven.repo.local=/opt/pwebster/lts/R4_localrepo

bash$ mvn -f maven-cbi-plugin/pom.xml \
  clean install \
  -Dmaven.repo.local=/opt/pwebster/lts/R4_localrepo

bash$ mvn \
  clean install \
  -Dmaven.test.skip=true \
  -Dmaven.repo.local=/opt/pwebster/lts/R4_localrepo

Once finished, your build repo is in R4_platform-aggregator/TMP/org.eclipse.sdk.epp/target/repository and some Eclipse SDK zips are in R4_platform-aggregator/TMP/org.eclipse.sdk.epp/target/products. These locations might move when we migrate some releng information over to eclipse.platform.releng.eclipsebuilder.

This is still a work in progress, so the build doesn't produce all of the build output of our tradition PDE-driven build. Work is still being done to get the compile levels correct, and figure out a replacement or callout to the customCallBacks.xml.

There are also some other hiccups, like having to run the build twice as the first time it fails with a compile error in org.eclipse.jface. But in short order you have yourself a product build.

For more information see the CBI wiki.

Wednesday, February 15, 2012

p2 can update your eclipse.ini

I regularly switch my base SDK install to try out new developer integration builds, and it's not to much trouble to get it set up correctly thanks to p2 and the director.

eclipse/eclipse \
-application org.eclipse.equinox.p2.director \
-noSplash \
-repository \
http://download.eclipse.org/eclipse/updates/4.2-I-builds,\
http://download.eclipse.org/e4/updates/0.12-I-builds,\
http://download.eclipse.org/releases/juno,\
http://download.eclipse.org/tools/orbit/downloads/drops/R20110523182458/repository \
-installIUs \
org.apache.commons.jxpath/1.3.0.v200911051830,\
org.apache.batik.xml/1.6.0.v201011041432,\
org.dojotoolkit/1.6.1.v201105210659,\
org.eclipse.nebula.widgets.gallery/1.0.0.R_0_20090212,\
org.eclipse.egit.feature.group,\
org.eclipse.jgit.feature.group,\
org.eclipse.emf.sdk.feature.group,\
org.eclipse.xtext.sdk.feature.group,\
org.eclipse.wst.xml_ui.feature.feature.group

As you can see I use the director to set up my eclipse/running target with orbit bundles I need, with EGit, and with SDKs that I need like EMF, XText, XML Editors and Tools. I love a good automated process.

With all functionality in use and with 23 platform and e4 git repositories for EGit/JGit to work through, it's easy to get an OOME with eclipse default settings. Because I'm testing downloads instead of upgrades, that means I have to update my eclipse.ini every time. It's minor, but it breaks the automated process.

Since p2 provides the touchpoints that can modify the eclipse.ini already, it should be simple to hand-craft a local repo with an IU that can update the eclipse.ini as part of my director call.

   <unit id='toolingorg.eclipse.e4.fix.jvmArg' version='4.2.0.I20120131-2200' singleton='false'>
      <provides size='2'>
        <provided namespace='org.eclipse.equinox.p2.iu' name='toolingorg.eclipse.e4.fix.jvmArg' version='4.2.0.I20120131-2200'/>
        <provided namespace='toolingorg.eclipse.sdk.ide' name='org.eclipse.e4.fix.jvmArg' version='4.2.0.I20120131-2200'/>
      </provides>
      <requires>
        <required namespace='org.eclipse.equinox.p2.iu' name='org.eclipse.sdk.ide' range='[0.0.0,5.0.0)'/>
      </requires>
      <touchpoint id='org.eclipse.equinox.p2.osgi' version='1.0.0'/>
      <touchpointData size='1'>
        <instructions size='2'>
          <instruction key='unconfigure'>
            addJvmArg(jvmArg:-Xms40m);addJvmArg(jvmArg:-Xmx384m);removeJvmArg(jvmArg:-Xms64m);removeJvmArg(jvmArg:-Xmx1024m);
          </instruction>
          <instruction key='configure'>
            removeJvmArg(jvmArg:-Xms40m);removeJvmArg(jvmArg:-Xmx384m);addJvmArg(jvmArg:-Xms64m);addJvmArg(jvmArg:-Xmx1024m);
          </instruction>
        </instructions>
      </touchpointData>
    </unit>



I added a .feature.group IU to make it stand out that the fix has been installed into this version of eclipse, and voila.

Tuesday, November 15, 2011

EGit and a shortlived topic branch

I've been experimenting with git workflows in Eclipse/EGit, trying to leverage some of the (supposed) power or functionality we get from using git for our SCM. Fortunately, the EGit folks were able to offer some hints as to how to make that happen.

The easiest way to get started is to create the topic branch off of your main remote branch, using Team>Switch To>New Branch. Because we want to avoid small merges, I already have branch.autosetuprebase=always set, and that's reflected in the defaults in the EGit Create Branch dialog. In my case I created pwebster/bug357532 off of origin/master. That way changes to master will be picked up and my local changes rebased automatically.

I also used the suggested pattern of <committerId>/branchName. We hope to have a receive hook in place soon that allows committers to delete their own topic branches if pushed to the public repo.

But if you aren't working on a feature branch, you probably want to push your topic branch commits back to your remote branch, in my case origin/master. You can do this by setting a push refspec on that remote. I went to my repository configuration in the Preferences>Team>Git>Configuration dialog and adding the following property: remote.origin.push=pwebster/bug358020:refs/heads/master.

Now for a short lived topic branch, you're set. Pulling will keep you up to date with origin/master, and you can push your fix to origin/master without having to do a lot of switching/fast-forward merging/pushing/deleting.

Wednesday, November 09, 2011

Where the git did that go?

I just came back from Paternity leave in the Eclipse SDK milestone (M3) week. The leave was great. Coming back was, as always, exciting. A slightly misconfigured tool had removed branches and tags (and an aggressive prune/gc had removed commits) during a push to our eclipse.platform.ui repo, leading to Bug 361707 - I have broken the platform-ui git repository. Because tags and commits were gone, we also broke all of the warmup I builds for our M3 week. My apologies to all that suffered.

These are all done by legitimate actions supported by git ... it just turns out that if you have a process backed by a central canonical repository, you really don't want your source disappearing out from under you. "I might have pushed back most of our repo" is not an acceptable level of support :-)

We were able to find a mostly complete repo, disable the Eclipse Foundation commit hook and push back the repo. We are discussing ways to limit this kind of damage in the future. See

  • Bug 362076 - Better policy to guard against deleting all branches and tags from our public repos
  • Bug 362361 - Better policy ... enforce receive.denyDeletes = true
  • Bug 362363 - Better policy ... provide hooks to allow a committer to delete /branchname branches

Tuesday, August 09, 2011

Git Migration and tagging

The Eclipse SDK, or Eclipse Classic as it now tends to be called, has been building and releasing for just about 10 years, using the PDE Build system. Our releases are based on map files, which contain the IDs, SCM, and co-ordinates of the plugins we need to build.

When moving from CVS, where you can tag individual files, to git, where you tag a commit (the repo), this can cause headaches.

The common practice of tagging the projects that change for each week's I-builds means that cvs2git generates a master branch that looks like a fir tree :-) The delete commits remove all of the projects that weren't tagged. That makes the commit accurate, but that's not the shape of the git repo we want. A little scripting and we moved the tags from the delete commits back onto the main branch.

The other problem comes from the practice of tagging the release (R3_6) but only branching projects that change (R3_6_maintenance). In order for cvs2git to make the branch correct, the first commit on the R3_6_maintenance branch is delete commit. To make the important maintenance branches look correct, you can fix this one of 2 ways. Either rebase the branch around the delete commit (before you publish it to the public), or pre-condition the CVS repo so that all projects that are tagged R3_6 are branched R3_6_maintenance as well.

The scripts I used to fix Platform UI are in the org.eclipse.migration/scripts directory.



Friday, July 29, 2011

Git Migration and ignore files

As I mentioned in Git Migration I've been involved in the Platform UI CVS to Git migration using the cvs2git tool.

After your first successful conversion of a test repo, you fire it up in eclipse and check it out. That's when you see ... a lot of untracked changes! Oh yeah, all our .cvsignore files are useless :-)

Obviously we wanted a .gitignore at the root of our repo, from the beginning of time ... just like the catchphrase "there's an app for that", git will allow you to re-write your history so that it looks like the .gitignore was in every branch and tag since day one (How do I make a git commit in the past?).  Obviously since this is a re-write, you need to do this as the last step before you publish your repo.

  1. create suitable file contents
  2. add it to your repo
  3. re-write all of the index trees in each commit to add the file
  4. reset your repo


Here's the example as fed into bash:

bash$ cat - >../gitignore <<EOF
bin/
*~
*.rej
*.bak
*.patch
javacore.*
heapdump.*
core.*
Snap.*
target/

EOF
bash$ new_file=$(git hash-object -w ../gitignore)
bash$ git filter-branch \
--index-filter \
'git update-index --add --cacheinfo 100644 '"$new_file"' .gitignore' \
--tag-name-filter cat \
-- --all
bash$ git reset --hard

Now I have a .gitignore at the root of my repo, and my status and EGit views are not cluttered with spurious untracked files. Rinse and repeat for more specialized .gitignores in various project directories that need them.

Wednesday, July 20, 2011

Git Migration

A large part of the Open Source world seems hot on git these days (and DVCSs in general). I can kinda see why. For tracking changes, the notion of a commit definitely makes more sense than updating the versions on files that CVS has to offer. The ease of creating git repos plays well with one of my own beliefs, which is "Even for one's own little projects, if it's not in a VCS it may as well be written on the back of a napkin". Social coding sites like GitHub offer an ease of contributing to projects that contributors love, and git makes it easier to keep contributions (commits) from going stale than when dealing with patches in email or bug tacking systems. Git itself has useful tools for visualizing your repo (like gitk), tracking down when a bad behaviour was introduced (git blame), and a dozen others besides.

And for those that miss their C roots, you can pretend your git hashes are pointers: 5eec2b04a70f704a7cdc1b77db59bb90ec03dd68 :-)

With the Eclipse Foundation also interested in supporting git as its main VCS, the Eclipse Project decided to move to git at the beginning of the Juno Release cycle. I've been involved with the Platform UI component migration.

I'll write a second post soon with more details about the problems we encountered and some of the steps we took, but after a grueling 2 weeks the conversion of this component is done. Woo Woo!

  • git://git.eclipse.org/gitroot/platform/eclipse.platform.runtime.git
  • git://git.eclipse.org/gitroot/platform/eclipse.platform.ui.git
  • git://git.eclipse.org/gitroot/e4/org.eclipse.e4.tools.git

Friday, June 17, 2011

p2 cheatsheet

I was writing a p2 mirror script suggestion for a colleague.  When it morphed into a sorta p2 cheatsheet, I thought I'd blog about it (so I can find it again when I need it :-).

repo syntax

Most repos are specified via URLs like http:, but they can be files on the disk or even zipped repos:

  • http://download.eclipse.org/eclipse/updates/3.7-I-builds
  • file:///home/data/httpd/download.eclipse.org/eclipse/updates/3.7-I-builds
  • jar:file:///opt/pwebster/emf-xsd-Update-2.7.0RC3.zip!/

Just a note: destinations are (almost) always normal directory paths, not URLs.

Mirroring repos

You can quickly (well, I/O bound :-) mirror 2 builds repos so you can compare them on your disk. Here's an ant script for current, important, build repos.

<project name="Mirror" default="mirrorRepos">
<property name="eclipse36Mirror" value="${baseDir}/eclipse36" />
<property name="eclipse36Repo" value="http://download.eclipse.org/eclipse/updates/3.6/R-3.6.2-201102101200" />
<property name="eclipse37Mirror" value="${baseDir}/eclipse37" />
<property name="eclipse37Repo" value="http://download.eclipse.org/eclipse/updates/3.7-I-builds/I20110613-1736" />

<target name="mirrorRepos">
<echo message="Mirror from ${eclipse36Repo} to ${eclipse36Mirror}" />
<p2.mirror destination="file:${eclipse36Mirror}" ignoreerrors="true">
<source location="${eclipse36Repo}" />
<!--slicingOptions includeOptional="false" includeNonGreedy="false" latestVersionOnly="true"/-->
<!--iu id="org.eclipse.sdk.ide"/-->
</p2.mirror>
<echo message="Mirror from ${eclipse37Repo} to ${eclipse37Mirror}" />
<p2.mirror destination="file:${eclipse37Mirror}" ignoreerrors="true">
<source location="${eclipse37Repo}" />
<!--slicingOptions includeOptional="false" includeNonGreedy="false" latestVersionOnly="true"/-->
<!--iu id="org.eclipse.sdk.ide"/-->
</p2.mirror>
</target>

</project>


You can run that from an eclipse instance, either in eclipse using an Eclipse Application, or from the command line (my preference):

bash$ eclipse/eclipse -noSplash \
-application org.eclipse.ant.core.antRunner \
-DbaseDir=/some/useful/basedir -buildfile mirror.xml

It'll spit out some messages about unsatisfied IUs, but that's OK because you're using the slicer, not the provisioner.

IUs and version numbers

If you're looking at plugin or IU versions, you can quickly get a list in the "IU=version" format:

bash$ eclipse/eclipse -noSplash \
-application org.eclipse.equinox.p2.director \
-repository http://download.eclipse.org/eclipse/updates/3.7-I-builds/I20110613-1736 \
-list

If you pick a composite repo like http://download.eclipse.org/eclipse/updates/3.6 you can list all of the IUs, and then install a specific version of org.eclipse.sdk.ide, for example.


Creating a product for your platform

You can use the same scripts that the PDE build uses to generate a complete install for a platform:

<project name="Build Product" default="buildProduct">
<property name="p2.repo.URL" value="http://download.eclipse.org/eclipse/updates/3.7-I-builds" />
<property name="iuId" value="org.eclipse.sdk.ide" />
<!--property name="iuId" value="org.eclipse.platform.ide"/-->
<!--property name="iuId" value="org.eclipse.rcp.id"/-->
<property name="profile" value="SDKProfile" />
<property name="os" value="linux" />
<property name="ws" value="gtk" />
<property name="arch" value="x86_64" />

<target name="buildProduct">
<ant antfile="${eclipse.pdebuild.scripts}/genericTargets.xml" target="runDirector">
<property name="p2.director.installPath" value="${baseDir}/p2_${os}_${ws}_${arch}/eclipse" />
<property name="p2.director.profile" value="${profile}" />
<property name="p2.director.iu" value="${iuId} " />
<property name="os" value="${os}" />
<property name="ws" value="${ws}" />
<property name="arch" value="${arch}" />
<property name="p2.repo" value="${p2.repo.URL}" />
<!--property name="p2.director.version" value="an.IU.version"/-->
</ant>
</target>
</project>



bash$ eclipse/eclipse -noSplash \
-application  org.eclipse.ant.core.antRunner \
-DbaseDir=/some/useful/basedir -buildfile  productBuild.xml

The exact same thing can be accomplished using the command-line director (just with a lot of arguments :-)

bash$ eclipse/eclipse -noSplash \
-application org.eclipse.equinox.p2.director \
-repository http://download.eclipse.org/eclipse/updates/3.7-I-builds \
-installIU org.eclipse.sdk.ide \
-destination /some/useful/basedir/linux_gtk_x86_64  \
-bundlepool /some/useful/basedir/linux_gtk_x86_64 \
-profile SDKProfile \
-profileProperties org.eclipse.update.install.features=true \
-p2.os linux -p2.ws gtk -p2.arch x86_64 -roaming

Installing into your own eclipse

This is the shortest command line of all, and the easiest way to take a new eclipse and install features from scripts.

bash$ eclipse/eclipse -noSplash \
-application org.eclipse.equinox.p2.director \
-repository http://download.eclipse.org/eclipse/updates/3.7-I-builds \
-i org.eclipse.releng.tools.feature.group

More info

More about the director command line: http://help.eclipse.org/helios/index.jsp?topic=/org.eclipse.platform.doc.isv/guide/p2_director.html

More about the p2 tasks and all the options they can take: http://help.eclipse.org/helios/index.jsp?topic=/org.eclipse.platform.doc.isv/guide/p2_repositorytasks.htm

Monday, March 28, 2011

maven and tycho vs an Orion feature

EclipseCon 2011 has come and gone, and I was lucky enough to be there.  One of the nice things about going to speak at EclipseCon is the opportunity to see the other projects and technologies congregating around the Eclipse Foundation.  I'll blog later about the talks I gave, but one of the things I tried while down there was using Maven and Tycho to build a p2 repo.

I picked one of the Orion features.  My efforts can be seen in git://git.eclipse.org/gitroot/e4/org.eclipse.orion.server.git in the mavenExperiment branch.

There are some things maven needs to get started, which I put in a parent pom.  A tycho-version property (I needed the 0.11.0-SNAPSHOT so a javax.servlet import package could resolve) and a pluginRepository so I could load that version of tycho.  The repositories to build from: 3 p2 repos in my case.  Some build plugin configuration I copied from the tycho RCP example, including configuring the qualifier with <format>'v'yyyyMMdd-HHmm</format>

The plugin poms are straight forward, as I want manifest first builds.  You just need to make sure the versions and IDs match.  Same with feature poms.

To generate the p2 repo you need to specify 2 things.  An eclipse-repository pom (relatively straight forward, although I've copied in extra maven plugin statements). And a category.xml that lists the features you want in the repo.

I list all of my modules in my aggregator pom, and then run "mvn clean install".

Ta da: a p2 repo with my 2 features in it.

I'm still encountering some problem either with my configuration or with tycho itself.  I open bugs for the $HOME/.m2/repository interfering with a second build: https://issues.sonatype.org/browse/TYCHO-606 and trying to add extra categories to the p2 repo using category.xml results in the generated repo missing things: https://issues.sonatype.org/browse/TYCHO-605

I think there's some work to be done on the Execution Environment side as well, although JIRA appears to be down at the moment.

Once the build has been configured to run with maven, the next person just has to check out the code and "mvn clean install".  That looks promising.  I'm glad I had the opportunity to try out maven and tycho.

Thursday, March 10, 2011

A working state from a p2 update

In one of our recent build/update cycles we made a change that was source compatible but not binary compatible.  In this case (there are no errors anywhere) it's easy to miss that you need to tag the consuming bundle as well.

The build ran fine, but the previous build's consuming bundle was used when creating the update site (since we didn't tag that bundle, the qualifier did not change).

When an unsuspecting co-worker updated to the latest build, his system would no longer start at all with a number of java.lang.NoSuchMethodErrors.

So how do you fix your install, given that you can't even bring up the workbench?  As it turns out, p2 can help you.  The SDK profile saves its state on each installation change.  You can usually find this information in:

bash-4.1$ ls eclipse/p2/org.eclipse.equinox.p2.engine/profileRegistry/SDKProfile.profile
1299467192263.profile.gz  1299499722731.profile.gz  1299761082054.profile.gz
1299467192395.profile.gz  1299499723473.profile.gz  1299761705917.profile.gz
1299467285942.profile.gz  1299499764937.profile.gz  state.properties
1299467287251.profile.gz  1299500329056.profile.gz

Then you just need to pick the last stable version of your installation, and you can revert your install and restart your system.

 eclipse/eclipse -noSplash \
-application org.eclipse.equinox.p2.director \
-repository file:$(pwd)/eclipse/p2/org.eclipse.equinox.p2.engine/profileRegistry/SDKProfile.profile \
-profile SDKProfile \
-revert 1299499722731

Obviously you'll need to use a different eclipse install and the necessary -destination arguments if the corrupt bundle is *in* the director part of your install's p2 framework :-)

The director won't run this command without specifying a -repository, but it's not important which one you use so I used the installations own SDKProfile repository.

For more director information see p2 director.

Wednesday, February 23, 2011

PDE build as workspace export

A question came up on the newgroups^H^H^H^Hforums recently about a headless way to build/export features and plugins from a workspace. It turns out it is possible, using the pluginPath property.

When you configure the PDE build build.properties:
  1. Your provided topLevelElementId has to be your feature, or a "master" feature.
  2. You must provide an elementPath property that's the absolute path to your master feature.
  3. Your pluginPath can then point to your workspace.
  4. You probably want to skipBase, skipMaps, and skipFetch
  5. p2.gathering = true will generate the p2 metadata
  6. p2.category.site=file://site.project/site.xml will include the site categories in the generated build repo
You also have to override "gatherLogs" in your customTargets.xml.  Your build repo is ready to install!

But I wouldn't recommend it.  Without digging into some arcane PDE build properties, it pollutes your workspace with build files, @dot directories, etc.

Friday, September 24, 2010

The simple RCP product build

A question came up the other day on the difference between building small RCP apps pre-3.5 with the delta pack, and what to do in the new p2 world. So I did what everybody wishes they could do ... I cornered Andrew Niefer and asked him a few questions.

I created the standard RCP mail app and then a feature to contain it. The I created a .product file that includes the RCP mail feature and org.eclipse.rcp (a few more plugins than the mail app technically needs, but a good starting point). The .product also had some branding information/splash information in it.

I want to use a product build, as I'm building a product. If you're going to be creating your runnable repo from p2 repos, you don't set baseLocation any more. You care about 2 different properties:
  • repoBaseLocation - a directory where you can mirror your p2 repos, or at least the pieces you need.  You can create multiple repo directories under this location
  • transformedRepoLocation - PDE Build will take your repoBaseLocation and convert it to a runnable repo (to be consumed by the build)
Out of the regular build properties, this looks like:

buildDirectory=${base}/buildDirectory
repoBaseLocation=${base}/repoBase
transformedRepoLocation=${base}/transformedRepos

You also need to set the location(s) of the repos you will consume, and provide a product file:

eclipsePlatformRepo=http://download.eclipse.org/eclipse/updates/3.6
product=${builder}/rcp_mail.product
runPackager=true
p2.gathering=true

There are other options that need to be filled in the build.properties file, but that's left as an exercise to the reader. The next step is to set up your customTargets.xml. Because I assume the defaults for most everything, I don't have to copy out the template any more. For the p2 part of this exercise, just provide a preProcessRepos target

<project name="RCP Mail customTargets overrides" >
        <import file="${eclipse.pdebuild.templates}/headless-build/customTargets.xml" />

        <target name="preProcessRepos">
                <p2.mirror source="${eclipsePlatformRepo}" destination="${repoBaseLocation}/launchers">
                        <iu id="org.eclipse.equinox.executable.feature.group" version=""/>
                        <iu id="org.eclipse.rcp.feature.group" version="" />
                </p2.mirror>
        </target>

        <target name="postFetch">
                <replace file="${buildDirectory}/pluginVersions.properties" token="HEAD" value="${timestamp}" />
        </target>
</project>



org.eclipse.rcp.feature.group will mirror everthing you need for the org.eclipse.rcp feature, including the different platform fragments. org.eclipse.equinox.executable.feature.group will mirror the features that a product build can use to create your specific executables (rebranded or renamed, etc). PDE build will run the p2.repo2runnable step for you automatically.

Now you just need to run it:

eclipse/eclipse -application org.eclipse.ant.core.antRunner \
 -buildfile eclipse/plugins/org.eclipse.pde.build_3.6.0.v20100603/scripts/productBuild/productBuild.xml \
 -Dbuilder=/opt/local/rcp/builder  \
 -Dbase=/opt/local/rcp/base \
 -Dtimestamp=20100924-1100


This generate 4 zips (the platforms from my configs= property) although the names were a little odd, they were complete product zips for each platform.

Thank you to Andrew for the examples and tips I used to set this up.

Monday, July 26, 2010

Import Projects from a Repository

I've been focused on e4/4.0 lately. A lot. Both on the modeled workbench infrastructure and on building the darn thing (I worship the ground Kim M walks on). But I discovered a feature in 3.6 that I find extremely useful. When importing plugins and fragments from any target platform, you now have the option to import them from (the CVS) repository.

Being able to import the source has always been useful if you want to see how something in JDT or PDE works, or if you want to tweak part of the behaviour of one of your plugins, or for the really brave, you want to see if you can fix a bug :-)

But while useful it has always been a problematic feature. Creating source bundles always lost the shape of the original project and its build information, since the source bundle is a representation of the binary bundle. The more complicated the layout or building of the original bundle, the further away the source bundle is from the actual plugin project.

And so "Import Project from Repository". Just grab the project. You have the option to take the tag that was used to build the bundle (the exact source in your target) or just grab it from HEAD (to get the latest). Fiddle with it. Reset it. Generate a patch for a fix that would actually apply to the bundle source! The committers will love you forever!!

Well, perhaps that's a little over the top, but I think it's a handy feature for plugin developers that need to poke around in someone else's bundles.

Friday, June 11, 2010

Maybe a comment policy

As much as I do enjoy the comments I get from such as "佩GailBohanan1蓉" I've decided to try comment moderation so I don't have to continually delete comments from my posts. So what's my moderation policy:

1) no adult site links
2) keep the language clean
3) stay more or less on topic

#1 is obvious

#2 well, this is a work related/technical blog. While sometimes eclipse might drive us to distraction, those moments can be captured using English (or the Language of your choice)

#3 this has a lot of flexibility (discussions will go wherever they want), but it is mostly in place to keep eclipse blogs from digressing into discussions about Chris Aniszczyk's Armani jorts... :-)

Friday, February 19, 2010

PDE will tell you

We're experimenting with a lot of new technology in e4 and with different development paradigms. It's all good. But development on 3.x and with PDE will continue for many years to come, and PDE has some often overlooked tools that can really help you develop your plugin and find your IDs.

1) Plug-in Selection Spy (ALT+SHIFT+F1) activate a part or dialog page and hit ALT+SHIFT+F1. Plug-in spy will open a popup and describe the contents (at least it will try). It will provide information about what ID and implementation class the focus part has, as well as which plugin contributed it, what are the active identifiers (menu, help, etc) and what type is the selection that part publishes.

2) Plug-in Menu Spy (ALT+SHIFT+F2). Hit ALT+SHIFT+F2 and then pick a menu item. The popup will provide information about where that item lives, action ID, command IDs if available, etc.

3) In the PDE Editor, the Browse... button. Many extensions need IDs provided in another extension. For example a menu contribution (org.eclipse.ui.menus) needs a commandId (org.eclipse.ui.commands). If you are asked to fill in an ID and there is a Browse... button, use it. It will give you a filterable list, and cuts down on cut&paste errors.

These 3 are examples of PDE tools that you probably don't use very often, but when you need them they're *really* helpful.

Friday, February 05, 2010

e4 and "early" compatibility

As the model for the e4 workbench stabilizes we're back working hard on the compatibility layer. Right now it consists of the gutted org.eclipse.ui.workbench plugin. The idea is to support the API we have in org.eclipse.workbench, but based on the e4 workbench model and e4 services, instead of the mass of internal code in parts, perspectives, and presentations.

We're taking a 2 pronged approach. Creating an e4 IDE application and slowly adding useful views and actions, seeing what is needed to bring them up. We want to support a useful number of views (like the Project Explorer and Problems view) sooner rather than later.

We're also running the org.eclipse.ui.tests.api.ApiTestSuite (after cleaning up internal references in the tests themselves with the aid of a tweaklet). ApiTestSuite covers the most common scenarios (opening and closing windows, perspectives, views, and editors), and supporting our API is a good way to help 3.x plugins run on e4 with the compatibility layer.

There's always a lot to do, so if you are interested please check out http://wiki.eclipse.org/E4/Compatibility/Running_the_compatibility_layer. You can also post to the e4-dev mailing list, or join us on irc at irc://freenode.net/#eclipse-e4

Tuesday, November 03, 2009

e4 and injectable command execution

I'm fond of most of the Command Framework in Eclipse 3.x (I have to maintain it after all :-) The notion of an abstract command (as opposed to the Command class) has 2 interesting responsibilities:
  1. Am I enabled?
  2. Do something.
In 3.x, commands delegate them to the command's currently active handler. Two aspects of the 3.x implementation aren't quite right in my opinion.

Responsibility #1 is implemented as state (isEnabled()) on the Command itself. Because a command is a global singleton (it's supposed to represent the operation) this has the unintended side effect that a command is enabled or disabled globally for the application. The active handler determines the state, but based on whatever information it wants (which may or may not be the same information it will execute against).

Responsibility #2 is implemented in the handler's execute method. It can then use the execution event to get the IEvaluationContext and extract information out of it. This mostly works well, and you end up treating the IEvaluationContext as a map.

In e4 we're looking IEclipseContext "Contexts: the service broker" which provides a localized view into the application state (i.e. what you can see from your view or editor makes sense). Information relevant to the state of the application is stored in your context in 2 forms. Some information is identified by class (ECommandService.class.getName(), EventAdmin.class.getName()) and some information is identified by name (selection, activePart).

Now the fun part. In e4 a handler for a command can define 2 methods:
  • canExecute(*) - This will be called by the framework before trying to execute a command
  • execute(*) - This will be called by the framework when appropriate (can you guess what it does? :-)
But in e4 you don't need to extract your variables from your context, you can ask for them directly:

public boolean canExecute(
@Named("selection") IStructuredSelection selection);
public void execute(
@Named("selection") IStructuredSelection selection,
Shell shell);

When the framework needs to call either method, it will inject the needed information from the appropriate IEclipseContext. It uses the @Named annotation if it is present, or the type of the parameter (for example, Shell.class.getName()) if @Named is not.

In e4 1.0 we're adopting the JSR 330 annotations for injection (plus a few extras that we need). This is still a work in progress, but it's moving right along. If you are interested, now is the time to get involved - http://wiki.eclipse.org/E4.

There's a lot more to this story, so stay tuned for the next update.

PW

Tuesday, October 06, 2009

Continuous integration of Platform UI tests

With the work I was doing for e4 builds I've had the opportunity to set up a PDE build and test environment. So I thought I would setup the Platform UI automated tests to run in a continuous build cycle on a machine in the lab.

I think it's interesting because:
  1. The tests need to run against the latest I build for that week
  2. I need to patch 2 features to install my plugins from HEAD. I just followed Andrew's instructions.
  3. I wanted to use p2 to configure the tests, not the Automated Test Framework
As there's no real packaging or reuse involved, I just set up 3 features to drive the build. My 2 patch features, which include all of the Platform UI plugins that need to be replaced in org.eclipse.rcp and org.eclipse.platform. And my test feature, which includes the 2 patch feature, all of the Platform UI test plugins that contain my automated tests, and the automated test framework plugins:
  • org.eclipse.core.tests.harness
  • org.eclipse.test.performance
  • org.eclipse.test
  • org.eclipse.ant.optional.junit
Setting up PDE build involves copying the headless build templates (build.properties, customTargets.xml) and setting the parameters to match your build environment. Then creating a map project that's pointing to the correct features/plugins, and away you go.

For continuous builds you tend to want to build from HEAD. The best way (especially if you have a real map that's is regularly tagged for something else) is to set to properties:

forceContextQualifier=v${buildId}
fetchTag=HEAD

That way you don't end up with a lot of plugins that look like com.example.plugin_HEAD :-)

I also want to generate a p2 repo so I can consume my build easily. That's the standard set of p2 properties at the bottom of the build.properties file.

Once that's up (and a few tweaks to the customTargets.xml to use the latest I-build I will provide) and then you can build with the org.eclipse.ant.core.antRunner application.

Now I want to use p2 to run my automated tests. First I had to use XSLT to fix the content.xml and relax the feature patch version ranges (feature patches will target a very specific feature qualifier combination). There's a potential p2 fix in the works, but simple XSL will do for now.

I'm running the automated tests in my postBuild, with modified code I swiped from the Eclipse SDK automated test framework. The difference is that instead of having to generate a test.properties with the correct test plugin to test plugin version mapping and trying to unzip the build plugins into the eclipse test instance, the setup to run the tests now involve:
  1. unzip the eclipse test instance
  2. Call the director to install the plugin under test, the support plugins, and the patch features
  3. call the plugin's test.xml
It manages all of the version numbers and dependencies for free, so your list of IUs looks like:

-installIUs ${testPlugin}, org.eclipse.test, org.eclipse.ant.optional.junit, org.eclipse.ui.test.platform.patch.feature.group, org.eclipse.ui.test.rcp.patch.feature.group

And now I have my headless automated test environment :-) The relevant build files and map files can all be found in CVS.

  • :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse
  • platform-incubator/ui/org.eclipse.ui.automated.build/org.eclipse.ui.releng
There are a lot of opportunities for improvement in this environment (continually updating the build target from http://download.eclipse.org/eclipse/updates/3.6-I-builds for example) but this is enough to get me started.

PW

Tuesday, August 18, 2009

Eclipse again

This last year has been very busy, working to finish Eclipse 3.5 and getting the Eclipse e4 Tech Preview into shape. Over the next few months, I hope to expound on some of the new techniques that can be used with Commands, Handlers, and Menu contributions as of 3.5

But really, I'll post something this time :-)

PW

Monday, October 30, 2006

Place command in menus in eclipse 3.2

In eclipse 3.2, there is good support for keybindings, commands, and handlers. There are some snippets available at Platform Command Framework.

But it's not obvious how to link your command+handler implementation to a menu or toolbar item. The <action/> elements have a definitionId attribute that can link the action to a command, but that allows the action's IActionDelegate to be linked to the keybinding (through the command). It turns the action's IActionDelegate into a handler for keybinding purposes through the use of an ActionHandler proxy class.

But what if you would prefer to implement your commands and handlers, and then just have the command called from your menus the same way it is through keybindings.

As it turns out, there is public API that will allow you to create a GenericCommandActionDelegate. I have an example available on the Platform Command Framework page.

The implementation takes advantage of the fact that the action element is created through IConfigurationElement#createExecutableExtension(*), which means that your action delegate can implement IExecutableExtension and receive extra configuration information from the plugin.xml.

It's not a perfect solution for 3.2 (there would need to be more work to support something like checked state or radio buttons), but it will allow you to focus on creating command and handlers, and then using the action based extensions to execute your commands.

The same strategy could be used for 3.1 as well, although some modifications would have to be made. In 3.1, you have to build your own ExecutionEvent and call the Command#execute*(*) method yourself.

PW