when building from source
Step-by-step guide
Install the tools
- Maven 3
- Java SE JDK 8
- Subversion Client (command-line or UI-based, like TortoiseSvn)
- Eclipse IDE (Kepler 3.8 or earlier, must be RCP Development Edition)
Set up your directories
Linux
$ cd ~ $ mkdir -p marketcetera/workspaces/base/code
Windows
Create the same directory structure from your root device, like C:\metc\workspaces\base\code. Keep it as short as possible as there's a limitation in Windows on the length of filenames.
Download the source
$ cd ~/marketcetera/workspaces/base/code $ svn co https://source.marketcetera.org/root/trunk public
Build the source
$ cd metc $ mvn -DskipTests clean install eclipse:eclipse
Start Eclipse
- Set your workspace to marketcetera/workspaces/base
- Set the target platform (Windows->Preferences->Plug-in Development->Target Platform)
- Select Add to create a new target platform
- Start with Nothing and click Next
- Name your target Marketcetera and click Add
- Choose Directory and click Next
- Browse to public/photon/maven/rcptarget/photon-target/eclipse and click OK
- Click Finish
- Click Finish
- Select the Marketcetera platform as the active platform and click OK
- Import projects (File->Import->General->Existing Projects into Workspace). Set the root directory to your top level code directory (marketcetera/workspaces/base/code/metc). Check Search for nested projects and select Finish. A popup may appear right away that says "build error". Just close this and ignore it. The code should build cleanly at this point.
Troubleshooting
If you have trouble compiling, check these items:
- Make sure you've built with Maven from the command-line successfully
- Make sure you start Eclipse with a Java 7 JRE/JDK
- Make sure you've included 'eclipse:eclipse' as one of the Maven goals (along with install)
- Make sure you've defined your Maven executable properly in Eclipse. You can double-check this by opening the Navigator view and running the Maven builder under the core project. This should dump Maven output to the Console View.
- Make sure you checked the 'search for nested projects' box when importing projects. There should be 45 projects when you're done.
49 Comments
Ralf Müller
So Eclipse is not optional anymore?
Colin DuPlantis
If you're going to build the UI component, Photon, then Eclipse is, more or less, not optional. Theoretically, everything, including Photon, which is based on Eclipse RCP, can be built from the command-line. However, the magnitude of pain involved makes it not worth pursuing, IMO. It's just going to be a lot easier to use Eclipse.
If you want to skip the Photon projects and just build the strategy engine and other server-side components, then Eclipse is entirely optional.
vinod verma
@Colin
Can you let me know how to use it compile and run photon from command line?
For some reasons i dont want to use eclipse.
Colin DuPlantis
You cannot build Photon without using Eclipse. It would be possible to build from the command line with P3, but you're still using Eclipse. Photon is literally built on top of Eclipse, so you can't get away from it.
Hadi
Hi everyone
Thanks for your great step-by-step guide. I tried to build Marketcetera 2.4.0 on Ubuntu 13.10. My mvn -v is:
Apache Maven 3.0.4
Maven home: /usr/share/maven
Java version: 1.7.0_51, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-openjdk-i386/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.11.0-18-generic", arch: "i386", family: "unix"
I passed all the troubleshooting's item but the 4 one! After I run copyTestsToPlugin.launch I get:
The file does not exist for the external tool named copyTestsToPlugin.
Also for item 5 I didn't have any "search for nested projects" option and it searched all nested projects(I hope). I think because my eclipse is old(3.8.1).
My first question is How can I pass this error and the second one is, as long as I'm new in java world, How can I run Marketcetera and Photon?
Colin DuPlantis
I think the "search for nested projects" option shows up only if you have m2e installed, which isn't required. As long as you have the right number of projects (45), you're fine. As for the external build thing, this seems to be something glitchy in Eclipse. Open the Navigator View,click on core to expand, click on .externalToolBuilders to expand, right-click on copyTestsToPlugin.launch and select run as copyTestsToPlugin. It should work ok from now on.
Once that works, run Photon and enjoy. Expand the org.marketcetera.photon.feature project and click on photon.product. Select the Launch an Eclipse application link. Use user/password to login.
There's an issue in trunk right now we're tracking down. It will take a couple of minutes to complete the login.
GeekySerge
Just a quick add to the guide.
In case you've never used Maven with Eclipse you need to set up the local Maven repository.
From Eclipse:
Window -> Preferences
Java -> Build Path -> Classpath Variables -> New
name will be "M2_REPO"
path will be:
Unix/Mac OS X: ~/.m2
Windows: C:\Documents and Settings\{your-username}\.m2
Click the OK button twice
dar nin
thx for your useful advice, but i think the path should be:
windows: C:\Documents and Settings\{your-username}\.m2\repository
Arul Krishnamoorthy
I am unable to checkout the code. Seeing the following message. Is the URL still valid?
C:\>cd metc\workspaces\base\code
C:\metc\workspaces\base\code>svn co http://source.marketcetera.org/root/trunk metc
svn: E175002: Unable to connect to a repository at URL 'http://source.marketcetera.org/root/trunk'
svn: E175002: OPTIONS request on '/root/trunk' failed: 504 Connection Timed Out
Colin DuPlantis
Valid and confirmed. Seems like a transient problem or a firewall issue on your end.
Hadi
Hi
Thanks for your hints. I can run copyTestsToPlugin.launch successfully, but photon.product is full of errors right now, I guess because it's under development. So, after learning how to use Subversion, I decided to compile 2.2-bugfix branch. By running:
mvn -X -DskipTests clean install eclipse:eclipse
give me these errors:
[ERROR] Error fetching link: http://commons.apache.org/cli/api-release/package-list. Ignored it.
[ERROR] Error fetching link: http://commons.apache.org/collections/api-release/package-list. Ignored it.
[ERROR] Error fetching link: http://commons.apache.org/io/api-release/package-list. Ignored it.
[ERROR] Error fetching link: http://commons.apache.org/lang/api-release/package-list. Ignored it.
and
[ERROR] Failed to execute goal org.codehaus.mojo:hibernate3-maven-plugin:2.1:hbm2ddl (default) on project core: Could not get ConfigurationTask. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:hibernate3-maven-plugin:2.1:hbm2ddl (default) on project core: Could not get ConfigurationTask.
As I googled, it seems this branch is too old. I tried to find out right versions by comparing pom.xml from this branch and trunk but didn't solve it. How can I fix these?
Colin DuPlantis
If you follow the instructions properly, the trunk builds and tests fine. I'm not sure what you mean by "photon.product is full of errors right now". Let's try a little harder to figure out where you've strayed from the path.
As for 2.2.x, the first batch of issues are because the doc links have changed for those sources. Like Maven, you can can ignore it. The second error is because you're using Maven3. For 2.2.x, you need to use Maven2. It's because of the DDL plugin we used at the time.
Hadi
Using JAVA_OPTS="-Xms1024m -Xmx2048m -XX:MaxPermSize=2048m" I compiled mvn -DskipTests clean install eclipse:eclipse successfully. Also copyTestsToPlugin.launch runs successfully. But launching photon.product gives me this error:
!MESSAGE Exception launching the Eclipse Platform:
!STACK
java.lang.ClassNotFoundException: org.eclipse.core.runtime.adaptor.EclipseStarter
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:626)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584)
at org.eclipse.equinox.launcher.Main.run(Main.java:1438)
at org.eclipse.equinox.launcher.Main.main(Main.java:1414)
revision: 16886
Colin DuPlantis
How are you launching Photon? Did you click on the Photon feature, select photon.product, and then click the link, "Launch an Eclipse Application"?
Hadi
yes, exactly! Is it possible svn cannot copy trunk? My internet connection is not so fast!! Maybe some information is missing! How can I check that? Is there any other tests that I can run? I have a plan to upgrade Ubuntu soon and reinstall everything! Maybe that solves the problem Thanks.
Colin DuPlantis
I don't know how to diagnose any internet problems you might be having. From the command line, try saying, "svn up" and see if that helps. Also, try saying, "svn stat". That should show no modified files.
Here's another thought: the error you're seeing might happen if you deleted some of the Photon workspace. Try recreating it. Select Run->Run Configurations in Eclipse. Delete everything under the "Eclipse Application" category. Rerun from the "org.marketcetera.photon.feature" step.
Arul Krishnamoorthy
will the following SVN checkout
svn co http://source.marketcetera.org/root/trunk metc
give the latest code (pertaining to version 2.4) or is it still version 2.2 code that is available in SVN?
Colin DuPlantis
Trunk contains 2.4.0 (right now). Later, it will probably be something else.
Manjunathan
Hello Arul Krishnamoorthy, Could you provide me your email...i have few basic doubts to ask to start with Marketcetra...Thanks
Colin DuPlantis
Post your questions here. That way, everybody can benefit.
Raj Ghosh
Is MarketCetera 2.2 still available in the SVN repository. I accidentally updated my SVN, and have upgraded to 2.4.Which revision do I need to revert to to return to Marketcetera 2.2? ... Thanks
Colin DuPlantis
It sounds like you are opposed to progress! Nonetheless, if you insist, have a look here.
Raj Ghosh
Thanks Colin. Also, would you be able to put in the separate instructions that were there for the mysql setup for the version 2.2 somewhere in the building from source page, so that we can try to build from source older versions if necessary.
Thanks.
Cui Yunchuan
[INFO] --- maven-compiler-plugin:2.0.2:testCompile (default-testCompile) @ photo n-bundle ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.4.3:test (default-test) @ photon-bundle ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-bundle-plugin:2.3.5:bundle (default-bundle) @ photon-bundle ---
[ERROR] Java heap space -> [Help 1]
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2271)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
at aQute.lib.osgi.EmbeddedResource.copy(EmbeddedResource.java:62)
at aQute.lib.osgi.EmbeddedResource.collect(EmbeddedResource.java:51)
at aQute.lib.osgi.EmbeddedResource.build(EmbeddedResource.java:33)
at aQute.lib.osgi.EmbeddedResource.build(EmbeddedResource.java:71)
at aQute.lib.osgi.Verifier.getBundleClassPath(Verifier.java:254)
at aQute.lib.osgi.Verifier.<init>(Verifier.java:203)
at aQute.lib.osgi.Builder.doVerify(Builder.java:424)
at aQute.lib.osgi.Builder.build(Builder.java:101)
at org.apache.felix.bundleplugin.BundlePlugin.buildOSGiBundle(BundlePlugin.java:480)
at org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:344)
at org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:261)
at org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:252)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
Cui Yunchuan
Hi Guys, When i compiling the photon, always get this OOM, no matter how big memory I set for mvn.bat, does anyone have an idea to solve it? thank you very much.
this is my config:
set MAVEN_OPTS=-Xms1024m -Xmx1024m -XX:PermSize=256M -XX:MaxPermSize=512M
Colin DuPlantis
When you say, "compiling Photon", I assume you mean that you're running from Eclipse. If so, you need to set the amount of memory Eclipse uses. Try this.
Alex Bodn
hello friends,
as a foreign user, i might be expected to appologize about my english.
but my java is even worse than my english, so please bear with my timid question:
i could apparently perform the build process, (btw the sting to substitute had to be maven_exec).
but i have no idea how to run photon, dare and the market data.
could you please point me to the right documentation?
thanks in advance for any advice,
alex
Colin DuPlantis
DARE is already running for you in the cloud - you just connect to it. Photon and the Strategy Engine (for market data) will run locally.
If you've just built from source, to run Photon, find the project called "org.marketcetera.photon.feature" in Eclipse. Expand it, click/double-click on the "photon.product" item. Select "Launch an Eclipse application".
For the Strategy Engine:
hammer
Hi,
It seems there is no such "strategyengine" directory downloaded from the source(
http:
//source.marketcetera.org/root/trunk
). Not sure if this is a recent change in the svn repository. Please kindly advise how to get thestrategyengine
downloaded. thanks.Colin DuPlantis
In the code, it's still called
strategyagent
but it's the same thing.hammer
Colin,
It works.
thank you.
Best Regards,
Hammer
OpenBMS JSC
Hi,
When checkout the source as describe above, I ran into problem below
svn: E155009: Failed to run the WC DB work queue associated with '/media/ubuntu-studio/9771-802B/Recovery-DiskD/Projects/metc/util/src/test/java/org/marketcetera/util/ws/stateless', work item 4194 (file-install util/src/test/sample_data/deleter/templates/unix/dir/dir_link 1 0 1 1)
svn: E000001: Can't create symbolic link '/media/ubuntu-studio/9771-802B/Recovery-DiskD/Projects/metc/util/src/test/sample_data/deleter/templates/unix/dir/dir_link.tmp': Operation not permitted
Could someone shed some lights on what was wrong?
Thanks in advance!
Phong
Colin DuPlantis
Most likely a toxic combination of FAT/VFAT and Ubuntu. Use a Linux-native filesystem under Ubuntu or a Windows-native filesystem under Windows.
Cihan T.
Hi,
I applied all steps but Eclipse showing a "Plug-in Problem". The main problem is "Bundle 'org.kxml' cannot be resolved" and other related errors. I tried on win and linux but its same. I installed Eclipse Luna (4.4.0) both of them. How can I resolve this problem?
Thank you,
Cihan
Colin DuPlantis
I've found that the most recent version of Eclipse isn't compatible and we haven't converted yet. Try Kepler, that's what I use.
Cihan T.
Hi Colin,
I built finally. Thank you. But when I run photon project as eclipse application I can't see photon ui, it coming as only empty eclipse welcome screen. How can I run? Thank you for your quick support.
Best,
Cihan
Danil Krivopustov
Hi!
There are some issues with showing images in the build instruction.
For example in "Define a String Substitution that points to your Maven executable (Window->Preferences->Run/Debug->String Substitution)" images shows as "Unknown Attachment".
The same problem with Troubleshooting guide at the 4 point.
Could you please help with it or just tell what name should have this substitution rule?
Thanks,
Danil.
Simon Rykers
Not sure what Im doing wrong, but I seem to have 2019 errors and 370 warnings during the build. Im using Kepler, and rest of my versions are below. Im doing this on a windows 8 machine. Is that the wrong platform for this?
Apache Maven 3.0.4 (r1232337; 2012-01-17 12:44:56+0400)
Maven home: C:\Program Files\Apache\maven
Java version: 1.7.0, vendor: Oracle Corporation
Java home: C:\Program Files\Java\jdk1.7.0\jre
Default locale: en_GB, platform encoding: Cp1252
OS name: "windows nt (unknown)", version: "6.2", arch: "amd64", family: "windows"
Colin DuPlantis
Make sure you've set your target platform as described in the instructions.
Pavithra Kuttiyandi Chandrakasu
Is Java 7 is mandatory for this setup?
Latest release is Java 8.
Colin DuPlantis
Java8 is fine
Pavithra Kuttiyandi Chandrakasu
There is no option called Plug-in-Development under Windows → Preferences
Out of available packages for eclipse Kepler which is one has to be installed?
any help is appreciated.
Colin DuPlantis
You need the RCP Development Addition of Kepler. I believe the package is called Eclipse for RCP and RAP Developers. There is a pre-built version in the downloads.
Ravi Kant
Hello Team,
I've checked out the latest version and trying to build the source ( Eclipse IDE Version : Oxygen , JDK : 1.8 ) but the code under the project "com.swtworkbench.community.xswt" is not compiled due to kxml2 library.
The original plugin dependency, in the checked out project, is "org.kxml" which is missing in the photon plugin lists and I've changed the dependency to "org.kxml2" and after this I get this access restriction error : Access restriction: The type 'KXmlParser' is not API (restriction on required library '/home/kiran/marketcetera/workspaces/base/code/metc/photon/maven/rcptarget/photon-target/eclipse/plugins/org.kxml2_2.1.5.jar')
To fix the access restriction compilation error , I've added the access rule for " org/kxml2/io/** " under project properties → Java Build Path → Plugin Dependencies.
But, when I launch the Photon FE , from eclipse IDE , if fails with KXmlParser ClassNotFound error as shown below. Could you please help me fix this issue.
!ENTRY org.eclipse.ui.workbench 4 0 2017-10-30 11:45:42.328
!MESSAGE Unable to create view ID org.marketcetera.photon.views.StockOrderTicketView: org/kxml2/io/KXmlParser
!STACK 0
java.lang.ClassNotFoundException: org.kxml2.io.KXmlParser
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:501)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:421)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:412)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.swtworkbench.community.xswt.xmlparser.MinimalPullParser.initParser(MinimalPullParser.java:33)
at com.swtworkbench.community.xswt.xmlparser.MinimalPullParser.<init>(MinimalPullParser.java:19)
at com.swtworkbench.community.xswt.XSWT.create(XSWT.java:173)
at org.marketcetera.photon.views.XSWTView.createPartControl(XSWTView.java:54)
at org.eclipse.ui.internal.ViewReference.createPartHelper(ViewReference.java:386)
at org.eclipse.ui.internal.ViewReference.createPart(ViewReference.java:240)
at org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:595)
at org.eclipse.ui.internal.PartPane.setVisible(PartPane.java:315)
at org.eclipse.ui.internal.ViewPane.setVisible(ViewPane.java:534)
Many Thanks for your help.
Colin DuPlantis
Good job on trying to solve the problem, even though it didn't work out.
The short answer is that Photon is built on the RCP SDK version 3.8, which means that it needs a corresponding version of Eclipse (3.8 or earlier) to build Photon. I use Kepler or Luna.
The longer answer is that we're in the process of switching over to a web-based client. When that work is complete, in the 4.x Marketcetera version line, you'll be able to use whatever version of Eclipse, indeed, whatever IDE, you wish.
Ravi Kant
Hi Colin,
Many Thanks for the response.
I've guessed there could be an eclipse dependency and I've used Kepler and the issue with org.kxml is resolved. But, when I tried to launch photon FE in Eclipse, I had OutOfMemory exception which I resolved by setting "-XX:MaxPermSize=1024m" in the run configuration arguments.
I did not encounter OutOfMemory issue with latest Eclipse-Oxygen because I was using JDK 1.8 and it appears that this Java version does not have Permanent Generation concept and is replaced by 'metaspace' and is part of over all heap used.
Thanks again..
Colin DuPlantis
Here's what my
eclipse.ini
looks like:I define a MaxPermSize and a larger heap that you are using.
Ravi Kant
Colin,
Thanks for sharing and have taken a note of it.
Anthony Lee
What's the command line command to run dare when built from source code?