In late April 2021 I had to install a Java development environment in my Ubuntu 20.04 machine and be able to use my Ant/Ivy-based build machinery with local IVY repositories. What follows is a writeup of how I proceeded.
The Java 11 runtime was already installed so I only needed to install the JDK with:
sudo apt install javacThis installed JDK 11
It should be noted that in the previous Java developement environment I had on the T460 ThinkPad machine I had Java 8. This caused a problem further down the road (read on).
I then installed Ant with:
sudo apt install antThat gave me version 1.10.7 (instead of 1.9.6 I had on T460)
The next question was how to install Ivy. Here, I tried various approaches and even though some initial ones were successful, in the end I used a different method. As documented, Ivy is simply a jar and it can be installed in two ways:
Furthermore, in Ubuntu 20 there's also a third way:
sudo apt install ivywhich places the ivy.jar under /usr/share/java
Next, I had to install some auxiallary Ant tasks (ant-contrib) which is some sort of abandonware available at: sourceforge. This project only makes the sources available (not the compiled library) and I had some issues getting the code to compile. Fortunately, once I copied over the /proj/DS.ots directory as mentioned above, this point too became moot as the ant-contrib library is available there too, as well, as an OTS dependency.
I then had to edit the various build.xml scripts as in the T460 laptop they used absolute paths to reach the home of the archetype-java-ant-ivy/build-system-machinery whereas in the P320 desktop I am using relative paths. In the same spirit, I also had to change certain symlinks to use relative, instead of absolute, paths (the latter never being a good idea).
In the save vein I had to change some hardcoded executables, e.g. the javac.executable property in file /repos/prj/archetype-java-ant-ivy/build-system-machinery/build-env.properties
Finally, another issue I had to solve was that in Java 11 the javax.activation package is not available (it has actually been retired since Java 9). As such I was getting a compilation error in the ~/repos/prj/js-react-leaflet/common project. The solution was found in SO. To implement the SO solution I had to add the following ivy dependency in the ivy,xml file of the above-mentioned project.
<dependency org="com.sun.activation" name="javax.activation" rev="1.2.0" conf="compile-time->master"/>
<property name="enable-ibiblio" value="true"/>
After the above modifications, I was able to fully build the js-react-leaflet project. However, I also wanted to recreate the entire LFSBIR from scratch as opposed to copying it over from the ThinkPad T460 machine and also include this new dependency there (in the LFSBIR) as opposed to setting the enable-ibiblio property to true. To that end I modified file ~/repos/prj/archetype-java-ant-ivy/ivy-importation-to-local-fs/build-ivy-install-local.xml and discoverd, in the process, that I had to further retrieve a couple of more additional packages there. During this process I also changed the properties in the /data/asc3/mperdike/ directory to make it writable. I was then able to fetch a fresh /data/asc3/mperdike/local-ivy-repository and I verified that its size profile was similar to the previous one.
I then removed the enable-ibiblio property from the build.xml file in the ~/repos/prj/js-react-leaflet/common project , cleaned the Ivy cache, re-build everything (including the sibling rest-backend project and so successfully concluded this process.
When trying to implement the new Java-based IACS backend I ended up adding:
<dependency org="javax.activation" name="activation" rev="1.1" conf="compile-time->master"/>
… which also worked. Initially I had tried to use rev="1.1.1" (from some other SO answer I won't bother linking to) but that failed as that exact version was not present in the LFSBIR.
The following worked like a charm in June, 2019
$ sudo apt-get install openjdk-8-jdk $ sudo apt-get install openjdk-8-jre $ sudo update-alternatives --config java $ sudo update-alternatives --config javac
With hindsight it was probably uncessary to install the JRE once I had already installed the JDK but then again it didn't hurt either.
NB: even after updating the alternatives, the JAVA_HOME environment in my shell was:
$ echo $JAVA_HOME /usr/lib/jvm/java-8-oracle… the above was on account of a file in /etc/profile.d with the following content:
$ cat /etc/profile.d/jdk.sh export J2SDKDIR=/usr/lib/jvm/java-8-oracle export J2REDIR=/usr/lib/jvm/java-8-oracle/jre export PATH=$PATH:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin export JAVA_HOME=/usr/lib/jvm/java-8-oracle export DERBY_HOME=/usr/lib/jvm/java-8-oracle/db
Java JAR files may be compared with the awesome package change analyzer pkgdiff — which in fact can be used to visualize changes in any kind of Linux software package.
$ pkgdiff dist/cxcaccount-1.0.0.war dist.safe/cxcaccount-1.0.0.war reading packages ... comparing packages ... creating changes report ... result: CHANGED (0.01%) see detailed report: pkgdiff_reports/cxcaccount/1.0.0_to_1.0.0/changes_report.html $ echo $? 1
While compiling (using Ant) on a machine with 12 cores I encountered the following failure while running ant build:
[javac] # There is insufficient memory for the Java Runtime Environment to continue. [javac] # Cannot create GC thread. Out of system resources. [javac] # An error report file with more information is saved as: [javac] # /tmp/jvm-21655/hs_error.log
Following the advice given in this SO answer I did:
export _JAVA_OPTIONS="-XX:ParallelGCThreads=2"NB: there is, indeed, an underscore at the beginning. The name of the variable is _JAVA_OPTIONS, not JAVA_OPTIONS. Read up more on this (undocumented) environment variable here.
That fixed it. Apparently the JVM was creating too many garbage collection threads (because of the high number of cores) and that caused the memory exhaustion.
On August 2016 I followed the following procedure (source mutatis mutandis) to install Oracle Java 1.7 after I had downloaded from the Oracle site file jdk-7u80-linux-x64.tar.gz
NB: I have copied the below instructions from the article linked above and I haven't replaced all filenames and version names, but the procedure should be obvious.
tar xvfz jdk-7u80-linux-x64.tar.gz… the JDK 7 package is extracted into ./jdk1.7.0_75 directory.
sudo mkdir -p /usr/lib/jvm sudo mv ./jdk1.7.0_75 /usr/lib/jvm/
sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0_75/bin/java" 1 sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0_75/bin/javac" 1 sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0_75/bin/javaws" 1This will assign Oracle JDK a priority of 1, which means that installing other JDKs will replace it as the default. Be sure to use a higher priority if you want Oracle JDK to remain the default.
sudo chmod a+x /usr/bin/java sudo chmod a+x /usr/bin/javac sudo chmod a+x /usr/bin/javaws sudo chown -R root:root /usr/lib/jvm/jdk1.7.0_75
sudo update-alternatives --config javaYou will see output similar to the one below - choose the number of jdk1.7.0_75 - for example 3 in this list (unless you have have never installed Java installed in your computer in which case a sentence saying "There is nothing to configure" will appear):
$ sudo update-alternatives --config java There are 3 choices for the alternative java (providing /usr/bin/java). Selection Path Priority Status ------------------------------------------------------------ * 0 /usr/lib/jvm/java-6-oracle1/bin/java 1047 auto mode 1 /usr/bin/gij-4.6 1046 manual mode 2 /usr/lib/jvm/java-6-oracle1/bin/java 1047 manual mode 3 /usr/lib/jvm/jdk1.7.0_75/bin/java 1 manual mode Press enter to keep the current choice[*], or type selection number: 3 update-alternatives: using /usr/lib/jvm/jdk1.7.0_75/bin/java to provide /usr/bin/java (java) in manual modeRepeat the above for:
sudo update-alternatives --config javac sudo update-alternatives --config javaws
jinfoNB: if the process was run as a service you need to
sudo
that.
$ sudo jinfo 1752 | grep arch.data.model Attaching to process ID 1752, please wait... Debugger attached successfully. Server compiler detected. JVM version is 24.80-b11 sun.arch.data.model = 32
java
program is the 32 or 64 bitSystem.getProperty("sun.arch.data.model")
java -d64 -version(will display a message if the version is not 64-bit)
java -d32 -version(will display a message if the version is not 32-bit)
java
and javac
javac
should play with either 32 or 65-bit java
.
netstat -a | grep TIME_WAIT | wc -lIn my case I was met with an NoRouteToHostException exception (actual trace at the end of this note) when the count exceeded 20,000 or so. This is more or less consistent with the available port range in my machine which was reported as:
$ cat /proc/sys/net/ipv4/ip_local_port_range 3276861000Apparently the solutions to this problem fall in four categories:
echo "1" >/proc/sys/net/ipv4/tcp_tw_reuse echo "1" >/proc/sys/net/ipv4/tcp_tw_recycle
Exception in thread "main" _int.esa.esavo.common.ShowStopper: org.postgresql.util.PSQLException: The connection attempt failed. at _int.esa.esavo.dbbusiness.Statistics.repoNameFromIvoid(Statistics.java:216) at _int.esa.esavo.dbbusiness.RepoNameFromIvoidTransformer.transform(RepoNameFromIvoidTransformer.java:54) at _int.esa.esavo.dbbusiness.RepoNameFromIvoidTransformer.transform(RepoNameFromIvoidTransformer.java:18) at _int.esa.esavo.consoleutils.valgraph.VALGRAPHStatsPlot.transform2(VALGRAPHStatsPlot.java:419) at _int.esa.esavo.consoleutils.valgraph.VALGRAPHStatsPlot.produce(VALGRAPHStatsPlot.java:255) at _int.esa.esavo.consoleutils.valgraph.VALGRAPHStatsPlot.produce(VALGRAPHStatsPlot.java:180) at _int.esa.esavo.consoleutils.esaknife.MonthlyReports.produceStatisticsByResource(MonthlyReports.java:234) at _int.esa.esavo.consoleutils.esaknife.MonthlyReports.produceStatistics(MonthlyReports.java:194) at _int.esa.esavo.consoleutils.esaknife.MonthlyReports.produceReports(MonthlyReports.java:184) at _int.esa.esavo.consoleutils.esaknife.MonthlyReports.produce(MonthlyReports.java:136) at _int.esa.esavo.consoleutils.esaknife.EsavoSwissKnife.main(EsavoSwissKnife.java:78) Caused by: org.postgresql.util.PSQLException: The connection attempt failed. at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:150) at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) at org.postgresql.jdbc2.AbstractJdbc2Connection.(AbstractJdbc2Connection.java:125) at org.postgresql.jdbc3.AbstractJdbc3Connection. (AbstractJdbc3Connection.java:30) at org.postgresql.jdbc3g.AbstractJdbc3gConnection. (AbstractJdbc3gConnection.java:22) at org.postgresql.jdbc4.AbstractJdbc4Connection. (AbstractJdbc4Connection.java:30) at org.postgresql.jdbc4.Jdbc4Connection. (Jdbc4Connection.java:24) at org.postgresql.Driver.makeConnection(Driver.java:393) at org.postgresql.Driver.connect(Driver.java:267) at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38) at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582) at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148) at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106) at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044) at _int.esa.esavo.dbbusiness.BaseDBFacade.getConnection(BaseDBFacade.java:16) at _int.esa.esavo.dbbusiness.Statistics.repoNameFromIvoid(Statistics.java:172) ... 10 more Caused by: java.net.NoRouteToHostException: Cannot assign requested address at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at java.net.Socket. (Socket.java:425) at java.net.Socket. (Socket.java:208) at org.postgresql.core.PGStream. (PGStream.java:62) at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) ... 25 more
~/.ant/lib/ivy-2.3.0.jar
).
build.xml
file:
<project xmlns:ivy="antlib:org.apache.ivy.ant" name="test" default="test-ivy"> <target name="test-ivy"> <ivy:settings /> </target> </pre> </project>… fails with
$ whoami voops $ ant Buildfile: build.xml test-ivy: BUILD FAILED /home/voops/test/build.xml:7: Problem: failed to create task or type antlib:org.apache.ivy.ant:settings Cause: The name is undefined. Action: Check the spelling. Action: Check that any custom tasks/types have been declared. Action: Check that any/ declarations have taken place. No types or tasks have been defined in this namespace yet This appears to be an antlib declaration. Action: Check that the implementing library exists in one of: -/usr/share/ant/lib -/home/voops/.ant/lib -a directory added on the command line with the -lib argument Total time: 0 seconds
<taskdef uri="antlib:org.apache.ivy.ant" resource="org/apache/ivy/ant/antlib.xml" classpath="${user.home}/.ant/lib/ivy-2.3.0.jar"/>… resulting in the following
build.xml
file that "builds" correctly:
<project xmlns:ivy="antlib:org.apache.ivy.ant" name="test" default="test-ivy"> <taskdef uri="antlib:org.apache.ivy.ant" resource="org/apache/ivy/ant/antlib.xml" classpath="${user.home}/.ant/lib/ivy-2.3.0.jar"/> <target name="test-ivy"> <ivy:settings /> </target> </pre> </project>Since Ant is supposed to look in
~/.ant/lib
for libraries in any case (and the error message indicates as such), it is
a mystery why the location of the Ivy jar has to be explicitly spelled out. See also my SO question
on the matter. At any rate, as noted in my question, this problem does not appear with Ant versions 1.8.2 and 1.9.4.
$ ls /usr/share/ant/lib/ivy-2.2.0.jar -rw-r--r-- 1 root root 947592 Jun 5 2013 /usr/share/ant/lib/ivy-2.2.0.jar
$ ls ~/.ant/lib/ivy-2.3.0.jar -rw-r--r-- 1 mperdikeas users 1222059 Jan 10 2013 /home/mperdikeas/.ant/lib/ivy-2.3.0.jar
#!/usr/bin/env bash EXPR=/usr/bin/expr /bin/false START=$(/bin/date +%s) RESULT=0 COUNTER=0 while IFS= read -r -u3 -d $'\0' file; do echo -n "building $file ..." ant -f "$file" build > "$file.log" 2>&1 && echo "success" || { echo "failure" && cat "$file.log" ; RESULT=1 ; break; } COUNTER=$((COUNTER + 1)) done 3< <(find a b c d -iname build.xml -print0) END=$(/bin/date +%s) DIFF=$($EXPR $END - $START) if [ $RESULT -eq 0 ] then echo "! build successfully concluded for all $COUNTER projects in $DIFF seconds" else echo "X build failed after $COUNTER projects" fisource
jstack -l 30600... should give you a pretty good idea of where the performance bottleneck is. SO posts: here and here.
jps... to also print the arguments passed to their main method use:
jps -p
java -d64 -version
ant_mod
in ~/tools
which echoes the java
executable used to spawn ant
ant -verbose
ant -diagnostics | grep java.vm
sudo add-apt-repository ppa:webupd8team/java sudo apt-get update sudo apt-get install oracle-java7-installer
apt-get update/install
part) and it
smoothly updated Java from 7u40 to 7u60. Also, after the update, the installation directory remained the same
(/usr/lib/jvm/java-7-oracle), so no updating of JAVA_HOME, PATH and CLASSPATH was required.
sudo cp ivy-2.2.0.jar /usr/share/ant/lib/
cd ~/esavo-reg/playground/ivy/02-install ant install
sudo apt-get install ivy... the ivy.jar doesn't show up where it's supposed to (/usr/share/ant/lib) but is instead found in folders like /usrshare/doc/ivy and /usr/share/java/. So one has to do a:
sudo ln -s -T /usr/share/java/ivy.jar /usr/share/ant/lib/ivy.jar
sudo mkdir -p /usr/lib/jvm/jdk1.7.0
sudo mv/* /usr/lib/jvm/jdk1.7.0/
sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 1 sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 1 sudo update-alternatives --install "/usr/bin/javap" "javap" "/usr/lib/jvm/jdk1.7.0/bin/javap" 1 sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0/bin/javaws" 1 sudo update-alternatives --install "/usr/bin/jar" "jar" "/usr/lib/jvm/jdk1.7.0/bin/jar" 1 sudo update-alternatives --install "/usr/bin/wsimport" "wsimport" "/usr/lib/jvm/jdk1.7.0/bin/wsimport" 1 sudo update-alternatives --install "/usr/bin/xjc" "xjc" "/usr/lib/jvm/jdk1.7.0/bin/xjc" 1
jvisualvm --openpid $(ps aux | grep java | grep tandal | awk '{print $2}'
String serverConfUrl = System.getProperty("jboss.server.config.dir") ;
javac -encoding utf-8 X.java // or utf8, UTF-8, UTF8 java -Dfile.encoding=UTF8 XEven the two settings above may not solve the problem if the font used by the terminal doesn't contain a rendering for a particular character (e.g. the 0x0222 character in the example below).
public class X { public static void main (String args[]) { System.out.println("\u0222"); } }Also, the file.encoding trick may be ok for a small console program but maybe not with a server. You may want to use a special PrintStream instead. That is because the JVM runtime does attempt to find out what character encoding the environment outside the JVM uses, and apparently in your environment it gets a native character set of something else that UTF8. So, even if you have funky UTF-8 characters in your source, Java may be able to print them out in environments with some other native character encoding, if that other encoding happens to have a code point for the same character glyph. For example, source code with UTF-8 may contain the byte sequence [0xc3, 0xa4], signifying lower-case a-diaeresis character glyph. Now, if that source code is compiled properly, letting the compiler know that the source is in UTF-8 character set, and subsequently the code is run in an environment with ISO-8859-1 character set, the program will output just one byte, 0xE4. Also, if the same code is run in an environment configured for plain US-ASCII character set, it will output only a question mark (as US-ASCII character set does not have a glyph for the a-diaeresis character). See here for the full discussion.
ANT_OPTS -Dfile.encoding=UTF8 JAVA_TOOL_OPTIONS -Dfile.encoding=UTF8
echo repo-wide-libs/*jar | sed 's, ,:,g'Alternatively one can use tr instead of sed:
java -cp $(echo lib/*.jar | tr ' ' ':') ..
<compilerarg line="-encoding utf-8"/>(see examplary java-se build.xml in archetypes/ant)
On some occassions I've also used:
<compilerarg line="-encoding ISO-8859-1"/>
sudo add-apt-repository ppa:webupd8team/java sudo apt-get update sudo apt-get install oracle-java7-installerAfter the installation finishes, if you wish to see if it was successful, you can run the following command:
java -versionIt should return something like this:
java version "1.7.0_04" Java(TM) SE Runtime Environment (build 1.7.0_04-b20) Java HotSpot(TM) Server VM (build 23.0-b21, mixed mode)