Monday, November 3, 2008

Create a WFS 1.1 service in java

I wanted to create an OGC Web Feature Service (version 1.1) from scratch in java. When I tried to use jdk6 xjc with the OpenGIS schemas I had lots of errors so I had to create a bindings file to resolve them. It all works fine now, here are the results of my labours:

Ant fragment:

<exec executable="${xjc}"
failonerror="true"
outputproperty="xjc.output"
errorproperty="xjc.error"
logerror="true">
<arg value="-extension">
<arg value="-verbose">
<arg value="-httpproxy">
<arg value="proxy:8080">
<arg value="-b">
<arg value="${opengis}/wfs/1.1.0/wfs-bindings.xml">
<arg value="-d">
<arg value="${generated}">
<arg value="${opengis}/wfs/1.1.0/wfs.xsd">
<arg value="${opengis}/cts/craft-feature.xsd">
<arg value="${opengis}/ows/1.0.0/owsExceptionReport.xsd">
</exec>


wfs-bindings.xml:

<bindings xmlns="http://java.sun.com/xml/ns/jaxb" version="1.0" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<bindings schemaLocation="wfs.xsd" node="/xs:schema">
<globalBindings
underscoreBinding="asCharInWord"
/>
<schemaBindings>
<package name="net.opengis.schema.wfs"/>
</schemaBindings>
</bindings>
<bindings schemaLocation="../../gml/3.1.1/base/gml.xsd" node="/xs:schema">
<schemaBindings>
<package name="net.opengis.schema.gml"/>
</schemaBindings>
<bindings schemaLocation="../../gml/3.1.1/base/defaultStyle.xsd" node="/xs:schema/xs:element[@name='topologyStyle']">
<class name="topologyStyleLowerCase"/>
</bindings>
<bindings schemaLocation="../../gml/3.1.1/base/defaultStyle.xsd" node="/xs:schema/xs:element[@name='geometryStyle']">
<class name="geometryStyleLowerCase"/>
</bindings>
<bindings schemaLocation="../../gml/3.1.1/base/defaultStyle.xsd" node="/xs:schema/xs:element[@name='graphStyle']">
<class name="graphStyleLowerCase"/>
</bindings>
<bindings schemaLocation="../../gml/3.1.1/base/defaultStyle.xsd" node="/xs:schema/xs:element[@name='featureStyle']">
<class name="featureStyleLowerCase"/>
</bindings>
<bindings schemaLocation="../../gml/3.1.1/base/defaultStyle.xsd" node="/xs:schema/xs:element[@name='labelStyle']">
<class name="labelStyleLowerCase"/>
</bindings>
</bindings>
<bindings schemaLocation="../../filter/1.1.0/filter.xsd" node="/xs:schema">
<schemaBindings >
<package name="net.opengis.schema.filter" />
</schemaBindings>
</bindings>
<bindings schemaLocation="../../ows/1.0.0/owsAll.xsd" node="/xs:schema">
<schemaBindings>
<package name="net.opengis.schema.ows"/>
</schemaBindings>
</bindings>
</bindings>

Thursday, October 2, 2008

Starting a background thread in Tomcat

I have long running processes that repeatedly do something (like check a directory for new files or a database table for new records) then sleep for a while. These processes are deployed to Tomcat (giving me jsp interfaces for monitoring and any other requests).

Ideally the Tomcat container should manage this long running thread. i.e Tomcat should be aware of the background thread particularly for stopping a web application completely (stopping the background thread) but also for metrics gathering.

I implement long running threads as HttpServlets and start them in my ant build via the task which does an http get to the servlet url. We want the to return control to the build process and not wait for the long running thread to finish. The trick is to simply close the response output stream and continue. i.e.

public abstract class RunnableServlet extends HttpServlet {

private Runnable runnable;

public RunnableServlet(Runnable runnable) {
this.runnable = runnable;
}

@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
String command = req.getParameter("command");

// continue with the task but with the container aware of this thread
if ("start".equals(command)) {
response.getOutputStream().write("started");
response.getOutputStream().close();
runnable.run();
}
}
}

By the way to use the above servlet extend it!
free hit counter

Friday, August 1, 2008

Homemade Digital TV Antenna

living in O'Connor, Canberra I should get good reception from Black Mtn tower but I haven't been able to suss it for a couple of years. Reception was very patchy and for my favourite station SBS was terrible. I just couldn't get my rabbit ears to work well so I made my own antenna. I used that strip tv antenna cable that costs about a dollar per metre from Bunnings and soldered it to a coaxial end.

In detail I cut a 1.8m piece of tv antenna cable and joined the paired wires on the cable together at the ends (so I end up still a 1.8m strip but it has little u turns at the ends). Then I broke one side of the strip and twisted in a bit of copper to then solder the two ends to the coax connector (one end to the centre prong and the other to the outer ring).

Worked ok but was still having some reception problems. Luckily I stumbled upon an article that suggested you make sure there are no sharp curves in your coax cable. When I smoothed that I had flawless reception!

So the deal is you can probably make your own antenna for maybe $10 including a length of coax.

If you have rabbit ears already smooth out the run of coax, no small radius curves. Worked for me.

Friday, January 18, 2008

Start Tomcat using an Ant target

I like using a direct call to the tomcat startup class so I can avoid shell scripts:

<!-- start tomcat -->
<target name="tomcat.start">
<mkdir dir="${tomcat}/temp" />
<mkdir dir="${tomcat}/logs" />
<echo message="starting tomcat" level="info" />
<java classname="org.apache.catalina.startup.Bootstrap"
fork="true"
failonError="false"
dir="${tomcat}/bin"
spawn="true">
<classpath location="${env.JAVA_HOME}/lib/tools.jar" />
<classpath location="${tomcat}/bin/bootstrap.jar" />
<classpath location="${tomcat}/bin/commons-logging.api.jar" />
<jvmarg value="-Dprocess.name=tomcat" />
<jvmarg value="-Xmx800m" />
<jvmarg value="-XX:MaxPermSize=350m" />
<jvmarg value="-Dcom.sun.management.jmxremote.port=9003" />
<jvmarg value="-Dcom.sun.management.jmxremote.ssl=false" />
<jvmarg value="-Dcom.sun.management.jmxremote.authenticate=false" />
<jvmarg value="-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager" />
<jvmarg value="-Djava.util.logging.config.file=${tomcat}/conf/logging.properties" />
<jvmarg value="-Djava.endorsed.dirs=${tomcat}/endorsed" />
<jvmarg value="-Dcatalina.base=${tomcat}" />
<jvmarg value="-Dcatalina.home=${tomcat}" />
<jvmarg value="-Djava.io.tmpdir=${tomcat}/temp" />
<arg value="start" />
</java>
</target>

How to deploy a Web Service to Tomcat 5.5 or 6.0

This example uses JDK6.

Adapted from http://saloon.javaranch.com/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic&f=51&t=004355:

Download the latest JAXWS Release Implementation (RI) from https://jax-ws.dev.java.net/ri-download.html. Unpack its jars as per instructions.

Tomcat 5.5
  • copy the jaxws jars to the shared/lib directory in tomcat.
  • move the jaxb-api.jar in shared/lib to common/endorsed.
  • start tomcat
Tomcat 6.0
  • create a shared directory in tomcat.
  • copy the jaxws jars to that directory.
  • create an endorsed directory in tomcat. If you are running tomcat using an ant task make sure you include the option
-Djava.endorsed.dirs=${catalina}/endorsed
  • Move jaxb-api.jar from shared to endorsed.
  • Edit conf/catalina.properties and set the shared.loader property as follows:
shared.loader=${catalina.home}/shared/*.jar
War

In the war, we update the web.xml to use a special listener that enables the web service. The web.xml should look like this:
    <listener>  
<listener-class>com.sun.xml.ws.transport.http.servlet.WSServletContextListener</listener-class>
</listener>
<servlet>
<servlet-name>MyExampleService</servlet-name>
<servlet-class>com.sun.xml.ws.transport.http.servlet.WSServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>MyExampleService</servlet-name>
<url-pattern>/example</url-pattern>
</servlet-mapping>
Create a sun-jaxws.xml file in WEB-INF directory which the listener uses to configure itself with the web service you want to expose:
    <endpoints xmlns="http://java.sun.com/xml/ns/jax-ws/ri/runtime" version="2.0">
<endpoint name="MyExampleService" implementation="moten.david.example.ws.MyExample" pattern="/example"/>
</endpoints>
After starting tomcat you can access the service by:

http://localhost:8080/{war-name}/example

Tuesday, December 18, 2007

Simple authentication over https with invalid certificate

Here's the story for opening an input stream on a url via https when the site has an invalid certificate (e.g. expired). You could place the certificate in your trusted store but this is a useful shortcut!

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.net.URL;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

public class SSL {

public static class SimpleAuthenticator extends Authenticator {
private final String username, password;

public SimpleAuthenticator(String username, String password) {
this.username = username;
this.password = password;
}

@Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password.toCharArray());
}
}

public static void main(String[] args) throws IOException,
NoSuchAlgorithmException, KeyManagementException {
SSLContext sc = SSLContext.getInstance("SSL");
TrustManager trustManager = new X509TrustManager() {

@Override
public void checkClientTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
// don't throw exception
}

@Override
public void checkServerTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
// don't throw exception
}

@Override
public X509Certificate[] getAcceptedIssuers() {
return null;
}

};
sc.init(null, new TrustManager[] { trustManager },
new java.security.SecureRandom());

HostnameVerifier hostNameVerifier = new HostnameVerifier() {
@Override
public boolean verify(String arg0, SSLSession arg1) {
return true;
}
};
java.net.URL url = new URL("https://hostname/stuff");
Authenticator.setDefault(new SimpleAuthenticator("username", "password"));
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setHostnameVerifier(hostNameVerifier);
con.setSSLSocketFactory(sc.getSocketFactory());

InputStream is = con.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
br.close();
isr.close();
is.close();

}

}

Thursday, November 29, 2007

XML Schemas - Best Practices to Manage Change

I create xml schemas based on my experience processing them in an automated fashion. Here are some pointers that make things easy for an automated xml consumer and also allow more robust management of (inevitable) schema changes:

When you write code to process xml based on a schema, use objects generated from the schema. This gives you protection from schema change in as much as a change in the schema and a rebuild of your code (including the code generation of course) will then indicate needed changes in your code with compile errors. This is a tremendous help, wherever possible look for compile time indication of errors rather than runtime errors after deployment!

For Java JDK 6 use the xjc tool to generate classes from a schema. Script it up say in Ant as part of your project build.

Designing an xml schema is like designing a relational database, and involves elements of object oriented design. Know these things before embarking on schema design!

Reuse types, both simple and complex. That way when you write code with generated objects you will be able to resuse code working on particular types.

Give maximum size limits to your data types, then when you design database fields to capture this information, provide bigger database field sizes than you initially need to accomodate change (e.g. put 50 chars down as a limit for a surname in the schema but use 100 chars in the database field). Remember, disk space is cheap having the same data in a varchar(100) won't take up more space than in a varchar(50). Somewhat larger field size limits won't slow your database down either.

Use lower case hyphenated element names (<start-time>) or upper camel case (<StartTime>) in the xml. Hyphenated names are still translated into object properties like "startTime" by tools such as xjc.

If an xml element has an associated unit, include the unit in the element name (either the SI unit abbreviation, or spell it out, e.g length-feet, mass-pounds). i.e rather than <time>54</time> use <time-ms>54</time-ms> or <time-milliseconds>54</time-milliseconds>. That way when someone produces xml using generated objects they will always know how to scale the value for the required unit (without having to check documentation for the field).