Mashing up RDF data with WSO2 Mashup Server

Okey so this is the fun part that I promised to write about :D. I managed to cook up a use-case to demonstrate RDF querying and making use of the semantic data. The data that I am using for querying, is the rdf data sources available in the UK site. With some analysis I figured out that this task can be fundamentally archived using the combination of Mashup and Gadget Technologies. My choice of tools were WSO2 Mashup Server and WSO2 Gadget Server for their great flexibility and of cause for other obvious reasons :D. However the Mashup Server does not natively support RDF data retrieval, hence I had to do some work to get such functionality integrated. The great fact about the mashup server is its extensibility, the concept of host objects and the ability to write custom host objects and its pluggable nature comes handy in such cases. The high level architecture of what I am trying to achieve is as follows.

RDF data retrival with WSO2 Mashup server / WSO2 Gadget Server

To implement the above architecture with the tools at hand I created a custom host object that can be plugged to the Mashup Server. When dealing with semantic web related tasks and RDF data handling HP’s Jena java library comes in handy. With the use of Jena-ARQ (for SPARQL) api I managed to get the host object working with few lines of code.

            Dataset dataSet = DatasetFactory.create(sparqlObject.rdfDataSource);
            // Create a new query form a given user query
            String queryString = sparqlObject.spaqrlQuery;
            Query query = QueryFactory.create(queryString);
            QueryExecution qe = QueryExecutionFactory.create(query, dataSet);
            ResultSet results = qe.execSelect();
           resultString = ResultSetFormatter.asXMLString(results);
..... OR.....
           ByteArrayOutputStream bos = new ByteArrayOutputStream();
           ResultSetFormatter.outputAsJSON(bos, results);

With the host object in place, the next task was to create a Mashup in-order to query the rdf data with a given source (EndPoint or data source). The javascript service (Mashup) is created to serve this purpose, where the consumer can specify the RDF endpoint or the data source with the SPARQL query and retrieve the dataset in XML or JSON.

function RdfDocQueryService(rdfDataSource, rdfQuery, resultType) {
   var sparqlObj = new SparqlHostObject();
   sparqlObj.rdfDataSource = rdfDataSource;
   sparqlObj.spaqrlQuery = rdfQuery;
   sparqlObj.resultType = resultType;
   return new XML(sparqlObj.getDataFromRdfSource());

Finally to bind everything together, lets try querying some data. My example usecase is to use the query at N2 blog to retrieve traffic monitoring points in UK roads. The query to retrieve the data set as follows,

#List the uri, latitude and longitude for road traffic monitoring points on the M5
PREFIX road:
PREFIX wgs84:
SELECT ?point ?lat ?long WHERE {
  ?x a road:Road.
  ?x road:number "A4"^^xsd:NCName.
  ?x geo:point ?point.
  ?point wgs84:lat ?lat.
  ?point wgs84:long ?long.

To visualize these points I have created a gadget with the aid of Google Maps api. This gadget can be hosted in the Gadget Server, where it can dynamically retrieve traffic monitoring points for each road in the UK and display them in the map as follows.

Traffic points in A4 road, UK

The Semantic Web

For the past week or so I was able to work with to find a possibility to make use of government/public data for a nice sample gadget, that will work in the Gadget Server. However it is quite disappointing to see most of these data are in lengthy PDF files, and without some expensive processing these data could not be consumed easily. The happy scenarios in my mind was, that there will be a bunch of RESTfull web services where I can request for public data and in return I will get the result set in XML or (and if am really lucky) in JSON ๐Ÿ˜‰

After some browsing over the data in the UK data site, There was one interesting area which is called SPARQL and that was my moment of joy :D. there were six government data endpoints which I can query against and the result set can be retrieved as XML or JSON. This was the ideal scenario, even the available data is limited, there it starts the story of the Semantic Web.

When I was reading for my bachelor’s thesis, under service orchestration and chirography, the concepts of semantic web (services) were discussed in length. However the idea is still in its early stages, and there is still no real business use-case for it. The very purpose of semantic web is that the resources of the WEB and there relationships can be self explained and explored.

Self Explainable RDF graph

I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web โ€“ the content, links, and transactions between people and computers. A โ€˜Semantic Webโ€™, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The โ€˜intelligent agentsโ€™ people have touted for ages will finally materialize.

โ€“ Tim Berners-Lee, 1999

The UK data site has offered chunks of eduction, transport, financial, environmental, analytical and statutory notices as RDF data endpoints which can be queried using SPARQL (SPARQL Protocol and RDF Query Language).

So why let this great opportunity to lay dry, my idea was simply to use these endpoints and query these data and display them in some gadgets, for instance with transport data we can draw traffic details in maps, or with financial and educational data we can create some graphs and charts.

But with some thought, WHY NOT go for mashing up, We have a great tool (WSO2 Mashup Server) just made for that. So I was cutting code for the past whole week to create a custom host object in the Mashup Server to query RDF (RDF Service endpoints or RDF Data sources) data, and am almost done, the host object is created, two mashup services are created (Query a RDF endpoint/and RDF data source) now its a matter of integrating the services with Google Maps API and visualize them as a gadget in the Gadget Server.

So yeah the good stuff will be in the next post. keep tuned ๐Ÿ˜€