Gadgets On the Cloud

There is no doubt that JavaScript/XML gadgets make a great presentation layer over the web with increasing amount of data floating around. The ability of which these gadgets can be embedded in any place over the web, provides a great flexibility, and a wider reach. Google does this quite nicely with their iGoogle gadgets, enabling the gadgets to be embedded in almost any web page. The success of this great idea, would be only logical if all the data, services and mashups are also available over the web with open access or maybe authenticated access. This is where a cloud story fits-in, and this the very reason why Google can do it quite easily.

However, what if you want to do everything from the scratch and also provide a great presentation layer for the users. For an instance, lets say you have a lot of financial data within your enterprise, and you need to provide some of these to your customers, to general public and some for your employees. To do this, you will have to create appropriate data services, maybe mediate or transform some data, integrate with some legacy data sources, create some business work flows, mashup them with some 3rd party services like Google finance or charts and finally expose the end results to the targeted user group. This is where WSO2 Stratos PaaS comes for your rescue 🙂

If your requirements are such, you will need a strong middle-ware platform to full fill all the above tasks, and if its all on the cloud, you will not have to worry about anything other than writing your business logic. Once the business logic is correctly compiled, you can Mashup some of your data with external service APIs, and then write the presentation logic purely on javascript and xml as XML Gadgets and expose them to the users you need. Once the gadgets are published on WSO2 Cloud Gadget Server its just a matter of linking them up in any web page you want over the web.

https://gadget.cloud.wso2.com/ifr?container=default&mid=3&nocache=1&country=US&lang=en&view=default&parent=https%3A%2F%2Fgadget.cloud.wso2.com%2F&debug=1&up_=undefined&st=john.doe%3Ajohn.doe%3A10717%3Ashindig%3Ahttp%253A//gadget.cloud.wso2.com%253A80/registry/resource/_system/config/repository/gadget-server/gadgets/ngeo_vid.xml%3A0%3Adefault&url=http%3A%2F%2Fgadget.cloud.wso2.com%3A80%2Fregistry%2Fresource%2F_system%2Fconfig%2Frepository%2Fgadget-server%2Fgadgets%2Fngeo_vid.xml#rpctoken=1304649864https://gadget.cloud.wso2.com/ifr?container=default&mid=0&nocache=1&country=US&lang=en&view=default&parent=https%3A%2F%2Fgadget.cloud.wso2.com%2F&debug=1&up_=undefined&st=john.doe%3Ajohn.doe%3A10197%3Ashindig%3Ahttp%253A//gadget.cloud.wso2.com%253A80/registry/resource/_system/config/repository/gadget-server/gadgets/soa.xml%3A0%3Adefault&url=http%3A%2F%2Fgadget.cloud.wso2.com%3A80%2Fregistry%2Fresource%2F_system%2Fconfig%2Frepository%2Fgadget-server%2Fgadgets%2Fsoa.xml#rpctoken=1304649864

The above two gadgets are taken from WSO2 Cloud Gadget Server and have linked in to this blog, to convince about the great flexibility and reach it can add-up. You do not need to use the Cloud Gadget Portal as the only place for your data to be presented (Of cause if you are not using other gadget server specific privileges such as inter-gadget communication etc). You can simply use the Gadget Server as your own gadget repository, and encourage users to discover the gadgets and embed them into their own web pages over the web.

To sum up the story I would say, try-out Stratos, try out the available services and you will definitely find out more use cases, and creative ways to use the platform and leverage the advantages of the cloud

Advertisements

Web Scraping & Parsing HTML to XML in Javascript

Today I was working on a customer POC and happened to create few Google gadgets to visualize selected data sets from *.gov.uk sites. The scenario which is implemented was, mixed with inter-gadget communication and content search over data.gov.uk sites. I created three simple gadgets which communicates with each other, and one acted as the controlling gadget which pushed the search parameters to other two gadgets. The two content gadgets showed UK (1) primary school information and (2) electoral information. The pushed parameter was the postal code of different parts of UK. The direct.gov.uk has a form based implementation of this.

The Requirements for the POC was, simple and we already had working samples of such a scenario at WSO2 library.

  1. Show how one gadget can pass the context to other gadgets
  2. How gadgets can harvest data in various formats (in my previous post I explained on how to get data from RDF endpoints, which are also available in *.gov.uk sites)

The building blocks for the implementation was the search url, which was quite straight forward. for all the requests based on postal codes the direct.gov site served in the same manner (because of this important fact, the automation process became trivial). for an instance the url for primary school information retrial was,

http://local.direct.gov.uk/LDGRedirect/LocationSearch.do?LGSL=13&searchtype=1&LGIL=8&Style=&formsub=t&text=SE1+7DU

Where the param “text” changed according to the postal code. So far everything seemed straight forward, however at implementation, while using Gadgets API for content retrial, I faced problems in parsing text with javascript. Hence the gadgets.io.makeRequest supported HTML as text and the API method returned the retrieved HTML document as string making it quite impossible to process.

With some thinking and advise, I brought the Mashup Server in to the picture and used it to retrieve the data from the gov site and returned the result in XML format. Using the Mashup Server web scraping seems to be a piece of cake, We created a simple mashup using the scraper host-object and captured the result set in the search result page. The mashup code as follows,

function search(searchUrl) {
	var scraper = new Scraper(
		
		    {searchUrl}
			
			    
				
				   
				
			     
			
		
	);
	return new XMLList(scraper.response);
}

And finally the two gadgets were making service calls to the mashup service and retrieved the data as an XML object, making the data processing painless. The final version at the Gadget Server looked quite appealing.

WSO2 Gadget Server with UK gov data
Gadget Server look - in the end

Special thanks goes to Ruchira for helping me out with the mashup service 🙂 You can download the Gadget code and the Mashup service and try the scenario yourself.