WSO2 ESB message corruption; False alarm

There seems to be a false alarm spreading over the online forums that WSO2 ESB cannot handle messages larger than 16K and results a corrupted response. This situation occurs only when a user enables one of the synapse properties which is disabled by default.

Streaming XPATH option is the aforesaid culprit, which is a newly introduced feature to optimize the performance in different message filtering / routing scenarios. However this does not effect the default workings of WSO2 ESB which does handle large messages successfully in thousands of enterprise deployments with the default XPATH implementation that ships out of the box.

As any new component achieve perfection with time, WSO2 ESB team have fixed all reported issues against streaming XPATH component (in ESB v4.8.0) and thrives to be the fastest ESB and the integration platform.

Following case study provide mode details on large message handling of WSO2 ESB in production.

UPDATE (02/17/2014) : WSO2 has released the latest performance stats, which details on the stream XPATH stabilization as well.

Agile Release Management & Product Engineering

I’ve been doing release management and product engineering for almost three years in WSO2. By no means am an expert. Am still learning and trying to tame the beast. However I do have my fare share of experience and knowledge which I have gathered by screwing it all up and also sometimes, doing it right as well 🙂

Recently we have done a successful 1.0.0 release of WSO2 User Engagement Server and I felt writing this post as the release retrospect.

Team

Team is the most important part of any product innovation. Without a strong, competent and hard working team you cannot achieve much (period !!). You feel it when it comes to agile release management. You cannot spoon feed your team. Each member will own a part towards success. You have to believe the quote “A Team Is Only As Strong As Its Weakest Link”, so your weakest got to be quite smart.

Milestones are short, quality of deliverables are high. The build *has* to pass always keeping the master stable. That’s a lot of work. A lot of smart work. So the team does matter and that’s the number one.

Well, WSO2 has the best from the Best and luckily UES had the dream team 🙂

1044748_10151485064622091_700075066_n

Vision

Number two is the Vision. Its not about the person whose leading the effort need to know the product vision. The team got to have a shared understanding about the vision of the product that they develop. Everybody has to be on the same page. If a button was moved from left to right everybody must know the reason behind that decision.

Non should develop a component without knowing the bigger picture of where that component fits in. (s)He should know the future of that component, what are the improvements can be done, how can it aid to market the product and how does it contribute to achieve the product’s vision.

Reviews

Reviews play a major role. They sync up the team. First you review the design, then review the architecture, the UI/UX and then the code. You get this right, you get a complete product feature.

During UES release, we did a lot of these. Our design review meetings were war zones. We talked about architecture and clean design patterns. Applied them appropriately and iteratively reviewed them. Our UI mockups were all over the whiteboard, we iterated until we achieve perfection. We reviewed our code, identified the improvements.

Continue reading Agile Release Management & Product Engineering

WSO2 User Engagement Server: How it kicked off from the whiteboard

We started re-engineering WSO2 Gadget Server during 2012 Winter. We were brainstorming of how we can change it to be more flexible and accommodate wider enterprise data/information presentation use-cases. We wanted to give more power to the developer, not just the ability to drag and drop widgets and select a layout. We wanted these pages to be discovered around the organization. With such ideas in mind we created multiple mind maps to get a clear idea

GS-2.0
GS 2.0 Mindmap (Nov. 2012)

We had plenty of email debates, F2F meetings of what we need to do ? Are we there yet ? Is this what developers/business users need ? etc. Continue reading WSO2 User Engagement Server: How it kicked off from the whiteboard

The Vision Behind WSO2 User Engagement Server

banner2

“A picture worth a thousand words” is a known fact which doesn’t need proof. At present we discuss about the value and the power of data. We use various tools to analyze them, we do business intelligence capturing trends or anomalies. However in the end you need to express the meaning in a sensible manner for the key decision makers. Would you go write thousand words or rather draw something, summarize the findings and let the stakeholder explore for more.

WSO2 Middleware stack has all the pieces you need to harvest, store & analyze (both real-time and periodical) vast amount of data sets, but when it comes to summarizing and presenting, the stack had somewhat a week tool set. We envisioned the idea of “SOA Last Mile” since latter part of 2009 with products such as WSO2 Gadget Server, WSO2 Mashup Server and frameworks such as viskit. We experimented, wrote code and delivered enterprise grade products to achieve this vision.  For instance WSO2 Gadget Server was one of the most downloaded products during 2010 – 2011 time span, and even now people are using it for various data presentation purposes.

Continue reading The Vision Behind WSO2 User Engagement Server

Reading an XML file into WSO2 ESB; Transform it and expose it as an API

Recently was working on a project where I had a to read an XML file from an FTP location, transform it and expose it as an API. Used WSO2 ESB 4.6.0 for this usecase; and I thought of documenting it for later reference. So here it goes

First the proxy that read the file from FTP and dump it to a defined location, (VFSProxy.xml)

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse" name="VFSProxy" transports="vfs" startOnLoad="true" trace="disable">
<target>
<inSequence>
<log level="custom">
<property name="STATUS" value="File received"/>
</log>
<property name="OUT_ONLY" value="true"/>
<send>
<endpoint>
<address uri="vfs:file:///home/nuwanbando/temp/files/out"/>
</endpoint>
</send>
</inSequence>
</target>
<parameter name="transport.PollInterval">10</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.FileURI">vfs:ftp://<ftpserver_url>/home/nuwanbando/temp/files/in?vfs.passive=true</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///home/nuwanbando/temp/files/processed</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///home/nuwanbando/temp/files/failed</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.xml</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>

view raw
VFSProxy.xml
hosted with ❤ by GitHub

This proxy will dump the file to "home/nuwanbando/temp/files/out" location.

This file need to be read on-demand, once requested. ESB by default does not have a mediator to read XML files so that the mediation flow can manipulate the content, Continue reading Reading an XML file into WSO2 ESB; Transform it and expose it as an API

{jaggery.js}

Is something I’ve been working on at wso2 for sometime, if you are following me I have tweeted the updates, releases etc. Jaggery went live with its fifth milestone, few weeks ago at jaggeryjs.org, its still young and has alot to improve.

Explaining a little bit about jaggery; Its NOT an alternative to node, its simply a toolkit to easily compose webapps and apis. Its a completely javascript way of writing all parts of the web application, and communicate throughout in json. Jaggery has out of the box features to create RESTfull applications, and its one of it’s value propositions. Jaggery can be easily extended via modules which can be written in javascript or java for that matter.

A very simple page that prints out some request properties in jaggery looks like,

<html>
   <body>
      <p>
      <%
          print("Method : " + request.getMethod() + "<br/>");
          print("Protocol : " + request.getProtocol() + "<br/>");
          print("User-Agent : " + request.getHeader("User-Agent"));
      %>
      </p>
   </body>
</html>

Jaggery docs site also provide a tool for you to try out some code and play around. So you can simply copy the above code and paste in jaggery try-it

I recently did a webinar on jaggery that explains the simple case, over looking its API and samples. Below is the recording hope it’ll help.

Events notification (presentation) using WSO2 technologies

The post is about creating a dashboard to display event notifications using few of WSO2 Technologies.

Requirement : Capture events from an event sender and display it in a dashboard

Used Technologies : WSO2 Enterprise Service Bus, WSO2 Application Server, WSO2 Gadget Server

Architecture : In brief, events are generated randomly, ESB will act as an event sink, and it will publish the events to the subscribed parties (to a service deployed in WSO2 AS). There is a gadget deployed in WSO2 Gadget Server where it will poll the received events from the service periodically.

HOWTO :

The ESB

At the ESB end we will have to set up the event sink as following,

https://gist.github.com/1653037.js?file=synapse.xml

Then you need to create a topic in the ESB and subscribe the created service. Steps in doing so is documented here

The Service

A very simple service to accumulate the events (this is not at all production grade, who whats to keep all the event in memory 😉 )

https://gist.github.com/1653077.js?file=TrafficEventReceiverService.java

The Client

Its too long to post, you can check it at https://gist.github.com/1653062

You can run the client like “ant trafficevent -Dtrpurl=http://localhost:8280 -Devent=traffic.is.crazy

The Gadget
https://gist.github.com/1653079.js?file=TrfficEventGadget.xml
In Action

 

“Jaggery” – A web{app/service} completely in Javascript

At WSO2 we are busy with something new for the new year 🙂 and we have just pushed the milestone-1 of it, not so complete but you can sure taste it. Everybody knows that WSO2 is big on Web Services, but over the years we have seen that the barrier between Web Services and Web Applications hasn’t been lowered. “Jaggery” is an effort to diminish this barrier and to merge the development process of services and applications.

By definition it is a framework to write webapps and HTTP-focused web services in pure Javascript for all aspects of the application: front-end, communication, Server-side logic and persistence, and yes as I mentioned one of the intents of the framework is to reduce the gap between writing web apps and web services.

In general web engineers are fond of javascript (and JSON of cause), so why not write the server piece, persistence logic and the communication in-between also in JS itself ?

“Jaggery” helps you to do just that! Right now, the Javascript you compose will be fetched by Rhino engine and execute at the server, At Milestone-1 we have pushed features such as,

  • Compose server side web pages (HTML templates) : Server logic written in javascript (.JSSP)
  • Compose server side scripts purely in javascript (.JSS)
  • Includes support for,
    • HTTP request/response and session
    • JSON send/receive
    • Server side XMLHttpRequest
    • Web Service invocation
    • File I/O
    • Email
    • Atom
    • Read/write feed
    • Relational database
    • WSO2 Registry/Repository
    • Open Social gadgets

A hello world server-side script looks as follows,
https://gist.github.com/1506790.js?file=HelloWorld.jss

and a HTML mixed, server page looks like,
https://gist.github.com/1514681.js?file=gistfile1.txt

For more information and samples you can visit Jaggery M1 Homepage, and stay tuned for more updates soon 🙂

WSO2Con is NOT just for “Hard Core” Techies

Well, maybe I am too late to write this post; but hopefully I will be able to discard a misconception in some of you. As you all know by now WSO2Con starts tomorrow and goes on till Friday (16/09). Its a great tech feast, and there weren’t many such in recent history in Sri Lanka. Maybe am not the best person to write all the good stuff about it and asking you to take my word. But if I forget the fact that I am a member of wso2 team, and look at it as a Sri Lankan techie, the agenda still sounds pretty awesome.

Yesterday I meat a friend of mine (lets say Bob 🙂 ) at a dinner, and Bob was saying, “you guys are doing a software conference right; I saw in media, sounds great !!”. I was head over heels (didnt wona say I am doing a talk; wanted to be bit modest 😉 ) I asked excitedly, “So are you coming ?”. But his reply surprised me. He said “Well I dont think its for us you know. Its too hard core; you guys are doing open source, SOA and cloud stuff. I don’t think we are there yet”. If I add little bit of context, Bob is the senior IT manager in a well reputed insurance company in Sri Lanka. So as I felt he had all the reasons to attend to WSO2Con.

I see several misconceptions in his comment, maybe people in Sri Lankan IT services (not directly software, but software is a supporting system like for IT systems for banks) industry are bit behind the game. For one, some people say SOA and Web Services are dead  🙂 it’s no news neither a buzz, Yes cloud stuff is kinda new, but you have it every where now, I guess its high time to check that out too. But whats so hard core about Open Source ??? !@#$.

So yeah WSO2Con is NOT just for “Hard Core” Techies. It certainly feels awesome that people think like that, but the truth is its a week of discussions on software systems. Maybe little bit more about distributed and service oriented systems, But isn’t everything distributed and service oriented these days ? :).

If you checkout the agenda its mostly about enterprise systems, issues in enterprise systems, how to solve them, how open source software can help and most of all, its a week of discussions on experience. Why experience in bold; well if you read the abstracts of many talks, its all about how people solved their enterprise integration problems. It a talk show of first hand experiences. There are, IT consultants, Architects and Visionaries coming from around the world to share there knowledge and experience. What else can you ask for ?

So I guess, if you are a manger who has even a slight influence in IT matters, this is something you goto check out; maybe you do have the same problems as some of these speakers had some time back. It would be a great opportunity to talk to them in person and get some insight.

Well hope I did make some change of your attitude; and I do hope to see you :).

FYI you can still register at wso2con.com