OSB Release Management Utility

Agile methodologies have increased radically the numbers of release events in organisations where it has been adopted.  Both development and operations teams need to collaborate more closely during production release events requiring more than ever the right supporting tools.

How many times have you been asked what has been deployed in environment X? 

If you are using CI tools and doing release management properly it should be a straightforward answer. However, software deployment comprises a set of steps involving multiple parties. As a result, the process isn't entirely automated requiring human intervention which is susceptible to errors.

Imagine you have just delivered an OSB Production bundle to the operations guys (DBAs, Weblogic Admins…).

How can you prove that the expected jar has been deployed without looking at the source code or running any tests ?

We are using a custom Release Management Utility that saves us a lot of time and headaches ☺

• No third-party software required; It uses Hudson, SVN, Maven and OSB
• Allows users without technical knowledge to view what’s deployed
• Identifies project changes between sprints
• Track back to source code based on SVN revision
• Displays build date and version for each individual project

Overall Architecture

Similar to a JAR file's manifest each OSB project will contain a build-version.xml file where all the required information is stored. When the build process is triggered Hudson pulls the project from SVN and the code is built. Additionally, maven replaces the Hudson runtime variables on the file ensuring the creation of a unique and traceable JAR. Once this is done, we aggregate and display the information as a web page to the user. We can now hit the OSB endpoint in any environment (DEV, SIT, UAT, PERF and PROD) and understand easily what’s actually deployed.

Implementation Details

Hudson holds runtime variables for each job execution which we use to replace the build-version.xml using the maven plugin “com.google.code.maven-replacer-plugin”. A snippet of the pom.xml can be found here. Additionally, a global variable called RELEASE_NUMBER was added to store the actual sprint number. This variable is incremented manually on the beginning of each sprint.

Once a project is deployed Weblogic stores the files under the following path:

<DOMAIN_HOME>osb/config/core/<PROJECT_NAME>/

All paths for the projects we would like to report are stored in a properties file.

<urls>

    <buildURL>osb/config/core/ProjectA/build-version.xml</buildURL>

    <buildURL>osb/config/core/ProjectB/build-version.xml</buildURL>

</urls>

The Reporting service provides data through an XML or HTML proxy.

XML Proxy

Regarding the properties file, the Java Callout reads all the build files and aggregates the information into a single xml. Beware that OSB wraps the files into CDATA and some parsing needs to be done.

HTML Proxy

Reuses the XML proxy and applies XSLT transformations to convert the data into HTML.

Simple extension

Having this implemented, we questioned ourselves about an easy way to compare this information across environments instead of hitting all the endpoints and fetch it manually.

So, from the delivery box we execute a Java HTTP client that consumes the XML proxies and outputs the comparison result (based on the SVN revision). As a result, code propagation between environments is more clear and traceable.

Note: Make sure you have connectivity to all the environments from the delivery box.

The revision of each project is compared across environments and the example below shows that two projects are out of sync.

Thanks to Wayne Highland and Kevin Ryall that participated on the development of this utility.

Drop me an email if you have any questions :)

Beyond Integration FAQs

Overview

Beyond Integration is a compelling vision for the future, an aspiration to take the challenges of modern enterprise applications integration and transform them into a giant step forwards.

It is, however, a complex topic that is difficult to explain adequately in a short overview - see previous post for the infographic.  We have therefore compiled this list of frequently asked questions to explain some of the concepts in a little more detail.

Frequently Asked Questions

1. Exactly how does Beyond Integration open up my Business Processes?
   All enterprise applications have at their heart a business process model; in legacy applications this is often ill-defined and locked away, but in a well designed modern SOA application it will be much more visible.  It may even be implemented directly in BPEL or BPMN tooling.  Almost certainly it will have service interfaces for interacting with that business process; for initiating and progressing process instances, and for exposing the state of those services in real time.  By combining these process services with your own enterprise business process management you can build truly enterprise wide processes that seamlessly span traditional organisational boundaries and bring the value in those processes to everyone who needs it.
2. My vendor assures me all the value in their app is unlocked already, am I OK?
   Of course they say that, and in some rare cases it might even be true. For most enterprise applications, however, things are rarely so promising.  Most were designed and built as self-contained systems, with the value (data, business logic and business process) exposed only through their own user interface.  Integration APIs would then be tacked on the side, giving access to the value that is easy to expose, or that most customers are clamouring for.  The rest remains locked away.
3. Is this just a sales pitch for Oracle Fusion Apps?
   Absolutely not.  Oracle have a strong sales team for that.  Fusion Apps is at most an enabler for the Beyond Integration vision, by virtue of being one of the first mainstream enterprise application suites built from the ground up on modern Service Oriented Architecture principles.  Simply buying or building modern service oriented applications won’t move you beyond integration; using them in a new way to unlock more value from within them will.
4. So, what are you trying to sell?
   Beyond Integration isn’t an attempt to directly sell anything.  We are trying to ignite discussion amongst IT leaders and set a direction for enterprise strategy as the technology world transforms around us.  We would like to be part of those conversations with you, and can provide consultancy services if you need help, but this journey is about unlocking value within your own organisation and ultimately you must walk your own path.
5. Can I move Beyond Integration right now?
   We’d love to answer “yes” to this but, unfortunately, we simply aren’t there yet.  Many of the enterprise applications we depend on have their architectural roots in the 1980s and 1990s, and our own in-house bespoke applications have often made too many compromises in order to hit project deadlines.  Things are improving, however, and we are confident that we will start to see large enterprises start to realise real value from this way of thinking in the next year or two.
6. Is this just about IT? Our silos are cultural as well as technological.
   Breaking down organisational silos and building seamless processes across the organisation is much more than an IT problem, and in order to truly benefit from moving beyond integration those organisational issues must be tackled.  However, the IT systems element remain a critical part of this.
7. Aren’t Enterprise Applications the best place for business process definition?
   For some areas where your organisation doesn’t seek competitive advantage there is often little value in inventing your own  business process, and buying in a process that is embedded in an application can make sense.  However, the rate at which best practice in business processes is changing at the moment often far outstrips the rate at which enterprise application vendors update their applications.
8. We are locked into our current apps for several years; what can we do?
   Beyond Integration is a vision to work towards, so as long as you can see a way to make progress there is value to be gained.  If you find yourself in this position you have plenty of time to prepare for the next round of vendor selection and to ensure that your Beyond Integration vision is a part of that.  Internal IT projects can also begin to work with service & process abstractions that are in line with your vision rather than with the applications directly, reducing the tight coupling that would likely pressure you to reselect the incumbent vendor in the future.
9. So what should I do next?
   As with any long term vision, understanding how to get there can be the hardest challenge.  Articulating your own vision is an important first step, as is defining a high level roadmap and identifying how you will measure your progress towards your goals.
10. What do all the little icons mean?
    The person with a screen represents an application’s user interface.  The water-cooler is a database.  The cogs show business processes, and the chain links denote service interfaces.
11. What is the funny shaped box that gets a question mark in it?
    Often when implementing service-oriented integration there is an intermediary such as an Enterprise Service Bus involved in service interactions.  When everything is in your own data centre this is easy to manage.  But what about interactions between services in the cloud? Where is your ESB hosted now?  Do the cloud service interactions have to route back into your data centre?
12. Why does moving to the cloud break database integration?
    Cloud data centres hold a huge amount of very valuable, and potentially very sensitive, data, and that data must be protected at all costs.  Opening up the data centre firewalls to allow direct database access for even one customer presents a risk to everyone’s data.  SaaS providers will have made specific assurances to their customers about the safeguards protecting that data.  Bear in mind that SaaS hosting is somewhat different to a traditional shared data centre model; you, as a SaaS customer, very likely won’t have specific infrastructure dedicated to your service, that can be firewalled-off separately.
13. Why don’t you show database access in “unlocking all the value”?
    Data tier integration is going to become increasingly difficult as SaaS becomes more pervasive, and has never been the first choice for integration in a Service Oriented strategy anyway.  SOA principles recommend integrating above the data tier, using services at the process or business activity level.  Of course all the valuable data in your databases should be unlocked, but we would recommend doing so via a well defined business activity service interface.
14. How do I know this is really possible?
    The problems we have been describing are largely confined to the world of enterprise applications.  Over the last few years there has been a revolution in consumer cloud applications, with agile startups (“digital disruptors”) using the mashup model very effectively to create value in ways that no-one could have foreseen previously.  Their culture is based on sharing their value with others; Facebook, for example, could have locked their systems away and been somewhat successful.  Instead, they built an API and an apps ecosystem, resulting in them becoming a global powerhouse almost overnight.

More Information

For further information, or if your questions aren’t covered here, please contact us through one of the following channels:-

Tel:	0208 614 7070
Web:	estafet.com
Email:	enquiries@estafet.com

Beyond Integration

Beyond Integration is a compelling vision for the future, an aspiration to take the challenges of modern enterprise applications integration and transform them into a giant step forwards.

Organisations that share this vision will naturally be wondering how to turn the vision into reality.  What actions can they take that will move them closer to the vision, how can they measure their progress towards it and who are the key stakeholders that need to be brought on board?

As with any vision, realising it is a journey rather than a single activity, and having a roadmap for that journey will be invaluable.

Estafet - Beyond Integration

Architecture in Scrum

Estafet's end-to-end delivery approach and service products include a number of architecture roles, including Enterprise Architects, Platform Architects and Solution Architects. We clearly believe that these architect roles add value to IT delivery.

However, our chosen delivery methodology, Scrum, does not recognise the role of architect at all. Instead, it has three roles: product owner, scrum master and development team member.

So, how do we reconcile these two seemingly conflicting views?

The answer to this is in two parts, one internal to the development team, and one external.

The first element is the inclusion of Solution Architecture skills in the development team. Having a manager (or Delivery Director or Scrum Master) assigning a solution architect within the team would be a violation of Scrum's "self organising team" principle. However, one of the responsibilities of the organisation is to assemble a team with all the necessary skills to complete the job, and this should include solution architecture skills. Normally we would address this by putting someone into the team that we believe will step into the role with the support of the rest of the team; however, we are sometimes surprised by a different team member stepping up in this area.

In this model, the Solution Architect isn't the person who makes all the architecture decisions. The team as a whole should own all design decisions. We prefer to say that the solution architect is the person who will typically explain and articulate those decisions and the solution outline to others outside the team. This can be either verbally, or in written documentation. Because they are the person who will normally be justifying decisions to others, the team may well award them the final say in design decisions. In addition, the first sprint for a scrum team may involve just a solution architect, doing "enough design up front".

The second architecture element in a Scrum engagement is external to the team. Here, the Scrum process says a lot more about what is not allowed than what is. For example, Scrum doesn't allow an architect outside the Scrum team to direct or control the development team. However, Scrum does allow (and encourage) the team to take the views of others into consideration, and to utilise capabilities provided from outside the team.

For Estafet this means two things.

First, we would expect there to be an enterprise architecture function providing strategic direction for the organisation, and we would expect the delivery team and the product owner to take these views seriously.

Second, the development team will typically be building their product on top of a technology platform, owned by a platform architect. The platform team, and platform architect, can reasonably expect that their platform will be used in the way that it was designed to be used, and the development team should take this into consideration as well.

Rate of Change Management

We hear a lot in IT discussions about change management, but maybe it's time to start talking about managing the rate of change as well.  I think we are starting to discover that every organisation has a limit to the amount of change it can sustain in any particular amount of time, and that rate does not necessarily match its budget.  As far as IT projects go, this limit will typically be defined by culture, availability of key individuals, the length of the development life-cycle and the amount of technical debt the organisation is carrying.

Where the change budget is lower than the rate-of-change (RoC) limit there isn't really anything new to manage, and simply managing budget is often sufficient.  I suspect most organisations have fallen into this category up until now.

However, as overall IT productivity rises through new tools and best practices we are starting to see budgets overtake the RoC limits, and this is raising new challenges.  Organisations now find themselves in the frustrating position of not being able to spend all of their budget effectively, because they simply cannot accommodate all the change it would deliver.

As is frequently the case in life and business, the first step in dealing with this is realising you have a problem.  If this really is an issue that real organisations are actually suffering from, we as an industry should start raising awareness of it.

Once an organisation has recognised that it has a problem with its RoC limit, it can then start to develop a plan to deal with it.  "Excess" budget that cannot deliver change itself should be refocussed on longer term initiatives designed to increase the rate of change that the organisation can sustain; paying off technical debt, for example, or shortening the development lifecycle with more agile practices.  

Contracts for Agile Delivery

I was thinking about the contractual basis for Agile bids and came across this on our internal site. It was posted by our chief architect, who highlights this as a particularly difficult area for IT contracts. I thought it deserved a wider audience. --- Estafet Chief Architect Internal Blog Yesterday I was at an IDC Enterprise Architects Conference at which one of the presentations was on the subject of IT contracts.  It was a fascinating presentation covering a lot of ground, but one point that stood out to me was that "agile development" was highlighted as a particularly difficult area for IT contracts. Contracts for agile development should therefore focus on ensuring that the methodology is followed, and then leave that methodology to ensure value for money is achieved.   Having mulled it over for a while, I see no fundamental reason why this this should be the case.  I suspect the perceived problem is that the people creating the contracts are used to defining required functionality in the contract, and agile development doesn't really support this.  But that definition of requirements isn't necessary for the contract, its just the way it has traditionally been done. The part of the contract in question is needed to ensure that the customer gets value for money.  The traditional way of doing this is do define "value for money" up-front; I will get X for £Y, and I am happy that  represents good value. However, the whole philosophy of agile development is about maximising value for money.  Agile methodologies embody this philosophy.  Value should be delivered every iteration (sprint), through a team working on the most valuable features. The contract should clearly state the methodology that is to be followed, the team size & composition, productivity and quality reporting requirements (both active and passive), the measures that will be taken when that reporting highlights issues, etc.   Contracts should also be defined in terms of the methodology: how many sprints are included, and how long are they?  After an initial startup period, a rolling contract can be useful, though there obviously needs to be a termination clause which should allow the team to "wrap up" the work they are doing.   An agile delivery contract framed in this way will allow a team (or teams) to keep delivering value for as long as they justify their cost, and once the team no longer does so, provides a simple exit.  Really effective teams will be around for a long time; teams that follow the process but aren't particularly effective (for whatever reason) will naturally be released fairly soon, but will still have provided value.  Teams that don't follow the process may not be providing any value, but will be in breach of contract anyway, and can be dealt with easily. In essence then, agile delivery contracts should focus much less on what the work will produce, and much more on how the work will be done.  The contract and methodology will support each other, rather than conflict, and allow good teams to concentrate on what agile teams do best: creating value for the customer. --- So, what do you think? Drop me a line below (or email me) with your thoughts. Perhaps you might also consider registering for our round table event on Agile next month.

Is your fragmented web user experience a good reason to build a fragmented mobile user experience?

Most IT organisations are now thinking to at least some degree about how to work mobile computing into their systems

source: Accenture Mobility Insights Report 2014

Some of the more forward-thinking of those organisations are trying to create a coherent plan for this, whilst for others it is happening in a piecemeal fashion, with individual application silos moving at their own pace.

Mobile Fragmentation

The prevalence of this piecemeal, siloed approach should not be a surprise, of course.  Many of the organisations will have built their web user experience in a similar way as each business unit - marketing, finance, HR, etc. - purchases or builds its own applications, each with its own user interface technology and design philosophy.  In recent years, these application UIs have tended to support some sort of “skinning”; changing of colours and text fonts, adding your corporate logo, etc. However, skinning only goes so far, and doesn’t really affect how the user interacts with the system.

It is worth taking a little time to think about how this would all translate to the mobile experience.  The mobile applications world is already fragmented along technology lines; mobile web, native and hybrid apps; Android, Blackberry and iOS. One could argue that additional fragmentation along functional lines can’t make things much worse.  Alternatively, there is a case to be made that it is precisely because of this technical fragmentation that users won’t be as tolerant of functional fragmentation as they are for web applications.

Taking a very simple example, think about the app icons on your employees' phones.  Assume you have an HR app, a CRM app and a finance app. Are they consistently named, with a consistent look for the icons themselves?  Is this even under your control, or do you just provide what the application vendor offers?

Of course, no-one sat down and designed the fragmented web user experience we have; it was never our strategy.  It was simply where we ended up, through countless small “pragmatic” decisions.

Each business unit has its own mobile app, creating a fragmented user experience

*   *   *

Designed for mobile

So, if we want to avoid making the same mistakes with our mobile user experience, what do we need to do?

Well, most organisations have, or are putting in place, an enterprise integration platform of some sort.  Even where it wasn’t really required before, the continuing rise of Software-as-a-Service (SaaS) is fast making it a necessity for 21st century businesses.  The most widely adopted approach is, of course, Service Oriented Architecture (SOA), but traditional Message-Oriented-Middleware, Event Driven Architecture and Extract/Transform/Load systems are also still relatively common.

These integration platforms create the perfect opportunity for a coordinated approach to mobile computing.  

Rather than having each application create its own mobile application, have them expose their functionality through the Enterprise Service Bus (ESB) or alternative integration platform, and then build a consistent, coherent mobile experience on those enterprise services.

A unified experience delivered through a common integration platform

By taking control of your mobile app you can ensure a much smoother user experience: integrated and seamless security; consistent structure and menus; uniform and up-to-date branding; efficient mobile resource usage.  

Mobile development is still a relatively immature discipline, and hence the realm of the specialist software craftsman.   The difference between a hybrid mobile application engineered by an enterprise developer and a native equivalent mobile app crafted by an experienced mobile developer is striking.  Enterprises that capitalise on this by forming development teams dedicated to the mobile platform will deliver a much more compelling mobile experience for their users.

*   *   *

Focus on value

Of course, exposing all the functionality in all those enterprise applications through a single mobile app is a massive undertaking; for most organisations it is probably far too expensive to be practical.  However, that should not be the objective.

Instead, focus on the specific scenarios where the mobile platform adds real value.  Take a look at the various processes that your applications support, and divide them into those that are best done in a full (web) application with a large screen and a proper keyboard, and those that lend themselves to mobile.

Consider a company’s expense claim process, for example.  The monthly process of collating, submitting and analysing expense claims is probably one that benefits from the convenience of the desktop.  However, snapping receipts with a camera and posting them to a pending expenses claim (with a timestamp and a geolocation code) is a great use of mobile technology. 

Once claims have been submitted they go into the approvals process, and here mobile can again add value: viewing a claim and approving/reject it is something that can be done on a small form factor device, and with a simple approve/reject button.  It can also be done offline, in the odd minutes while waiting for a taxi, for example.

Some final thoughts

Users have chosen mobile the key platform for reviewing and actioning content. We have little choice but to embrace these changes, otherwise we frustrate or alienate those we are trying to reach, be they customers, employees or partners. A pragmatic integration strategy to incrementally expose business functions through a common integration layer provides a platform for that mobile strategy the execs are clamouring for.

Questions for comments:

• Do you support BYOD?
• Do you provide corporate devices?  
• Do you promote or enforce device standardisation, or support multiple platforms?