Architecture in Scrum

Estafet's end-to-end delivery approach and service products include a number of architecture roles, including Enterprise Architects, Platform Architects and Solution Architects. We clearly believe that these architect roles add value to IT delivery.

However, our chosen delivery methodology, Scrum, does not recognise the role of architect at all. Instead, it has three roles: product owner, scrum master and development team member.

So, how do we reconcile these two seemingly conflicting views?

The answer to this is in two parts, one internal to the development team, and one external.

The first element is the inclusion of Solution Architecture skills in the development team. Having a manager (or Delivery Director or Scrum Master) assigning a solution architect within the team would be a violation of Scrum's "self organising team" principle. However, one of the responsibilities of the organisation is to assemble a team with all the necessary skills to complete the job, and this should include solution architecture skills. Normally we would address this by putting someone into the team that we believe will step into the role with the support of the rest of the team; however, we are sometimes surprised by a different team member stepping up in this area.

In this model, the Solution Architect isn't the person who makes all the architecture decisions. The team as a whole should own all design decisions. We prefer to say that the solution architect is the person who will typically explain and articulate those decisions and the solution outline to others outside the team. This can be either verbally, or in written documentation. Because they are the person who will normally be justifying decisions to others, the team may well award them the final say in design decisions. In addition, the first sprint for a scrum team may involve just a solution architect, doing "enough design up front".

The second architecture element in a Scrum engagement is external to the team. Here, the Scrum process says a lot more about what is not allowed than what is. For example, Scrum doesn't allow an architect outside the Scrum team to direct or control the development team. However, Scrum does allow (and encourage) the team to take the views of others into consideration, and to utilise capabilities provided from outside the team.

For Estafet this means two things.

First, we would expect there to be an enterprise architecture function providing strategic direction for the organisation, and we would expect the delivery team and the product owner to take these views seriously.

Second, the development team will typically be building their product on top of a technology platform, owned by a platform architect. The platform team, and platform architect, can reasonably expect that their platform will be used in the way that it was designed to be used, and the development team should take this into consideration as well.

Rate of Change Management

We hear a lot in IT discussions about change management, but maybe it's time to start talking about managing the rate of change as well.  I think we are starting to discover that every organisation has a limit to the amount of change it can sustain in any particular amount of time, and that rate does not necessarily match its budget.  As far as IT projects go, this limit will typically be defined by culture, availability of key individuals, the length of the development life-cycle and the amount of technical debt the organisation is carrying.

Where the change budget is lower than the rate-of-change (RoC) limit there isn't really anything new to manage, and simply managing budget is often sufficient.  I suspect most organisations have fallen into this category up until now.

However, as overall IT productivity rises through new tools and best practices we are starting to see budgets overtake the RoC limits, and this is raising new challenges.  Organisations now find themselves in the frustrating position of not being able to spend all of their budget effectively, because they simply cannot accommodate all the change it would deliver.

As is frequently the case in life and business, the first step in dealing with this is realising you have a problem.  If this really is an issue that real organisations are actually suffering from, we as an industry should start raising awareness of it.

Once an organisation has recognised that it has a problem with its RoC limit, it can then start to develop a plan to deal with it.  "Excess" budget that cannot deliver change itself should be refocussed on longer term initiatives designed to increase the rate of change that the organisation can sustain; paying off technical debt, for example, or shortening the development lifecycle with more agile practices.  

Contracts for Agile Delivery

I was thinking about the contractual basis for Agile bids and came across this on our internal site. It was posted by our chief architect, who highlights this as a particularly difficult area for IT contracts. I thought it deserved a wider audience. --- Estafet Chief Architect Internal Blog Yesterday I was at an IDC Enterprise Architects Conference at which one of the presentations was on the subject of IT contracts.  It was a fascinating presentation covering a lot of ground, but one point that stood out to me was that "agile development" was highlighted as a particularly difficult area for IT contracts. Contracts for agile development should therefore focus on ensuring that the methodology is followed, and then leave that methodology to ensure value for money is achieved.   Having mulled it over for a while, I see no fundamental reason why this this should be the case.  I suspect the perceived problem is that the people creating the contracts are used to defining required functionality in the contract, and agile development doesn't really support this.  But that definition of requirements isn't necessary for the contract, its just the way it has traditionally been done. The part of the contract in question is needed to ensure that the customer gets value for money.  The traditional way of doing this is do define "value for money" up-front; I will get X for £Y, and I am happy that  represents good value. However, the whole philosophy of agile development is about maximising value for money.  Agile methodologies embody this philosophy.  Value should be delivered every iteration (sprint), through a team working on the most valuable features. The contract should clearly state the methodology that is to be followed, the team size & composition, productivity and quality reporting requirements (both active and passive), the measures that will be taken when that reporting highlights issues, etc.   Contracts should also be defined in terms of the methodology: how many sprints are included, and how long are they?  After an initial startup period, a rolling contract can be useful, though there obviously needs to be a termination clause which should allow the team to "wrap up" the work they are doing.   An agile delivery contract framed in this way will allow a team (or teams) to keep delivering value for as long as they justify their cost, and once the team no longer does so, provides a simple exit.  Really effective teams will be around for a long time; teams that follow the process but aren't particularly effective (for whatever reason) will naturally be released fairly soon, but will still have provided value.  Teams that don't follow the process may not be providing any value, but will be in breach of contract anyway, and can be dealt with easily. In essence then, agile delivery contracts should focus much less on what the work will produce, and much more on how the work will be done.  The contract and methodology will support each other, rather than conflict, and allow good teams to concentrate on what agile teams do best: creating value for the customer. --- So, what do you think? Drop me a line below (or email me) with your thoughts. Perhaps you might also consider registering for our round table event on Agile next month.

Is your fragmented web user experience a good reason to build a fragmented mobile user experience?

Most IT organisations are now thinking to at least some degree about how to work mobile computing into their systems

source: Accenture Mobility Insights Report 2014

Some of the more forward-thinking of those organisations are trying to create a coherent plan for this, whilst for others it is happening in a piecemeal fashion, with individual application silos moving at their own pace.

Mobile Fragmentation

The prevalence of this piecemeal, siloed approach should not be a surprise, of course.  Many of the organisations will have built their web user experience in a similar way as each business unit - marketing, finance, HR, etc. - purchases or builds its own applications, each with its own user interface technology and design philosophy.  In recent years, these application UIs have tended to support some sort of “skinning”; changing of colours and text fonts, adding your corporate logo, etc. However, skinning only goes so far, and doesn’t really affect how the user interacts with the system.

It is worth taking a little time to think about how this would all translate to the mobile experience.  The mobile applications world is already fragmented along technology lines; mobile web, native and hybrid apps; Android, Blackberry and iOS. One could argue that additional fragmentation along functional lines can’t make things much worse.  Alternatively, there is a case to be made that it is precisely because of this technical fragmentation that users won’t be as tolerant of functional fragmentation as they are for web applications.

Taking a very simple example, think about the app icons on your employees' phones.  Assume you have an HR app, a CRM app and a finance app. Are they consistently named, with a consistent look for the icons themselves?  Is this even under your control, or do you just provide what the application vendor offers?

Of course, no-one sat down and designed the fragmented web user experience we have; it was never our strategy.  It was simply where we ended up, through countless small “pragmatic” decisions.

Each business unit has its own mobile app, creating a fragmented user experience

*   *   *

Designed for mobile

So, if we want to avoid making the same mistakes with our mobile user experience, what do we need to do?

Well, most organisations have, or are putting in place, an enterprise integration platform of some sort.  Even where it wasn’t really required before, the continuing rise of Software-as-a-Service (SaaS) is fast making it a necessity for 21st century businesses.  The most widely adopted approach is, of course, Service Oriented Architecture (SOA), but traditional Message-Oriented-Middleware, Event Driven Architecture and Extract/Transform/Load systems are also still relatively common.

These integration platforms create the perfect opportunity for a coordinated approach to mobile computing.  

Rather than having each application create its own mobile application, have them expose their functionality through the Enterprise Service Bus (ESB) or alternative integration platform, and then build a consistent, coherent mobile experience on those enterprise services.

A unified experience delivered through a common integration platform

By taking control of your mobile app you can ensure a much smoother user experience: integrated and seamless security; consistent structure and menus; uniform and up-to-date branding; efficient mobile resource usage.  

Mobile development is still a relatively immature discipline, and hence the realm of the specialist software craftsman.   The difference between a hybrid mobile application engineered by an enterprise developer and a native equivalent mobile app crafted by an experienced mobile developer is striking.  Enterprises that capitalise on this by forming development teams dedicated to the mobile platform will deliver a much more compelling mobile experience for their users.

*   *   *

Focus on value

Of course, exposing all the functionality in all those enterprise applications through a single mobile app is a massive undertaking; for most organisations it is probably far too expensive to be practical.  However, that should not be the objective.

Instead, focus on the specific scenarios where the mobile platform adds real value.  Take a look at the various processes that your applications support, and divide them into those that are best done in a full (web) application with a large screen and a proper keyboard, and those that lend themselves to mobile.

Consider a company’s expense claim process, for example.  The monthly process of collating, submitting and analysing expense claims is probably one that benefits from the convenience of the desktop.  However, snapping receipts with a camera and posting them to a pending expenses claim (with a timestamp and a geolocation code) is a great use of mobile technology. 

Once claims have been submitted they go into the approvals process, and here mobile can again add value: viewing a claim and approving/reject it is something that can be done on a small form factor device, and with a simple approve/reject button.  It can also be done offline, in the odd minutes while waiting for a taxi, for example.

Some final thoughts

Users have chosen mobile the key platform for reviewing and actioning content. We have little choice but to embrace these changes, otherwise we frustrate or alienate those we are trying to reach, be they customers, employees or partners. A pragmatic integration strategy to incrementally expose business functions through a common integration layer provides a platform for that mobile strategy the execs are clamouring for.

Questions for comments:

• Do you support BYOD?
• Do you provide corporate devices?  
• Do you promote or enforce device standardisation, or support multiple platforms?

BYOD experience made better with Oracle Mobile Security Suite

In the current corporate world more and more emphasis is being placed on mobile productivity through the concept of Bring Your Own Device (BYOD) where employees will use their personal devices such as tablets and smartphones increasingly to make use of corporate data.

This appeals to the enterprise of course as it can decrease costs to buy and maintain equipment as most employees will have some kind of smart device available to them already.

Clearly security is crucial as corporate applications may have to coexist with essentially unsecured (at least in a corporate sense) personal applications such as facebook, twitter and other personal and social media. The two approaches to securing corporate data and apps on a personal device are MDM and MAM.

MDM (Mobile DEVICE Management) is a sledgehammer approach and consists in securing the whole device - essentially locking it down and allowing only corporate data and apps.

Oracle's new approach with OMSS is much more refined: MAM is Mobile APPLICATION Management. In this scenario personal, 'unsecured' apps can coexist with a secure container which will allow corporate data and apps to run on the device alongside the personal apps, which will not be affected. Thus employees can retain their access to Twitter, Facebook, personal photographs and media while working on secure corporate apps alongside. This is a win-win situation, with both companies and employees benefitting.

Oracle's mobile solution includes a 'secure mobile workspace' which includes a set of basic productivity apps out of the box including secure internet browser, secure mail client, secure file access. Secure apps can be created and made available through a corporate catalogue - much like an appstore - where access and visibility of downloadable corporate apps can be controlled by security policies. The technology makes use of existing corporate authentication and authorisation capabilities and includes an SSL ‘tunnel’ between the corporate container on the device and the server in the cloud.

Rather than wipe entire devices if they are lost or stolen, or when an employee leaves the company, organizations can use secure containers to selectively delete only corporate applications and data. This enables employees the added flexibility of using personal devices and content, without interference by, or to, enterprise data and applications.

This technology will, I am sure, prove popular with both organisations and employees.

Find out more here : http://www.oracle.com/us/products/middleware/identity-management/mobile-security/secure-adoption-of-byod-wp-2199321.pdf

APIs and SOA Services

Introduction

We’ve been talking about Service Oriented Architecture (SOA) and the services that are at the heart of the concept for years now.  A large body of standards, tooling and best practice has evolved over this time, and SOA is now considered a relatively mature discipline.

Web APIs, at least in the sense that they are currently being discussed, are a somewhat newer phenomenon.  There is clearly a lot of overlap between the two concepts, but just how different are they really?  What do we need to do for APIs that we weren’t already doing for our SOA services?  What can we reuse from our SOA programmes?

Similarities

We’ll start with the similarities between SOA and API Management.

First of all, they are both about exposing functionality from one computer system to another through a well-managed published interface that works well with the modern Internet technologies.  They both focus on managing the long term costs of ownership through standardisation, monitoring and control, loose-coupling and facilitating reuse.

There are remarkable similarities between SOA management products such as the Enterprise Service Bus and API management products.  In many cases, a vendor’s API Management product suite contains many of the same components as their SOA product suites.  In addition, established vendors often group their API Management products as part of their overall SOA product portfolio.

Differences

Probably the biggest difference between web APIs and the more mature SOA service concept is that APIs are generally public and published.  The SOA services that enterprises have traditionally built have largely been consumed internally, or by a limited number of known partner organisations.

• You don’t know in advance how many consumers of your API you will have
• There are probably a lot more of them
• You often don’t have a direct relationship with your API consumers
• You can’t really tell those consumers what to do

These are all important differences, and will require a change in approach to some degree from what we have been doing so far with SOA.  

In terms of maturity, SOA and the products that support it have been around for ten years or more, whereas API Management and API Management products are still a rapidly evolving “new” discipline.  This has allowed API Management tools to avoid some of the issues that SOA has seen, but there is also a risk that it is failing to accommodate some of the lessons learned as well.

A more technical difference is that enterprise SOA is typically based on SOAP Web Services, whereas web APIs are more likely to use RESTful services.  However, both disciplines can use either interface type, so this is less of a difference than it might seem.  Bear in mind that SOAP interfaces are a lot more constrained than RESTful interfaces; is this level of constraint an example of a SOA mistake that API management has avoided, or of a SOA best practice that API management has failed to learn from?

While some people, particularly product vendors, claim that SOA is a set of tools to be purchased, most architects, analysts and IT decision makes realise that it is more about the way an organisation operates than the specific tools used.  In organisations with effective SOA programmes the discussions tend to focus on governance, maturity models, roadmaps and centres of excellence.  API Management still seems to be very much focussed on the tooling and products used. Is this a sign that API Management is a much simpler problem for which “canned” solutions are appropriate, or a symptom of the lower level of maturity in this space?

Conclusions

It is clear that there are significant differences between API Management and enterprise Service Oriented Architecture, but that there is also a lot of overlap.  Organisations looking to strengthen their API Management strategy should be wary of constraining that strategy by assuming that their existing SOA tooling and approach is sufficient for API Management.

At the same time, organisations with a mature SOA initiative - or that have already embarked on a programme of SOA adoption - would be advised to look to reuse as much as they can and to only change what they can demonstrate needs to be changed.  Assuming that an element of a SOA strategy or platform is not necessary for API Management simply because no-one is talking about it in that context (yet) has the potential to cause significant pain and embarrassment in the future.

The best approach is surely to assemble a team with the experience built from successful SOA adoption, but with the open minds and agility required to adapt and change what they know to accommodate the new challenges and opportunities inherent in API Management.

With the right team in place it won’t matter whether we are in a brave new world or just rediscovering what we already knew.  

Thoughts on Ignite 2014

Rich Walker and Noel Sharkey with Shadow Robot's "Dextrous Hand"

Inspired by Ignite 2014? With talks on digital disruption, robotics, lean & agile start-ups, and transformation what will change as a result of the conference? We were one of the sponsors of Ignite. We transform companies by working with them to become Agile at delivering innovative applications with a new collaborative approach. I was intrigued by some of the speakers (and who doesn't find Peter Cochrane or Noel Sharkey - above right with Rich Walker - engaging and fascinating?) but also by the conversations we had with a number of you about how you are dealing with disruption in your markets. We would like to talk to you if you want to Transform and Deliver a new application into your business. Please drop me a line (even a few words) at alistair.park@estafet.com Here's what inspired us... British Gas creates a lean start-up for active management of your home heating

A really simple and practical lesson in to how to run a lean start-up was given by Kass Hussain of British Gas. They've taken a new market, new technology and have literally gone out of their way (locating offices near talent, rather than parent company) to create a new brand and new systems to support it. We're working with a smart metering company that works to lean principles and continuously integrates the work of multiple, international teams. When the Sprint Review is happening, everyone from developer to CEO stands around big screens using a Google Hangout to share what has been achieved. Progress is quick, but goals are always achievable. "78% of professionals say their industry sector is currently being disrupted"

The Ignite survey of members found a high level of disruption within their industry sector. Is this also your conclusion? We are seeing disruption across a number of industries from card payments to retail to smart metering with a common theme of lean start-up. What's exciting is the creativity that is being found everywhere to find new ways to attack existing markets. To maintain the initial agility, they key has been to focus on the pragmatic delivery of customer-facing functionality: release early, release often. We've helped through ruthlessly eliminating technical debt and reducing time-to-market through continuous integration and delivery. Uber's private hire service is disrupting the London Taxi market

Another interesting talk on disruption was by Uber, a San Francisco start-up which is now taking the battle to the streets of London. London cabbies have a protected status (unchanged in many ways for a hundred years) whereby meters are closely related to time and distance travelled. New technology means that the existing laws can be circumvented using GPS and a bit of technology.

The cabbies don't like it, but I can't see how they can win this battle. Also, with Google backing Uber whilst releasing it's first driver-less car there's a very clear route ahead.