Well, judging by work done by Amazon and Microsoft, you’ll struggle. Amazon believe that 100mS extra delay costs 1% of sales, while Microsoft believe that 500mS costs 1%. Given this – not uncommon – ambiguity, you might determine that that only strategy available to you is to make your system respond as fast as time and money will allow.

But this has clear cost implications and, afterwards, you still won’t know whether users have noticed any difference. You still won’t know whether you business has benefited economically.

This is where Perceptual Quality helps. PQ is a measure of what people notice. If people notice deteriorating performance, their productivity falls. PQ is application specific, which means that, for the key applications that drive your business, you can now measure and manage the performance of your entire IT estate in economically meaningful terms.

More importantly, Perceptual Quality is a powerful new tool in these austere times. Combining PQ with Actual Experience’s fully automated Analytics, can save your business money AND increase revenue.

Notes – Amazon: http://sites.google.com/site/glinden/Home/StanfordDataMining.2006-11-28.ppt; Microsoft: http://velocityconf.com/velocity2009/public/schedule/detail/8523

Andy worked with Actual Experience on the Digital Britain programme where Cisco Systems are deploying Actual Experience services to help ISPs quantify the quality of their services, in human terms, enabling business leaders to make high impact investment decisions, and support service differentiation monetization.

Macleod says, “I have worked in the industry for over 25 years and have seen nothing like this. Actual Experience improves what is important to the user – their personal experience of the applications they use – whether on a corporate network at home or mobile. So it’s an incredibly powerful tool for both CIOs and business leaders, and it also has a serious future in the consumer internet market.”

Dave Page, Co-Founder and CEO of Actual Experience commented, “We’re thrilled to have Andy on board – his pedigree, reputation and broad experience of sales, marketing strategy and channel development are exactly what we need to look after our growing family of customers and scale the business.”

A highly respected figure in one of our Managed Service Provider customers said, “I’ve never before seen a product or service that can be utilised at all levels throughout any business… to communicate effectively with all people in all areas within a business thanks to the uniform language that it has inspired.”

More specifically, “The scope of your product and service is so wide – anything from assisting an engineer in identifying a particularly troublesome network component… presenting tangible evidence of recent ROI to a CFO,.. contributing to business cases… to company 5/10 year investment and growth plans… it’s a value-add service to complement any other IT service… a solution that can assist with capacity planning… it can contribute towards demonstrating leading market edge capabilities, or identify what would need to be improved to achieve them… it can create justifications for the necessary investments for all of these things… all from the same data and all due to it being presented back in the same way.”

Actual Experience Benefits Many Stakeholders

One of our goals when we started Actual Experience, was for the language of Perceptual Quality to enable business leaders to engage more meaningfully with IT decision-making. Our customer went on, “CEOs, CIOs, CFOs – they’ll all love it. But the key differentiator here is that they’ll all love it not only because of what it can do and what it achieves for them in a tangible way, but because they can and will all understand it without the need for techno-speak translators… this gives what you do total and universal market appeal.”

As we meet more and more customers and prospects, the company-wide appeal of Perceptual Quality is gaining momentum. The graphic summarises this.

The Problem with Today’s SLA and How to Solve it

The Issue

Over the past 15 years I have worked in many different roles for both Managed Service Providers (MSP) and End Customers alike. The single biggest issue I have faced in both roles is the accuracy of measurement of technical Service Level Agreement (SLA) data and how that translates to the human experience.

As a Customer Services Manager for a Global MSP, I relied heavily on the technical SLA information collected by machines (and the interpretation of said data) to be an accurate portrayal of what my customers end users were telling them. All too often I would stand in front of a customer with an ‘all green’ dashboard only to have him bemoan the fact that his users were complaining of poor application performance!

My response was to explain (ever so politely) that our data was accurate and that it had to be an issue out of our visibility and control.

A number of years later, I joined a global financial services organisation where I was in charge of vendor management. One of my job functions was to meet with Customer Services Managers (CSM) from various vendors and discuss service management reports (…can you see where this is going!!).

Each month I would have any number of vendor CSM’s tell me that the dashboard was ‘all green’ and (ever so politely) explain to me that their data was accurate and that it had to be an issue out of their visibility and control!

For the first time I experienced the issue that plagues the vast majority of Service Level Agreements in operation today. That is to provide the customer and the vendor with an accurate measurement of the Technical Metric AND the Human Experience for any given service!!

The Solution

So off I went and scoured the 4 corners of the earth (…well just the Internet mostly) to find a means to fix the SLA conundrum and am happy to report that my search led me to one such piece of software that does just that…Actual Experience.

The product that they have spent the last 10 years developing not only allows me to accurately quantify the End User experience of any application, it also gives me visibility as to which pieces of infrastructure, in the end-to-end service chain, are contributing to their dissatisfaction.

I now have this clever piece of software deployed across our IT estate. It not only highlights to our own engineering teams where the problems are, it also gives me the collateral I need to explain (ever so politely) to the vendor CSM’s that although their dashboard is ‘all green’, there is a definite problem within their infrastructure and this is the device causing it.

I am happy to report that they liked our chats so much, they are now actively looking at Actual Experience as a means to capture and deliver a better quality of service to all of their end customers.

So End Customers and Service Providers alike, if improving internal and external customer satisfaction is of critical importance to you and your business (according to Aberdeen Group, at least 55% of you agree), then take a serious look at Actual Experience.

I did and it is solving my problem!

The Thinking Man sculpture at Musée Rodin in Paris. Photo by Satyakamk (Wikimedia Commons)

Are we an analytics company? I have begun to wonder. According to Random House Webster’s Dictionary, analytics is ‘the science of logical analysis’. When you look at what other so-called analytics companies and services provide, it seems to me that they collect data and visualise it in tabular or graphical form. This can be made to look interesting, but I believe this has more in common with reporting than logical analysis. In contrast, we are automating Prof’s expertise in order to analyse Agent #performance #measurement data. Auto-Prof. We are an Automated Expertise company.

Human experience change during data centre migration

The white line on the graph represents the performance ceiling – as defined by laws of physics – of the network and compute system observed by one of our Agents. When there are no impairments in the system, the graph converges on the white line. However, if there are network impairments affecting human experience, the graph diverges above the white line. If human experience is impaired by servers, the graph diverges below the white line.

It had been clear from our analytics that both network and compute infrastructure in the data centre were impairing human experience. The graph captures the moment beautifully when the application migrated to the new data centre, and impairments to human experience more or less disappeared. As expected, the ‘laws of physics’ performance ceiling of the system improved too. Happier users. A satisyfying outcome of the use of AE Analytics.

Internet Map by Matt Britt (http://commons.wikimedia.org/wiki/File:Internet_map_4096.png) Shows Internet Complexity

The attributable part relates to the fact that broadband service providers compete in a very competitive, price-sensitive market. But, because of the complexities of broadband, it is very difficult for providers to differentiate themselves in a way that is meaningful to customers. Consequently, broadband providers have simplified broadband marketing to the simple notion of speed. Sadly, having simplified things, it turns out that speed is misleading to customers too.

There is no standard for speed that the industry can rally around. Even if there were, measured broadband speed varies by time of day – in fact each time tests are run – and geographic location, and seems, in any case, always to be less than marketed speed. Worse, the variation of performance over time and location is important to users, but lost in aggregated national performance rankings. See, it’s got complex again.

Even worse! There is no reliable relationship between speed and the subjective human experience of a specific application. This is critical, because customers judge broadband service providers by the experience they have of the applications they use. For all applications, speed on its own is not a determining factor in subjective experience. There is so much more going on between a broadband user and the content or person at the other end.

This is where we hope to help with the BBFix Project. We will be able to provide the industry with a language and tools that allow broadband quality to be described in a way that everyone understands. In particular, a language that is specific to an individual user in terms of time, location and the applications they use. This, we believe, will be a step in the direction of closing the gap between expectation and experience.