Month: June 2012

Cloud Is The Buzz, But What About The SLA?

Published on by Howie Cohen

Cloud Is The Buzz, But What About The SLA? http://www.servicetechmag.com/I63/0612-1

Published: June 26th, 2012 • Service Technology Magazine Issue LXIII  

 

Abstract:

Everyone is talking about cloud computing but a great deal of writing is spent on the advantages of the technology. Moving services off premise can create a great deal of savings but there are risks that should be considered. Most of the risks are associated with the fact that services and service agreements favor service providers. The key to lowering the risk is thinking about the right questions to ask up front and considering the implications of the service-level agreement (SLA). Cloud is the buzz, but what does that mean to you? Understanding what is in an SLA and what questions should be asked of service providers is key to the stability, sustainment and future of any cloud based consumer. What should we consider?
Cloud Is The Buzz, But What About The SLA?

Introduction:

The cloud is the buzz! Now go out and get yourself some cloud. But what does that mean to your business or organization? It is critical to ask questions of service providers up front and understand the implications of doing business with them. The service-level agreement (SLA) should be considered up front as part of the business case. The idea behind this article is to spark some thinking concerning the SLA. In addition to seeing answers, implementers and decision makers should think about the best questions to ask. Before jumping into a cloud without a parachute, take some of these ideas and concepts into consideration.

Recently the Cloud Council [REF-1] released a practical guide to cloud SLA’s. It looks at 10 steps to evaluate cloud service level agreements [REF-2].

Online, you can find various calculators and planning tools to help identify and define the various properties and areas of concern with SLAs. What you don’t see is a lot of information concerning the implications of services, regardless of the SLA. One example of this could be international law as ub what happens when you do business with China [REF-3].

Does the SLA specify where cloud service transactions will take place? Should it? What kind of legal implications are there for doing business in the cloud? What are the costs of having legal representation overseas? What about downtime? How much will it cost to have more than one company with managed services? How many companies have lawyers involved during the evaluation and purchase phase of acquiring cloud services? How many companies have shadow IT involved in cloud services without even knowing the implications?

What about the US Government? How many experts are actively engaged at the program or project level in understanding the implications of cloud resources?

You need to understand what legal questions to ask as well as the technical questions. Today, SLA’s are heavily weighted in favor of the providers and that will need to change in order to meet the needs of service consumers.

Cloudbus offers up many considerations [REF-4]. What the agreement does say is that you can terminate the agreement. However, what it doesn’t say is what happens to your business or work.

There is a problem and it is a serious one; consumers are not controlling the market. The idea of this doesn’t make much sense, but it is the reality and you need to know what can be done about it, if anything. You also need to know what questions should be asked from the boardroom to the lab.

How many times have you installed a piece of software and scrolled down through 60+ pages to install or use the product? Let’s get this straight. Generally you go someplace or purchase something and you have “no say” in the conditions of the exchange other than if you don’t like it, you don’t accept. What if your husband or wife agreed and you didn’t know? What about your children? Just say, maybe, they click faster than you.

In the past, it was common to negotiate a price on an item and the conditions of sale. To a degree you can still do that! This is very important because it directly correlates to your ability to make choices on what you are willing to purchase, and what you aren’t. If I told you that somewhere in the agreement that you didn’t read, it says “If xxx.inc finds out that you have died, xxx.inc has the right to recover software sold under licensing and any hardware or media that software resides on.” Would that be okay? This may seem like a far-fetched example, but it can actually be shocking what is “technically” in some agreements, so read them.

This applies to the service level agreement as well. This is critical because lawyers are not making these agreements on your behalf; they are making these agreements on behalf of the service provider. It is like going to court without representation against a sort of ‘legal’ mafia.

Last year, I bought a phone from a phone service provider that buys its service from another phone service provider. I had to pay for the phone outright because this was a no-contract deal. Interestingly enough, they had unlimited voice and data with a payment model that would decline over time as long as I paid on time. What they didn’t tell you was that if you made payments early it didn’t count. They also didn’t tell you that the price would decline every six months, not every month. They didn’t tell you outright and clearly that the service they offered had bandwidth limitations. They didn’t tell you clearly that any of the phones offered for this service were going to be of the lowest quality compared to the parent services.

I started the service in December and the phone was only for business use. The first problem I had was the Bluetooth headset wouldn’t operate properly. I bought the headset with the phone as part of a package. You would think that the commands from the headset would work with the phone. I spent about $250.00 US on the package for the phone and about $55.00 US for the service. At this time I was primarily working from my home where I didn’t get service on the phone. I had to use the Google phone service and give everyone the Google number so that it would ring either my home office phone or my cell when I was out of the office. In other words, I had to use a service to use my service and in extremely limited conditions.

Frequently while driving from point A to B I would lose signal. I had been on some business calls and they calls were dropped. Not only was it embarrassing, it disrupted my business. Ironically, this service was not purchased from a small local carrier; it was an international service.

I called the company and asked for help. It was a challenge to get through to a person; it took time out of my day. I actually had to call back several times because I couldn’t wait on the phone as I had other things I needed to do. I had to find alternatives and work around the situation. And, of course, when I finally was able to talk to a human being, they were unable to help me.

Of course I am not going to name the company but here is an excerpt from their service level agreement:

“YOU AGREE THAT WE ARE NOT RESPONSIBLE FOR CERTAIN PROBLEMS:

YOU AGREE THAT NEITHER WE NOR OUR SUBSIDIARIES, AFFILIATES, PARENT COMPANIES, VENDORS, SUPPLIERS, OR LICENSORS ARE RESPONSIBLE FOR ANY DAMAGES RESULTING FROM: (A) ANYTHING DONE OR NOT DONE BY SOMEONE ELSE; (B) PROVIDING OR FAILING TO PROVIDE SERVICES, INCLUDING, BUT NOT LIMITED TO, DEFICIENCIES OR PROBLEMS WITH A DEVICE OR NETWORK COVERAGE (FOR EXAMPLE, DROPPED, BLOCKED, INTERRUPTED SERVICES, ETC.); (C) TRAFFIC OR OTHER ACCIDENTS, OR ANY HEALTH-RELATED CLAIMS RELATING TO OUR SERVICES; (D) DATA CONTENT OR INFORMATION ACCESSED WHILE USING OUR SERVICES; (E) AN INTERRUPTION OR FAILURE IN ACCESSING OR ATTEMPTING TO ACCESS EMERGENCY SERVICES FROM A DEVICE, INCLUDING THROUGH 911, ENHANCED 911 OR OTHERWISE; (F) INTERRUPTED, FAILED, OR INACCURATE LOCATION INFORMATION SERVICES; (G) INFORMATION OR COMMUNICATION THAT IS BLOCKED BY A SPAM FILTER; (H) DAMAGE TO YOUR DEVICE OR ANY COMPUTER OR EQUIPMENT CONNECTED TO YOUR DEVICE, OR DAMAGE TO OR LOSS OF ANY INFORMATION STORED ON YOUR DEVICE, COMPUTER, EQUIPMENT, OR COMPANY A STORAGE SPACE FROM YOUR USE OF THE SERVICES OR FROM VIRUSES, WORMS, OR DOWNLOADS OF MALICIOUS CONTENT, MATERIALS, DATA, TEXT, IMAGES, VIDEO, OR AUDIO; OR (I) THINGS BEYOND OUR CONTROL, INCLUDING ACTS OF GOD (FOR EXAMPLE, WEATHER-RELATED PHENOMENA, FIRE, EARTHQUAKE, HURRICANE, ETC.), RIOT, STRIKE, WAR, TERRORISM, OR GOVERNMENT ORDERS OR ACTS. YOU SHOULD IMPLEMENT APPROPRIATE SAFEGUARDS TO SECURE YOUR DEVICE, COMPUTER, OR EQUIPMENT AND TO BACKUP YOUR INFORMATION STORED ON EACH.”

“YOU AGREE OUR LIABILITY IS LIMITED – NO CONSEQUENTIAL DAMAGES:

TO THE EXTENT ALLOWED BY LAW, OUR LIABILITY FOR MONETARY DAMAGES FOR ANY CLAIMS THAT YOU MAY HAVE AGAINST US IS LIMITED TO NO MORE THAN THE PROPORTIONATE AMOUNT OF THE SERVICE CHARGES ATTRIBUTABLE TO THE AFFECTED PERIOD. UNDER NO CIRCUMSTANCES ARE WE LIABLE FOR ANY INCIDENTAL, CONSEQUENTIAL, PUNITIVE, MULTIPLE, OR SPECIAL DAMAGES OF ANY NATURE WHATSOEVER ARISING OUT OF OR RELATED TO PROVIDING OR FAILING TO PROVIDE SERVICES IN CONNECTION WITH A DEVICE, INCLUDING, BUT NOT LIMITED TO, LOST PROFITS, LOSS OF BUSINESS, OR COST OF REPLACEMENT PRODUCTS AND SERVICES.”
What Does This Have To Do With Cloud Computing?

The situation is the same for cloud services. Imagine if you will, for a moment, that this was a service you purchased for your corporation and you had a corporate agreement. When you buy cloud services, do you consult a lawyer? Do you present the service provider with a contract or does the service provider present a contract to you? Would you enter into a contract with someone that you don’t know with no mediation and no active discussion on terms? Would you enter into a contract with an international company without understanding the implications? We do this all the time and don’t realize it.
What Are The Implications?

Last year Amazon had a service outage that put companies down hard, but it didn’t violate any SLA and further there was no remedy offered to these companies [REF-5].

According to Ray Wang [REF-6]:

As calmer heads prevail, most CIOs, business leaders, and analysts realize that:

The intent concerning my phone service was to save my company money. I was under the impression that if I went with a global carrier that my service would meet my expectations. I also believed that if I had a problem, it would be resolved through a call to the organization. The result was that I spent more money up front, I lost money over time and I adversely impacted the relationship with the people I conducted business with due to service disruptions. While I was able to return phone calls and apologize, it was at minimum embarrassing. I wound up changing services to another global provider that had a good reputation and my overall service costs more than doubled. While the total costs of services are now more expensive, the tradeoff is that I have lowered risk. I have a better quality phone and rarely lose connectivity. The fact is that even with this carrier, I can suffer some of the same issues. My current carrier has a service level agreement similar to the other carrier. As we move towards geographically dislocated services we have to consider the service and legal implications. Having a service level agreement will do no good if it is weighted to the service provider and if there is nothing you can do to take action to remedy poor service.

Some folks say, “Just change your service.” Let us think about that for a moment. If you spent a lot of money moving to cloud services and you found yourself leveraging some of the proprietary options, you may find yourself in a bind. As a consultant, I have seen companies moving into cloud solutions without consideration of the SLA and without a lawyer in hand. Why? Because not unlike my effort to save my company money, a lot of cloud services are approached by people with good intentions from the bottom up. As these teams work to come up with cost savings and prove that leveraging cloud services can really cut costs and improve services, they may not have the expertise to understand the implications of the total cost (monetarily or otherwise) of implementation to the business.

  • Cloud outages are rare but can happen. While most organizations cannot deliver 99.5% up time let alone 90% performance, disruptions can and will happen. The massive impact to so many organizations last week highlights potential vulnerabilities of betting 100% of capacity in the cloud. More importantly, it showed that broad adoption does not equate with bullet-proof reliability. Most organizations lacked a contingency plan.
  • Cost benefit ratios still favor cloud deployments. For most organizations, the cost of deploying in the cloud remains a factor of 10 cheaper than moving back to the traditional data center or even a private cloud. Capital costs for equipment, labor for managing the data center, excess software capacity, and the deployment time required to stand up a server create significant cost advantages for cloud deployments.
  • Current service level agreements lack teeth and should be improved. Most organizations lack teeth in the cloud/SaaS contracts to address service level agreement failure. Despite all backups and contingency plans, clients should consider scenarios where core business systems go down. What remedies are appropriate? What contingencies for system back up are in place? Who is responsible for disaster recovery? Will the vendor provide liability and for what?

The bottom line is that you must proactively account for breaches In service level agreements In SaaS/cloud contracts. This can be done through a combination of contract provisions and contingency plans. Here are some suggestions recommended to clients:

  • Apply provision from the SaaS/Cloud bill of rights [REF-7]. Though written in late 2009, this document remains a best practices guide to SaaS contracting. Key provisions to apply include: Quality guarantees and remuneration, stipulate data management requirements, and on-going performance metrics.
  • Include service level agreements with teeth. Credits for free licenses for down time sound good on paper. In reality, down time when critical systems fail could result in massive financial losses. Contracts should apply risk on the potential business loss. Some clients include a provision that identifies compensation for a percentage of average daily business revenue during the time period of down time.
  • Reevaluate your Amazon deployment strategy. Believe it or not, Amazon technically did not violate its service agreements [REF-8]. To deploy a true backup strategy, organizations should add copies of their server instance in multiple regions and data centers as an added layer of protection. This ensures that a proper fail over occurs even if multiple regions experience outages.
  • Implement a real disaster recovery strategy. The Amazon outage exposed that many start ups failed to have a disaster recovery strategy [REF-9]. A number of solution providers now provide cloud disaster recovery. More importantly, these providers can recover physical or virtual machines in a cloud within minutes. Whether organizations can fire up a backup server in time remains the open question.

How many organizations are responsible for more than just themselves? Local, state, and federal government are moving into cloud strategies, banks and financial institutions are making groundbreaking moves on cloud computing and finally medical communities are moving “to the cloud.”
Conclusion:

It is critical to take into consideration the SLA and have a clear understanding of any user agreements. Most of us have the end in mind when we look to leverage technologies but we have to take into account the legal ramifications of glossing over these agreements. The SLA is generally presented by the vendor but there is no reason that an organization has to agree to terms without discussion or additional interaction with an organization. The bottom line is that consumers are responsible to make sure that the SLA is fair and that it meets their needs. The SLA should be addressed as part of the technology analysis while evaluating cloud services. If the SLA is overlooked a catastrophic technical event could make for a catastrophic business event.
References

[REF-1] Cloud Council, http://www.cloud-council.org/

[REF-2] Cloud Standards Customer Council.,https://cohenovate.files.wordpress.com/2012/04/pgcloudsla040512mgreer.pdf

[REF-3] James Fallows, “Doing Business in China ,” The Atlantic.http://doingbusinessinchina.theatlantic.com/

[REF-4] Linlin Wu and Rajkumar Buyya. “Service Level Agreement (SLA) in Utility Computing Systems,” CloudBus, http://www.cloudbus.org/reports/SLA-UtilityComputing2010.pdf

[REF-5] Arnal Dayaratna, “Why Amazon’s Cloud Computing Outage Didn’t Violate Its SLA,” Cloud Computing Today, April 29 2011, http://cloud-computing-today.com/2011/04/24/why-amazons-cloud-computing-outage-didnt-violate-its-sla/

[REF-6} R Ray Wang, “Monday’s Musings: Lessons Learned From Amazon’s Cloud Outage,” Forbes, May 25 2011, http://www.forbes.com/sites/ciocentral/2011/04/25/mondays-musings-lessons-learned-from-amazons-cloud-outage/

[REF-7] R Ray Wang, “Research Report: Customer Bill of Rights – Software-as-a-Service,” A Software Insider’s Point of View, October 12, 2009, http://blog.softwareinsider.org/2009/10/12/research-report-customer-bill-of-rights-software-as-a-service/

[REF-8} Arnal Dayaratna, “Why Amazon’s Cloud Computing Outage Didn’t Violate Its SLA,” Cloud Computing Today, April 29 2011, http://cloud-computing-today.com/2011/04/24/why-amazons-cloud-computing-outage-didnt-violate-its-sla/

[REF-9] Krishnan Subramanian, “Some Lessons From AWS Outage,” Cloud Ave, April 22 2011,http://www.cloudave.com/11886/some-lessons-from-aws-outage/

People Filters (TURNED OFF) and Deceptive Leadership

Published on by Howie Cohen

As a Community Manager I manage nothing.   I think the title is wrong actually, I should be called “Community Motivator” or “Community Influencer” or something else but manager is not what I am or what I do.   I listen to stakeholder problems most of the time and unless asked at this point I work hard to refrain from providing solutions.

I have discovered that as organizations have become more horizontal (FLAT) that people individually think of themselves as being smarter than others.   Being smart is one thing, but there is a dangerous trend that I am observing (not scientifically).  There are two areas that I am specifically addressing today.  The first is “the Facebook effect” which I mean that people are losing their ability to filter their thoughts when communicating over technical mediums. We can see this everywhere, reality television, news, in the office, with our children etc.  We are becoming a “say anything world” which is harmful.  This last week a few middle school children said horrible and disgusting things to a bus monitor.   This unfiltered behavior is not being addressed or brought to light enough in my opinion.   There are business effects as well.   One of the issues or trends is trickle down deceptive leadership.  As organizations are becoming flatter and more money and power is filtered to middle management, people are empire building at a higher frequency, but since it is distributed it looks more like this:

A common conversation would go something like:

Dave: Have you heard about x,y,z? Can I talk to you about x?

Robert: No, but I know all about that.  I read something on Google about that once.

Dave: oh, ok.

Robert: I have made a decision and we are going to do something.

Dave: What are we going to do?

Robert: We are going to do things and then other things, and once we do that and we become popular for the things that I just thought of we are going to do other things.

Dave: What other things?

Robert: I don’t know yet but I will be sure to let you know when I think of them, oh by the way I don’t like you because you are not my friend so I am going to have to let you go next week.

These are real conversations.  If you have experienced them or spoken to the Dave’s of the world you understand.   As people are becoming more and more connected through technology, they are becoming more and more disconnected socially through real life and real world interactions.   Their filters are turned off and they are becoming drunk with power.   Dave never had a chance, and maybe he has a family and maybe he cares about Robert’s success.  Dave will have to figure out what is next for himself.

But it is worse than that, Robert didn’t care and didn’t have any sense of guilt or sorrow.  The reason he didn’t care was because of the (WIFM) or what’s in it for me?  What Robert has today is some money and some ability to execute (a little power).  He doesn’t have enough for him to believe that he can control all of the people in his organization, but he has enough for him to believe that he can change the world with his ideas.  Did you get that?   He can’t influence the people who know him, but he can influence people who don’t. Through technical exchange like email, blogs, and social mediums.   Most of the people who actually know him don’t respect him.  He doesn’t listen and he doesn’t lead.  He just knows what he knows and that knowledge or his answers are the end of all discussions.    Over the years I have seen a trend of deceptive leadership.   I would say deceptive leadership is when someone pretends they are a leader but they truly make no effort to lead or follow for that matter.  They play a part and slink through an organization while stabbing people in the back, making back office drug deals while betraying people who trusted them.    Deceptive leaders destroy organizations like a cancer from the inside out because the people who work for them become victims.  Ironically, unless they do something really noticeably wrong the people who they work for become victims as well.

I think, I want, I need, I, I ,I , me, me, me.    They are so smart that they wind up focusing on the wrong problems precisely (see type 4 error)

I am writing this because I believe it needs to be said.   I am writing this because I believe this to be true.  I see it more often now than I have ever seen it in my career. These days I am in touch with a lot of people and this appears to be a trend(not scientifically proven) but through casual observation and various discussions.

I started discussing this with my uncle Mark Friedman who is a well-known Organizational and Industrial Psychologist.  Mark also blogs at http://bizcoach5.wordpress.com/.   Through my life and my career I have had the benefit and opportunity to talk about my experiences with Mark.  We always have interesting and productive discussions.   Recently we started talking about this trend of disconnected behavior and accountability.  One of the critical concerns is that organizations don’t invest enough in understanding the relationships between staff.   When dealing with people issues even though there is a cost with losing people or in a lack of productivity, there is no directly correlated line item number in a budget.  This lack of accountability is very costly but since there is no direct relationship with the budget it simply winds up costing the organization in the bottom line.   People are our most valuable resource but realistically these least accounted for.   As a matter of fact people don’t depreciate in value over time, they accrue value.  The organizational knowledge they gain will make a great difference to business operations.   Social networking and collaboration is now identified as one of the fastest growing and most profitable areas in business, yet we are leaning on technologies to make this happen and not looking at people and understanding relationships.    This makes it easy for the Robert types of the world to devalue and dispose of the Dave’s.

I believe this is a cascading socio-economic challenge that WE need to talk about and we need to start thinking of people oriented solutions as opposed to technical solutions to this seriously costly and harmful wave of behavior.

Question over Answer

Published on by Howie Cohen1 Comment

This week a study was published and highly publicized concerning microbes on the human body.  Through science and technology we constantly learn that we don’t really know a lot about ourselves.   Yet, I find that people talk in absolutes all the time.  When presented with problems in the world of IT most if us come up with THE answer.   I have personally observed what I call “I know” syndrome.    I would venture to say that we don’t even know what is really going on around us http://www.youtube.com/watch?v=MkANniH8XZE David Eagleman has a pretty interesting perspective on time relative to our understanding of the world around us.

I realize that I keep presenting and asking questions and I don’t provide a lot of answers.  This is because as I am getting older things are becoming more clear to me and by clear I mean abstract.   We humans really don’t know a lot of anything.  We are becoming fixated with technology and what we consider “being smart” but from my perspective becoming smarter narrows our ability to be dynamic.  I would say that the more answers we believe we are capable of providing actually limits our ability to solve problems.

A 5 second commercial on the Science HD channel with Morgan Freeman really affected my perspective of things a few weeks ago.  He said, “answers are terminus, the questions are where it’s at.” I was taught at an early age to ask questions and the result was initially not great.  As a kid, I would ask “why” to everything and then challenge everything.  I really can’t speak to my logic although I can say I enjoyed the feeling of walking outside of the group.  Now I believe there are times to ask tough questions and there are times to go with the flow.  Going with the flow doesn’t mean that going with an answer to a question holistically.  It means that we may recognize an answer and question this answer when the time is right.

The answer is that there is no answer, well not 100% because we don’t know what we don’t know.  If we stop asking questions we will have a hard time being dynamic and flexible.   That is the message for anyone reading this today.  Stop being so smart and knowing it all and value your questions more than your answers.

Happy Fathers Day Dads!

(COHEN) Trust GONE VIRAL! (SUTTON) My point is this: Trust is a violent act.

Published on by Howie Cohen3 Comments

You think that we should have a conversation about TRUST?   I talk about trust and leadership a lot and I believe that you need both but as pointed out to me by my good friend Matt Sutton, we are diluting the value of these words.  In a paper put out by Harvard Business Review they talk about leadership and of course all of us collaboration types run to the well.

You know what the truth is though?   The truth is that leadership is mostly dead.   It is like that Pixar movie Wall-e trying to find trust and leadership in an environment that we consume and kill everything in our path.   A few years ago I asked leadership in my organization how many Industrial Psychologists we have on staff in order to help teach leadership how to understand and work with people better, the answer was none.   Not one, because while people are the most important asset, they are only important when the organization is lacking in some capacity.  It is like air, you aren’t missing it until you have none.   Leaders today are more than not politicians that say things they don’t mean or can’t support and trust is the word they use to make it sound as-if.

In my years of working I have seen great leaders and they are few and when I worked for them or worked with them, I knew because trust was an activity, more than just a word.

Have you ever had your life in someones hands?   Have you even had someones life if your hands?  We all have but we have forgotten because it is easy to forget who and what we are.   It is easy to put out the list of 5 things you need to know to be a great leader.  It is hard to do the work and lead from where you are.

Sutton Says

If you’re a worker in the trenches trust means two things that are important to both leadership and to environments were you work with people remotely.

1. Knowing Right from Wrong.  In the absence of religion, corporations establish “Core Values”, “Morality Clauses”, “Codes of Conduct”.  You get the idea.  Something to provide a pseudo moral compass.  Ironically I have never seen the number one core value (or whatever you want to call them) on corporate literature.  Making money.

We know the business needs to make money so the real question a person working has to ask, “Will meeting the profit goals of the company come at my expense?”  In order for workers to even begin having trust in their leadership they need to know that the powers that be aren’t going to be the ones lighting our final cigarette in front of the firing squad of profit.

2. Gauging whether our coworkers or leadership know right from wrong is tricky.  Historically that kind of insight usually comes from life-n-death situations where self-preservation strips away all of our pretenses.  In the corporate world it usually comes from an equally scary dice roll of putting ones self out there to be judged by your peers and see who steps to support you. But isn’t that the whole point of trust?  To know whether your peers and leadership has your back when you do take risks or get in tight spots?  Few are the people who take such risks and fewer are the ones who survive it.

Finally,  if you are at all interested here is the discussion that ended with more discussion.  Don’t look for correct grammar it is a chat but the conversation was interesting.

Matt:

can I comment on how much I detest blogs and articles where the

writer begins by commenting on how smart they are?

The Four I’s of Leadership Communication

this link is a great example

Howie:

I didn’t have a chance to blog yesterday and I need to… maybe this

should inspire me

Matt:

And I need to express this: I want to punch the next person who talks

about “Building Trust” in terms of Leadership and Communication

Howie:

You would punch me then, when I see you, you can punch me.

Matt:

My thing with “trust” is that it is a nebulous and vague attribute.

people know it’s essential, but no one knows really how to do it

Howie: Integrity, intent, capability, results (trust) we can break it down more (M.R.COVEY)

who are we? what do we believe?

what do we want?

What tasks are associated with that?

What have we done? What are the results of our past interactions?

Do we bad mouth others?

Are we honest?

Are we too honest?

Are we enablers for us or enablers for me?

Does this dress make me fat?

the answer.. is.. not the answer itself but your response (overall)

in other words.. I trust you because I know that you will tell me what

I need to know from your perspective even if it hurts a little

but the pain of my self-consciousness is less important than knowing

what I can / should do to right myself.

You have my trust and that is important. It is number 1 important.

If you call me and you need my help, I will be there for you no matter

what time and what circumstance.

That is trust.. it is clear and to the point and detailed not vague.

Trust is not something you give away like candy, it has to be earned

or give via a token.

It has to be nurtured like a flower.

If you leave it without water it will die, if you don’t tend to your

relationships trust diminishes over time.

Matt: I am calling you and talking to your wife right now.

Howie: UH OH!

Collaboration Pattern

Published on by Howie Cohen

Collaboration Pattern.

Collaboration Pattern

Published on by Howie Cohen2 Comments

I get a lot of phone calls and emails about the “secret sauce” with community management.  The key to having a successful community is facilitation in context.   Matt Sutton, Wendy Woodson and I have been working on the challenges of collaboration for a long time.   As a Community Manager in the DoD, I have found that everything begins with People, Process, Methods first and Tools last.  Most programs and projects start with tools and try to make their people, process and methods fit.    I am posting some slides which is an introduction to the collaboration pattern and I will follow with some short notes on the different areas and lessons learned.

CollaborationPattern

Community Management Metrics

Published on by Howie Cohen1 Comment

Community Metrics in Context

In late 2011 I started working for DISA on the Forge.Mil project.   Forge.Mil is a really great concept that was born out of the need for a change in the Department of Defense concerning software development and application life-cycle management.  Some the concepts that attracted me to Forge (that is what I call it), was that it integrates knowledge management, information management, software development and object relational mapping to authoritative data.   As a System Integrator, I would always have a challenge when it came to documentation and information management aside from required documents for information assurance.   Forge enables teams to share data in context of IT in ways that are similar to other knowledge systems but essentially designed for software development and software projects.

This is a unique and challenging situation because most of the time in social communities people have a desire to communicate and collaborate.   In this environment with hundreds of different projects and thousands of users motivators for the individual developer varies.   In other words, some kids may not want to play in the sandbox, while others excel at building castles together.    As Morgan Freeman says while discussing the universe “Answers are terminus, it is the questions that are where it’s at.”   With that said, I had and have a lot of questions about the community management concerning IT related work.

How do you know what you need to look at to determine what is happening in your community?

I have spoken to a lot of people about this in the past few weeks and I have scoured the internet looking for questions and answers to consolidate a nice list.   One of the first things to consider though is asking yourself about the purpose of your community.   Who are my community members? What does the population look like?  What do they do on a daily basis?  What makes them different from each other?  What makes them the same?  Questions like these really help flesh out what you need to measure.

I put together a document on metrics and it was really a mashup of data from a lot of community managers, blogs, discussion posts and some publications.   I am posting most of the content here so that if you happen to stumble along here, you can take what you need and put together your own thing.   I have links to most of the authoritative data sources but if you come across something you wrote and I didn’t source it properly (PLEASE) let me know and I will include the proper reference.  I am not looking to take credit for the hard work and thought of others.

Think -Wait-Think- Do..

“We need to first define the problem.
Albert Einstein once said:
“If I had an hour to save the world
I would spend 59 minutes defining the problem
and one minute finding solutions”
And I find in most organizations
people are running around spending sixty minutes
finding solutions to problems that don’t matter.”
~ Stephen Shapiro

The information included in this post is a primer to get you thinking about what you need to do in your organization.  These are not the answers, these are the basis for the questions.    Most technical people think about the answers as quickly as the problems present themselves.   As a “leader from where you are” it is your job to help keep the solutions at bay until you determine that enough questions have been asked.   Additionally, the process should never end.  We should always ask questions, tune, adjust, qualify, quantify and wonder.  If you want to talk about the technical side of this work contact me separately.

Context First

Just remember to always keep the qualitative metrics in mind first as you consider the quantitative metrics.   One of the first posts that I came across that I thought was very interesting located here talks about a table with some basic attributes of consideration.  I added some attributes in the paper I put together but I could have very well just stuck with these.

“It’s either quantifiable or it’s not measurable”

“It’s either quantifiable or it’s subjective

“We have to quantify it in order to measure it” – Dr. Mel Schnapper, PhD

Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force:

“There are three kinds of lies: lies, damned lies and statistics.” – Autobiography of Mark Twain

Program Managers must understand what the total cost of implementation is. How do we capture a baseline? What is the total cost of ownership? What is the total cost in investment? What are the estimated savings? How do we know that we are achieving our goals? How do we know that we have growth? How do we know that this project is sustainable? How do we know where to course correct? What are the measures of performance? What are the measures of effectiveness? How do we define “effective”? How are we measuring success?

Community Goal: Drive Reuse

It is vague to consider a generic goal of “reuse.” There are a number of community components that can be measured for their reuse-ability.

• How do we know if people are informed about new features?

• How do we offer training on these features?

• How do we know if people are using these new features? If not using – Why?

• Are they using other web sites? Are they linking to them?

• Are they using their tool as a SharePoint or as a code collaboration tool?

• What are linkages / commonalities between projects?

1:Pre-Built Components Reuse

• Goal: Reuse Pre-Built Components

• Question: How much interest is there in each component that is meant for reuse?

• Metrics: Number of downloads, number of posts on the discussion posts around the component

2: Code Snippet Reuse

• Goal: Reuse Code Snippets

• Question: How many different places do code snippets intended for reuse appear within the community?

• Metrics: number of projects where this code snippet is being used, and/or where it has been “forked”

3: Knowledge Reuse (non-source code)

• Goal: Community Members are sharing knowledge within the community.

• Question: How often are users looking for information among the entire community?

• Metric: number of attempts to search for information community-wide on a particular topic.

4: Expertise Reuse/Reputation Management

• Goal: Users are able to find community experts that can help or guide them.

• Question: How would a user find an expert on a specific topic?

• Metric: determine which are the most popular community-wide search topics -> determine which community members post the most information about the most searched topics.

5: Trust in Reuse

• Goal: Users within the community trust that the community is a place to find valuable, trustworthy answers

• Question: Do users believe that they can rely on the information they are getting outside of their silo?

• Metric: while this is probably best learned through surveys, the number of hits/interest around specific topics that are demonstrating a high level of activity can also provide evidence of success in reuse trustworthiness.

(https://ctf.open.collab.net/sf/wiki/do/viewPage/projects.community-mgmt/wiki/DriveReuse)

Creating and Managing a Healthy Community

Growth

For growth, it is how many people are invested in what you’re doing that matters.

In terms of Growth, we look at:

• Twitter followers/Fan (LinkedIn, Milsuite, other) page members/social media friends.

• Blog Subscribers.

• # of Active commenters.

• Member registrations.

• Unique visitors.

• Ratio of posts to comments, types of comments.

• # of Message posts, if a forum.

• # of Conversations over a month period.

Presence

How visible are you in your space and how does your visibility measure up against that of other defense communities?

In terms of presence, we look at:

• Buzz over a 30 day period.

• Types comments/posts written about Forge.Mil – mentions (linked or unlinked).

• Who authored the mention – client, colleague, recognized social media contact, influencers, etc.

• Where was the mention located?

• How often does your community share your content?

Conversation

Presence is who’s talking about Forge.Mil, Conversation looks more at who Forge.Mil is talking to and the effectiveness of those conversations. Measuring the types of conversations Forge.Mil leadership and Community is having is an important metric because it shows leadership where your time is being spent and how people are engaging with you.

In terms of Conversation, we look at:

• Breakdown of the types of conversations being had– support-based, link sharing, friendly banter.

• Time spent on each conversation group. What’s more cost-effective – social media or phone/email, defense connect online chat?

• Whom you’re conversing with – customers, prospective customers, colleague, outsiders.

• Conversation spread and growth?

• Actionable knowledge learned about core audience.

Sentiment

More important than simply knowing Forge.Mil is being talked about knowing what people are saying about Forge.mil and how that’s changing over time.

In terms of Sentiment, we look at:

• Emergence of Evangelists – onsite and off.

• Ratio of positive/neutral/negative mentions (i.e. satisfaction).

• Forge users recommending the community, passing it on to friends.

• Frequency of community members responding to/helping other community members, overall “vibe” of the room based on tracked interactions.

• Community members defending Forge.Mil on negative blog posts and feedback.

Conversions

In terms of Conversions, we look at:

• Community member, followers, frequent blog commenter, etc).

• Customer loyalty forum (Charter) community conversions – how many times do they refer?

http://outspokenmedia.com/social-media/how-to-measure-community/ (great content)

Metrics as an indicator

A community must support business goals and the current (and prospective) community members themselves.

  • If the business goals are not defined, the community risks being feature-driven and may suffer from shiny-object syndrome.
  • If the community members are not involved in the success definition process, the community risks being irrelevant to its members.  (Community Charter)
  • If business goals are undefined, or if community members themselves are not involved in the definition of the community (it’s for them, after all), the community’s risk of failure grows substantially.

There are generally two types of metrics

Qualitative Connecting the dots can be challenging, since the points of data capture for qualitative metrics are often two or more degrees of separation from the data.  Qualitative information is contextual information derived by the relationships of data points.   Qualitative metrics takes variables, attributes, values and relationships into consideration in order to determine the likelihood of a condition.  Qualitative information / data is based on a deep understanding and the correlation of data.

Community example:

There are various interactions in the community that can be measured by frequency of visits or downloads but we may not know why.   This is an important factor as understanding the drivers can give us the ability to reinforce successful behaviors.

What does it mean? Interpretation of the data once captured.

Attraction -The ability to attract an initial audience.

Attention – The ability to ‘reel them in’ and have them go deeper into community content

Adoption – the ability to ‘convert’ them into Community users have them contribute to discussions.

When placing the aforementioned metrics into the 3 framework categories above there is a clearer understanding of what we are actually measuring:

Each of these has a quantitative component although it is difficult to measure external connectivity (meaning interactions that occur as a predecessor or successor as a result of the interaction in the community outside of the environment.

Quantitative Quantitative metrics are gathered directly through the observation and measurement of data.  There is a high degree of transparency and a direct correlation between action and outcome with quantitative metrics.

The data points can be classified as large Q for quantitative or small q for qualitative.

Attraction – The ability to attract an initial audience.

Attention – The ability to ‘reel them in’ and have them go deeper into community content

Adoption – the ability to ‘convert’ them into Community users have them contribute to discussions.

When placing the aforementioned metrics into the 3 framework categories above there is a clearer understanding of what we are actually measuring:

Attraction (Q,q) Attention (Q) Adoption (Q,q)
Visits Page views per visit User Creation
Unique Visitors Bounce Rate % Returning Visits
% New Visits Length of Time on Site Interactivity Rate (q)
% Users by service First time contributions Account age
% Users by organization Most active members Content ratings
% Users by agency Mentions by influencers New posts per month
% Users by unit Overall project activity Reputation changes
Page views overall Ratio: Views / Post Topic activity by project
Awareness (q) Ratio: Posts / Thread Individual project activity
Inbound Links Ratio: Searches / Post % Content with “tags”
    Innovation (e.g. # new product ideas sourced from community) (q)
    Member Satisfaction (q)

Other potential factors (some duplication)

Visitors Metrics: Such as Unique Visitors, Bounce Rate, Pages per Visit, Pageviews, Time on Site, Keywords, and Referring Sites

Members Metrics: Such as New Registrations, # of Active Members, Completed Profiles, Pages per Visit, Pageviews, and Time on Site

Contributors Metrics: Such as # of Edits, # of Comments, and # of New User Generated Content

Evangelists Metrics: Such as # of External Invitations, # of ShareThis external shares, # of Mentions on social media sites (e.g. Twitter)

Leaders Metrics: Such as # of Active Admins, and # of Active Moderators

http://blog.wiser.org/metrics-for-the-busy-community-manager/

Getting a Baseline

If we have 1,000 members in the Forge.Mil community and we do some basic analysis and learn that each week there are approximately 50 new threads posted in our project discussions and about 500 replies.

These three pieces of data are a good baseline for starting to look at ratios. A bit of quick division and your ratios come out like this:

Ratio of posts per member, per week: 0.05 (50/1000) (baseline)

Ratio of replies per member, per week: 0.5 (500/1000) (baseline)

Scenario 1 – Members Up But Engagement May Not Be Keeping Pace

Fast forward one month and let’s say our community has grown to 1,500 members. This is a huge increase, but are the ratios keeping pace with the growth? Let’s say, for example, that there are now 100 new threads posted (a lot of the new members introduced themselves) and about 650 replies each week.  Your new ratios look something like this:

Ratio of posts per member, per week: 0.0666 (100/1500) (increased)

Ratio of replies per member, per week: 0.4333 (650/1500) (decreased)

The ratio for posts per member (per week) has gone up thanks to all those new members introducing themselves to the forum. However, the ratio for replies per member has decreased and hasn’t kept pace with the growth. Using these findings as a starter, a quick look around the community may reveal that all those new members are introducing themselves but are not engaging or replying as much.

If you can not measure it, you can not improve it.
– Lord Kelvin, 1883

Scenario 2 – Members Down But Is The Quality Better?

In another example, let’s say community experiences a slight loss in membership, dropping down to 900 members, but the number of posts and replies stays the same. The new ratios are:

Ratio of posts per member, per week: 0.055 (50/900) (increased)

Ratio of replies per member, per week: 0.55 (500/900) (increased)

The ratios above show that community is actually looking very healthy, and so the decrease in membership is likely represented inactive members.

Ratios are a simple way to start putting context to data.  In the first example, the community numbers increased, which looks great, but the number of replies didn’t keep pace with the growth— leading to some actions or follow-up with the newer members. In example two, the overall numbers dropped, which looks negative on the surface, but the ratios show us that the community is actually healthier now.

(http://thecommunitymanager.com/communities-and-the-ratios-that-bring-insight)

Metrics Review: reviewing your performance at least every month, and compare month-over-month (MoM%) and year-over-Year (YoY%) growth rates in a spreadsheet

Other notes:

NOTES:

Here is some other relevant information.  This comes from some of the collabnet notes in their community wiki https://ctf.open.collab.net/sf/wiki/do/viewPage/projects.community-mgmt/wiki/CommunityMetrics

Community Metrics should be

  • Based on agreed upon, communicated community goals
  • Centrally managed by the Community Manager and Internal Community PM
  • Communicated ‘up’ to stakeholders in a way that provides information about community goal progress, community health, and ROI
  • Communicated ‘down’ to the community in a way that promotes the community and identifies opportunities for community growth

Community Metrics should be planned to come to an agreement on

  • The various aspects of the goal: For example, the goal reuse can include: code component reuse, code snippet reuse, knowledge reuse. Each aspect of the goal is itself a sub-goal that will be further analyzed and measured differently
  • The questions around the goal: For example, the sub-goal of ‘code component reuse’ begs the question “How many times have reusable code components been reused”
  • The metrics to be collected regarding the goal: For example, code component reuse’ could be measured by collecting: # of times reusable code components have been downloaded
  • The methods employed to provide the goal results: For example, how/when will the metric/s data be captured
  • Should the metrics data be trended over time: For example, Should the metrics data be compared against other data for relevance and further analysis

Measuring Community Health

Measuring community health is not looking at traffic numbers or page views. A community is considered healthy when:

  • The community goals are being met, such as support questions are being answered
  • Attitudes in conversations are light and friendly
  • Conversation is two-way or more, but not just single posts
  • Issues are being resolved and needs are being met

Community health can NOT be determined by simple numbers. It takes a community manager, or several, to read through conversations.

A community is considered unhealthy when:

  • Community goals are not met
  • Conversations are non-existent
  • Flame wars and arguments are taking over the community
  • Community members are unhappy with the interaction, or lack there of

Type of messaging that reveals the health of a community:

  • Messages of appreciation
  • Messages with solutions to problems posed by community members
  • Requests for information are answered
  • Conversations are friendly and sometimes lengthy
  • Complaints are directly addressed by the community manager or other company staff
  • Community leaders emerge from the conversations

Good luck!

Keep reading and writing!  Keep asking questions and publish what you find, we can all use the help.  Remember people first, context and scope (qualitative) and the quantitative should help with the qualitative.    If there is an interest, I can post some information about tools but most of the focus is on people, process, and methods.. cheers!