Response to Heather's Question on Metrics

I want to thank Heather for commenting on the previous post. And while I cannot take her up on a dance off (we haven’t met face-to-face Heather, but large overweight bald white guys are not a pretty sight on the dance floor... I could trip over my two left feet and kill somebody accidentally), I felt her question deserved a response. I decide to do this as a post rather than a comment in reply because long comments are even more tedious (if that’s possible) than long posts.

Also, I apologize for any errors in this post. I had exactly 30 minutes to throw it together before my next meeting.

Investors decide the validity of metrics. They usually decide by evaluating the return along two factors : efficiency and effectiveness.

Efficiency is about reducing the cost and risk to capital. Said another way, whoever can do more with less wins. Effectiveness is about value: with the same amount of capital I can extract more value than my competitors. Both measures are relative to competitors. They are not static.

In Employment Branding, I believe most investors think in the following ways:

Efficiency: By investing in Employment Branding I reduce transaction cost per dollar employed, or, conversely, get more transactions done with less dollars. So if its between putting another dollar into hiring a recruiter or putting a dollar into blogging, I want to know that I can get more done with a blog than I can with a recruiter. Everyone starts to chuckle “Now you are just off in left field.” Actually, no. Look at the total end-to-end recruiting and hiring transaction. In a “bad market” (labor demand outpaces labor supply in positions that are core to competitiveness), more dollars go to front-loading the transaction than to maximizing throughput later in the process. In English: it costs more to source relative to how much you spend on screening, interviewing, closing and on boarding.

If more dollars are front loaded into the system, then the efficiency measure is “Can you source more people with less money, or more people for the same amount of money?” In this respect most corporate blogging (but especially your blog), provides a good return. Microsoft competes against a lot of different companies for marketing talent. At the same time, Microsoft is ever more dependent on marketing talent to expand its markets (there are only so many office suites any one person can buy). In a buyer’s market, differentiation between sellers is key. As the volume and velocity of information increases, rising above the noise is absolutely critical. In a hiring transaction there are only two ways to do this: develop low-load / high value relationships and provide a convenient location for buyers to gather data that is meaningful for them in order to differentiate various competing offerings. Blogs do a really good job of both.

“But wait – how do we measure that?” In this case repeat traffic is not a bad measure. Perfect? Absolutely not. But better than a guess. Repeat traffic from IPs that you want to target is an even better measure. Creating an association between your topic, IP targets and source of hire would be yet another.

Effectiveness: By investing in Employment Branding I (investor) increase the quality of the candidates who will actively consider employment opportunities at Microsoft. Of course this is tricky, but not because the net effect of blogging is difficult. It is difficult because “quality” is a measure of variance to specification, and our specs (job descriptions) are not highly predictive of value delivery probabilities. So it is difficult to determine whether you get “better” people if you don’t know what “better” means.

But let’s get beyond that for a second and assume that we know who the stars are within Microsoft. The “existence proof” you are seeking for the effectiveness measure of your blog is a conversation. Talk to the new hires in marketing that started at least 3 months (an arbitrary number, but intuitively long enough to allow your blog to get traction and inbound links) after you started blogging and ask them whether:

1. They had read your blog prior to starting work.
2. The blog played any part in their decision to join Microsoft.

Now wait for six months and go talk to the hiring manager of all the new hires. How do they stack rank (which is a baloney methodology, but most investors will buy the output), and what is the correlation of those who are “A”s to those who said “Yes” to both questions. Compare that to the people who said “No."

All the statisticians and social scientists who may be reading this can blow holes all day long through this methodology. And they would be right as far as the science goes. But empirical validity is not the problem you are trying to solve. Continued investment is. And most investors would take a positive correlation as “proof” that they are getting a good return for their bucks.

Overall, you have to remember that when a new system is created the measurement is largely along “belief proof” and “existence proof” coordinates. Do you have any data to support the claim that you were able to source more people than before you started blogging? It could be as flippant and anecdotal as one sourcer’s word. It could be more detailed: you could do an alignment survey of the downstream clients of your service “On a scale of 1 to 10 rate how effective blogging is in getting you more and better candidates.” Where you have a high degree of standard deviation in response you have little data. But where you have low standard deviation, either good or bad, you have basic proof.

One of the hardest things that we do in business is to take intangibles and turn them into tangibles. Harder still is to take indicators and turn them from lagging (traffic) to leading (probability that any future candidate will have made a buying decision based on your content). And then there is the hardest job yet… making all meaningful to investors without killing the goose that lays the golden egg. I understand that this, more than anything, is your basic concern. But just because it is really, really difficult does not mean that it is not worth pursuing. We need to get more investors to keep this train rolling, and at the end of the day we are accountable to the value they perceive they get from us.

(And that’s why I pay for my own blog and do it on my own time – I don’t like most investors. ;-) )

 

Comments

[martin snyder]

hey hey- what's with the chubby bald guy references ? That's MY avatar your talking about.....

this topic has been cross posted all over the place, but here is a response I added to another blog:

Sure you can count page views, links, hires, etc. etc. but how do you measure influence? How do you measure the impact of an idea? It's like trying to measure a single snowflake in an avalanche; it may or may not tell you anything about the larger phenomena.

If Heather, or any other blogger, posts a highly novel or provocative idea, and receives one single click, what does that tell you? Well, if that click is the Secretary of Labor or the CEO of GE, and that person takes action based on the idea, your 'metric' is a joke. If that person was looking for information on pimping her ride and landed on the wrong site, that one click means less than nothing.

This is not even to speak of the effect of blogging on the blogger themselves; the possible changes wrought by study or deeper thinking about various topics, or the clarification and validation or rejection of certain positions that can change future efforts.

Measurement in media and throughout business, is critical, and there are a lot of things that can be done better by better measurement. But blogging is one of those things where the 'metrics' can't yet be a reliable mirror or indicator of results.

I think Heather was closer to the real truth than Sumser. Check this article for an interesting summation of this concept:

http://en.wikipedia.org/wiki/Emergent_property

[Jeff Hunter]

Martin -

Let me say that it is an honor to have someone of such intelligence and thoughtful purpose commenting on the blog. I don't agree with you, but that is the beauty of this enterprise. I would ask: when is that magic moment when 'metrics' can be a reliable mirror or indicator of results. If you don't need an existence proof now then when will you? Or will this be the kind of creative acitivty that needs no justification, because to justify it will be to denigrate it?

Seriously, one bald guy to another (we have to stick together you know), the conversations we have had about the video game industry have been inspired. Are you saying that metracritic ratings on games shouldn't count, because the games are billed as "movable art"?

Here is where people with your depth and intelligence need to explore, because it is in fact the crux of our collective business future: how do you make the intangible tangible without destroying it? How do you take something as delicate and beautiful as a flower and not rip it out of the ground in order to find out how much pollen it contains? I don't know the answer to that question, but I don't buy that just because it is difficult that seeking that course will necessarily be destructive to creative enterprises. I am more of an "and" guy.

Again, thanks Martin. It is really great to be out here talking with you.

[Rob McIntosh]

Now we are finally getting somewhere. I have been watching this whole discussion around the blog network for the last little while and poking my nose in from time to time but finally I think we are starting to hit on the essence of the real issue here. Just because it is squishy or hard to put a finger on it does not mean we should not try and measure it.
I am not saying there is a magic solution here to measuring what some feel is immeasurable, but since blogging is now being considered by many as a viable marketing and talent attraction tool, then it is time we at least started trying to measure what works and what does not.
Heck, if we all sat back and cried "oohhh, that is too hard", then we would have had Edison give up on inventing the light bulb after a few attempts vs. the many he took to get it right.
Now is the time that we starting poking and prodding on how to duplicate/replicate the success of what blogging can and should be!

[Heather]

Oh yay, I love hearing Martin's comments!

So Jeff, first, we know you are smart (and you say you are large and bald). I haven't read anything like that since I was in college. Yowza! Your theory breaks down (early) at the point that you 1) assume that investors really care what a single employee does and 2) assume that we don't hire people that are competent to make the determination of HOW to get their jobs done themselves. What you describe is my worst freaking nightmare. Why make it so hard when it doesn't have to be? "Hard to measure" does matter when you have enough anecdotal info to make a go/no go decision and the data gathering sucks up time.

The fact of my situation is that nobody asked me to blog, ever. I was given a job to reach out to people in marketing, make them think about working here, make them *want* to work here. How I decided to do that was up to me (and I was barely able to scrape together the few brain cells it took to figure it out...as if). Investors want to know whether they can get more done with a recruiter or a blogger? Really? Do you think that actually comes up at shareholders meetings? Shareholders want the best senior leaders driving business decisions. They couldn't give a rip about me and my blog, nor should they. Now, I'm not sure you are meaning "investors" literally, but if you mean "management", my simple recommendation to them (in the generic scenario yu describe) is that they hire better people that can make decisions for themselves, if that is not what they are seeing and if they feel the need to have that level of control over employees blogging.

Repeat traffic is a measure? How so? (and how do you measure "repeat" traffic?) I know for a fact that at least one person comes to my blog regularly to look at my picture (hey, life is weird). I'm confident that some people come to my blog because they don't like me. My mom visits my blog regularly to find out what I am up to. There are tons of people that visit my blog that don't have the kinds of marketing backgrounds that we would hire at Microsoft. These folks are traffic. Yet, there's some value there that's equal to the value of traffic from people that match the target profile I'd look for? What if my posts suddenly stink and I start to behave very oddly? People might come by for the spectacle. My traffic will spike (have you seen all the Alaska Airlines comments on my blog...and don't even get me started on blog spam traffic). Traffic is not a generic measurable that equates to value for bloggers or shareholders. I think this is Martin's point. It's how they *feel* about what they see and whether that compels them to do something now or later.

So let me give you a scenario. Let's say that these "investors" of yours say that the number of hires they would want to see (because they are intensely interested in my job?) as a result of my blog is 2 (just an example); the number of hires that justify the time (my salary) investment. Let's say that during the year, five people e-mail me and tell me that they just joined Microsoft BECAUSE of my blog this year. To me, that's anecdotal but the investors should be satisfied, no? Would they prefer that I then spend all my time interviewing all the marketing hires for the year (not possible, by the way, it would be my full-time job)? See the thing is, we do have team metrics. But what you have to consider is that blogging is part of an integrated outreach program. It's one spoke in a wheel. If I am doing my job, my target blog reader is also seeing me via their professional organizations, they are talking on the phone to Maria, they are part of my Jobster network, they respond to job postings that I facilitated through deals with theLadders, they are meeting recruiters at events, they see announcements about Fortune's best companies to work for (and so on). Our metrics look at whether we influence hires through our meta efforts(meaning we get to keep our jobs), how we impact the best hires (top performer analysis), but to try to carve of one relatively minor piece like blogging is silly. It's like measuring "talking" for recruiters. It depends on what you say. You are measuring quantity but I care about quality. As I keep mentioning, blogging is not a sourcing tool, it's a brand building tool. Hires are the bonus.

Jeff, you mentioned that you blog on your own time. I think where we aren't connecting is that you are approaching blogging as a business leader that wants to decide whether he can justify his staff blogging at work; I'm approaching it as a blogger who has the luxury (meaning trust of my management) of deciding whether blogging is an effective part of my job (it's the fact). We don't participate in the same blogging microcosm. In my environment, I'm trusted/empowered to make decisions about what work to do to accomplish my goals. This is why I feel so free to blog about myself (personally). I'm not hearing the same thing in your scenario (not necessarily meaning EA, just the scenario you describe)So, I'll say what I have said before: blogging is not for everyone; a successful blogger MUST have the trust of their management to make decisions about their blogging activity (meaning IF they blog, WHAT they blog about and how often).

My advice: if management is intensely interested in having control of blogging activity, then don't do it (bloggers, run, don't walk). managers/investors, take your money and put it into something measurable so you can have your needs for quantification met. Because the metrics turn a blog into something completely different. If it's about filling individual jobs, invest the money in your career site, buy some LinkedIn job postings. But don't tread on my blog. To put it bluntly: "hands off or the blog gets it!"

By the way, I love that we can share our thoughts about this (I know that we are both very passionate about the subject, though I wish I had Martin's eloquence). You know I have a ton of respect for you and your intellect and opinion. We are just miles apart on this. At the very least, this discussion has really made me appreciate my chain of command and the corporate culture at Microsoft that allows me to do my work. Because the scenario you describe represents something very different.

[Jeff Hunter]

Heather -

Thanks very much for your great comment. You are right - we are worlds apart. I'll let our respective posts and comments speak for themselves.

One thing that I want to make clear: I started blogging before I got to EA. I don't blog on EA's behalf, and I am not held accountable for my blog achieving anything for EA. However, in the things that I am accountable for, I like to have proof that I am actually executing. That has nothing to do with my management - when I have been a CEO I wanted the people who worked for me to have proof that I made their lives better too. We can agree to disagree about that, but that is perhaps where the gulf between us is the widest.

I look forward to seeing you down at ERE! And thanks again for the comment.

[Martin Snyder]

Folks thanks for the kind words about my writing- it's kind of a Parlor Trick because I use Dragon Voice Software much of the time- its quite surprising how different the syntax and flow changes between speaking my ideas and typing them. Also I was born with a bizarre left-brain only configuration, so I can barely do simple arithmetic and the idea learning of a foreign language makes me queasy. No wonder I have in inherent distaste for metrics !

I'm still not that sure that Jeff and Heather are not more in agreement than not. Heather brings up the key qualifier with "meta efforts”: the sum of her and her team's impact. She is saying that her blogging takes up an immaterial percentage of her work time, and that she does not really need a precise measurement about impact because she knows its important from several independent results, any of which exceed the value of that time. Jeff comes from the standpoint that if the blogging is using a material amount of time, than it must be justified, because it’s simply un-businesslike to use resources without the best information regarding the ROI situation.

I think its possible that if Heather would come out and say that her blog takes up two minutes a day and has no marginal cost to host, and that she is finding those two minutes by drinking one less cup of coffee, Jeff might say, yea, its possibly not worth measuring. Now of course it takes more than two minutes, but it still may not take more than 15 or 20 minutes, and you can find the borders there that you wish- as a management exercise.

Which is another key point raised by all the good comments here; how your firm management feels about the blogging is a key factor. This speaks to the actual ownership of the blog. Jeff’s blog clearly ‘belongs’ to him 100% because he does it on his time, under his sole control. It may help EA in any case; I think more highly of that company because I know Jeff through his writing, but its still his blog. I have the luxury of directing some of my own time (please note the necessity of that arrangement as otherwise I might lack employment) and I don’t expect any benefit to explicitly accrue to my company, although some very well may. Heather also has great latitude to determine the mix of her “meta efforts”, but clearly she is blogging for Microsoft, on their time, and they certainly would and could have some input to that blog’s shape and future. It’s a very individual situation.

Jeff you said something provocative about art in agreeing with Gates that “real artists ship”. I could not disagree more. Real designers ship, and real decorators ship, and SOME real artists ship, but many real artists could give a crap about shipping, and rightfully. How much of our cultural heritage has arisen from people who did not see rewards in their time? Plenty. How much of the genesis of individual artistic skills and output was not based on expectation of economic reward? I’d venture to say that quite a high percentage was.

Now blogging may be an art for some, and a product for others- but it’s again an individual thing, and thus the metrics attached are as well. In my case, I would not have encountered Jeff or Heather or so many of the great people who I have the pleasure of sharing ideas with if it had not been for the blogging that I do. There is no ROI question in my mind at all- with or without any metrics to back up my conviction.

May we re-define this conversation in terms of intent, scale, and ownership in relationship to the need for metrics? I would guess that we would probably all be very aligned once the parameters of each situation were a given.

[Heather]

Check out Martin bringing people together. I want to be martin when I grow up or...better yet....Martin for President.

Anyway, Martin, you nailed the common ground. It depends on WHY you are measuring. Is it for a go/no go decision or is it to evaluate how to blog. For the former, sometimes you already have enough anecdotal info to make that decision. I personally think the latter is wrong (or in Jon Stewart's words, "it *hurts* America"...I joke). It's a blogging integrity issue for me.

Jeff-with regard to Ian, can't you just go to his manager and say "Ian is doing something great and really innovative that will have huge impacts to our employment brand on campus over the long term. He's a great employee who I think we can trust to evaluate opportunities to reach his target talent pool; let's let him blog for work". I don't know...now I am kind of feeling bad for Ian (the idea of seeing something that can make a huge impact and then having to justify it with numbers that don't really represent it's value...in my opinion).

Well, this is the kind of interesting debate I was looking for. Kind of wish it was happening on my blog but, oh well (big sigh). ; )

[Brian Toland]

So how do you quantify me? I'm a recent addition to Microsoft but am definitely not a marketer. On the other hand Heather's and Gretchen's blogs were a key reason that I was willing to commit to a very, very long commute. It was only one factor, but I decided that any company that was aspiring to have such transparency in its recruiting efforts - I wanted in.

To Martin's points, Heather's demonstarted influence has now spread beyond marketing and is affecting executive search efforts in the tech space for Microsoft. Before this comment, how would that be measured?

Since her blog is a 'marketing' blog (it's really much more), does her contribution to my decision have less value. As I reach out and recruit high impact tech players to Microsoft, how do we quantify how much credit goes to Heather and should that have any influence on whether she gets to keep blogging?

To Heather's point, you can spend a lot of time gathering a lot of data points and still come to the same decision of doing a managerial gut check (much quicker, less defendable to investors) of saying 'does this add value? Keep doing it.'

I love the time Jeff has taken to explain the thought process that you could take to determine the value of a blog. He's done what a good conversation can do and made me more open to his point of view.

This is what the conversation should have been in the first place.

[Jeff Hunter]

Brian -

I take back the comments about the funny hats. Nicely said.

I think I am starting to understand the hub-bub. It's the term "quantify" that has this discussion wrapped around the axle. Since you both proved and questioned my point in the same comment, I thought I would take some time to explain.

Quantify has two definitions.

Quantify (http://dictionary.reference.com/search?q=quantify)

1.) To determine or express the quantity of.
2.) Logic. To limit the variables of (a proposition) by prefixing an operator such as all or some.

I am dealing with the second definition and have left you with the impression that I am advocating the first at this point in time. I am not.

Brian you said “To Heather's point, you can spend a lot of time gathering a lot of data points and still come to the same decision of doing a managerial gut check (much quicker, less defendable to investors) of saying 'does this add value? Keep doing it.'”

Expressed another way, you said “Does this add value? None, some, a lot? Rudimentary analysis says ‘some.’ “

Therefore you have quantified. You have limited the expression of variables in the equitation. This was my point from the start, but I expressed it as “existence proof.” At the start of any system we seek to understand a binary equation: value or none? Value is determined by the investors (I don’t mean shareholders, I mean investors – as in people who commit their capital or talent in expectation of a return that they determine to be of greater value than that which they provided). Heather’s management (the investors) believe that Heather’s blogging has value. Why? It could be because of comments like yours, which establishes an existence proof and justification of the hypothesis. It may be that an investor wants more (as in the first definition: the investors wants quantities), and then the individual blogger will have to determine whether they want to "produce" given those conditions. But every single one of us who is involved in blogging is in fact measuring value: either none or some. That is a measurement. And because it goes from “any variable at any level” to “some variable (value) and some level (some)” it is a quantification.

I know this may all sound a bit esoteric, but it is important to understand. All innovation (and the tool of blogging represents an innovation is the scope and ease of the personal press) starts this way: somebody with passion and commitment determines the value they derive from the exercise (as in "it feels good" or “a way to express myself” or “building a community”, all of which are expressions of value), somebody else finds value in their output (but perhaps for different reasons) thus creating a market, the market attracts capital (as in dollars and talent), the capital providers demands rationale for continued investment (as in “I get value out of return on my investment, not in the spiritual fulfillment of the product itself"), measurement begins, the innovator believes that the measurement destroys the engine that delivers the value at which point the market either succeeds or fails based on the contribution of other producers who are inspired by the original innovator's contribution.

I have lived through this cycle many times myself (and in fact am living through it now with my blog). In some ways that is what is happening here: people who are blogging because of their passion and commitment to their vision believe that if dunderheads (like me?) demand that there be some measurement of the activities' relative value / success that the very nature of the possibility of that value is destroyed.

Sometimes the innovator is correct. Many times they are wrong. The innovation always changes in response to it's introduction to the market, and what it changes into may not be interesting or compelling to the original adopters of the innovation. Therefore the early adopters claim defeat. But the innovation continues to be important, but in different ways.

The wonderful thing about a market as huge as communication and media is that the tools used for communication (in this case blogging) are unlikely to destroy the desire to connect and create a loosely affiliated community no matter who employs them in what manner. Just because the printing press was used to create the National Enquirer doesn’t mean that nobody cares about the Guttenberg Bible.

Thanks again Brian. I am sure I have just clouded up the issue more than clarified it, but your admission that Heather and Gretchen are part of the reason that you take the long community is an admission that blogging has measurable value. We may not yet be able to count, but we are surely limiting the variables.

[Brian Toland]

Should we tackle World Peace next?

When you talk about metrics, I inherently think about numbers (page views, unique visitors, etc.) Now that we're agreed that we are closer to establishing metrics in Martin's more amorphous realm (circles of influence, impact of an idea - e.g, transparency in recruiting at Microsoft) than I agree with you absolutely.

If you can get to the determination of value by saying:

1. Does it have no value? No, it clearly is have concrete impact on perceptions of Microsoft, so it has 'some' value.

2. Is it efficient? Yes, since one persons blogging has affected the decision process of six others, who likely would have chosen otherwise.

3. Is it worth continuing? Given the relative low cost of continuing this blog rather than the associated time cost of building a community by other means, than yes.

No hard numbers are attached yet we've 'measured' that this particular blog is worthwhile.

Thanks Jeff, I'm a convert. Halleluiah and pass the fried chicken.

PS - Show this thread to Ian's Boss and if the decision is no go on his blogging, I'll hire him here at Microsoft.

[Heather]

I agree with Brian.."metrics" bring to mind numbers, stats, things that take effort to measure. Well, I thought I explained the "investor gut check" in my original post but must not have done a good job of it (otherwise, why all the fuss?...that's a rhetorical questions, actually).

At the same time, the place where I differ a little, still, is this notion that investors say "more!" or "less!". Mine don't and if they did, there'd be a conversation about blogging integrity (once I comply with demands on when/what to blog, it's no longer my blog, it's theirs). I will fiercy protect my blog from becoming a "staffing tool". Right now, it's my opinions, my blog. The transparency doens't exist if it doens't stay that way (and again, in that regard, I am very lucky because nobody internally here is trying to influence me to do anything with my blog than whatever I choose).

Anyway, we're getting there ; )

[John Sumser]

Measurement is not a one time thing, it's an ongoing process. The question is not, "Does this blog provide value today?".

Measurement is an ongoing process designed to make management easier. Metrics are a part of a deep, ongoing conversation about the thing, person or process which is under management.

The mechanisms discussed so far have an enormous amount of overhead. If you conducted them routinely, the management of the blogging process would simply be too time consuming. Charted and graphed, ongoing metrics provide a usable shorthand for that important but expensive underlying conversation.

Let me say it again: Metrics are a sign of an ongoing conversation. We use them to allow the entire organization to participate in a deep conversational process. We use them to fight the creeping tide of mediocrity. We use them to make what we do better and better.

The core conversation involves issues like:
- What is the value and is that still valuable? (How is it helping the organization?)
- How can we achieve that value more effectively? (Can we do this faster, cheaper and or better?)
- How is the value changing over time?" (Is it still relevant, should we change something?)

I still have my original problem with the way the Microsoft folks articulate their vision. The idea that blogging should be left unmanaged and unmeasured assumes a couple of things about the organization and the blogger:

- That the organization is wealthy enough to afford to do unmeasured things;
- That the blogger is trusted enough to be allowed to impact the company's image; and,
- That both of these things are inherently good ideas.

We're all early adopters of technology. The idea that our toys can be justified on the basis of the fact that we like them is the way that early adopters see things. The "chasm" between early adopters and the rest of the world always involves quantification.
For blogging to spread as a useful tool for businesses, it needs to cross the "chasm". My interest is in causing and doing the things that will make blogging spread into small and medium sized operations as a matter of course. This "chasm crossing" only happens when value and process can be quantified.

That's really just a synopsis of Geoffrey Moore's widely accepted argument. http://en.wikipedia.org/wiki/Crossing_the_Chasm

In the end, usable metrics make things like blogging easier to implement. Rather than fully change consciousness to engage in a new subject, a manager can simply review an agreed upon measurement. This opens the management process to take on more pressing things.

In a strange way, not having metrics can be a good thing. The situation forces the manager to either pay very close attention to blogging and the blogger or forces her to let go and trust. That might be a good way to preserve editorial integrity.

It's a very difficult example to follow.

[Andrew Marritt]

As a measurement fan this has been a conversation that I've been watching with interest - now let me see if I can contribute to it.

It will come as no surprise that in my working life I am heavily measurement-driven. Working for a big bulge-bracket bank certainly reinforces this mindset. A big chunk of our yearly budget and our time goes into candidate research. As you can imagine, it is an area where we develop competitive advantage so I won't go into details. Needless to say it's multi-faceted, global in scale and drawing in the likes of marketing specialists, our brand research team, external consultancies and internal data-analysis teams.

The central question I guess is 'Should Heather be measuring her blog?' I guess my response is 'Probably not'. The secondary question could be 'Should the effects of Microsoft blogs be measured?' The answer is 'Yes I think they should'.

First let's consider web traffic reports. Are they useful in this instance? Probably not. For me the most important information you can get from web traffic are things such as user paths and therefore segmentation. They're good for refining how the site functions. Unfortunately blogs don't work this way. It's hard to analyse user paths because the site is pretty flat. User traffic probably serves the ego best in this instance.

What could be useful in comparative stats compared to competitors. Unfortunately getting this data would be almost impossible, not least because few competitors are replicating the approach.

Overall I guess there are two places where the activities could be measured:

* As part of a campaign-post test to look at overall effectiveness in conjunction with other materials (all Microsoft Recruiting blogs)
* As a contributing part of a brand-reputation index for Microsoft as a whole (this would be all Microsoft Blogs)

What you're really trying to get at here is 'what is this activity trying to achieve?' (Probably increase in an image of transparency and greater relationship building) followed by 'does this activity contribute to that change?'

Both can be answered. Both probably can be tracked over time should you wish.

So coming back to the first question 'Should Heather be measuring her blog?' - answer: 'Probably not'.

Another question 'Should the blog be tracked as part of another Microsoft brand reputation measurement?' - Answer 'Almost certainly yes / I would be shocked if it wasn't already / It should not be Heather's worry / It should be part of an overall approach to Microsoft blogs.'

Final question 'Should the blogs be measured in an overall recruitment marketing materials post-campaign test?' Answer 'Yes, but the driver shouldn't be to quantify a number of recruits but to identify whether it is effective at delivering the brand promise / changing image.'

It's primarily a qualitive issue, not a quantitive one.

[Martin Snyder]

I have been away at the ERE event in San Diego (I can't belive it- but I missed meeting both Heather and Jeff.....yet the hottub seemed soooo big!) and I was not blogging much from there.

I wanted to add another element to this discussion because I see that it has evolved quite well into Andrew's excellent summation.

First a detail item; Everett Rogers labeled first stage users as simply "innovators" and are relatively risk seeking.

"Early adopters" on the other hand are mainstream individuals with much more typical risk aversion.

Now to the bigger point that I would like to make; even though I agree with Heather's position completely and do belive blogging is only currently subject to an amorphous idea of measurement, I don't believe that large-scale corporate blogging is likely to be a very good idea for most corporations.

The question of whether it's a good idea or not was not the subject of this discussion, but it does deserve discussion as a part of any exploration of the metrics involved.

The hierarchy flattening effects of blogging are well-known- but something probably less understood is the negativity effect, which is a major risk factor for business.

http://en.wikipedia.org/wiki/Negativity_effect

We all know from personal experience that negative events and perceptions seem to carry more weight than positive events and perceptions. This is certainly visible in business; for example, a stock may climb over many months on generally good news, but can crash in an hour on negative news that is ostensibly equally offsetting to the good news.

The problem with uncontrolled or semi-controlled blogging is it's potential to allow for interpretations of events or perceptions into a frame that the corporation may not desire.

Rightly or wrongly, the responsibility and power of corporate management is especially focused in the control of the frames that the corporation uses.

Business may be a lot of things, but it's not a democracy. Some businesses may be more democratic than others, but from time to time, especially in controversial or risky areas, the framing decisions of corporate managers should not be undermined without great cause and presumably great benefit.

I say that because the risk of the negativity effect is real, and equally amorphous from metrics standpoint as the benefits of blogging.

So to qualify my position on metrics in corporate blogging, I am assuming that any corporate blogging has already been thoroughly vetted by management to be trustworthy in protecting the frames preferred by management. If that person, for whatever reason, needs to push back against those frames, it should be assumed that it's a serious situation that goes beyond the managers and perhaps should be addressed by the investors.

Yes I'm a bombthrowing anarchist, but only until it costs real $$$ !

[Heather]

Andrew-well done!

Martin-the hot tub of the east tower was right under my window. Was that you splashing around in there? You wer the one person that I wanted to meet but didn't get to. Grrr, next time for sure, OK?

Both-We do have a community team that *owns* our blog portal and it was up and running before I ever knew it existed (though it was set up to support community building in the ITPro and developer segments, not for marketing). I have no doubt that there was some vetting that happened to get it up. My sentiments on wanting to be part of the conversation if someone is talking about my brand aren't original. I definitely think it's the mindset behind our blogging efforts. As for blogging not being right for some commpanies, I could not agree more. That's something I warn of when I've presented on the topic of blogging in the past. If you have a culture that is not comfortable with risk taking, controversy, if you have a PR that wants to manage all of your online communications...don't blog! Also, central to the blogging success, I believe, is that the blogger should have some responsibility for an aspect of your brand (otherwise, it's a time suck that cuts into other work). I'd be surprised if we weren't doing some kind of work to understand the impact of blogging on Microsoft in general, but in that sense, I really think they are all marketing blogs and they are all recruiting blogs too. I agree that you have to look at the big picture impact.

[Jeremy Langhans]

Jeff is right.
'nuff said.

[Heather]

Gee Jeremy, that's a convincing argument. Care to elaborate?

[Jeremy Langhans]

don't have time right now ...but,
feel free to ring me up :-)

949.872.2328
~jer

[Heather]

No, I meant elaborate on the blog. The point of having a blog conversation is that everyone can participate. Sounds like you have an opinion you want to share.

[Dave Lefkow]

The challenge of not having metrics is that recruiting is a very cyclical industry with highs and lows, boom times and survival modes - and cuts always come first to the areas without metrics. Anyone remember 2002???

First off, I LOVE the fact that recruiting is branching out as an industry, that it's taking more and more lessons from marketing and sales and specializing in a few key disciplines (like sourcing and branding and personalization through things like blogs).

But if I'm a CFO or a VP of HR looking to cut costs in a recession, what would I cut first? The recruiter who still has 20 req's to fill? The sourcer who is finding us hundreds of people that aren't coming to us? Or the blogger that is building an invisible employer brand not tied to hires or profits?

The first two people have detailed metrics to support their existence. A blogger without metrics is truly vulnerable. Heather - we love ya too much to leave you exposed like that.

Just as there are ways to quantify consumer branding (not just expensive ways either), there are absolutely ways to quantify employer branding initiatives like blogs. It has to be done objectively and without bias, and you shouldn't fear what the results might be.

Heather - we all want to see you around for a long time doing what you're doing. Embrace your inner statistician :)

[Heather]

Et tu Lefkow? ; )

OK, seriously, I have data that quantify the contribution of many of my programs. I use this data to select programs going forward. There's a clear connection between the program and the hire. So it's pretty straight-forward. The data just isn't blog metrics. I look at my work as a whole and the impact it has on our pipe hiring. I don't generally operate in CYA mode. Much (not all) of my work is designed to drive hires in the short-term. Totally trackable and the metrics are meaningful ones. I create an event and x number of people get hired from the event, it's easy. How many hires did I influence directly and what were the programs I developed to get them? Those are the kinds of metrics I track. That's a certain % of my job. Blogging is "buzzy" and tracking "influence" is squishy. Hopefully, I'm driving prospects and candidates to engage with recruiters, helping them prepare better for interviews. Quantifiable?

I just don't feel the same way about blog metrics as I do about other metrics. But the fact that the blog has resulted in some hires (which isn't the main goal of the blog) is justification enough to keep it going. The fact that it's generated an incredible amount of free press. I'm not paranoid about my employment prospects. And blogging is a tiny part of my job. It's the part that other people seem the most interested in but it isn't my main focus. If there was a good business reason for me to stop blogging, I'd actually be OK with it. I'm not married to the thing. I just happen to think it's good work and so do a lot of other people.

I just don't live in a world where ALL aspects of my job have to be measurable. If my job was 100% blogging (right now, I'd put it at about 15%...probably a little more lately), if I was more paranoid about keeping my job (I don't flourish under those circumstances and doubt I would be blogging under them)and someone could present some meaningful metrics that actually reflect the full value of blogging, then I'd be inclined to take notes. I'm still waiting.

It's like trying to track the value of your participation on the ethics panel. How are you doing that? How do I place a value on the fact that publications seek me out for quotes (because of my blog). How do I measure perception change that goes on in peoples' heads? Perception change is subtle and not particularly quantifiable.

Anyhoo, I appreciate the concern (though I know you are teasing). When I heard Lisa Brummel mention blogging as something that we were doing that's innovative in recruiting and that we don't have to measure it (I'm paraphrasing), that was enough for me. Not everything that is good or right to do is measurable. That's my reality. So it's all good.