Response to Heather's Question on Metrics

I want to thank Heather for commenting on the previous post. And while I cannot take her up on a dance off (we haven’t met face-to-face Heather, but large overweight bald white guys are not a pretty sight on the dance floor... I could trip over my two left feet and kill somebody accidentally), I felt her question deserved a response. I decide to do this as a post rather than a comment in reply because long comments are even more tedious (if that’s possible) than long posts.

Also, I apologize for any errors in this post. I had exactly 30 minutes to throw it together before my next meeting.

Investors decide the validity of metrics. They usually decide by evaluating the return along two factors : efficiency and effectiveness.

Efficiency is about reducing the cost and risk to capital. Said another way, whoever can do more with less wins. Effectiveness is about value: with the same amount of capital I can extract more value than my competitors. Both measures are relative to competitors. They are not static.

In Employment Branding, I believe most investors think in the following ways:

Efficiency: By investing in Employment Branding I reduce transaction cost per dollar employed, or, conversely, get more transactions done with less dollars. So if its between putting another dollar into hiring a recruiter or putting a dollar into blogging, I want to know that I can get more done with a blog than I can with a recruiter. Everyone starts to chuckle “Now you are just off in left field.” Actually, no. Look at the total end-to-end recruiting and hiring transaction. In a “bad market” (labor demand outpaces labor supply in positions that are core to competitiveness), more dollars go to front-loading the transaction than to maximizing throughput later in the process. In English: it costs more to source relative to how much you spend on screening, interviewing, closing and on boarding.

If more dollars are front loaded into the system, then the efficiency measure is “Can you source more people with less money, or more people for the same amount of money?” In this respect most corporate blogging (but especially your blog), provides a good return. Microsoft competes against a lot of different companies for marketing talent. At the same time, Microsoft is ever more dependent on marketing talent to expand its markets (there are only so many office suites any one person can buy). In a buyer’s market, differentiation between sellers is key. As the volume and velocity of information increases, rising above the noise is absolutely critical. In a hiring transaction there are only two ways to do this: develop low-load / high value relationships and provide a convenient location for buyers to gather data that is meaningful for them in order to differentiate various competing offerings. Blogs do a really good job of both.

“But wait – how do we measure that?” In this case repeat traffic is not a bad measure. Perfect? Absolutely not. But better than a guess. Repeat traffic from IPs that you want to target is an even better measure. Creating an association between your topic, IP targets and source of hire would be yet another.

Effectiveness: By investing in Employment Branding I (investor) increase the quality of the candidates who will actively consider employment opportunities at Microsoft. Of course this is tricky, but not because the net effect of blogging is difficult. It is difficult because “quality” is a measure of variance to specification, and our specs (job descriptions) are not highly predictive of value delivery probabilities. So it is difficult to determine whether you get “better” people if you don’t know what “better” means.

But let’s get beyond that for a second and assume that we know who the stars are within Microsoft. The “existence proof” you are seeking for the effectiveness measure of your blog is a conversation. Talk to the new hires in marketing that started at least 3 months (an arbitrary number, but intuitively long enough to allow your blog to get traction and inbound links) after you started blogging and ask them whether:

1. They had read your blog prior to starting work.
2. The blog played any part in their decision to join Microsoft.

Now wait for six months and go talk to the hiring manager of all the new hires. How do they stack rank (which is a baloney methodology, but most investors will buy the output), and what is the correlation of those who are “A”s to those who said “Yes” to both questions. Compare that to the people who said “No."

All the statisticians and social scientists who may be reading this can blow holes all day long through this methodology. And they would be right as far as the science goes. But empirical validity is not the problem you are trying to solve. Continued investment is. And most investors would take a positive correlation as “proof” that they are getting a good return for their bucks.

Overall, you have to remember that when a new system is created the measurement is largely along “belief proof” and “existence proof” coordinates. Do you have any data to support the claim that you were able to source more people than before you started blogging? It could be as flippant and anecdotal as one sourcer’s word. It could be more detailed: you could do an alignment survey of the downstream clients of your service “On a scale of 1 to 10 rate how effective blogging is in getting you more and better candidates.” Where you have a high degree of standard deviation in response you have little data. But where you have low standard deviation, either good or bad, you have basic proof.

One of the hardest things that we do in business is to take intangibles and turn them into tangibles. Harder still is to take indicators and turn them from lagging (traffic) to leading (probability that any future candidate will have made a buying decision based on your content). And then there is the hardest job yet… making all meaningful to investors without killing the goose that lays the golden egg. I understand that this, more than anything, is your basic concern. But just because it is really, really difficult does not mean that it is not worth pursuing. We need to get more investors to keep this train rolling, and at the end of the day we are accountable to the value they perceive they get from us.

(And that’s why I pay for my own blog and do it on my own time – I don’t like most investors. ;-) )

 

Comments

[Heather]

I agree with Brian.."metrics" bring to mind numbers, stats, things that take effort to measure. Well, I thought I explained the "investor gut check" in my original post but must not have done a good job of it (otherwise, why all the fuss?...that's a rhetorical questions, actually).

At the same time, the place where I differ a little, still, is this notion that investors say "more!" or "less!". Mine don't and if they did, there'd be a conversation about blogging integrity (once I comply with demands on when/what to blog, it's no longer my blog, it's theirs). I will fiercy protect my blog from becoming a "staffing tool". Right now, it's my opinions, my blog. The transparency doens't exist if it doens't stay that way (and again, in that regard, I am very lucky because nobody internally here is trying to influence me to do anything with my blog than whatever I choose).

Anyway, we're getting there ; )

[John Sumser]

Measurement is not a one time thing, it's an ongoing process. The question is not, "Does this blog provide value today?".

Measurement is an ongoing process designed to make management easier. Metrics are a part of a deep, ongoing conversation about the thing, person or process which is under management.

The mechanisms discussed so far have an enormous amount of overhead. If you conducted them routinely, the management of the blogging process would simply be too time consuming. Charted and graphed, ongoing metrics provide a usable shorthand for that important but expensive underlying conversation.

Let me say it again: Metrics are a sign of an ongoing conversation. We use them to allow the entire organization to participate in a deep conversational process. We use them to fight the creeping tide of mediocrity. We use them to make what we do better and better.

The core conversation involves issues like:
- What is the value and is that still valuable? (How is it helping the organization?)
- How can we achieve that value more effectively? (Can we do this faster, cheaper and or better?)
- How is the value changing over time?" (Is it still relevant, should we change something?)

I still have my original problem with the way the Microsoft folks articulate their vision. The idea that blogging should be left unmanaged and unmeasured assumes a couple of things about the organization and the blogger:

- That the organization is wealthy enough to afford to do unmeasured things;
- That the blogger is trusted enough to be allowed to impact the company's image; and,
- That both of these things are inherently good ideas.

We're all early adopters of technology. The idea that our toys can be justified on the basis of the fact that we like them is the way that early adopters see things. The "chasm" between early adopters and the rest of the world always involves quantification.
For blogging to spread as a useful tool for businesses, it needs to cross the "chasm". My interest is in causing and doing the things that will make blogging spread into small and medium sized operations as a matter of course. This "chasm crossing" only happens when value and process can be quantified.

That's really just a synopsis of Geoffrey Moore's widely accepted argument. http://en.wikipedia.org/wiki/Crossing_the_Chasm

In the end, usable metrics make things like blogging easier to implement. Rather than fully change consciousness to engage in a new subject, a manager can simply review an agreed upon measurement. This opens the management process to take on more pressing things.

In a strange way, not having metrics can be a good thing. The situation forces the manager to either pay very close attention to blogging and the blogger or forces her to let go and trust. That might be a good way to preserve editorial integrity.

It's a very difficult example to follow.

[Andrew Marritt]

As a measurement fan this has been a conversation that I've been watching with interest - now let me see if I can contribute to it.

It will come as no surprise that in my working life I am heavily measurement-driven. Working for a big bulge-bracket bank certainly reinforces this mindset. A big chunk of our yearly budget and our time goes into candidate research. As you can imagine, it is an area where we develop competitive advantage so I won't go into details. Needless to say it's multi-faceted, global in scale and drawing in the likes of marketing specialists, our brand research team, external consultancies and internal data-analysis teams.

The central question I guess is 'Should Heather be measuring her blog?' I guess my response is 'Probably not'. The secondary question could be 'Should the effects of Microsoft blogs be measured?' The answer is 'Yes I think they should'.

First let's consider web traffic reports. Are they useful in this instance? Probably not. For me the most important information you can get from web traffic are things such as user paths and therefore segmentation. They're good for refining how the site functions. Unfortunately blogs don't work this way. It's hard to analyse user paths because the site is pretty flat. User traffic probably serves the ego best in this instance.

What could be useful in comparative stats compared to competitors. Unfortunately getting this data would be almost impossible, not least because few competitors are replicating the approach.

Overall I guess there are two places where the activities could be measured:

* As part of a campaign-post test to look at overall effectiveness in conjunction with other materials (all Microsoft Recruiting blogs)
* As a contributing part of a brand-reputation index for Microsoft as a whole (this would be all Microsoft Blogs)

What you're really trying to get at here is 'what is this activity trying to achieve?' (Probably increase in an image of transparency and greater relationship building) followed by 'does this activity contribute to that change?'

Both can be answered. Both probably can be tracked over time should you wish.

So coming back to the first question 'Should Heather be measuring her blog?' - answer: 'Probably not'.

Another question 'Should the blog be tracked as part of another Microsoft brand reputation measurement?' - Answer 'Almost certainly yes / I would be shocked if it wasn't already / It should not be Heather's worry / It should be part of an overall approach to Microsoft blogs.'

Final question 'Should the blogs be measured in an overall recruitment marketing materials post-campaign test?' Answer 'Yes, but the driver shouldn't be to quantify a number of recruits but to identify whether it is effective at delivering the brand promise / changing image.'

It's primarily a qualitive issue, not a quantitive one.

[Martin Snyder]

I have been away at the ERE event in San Diego (I can't belive it- but I missed meeting both Heather and Jeff.....yet the hottub seemed soooo big!) and I was not blogging much from there.

I wanted to add another element to this discussion because I see that it has evolved quite well into Andrew's excellent summation.

First a detail item; Everett Rogers labeled first stage users as simply "innovators" and are relatively risk seeking.

"Early adopters" on the other hand are mainstream individuals with much more typical risk aversion.

Now to the bigger point that I would like to make; even though I agree with Heather's position completely and do belive blogging is only currently subject to an amorphous idea of measurement, I don't believe that large-scale corporate blogging is likely to be a very good idea for most corporations.

The question of whether it's a good idea or not was not the subject of this discussion, but it does deserve discussion as a part of any exploration of the metrics involved.

The hierarchy flattening effects of blogging are well-known- but something probably less understood is the negativity effect, which is a major risk factor for business.

http://en.wikipedia.org/wiki/Negativity_effect

We all know from personal experience that negative events and perceptions seem to carry more weight than positive events and perceptions. This is certainly visible in business; for example, a stock may climb over many months on generally good news, but can crash in an hour on negative news that is ostensibly equally offsetting to the good news.

The problem with uncontrolled or semi-controlled blogging is it's potential to allow for interpretations of events or perceptions into a frame that the corporation may not desire.

Rightly or wrongly, the responsibility and power of corporate management is especially focused in the control of the frames that the corporation uses.

Business may be a lot of things, but it's not a democracy. Some businesses may be more democratic than others, but from time to time, especially in controversial or risky areas, the framing decisions of corporate managers should not be undermined without great cause and presumably great benefit.

I say that because the risk of the negativity effect is real, and equally amorphous from metrics standpoint as the benefits of blogging.

So to qualify my position on metrics in corporate blogging, I am assuming that any corporate blogging has already been thoroughly vetted by management to be trustworthy in protecting the frames preferred by management. If that person, for whatever reason, needs to push back against those frames, it should be assumed that it's a serious situation that goes beyond the managers and perhaps should be addressed by the investors.

Yes I'm a bombthrowing anarchist, but only until it costs real $$$ !

[Heather]

Andrew-well done!

Martin-the hot tub of the east tower was right under my window. Was that you splashing around in there? You wer the one person that I wanted to meet but didn't get to. Grrr, next time for sure, OK?

Both-We do have a community team that *owns* our blog portal and it was up and running before I ever knew it existed (though it was set up to support community building in the ITPro and developer segments, not for marketing). I have no doubt that there was some vetting that happened to get it up. My sentiments on wanting to be part of the conversation if someone is talking about my brand aren't original. I definitely think it's the mindset behind our blogging efforts. As for blogging not being right for some commpanies, I could not agree more. That's something I warn of when I've presented on the topic of blogging in the past. If you have a culture that is not comfortable with risk taking, controversy, if you have a PR that wants to manage all of your online communications...don't blog! Also, central to the blogging success, I believe, is that the blogger should have some responsibility for an aspect of your brand (otherwise, it's a time suck that cuts into other work). I'd be surprised if we weren't doing some kind of work to understand the impact of blogging on Microsoft in general, but in that sense, I really think they are all marketing blogs and they are all recruiting blogs too. I agree that you have to look at the big picture impact.

[Jeremy Langhans]

Jeff is right.
'nuff said.

[Heather]

Gee Jeremy, that's a convincing argument. Care to elaborate?

[Jeremy Langhans]

don't have time right now ...but,
feel free to ring me up :-)

949.872.2328
~jer

[Heather]

No, I meant elaborate on the blog. The point of having a blog conversation is that everyone can participate. Sounds like you have an opinion you want to share.

[Dave Lefkow]

The challenge of not having metrics is that recruiting is a very cyclical industry with highs and lows, boom times and survival modes - and cuts always come first to the areas without metrics. Anyone remember 2002???

First off, I LOVE the fact that recruiting is branching out as an industry, that it's taking more and more lessons from marketing and sales and specializing in a few key disciplines (like sourcing and branding and personalization through things like blogs).

But if I'm a CFO or a VP of HR looking to cut costs in a recession, what would I cut first? The recruiter who still has 20 req's to fill? The sourcer who is finding us hundreds of people that aren't coming to us? Or the blogger that is building an invisible employer brand not tied to hires or profits?

The first two people have detailed metrics to support their existence. A blogger without metrics is truly vulnerable. Heather - we love ya too much to leave you exposed like that.

Just as there are ways to quantify consumer branding (not just expensive ways either), there are absolutely ways to quantify employer branding initiatives like blogs. It has to be done objectively and without bias, and you shouldn't fear what the results might be.

Heather - we all want to see you around for a long time doing what you're doing. Embrace your inner statistician :)