In the recent weeks a number of blog posts dealing with measuring influence have stirred up quite a debate.
Jeff Jarvis and
Steve Rubel aired their thoughts on the issue.
David Brain and Jonny Bentwood from Edelman’s London office
published something termed “Social Media Index” where they propose a brand new method for measuring on-line influence.
Most of my feelings after having read the article were (far better) summed up by
Jennifer Mattern in her
comment.
The article offers no research, no references to prior research, no logical reasoning for its claims and proposes no way of testing the model.
Can you imagine if the CEO and the Head of Research of a large investment bank published a new options pricing model without actual research and without mentioning the
Black–Scholes model? Of course not. So why in PR?
Actually the very notion that influence is not related to a topic is comical. As I understand it, the article proposes that someone has the same influence regardless of the topic discussed. In other words, TechCrunch (for example) is as influential on the topic of fly fishing or jogging as it is on Silicon Valley gossip. I don’t think that’s the case. We know both from intuition and from research that influence is topical.
At best, the Social Media Index is an indication of popularity. But popularity is not influence. Those who are very popular often have good influence but you don’t need to be very popular to have a lot of influence.
When you measure popularity, all “votes” count the same. However, when measuring influence each “vote” counts with the weight of all the votes leading to the voter.
Take a look at the figure below.
Person A is clearly more popular than person B. However, those who listen to person A do not, to any large extent, go on to influence others.
Quite differently so for B. Those who listen to person B they go on to influence others, who go on to influence others, and so on. The aggregated impact (or influence) of B is in this case bigger than A.
The figure (above) indicates why models that only take the first layer into account will come up short when trying to explain the overall outcome.
In pre-Internet times measuring popularity (or “reach” or “circulation”) was a relatively good proxy for influence because those who read a particular news paper or watched a TV programme did mostly not go on to influence anyone outside their close social circle.
But because of the Internet this has all changed and the social
event horizon is now defined by language rather than physical distance. Both direct and indirect influence must therefore be taken into account when trying to measure influence correctly.
Relative influence correlates better to outcome than relative popularity simply because influence takes the indirect effect into account.
So how to measure influence?
Well, it’s been done for a long time. In the academic community “citation analysis” has been used for decades to measure the influence of academic journals, articles, scholars and universities.
The principle is quite simple: You collect all references made between articles about a particular topic. The references are transformed into a large set of simultaneous equations that, when solved, provides the relative influence of each journal.
Thomson Scientific is probably the leading provider for the academic community.
However, the original science for measuring influence in a linked network was developed by
Wassily Leontief who devised something called the
Input-Output Analysis. It was originally (and still is) used to measure how sectors of the economy directly and indirectly influenced each other.
Leontief won the
1973 Nobel Prize in Economics for this specific work.
Now, you may doubt that knowledge gained in pre-Internet times more than 60 years ago can provide any useful input to explaining how to measure online influence.
But in much the same way as engineers at NASA draw on
Isaac Newton’s work, those who shape the on-line world draw on Leontief’s.
In 1965 Random House published a book by W. H. Miernyk titled “The elements of input-output analysis”. The book deals, as the title says, extensively with Leontief’s work.
This book is cited as a source in the article “Measuring the relative standing of disciplinary journals” by P. Doreian (1988).
Doreian’s article then goes on to be cited as a source by Jon M. Kleinberg in his article “Authoritative Sources in a Hyperlinked Environment” (1998).
Then, same year, an article, which you may have heard of, cites Kleinberg’s:
Lawrence Page, Sergey Brin, Rajeev Motwani, Terry Winograd. The PageRank Citation Ranking: Bringing Order to the Web.
There you have it: From Leontief to Google in 4 easy steps. (There may be shorter/more paths).
Ok, end of part 1.
In part 2: The list of the 80 or so most influential PR blogs measured using citation analysis including their relative influence. Stay tuned.
Edelman, a PR company, and Technorati, a blog-search-engine, has released a list of "the most influential bloggers".
According to
Edelman’s own release the methodology has been to count the number of links each blog receives from other bloggers.
If that is a correct description of their methodology it can be deduced that their analysis rests on two central conjectures that are both wrong.
1st wrong conjecture: Influence is defined using one factor only: The number of endorsements (links) an actor receives.
2nd wrong conjecture: Influence is independent of issue.
In a social context influence (often in the literature referred to as “prestige”) is normally defined using at least two factors: The number of endorsements an actor receives and the prestige of each actor awarding the endorsement. Lately (in the last 30 or so years) a third factor is usually also included: The number of actors each actor is awarding his or her prestige to (linking to) (i.e. how thin they spread their total endorsement).
Popularity, on the other hand, is usually defined using one factor: The number of endorsements an actor receives.
The Edelman/Technorati study doesn't attribute different weights to the links/endorsements by different bloggers. One could say that they regard a link from any two bloggers as contributing equally to the influence of the blogger linked to.
This (1st wrong conjecture) is intuitively wrong as well as logically flawed.
Intuitively wrong because what they are saying is that it would give a blogger equal influence to receive the endorsement of the most credible and well respected blogger in the world as an endorsement made by someone who hasn't got a clue and that nobody has ever heard of. Intuitively it doesn't make sense.
The conjecture is also logically flawed. The very foundation for their methodology is that all bloggers are equally influential (as all links count equally). And what do they produce: A ranked list of bloggers stating that some are more influential than others. But hey - what do you actually mean guys: Should a link from the number 1 on your list count the same as a link from the last guy on the same list? It does when you start counting but it doesn't when you're done. Clearly there was a change of mind (and logic) somewhere along the way..
As for the 2nd wrong conjecture it is even more illogical than the first. (This is the one that says that influence is universal).
Are you really saying that, if say David Beckham is influential on football he is also influential on the developments in British politics and on say, wine?
Influence is issue-based because the endorsements/links are related to the issue discussed. If not, bloggers would link to the same blogs in any context and this is obviously absurd.
Reading that influence is independent of issue reminds me of the time (Feb 17th, 2006) I was researching a story on bird flu and searched Technorati to see who they regarded as the most authoritative blog on this issue. The result: Engadget – a blog on gadgets.
The whole thing leaves me with a number of questions I don't seem to be able to answer:
Why don't they just call it "popularity"? Then they would be totally right. They could have said "We have made a list of those bloggers who are most popular with other bloggers". That could have been a respectable piece of research and an interesting list. as those who are popular usually also have some influence (although you don't need to be popular to have influence).
Why do they call it "influence" when it's clearly not?
Thanks to
Antony Mayfield for pointing me to this.
It looks like Edelman is winning the
PR War for Walmart.
Figure 1 (below) shows the number of blog posts about Walmart in the previous 30 days along with the accumulated sentiment (Net Promoters Index).
Some 2 weeks ago something interesting happened: The number of daily posts went up and the majority of the posts became positive.
I wonder what Edelman’s magic formula is because it sure is working.
The NY Times has a new and interesting
article about Wal-Mart’s PR war.
When the
previous article was published
we analysed Wal-Mart’s online stakeholder universe to understand the strength of the players.
Perhaps it is time for a follow-up analysis..?