There's a reason why a few creative people wanted to build entire websites in Flash during the pre-Ajax/Jquery era: they wanted a seamless user experience, without that pageload.
It might only take a fraction of a second for a page to load on most sites, but that interruption in the experience has been identified as unnecessary. Inelegant.
More can be experienced these days without a pageload.
And it's through the pageload that most web analytics tools capture data.
The writing was on the wall for a long time - perhaps since the begining, starting with real player content, quicktime, and then the onset of flash video. As experiences become more distributed, the pageview paradigm decays that much more.
The denial cock won't crow anymore. In a world where in a single page visit, a user can view, read, rate, and comment and then leave the site - and none of it captured unless the script is artificially forced to fire - there can be no tolerance for the pageview paradigm.
In the previous blog post on this, elegantly named "Screw Pageviews", Jacques Warren, in the comments section, predicted that "Pageviews" are essentially "Clicks", and that the whole vocabulary would be dominated by the word "interactions".
The problem with breaking the current model is that the server load on the third party side goes up, and since vendors are typically compensated on the basis of pageviews: server calls - the costs escalate.
That poses a barrier. I can see the vendor arguing: "Want three times the granularity? Pay three times the amount". And I want vendors to get paid. They do good R&D and there are some really good guys out there.
It's at this point that I have to question the cost of a pageview.
I keep on hearing how cheap memory and processing power is. It's really cheap. Google gives so much of it away for 'free' in exchange for being a lookie-loo. It's truly cheap.
But if the vendors are charging a lot on the processing power - because they sure as hell arn't investing processing power into actual analytics - where is all the money going?
R&D?
Acquisitions?
In a post-pageview paradigm world - where we will be gathering a lot more information and richer information - what is the right monetization model?
What are we, as practitioners and clients willing to pay?
What are vendors, as investors and researchers, willing to charge?
Friday, May 29, 2009
Monday, May 25, 2009
Analytics Evolution and Growth of the Industry
John Lovett over at Analytics Evolution talks about the growth of our analytics industry. It's worth a read.
It's one of the most succinct postings I've seen on all our ills.
The inflated expectations, the skill gaps, the talent gaps...it's great stuff - and that's not just because we happen to agree on so much.
There's a market maturity issue with Google Analytics. GA is great at telling people 'what' is going on. In effect, that's what 90% of web analyst did between 1995 and 2007 - fight with in an effort to determine 'what' is going on. The journey ended at the dashboard.
Since most people assume that since they have Google Analytics on their blog, and they can tell 'what' is going on - and with the help of Avinash - who tells them how interpret specific GA terms, like 'bounce rate', they can conclude that really, analytics is truly easy.
'Why' is seldom asked.
The current suite of analytics tools are unable and/or unwilling to answer those questions even if that question is asked.
We have deductive methods of testing hypotheses of 'why' something is happening, but it is extremely seldom that organizations are set up to answer that on a periodic basis.
Setting up those routines - that culture of analytics - is but one component of the bigger picture.
John is right in the end though: it's time for talented practitioners to shine.
It's one of the most succinct postings I've seen on all our ills.
The inflated expectations, the skill gaps, the talent gaps...it's great stuff - and that's not just because we happen to agree on so much.
There's a market maturity issue with Google Analytics. GA is great at telling people 'what' is going on. In effect, that's what 90% of web analyst did between 1995 and 2007 - fight with
Since most people assume that since they have Google Analytics on their blog, and they can tell 'what' is going on - and with the help of Avinash - who tells them how interpret specific GA terms, like 'bounce rate', they can conclude that really, analytics is truly easy.
'Why' is seldom asked.
The current suite of analytics tools are unable and/or unwilling to answer those questions even if that question is asked.
We have deductive methods of testing hypotheses of 'why' something is happening, but it is extremely seldom that organizations are set up to answer that on a periodic basis.
Setting up those routines - that culture of analytics - is but one component of the bigger picture.
John is right in the end though: it's time for talented practitioners to shine.
Wednesday, May 20, 2009
A pretty big day for #wa and #analytics in terms of interdisciplinary convergence
It's a pretty cryptic tweet, yes.
A paper was published in the Journal Marketing Science that just came to my attention. A review will be forthcoming on the Web Analytics Association website. I'll publish the link once I write it.
The article in question is very theoretical and mathematical, but it leverages an area of mathematics that has yet to be applied, practically, to web analytics. What's most exciting is that the physical technology to make it happen exists. The case study that it works is there.
The one sentence summary is: "There's a way to radically increase the tempo of optimization while drastically lowering both the monetary and personhour cost".
Now the bad news:
The social technology is another question.
It requires an interdisciplinary team of web analysts/marketing scientists, data miners/marketing scientists, information architects, technologists, and creatives. These five groups are going to have to work together to realize the full competitive advantages offered by the physical technology. In many ways, the social pathways simply don't exist within most companies to take advantage of the technology. The technology will move faster than traditional human institutions will be able to keep up.
Yes, we will all start off very small to start with. But these things grow.
As it is now, anways, I don't think there is nearly enough interdisciplinary chatter to fully leverage the technology.
Therein lies the competitive advantage.
Profit to the people capable of overcoming their disciplinary barriers: replete with all their inherent jargon, prejudices and cannon.
A paper was published in the Journal Marketing Science that just came to my attention. A review will be forthcoming on the Web Analytics Association website. I'll publish the link once I write it.
The article in question is very theoretical and mathematical, but it leverages an area of mathematics that has yet to be applied, practically, to web analytics. What's most exciting is that the physical technology to make it happen exists. The case study that it works is there.
The one sentence summary is: "There's a way to radically increase the tempo of optimization while drastically lowering both the monetary and personhour cost".
Now the bad news:
The social technology is another question.
It requires an interdisciplinary team of web analysts/marketing scientists, data miners/marketing scientists, information architects, technologists, and creatives. These five groups are going to have to work together to realize the full competitive advantages offered by the physical technology. In many ways, the social pathways simply don't exist within most companies to take advantage of the technology. The technology will move faster than traditional human institutions will be able to keep up.
Yes, we will all start off very small to start with. But these things grow.
As it is now, anways, I don't think there is nearly enough interdisciplinary chatter to fully leverage the technology.
Therein lies the competitive advantage.
Profit to the people capable of overcoming their disciplinary barriers: replete with all their inherent jargon, prejudices and cannon.
Sunday, May 17, 2009
On Analytical Business Question Spaces
Begin with five statements:
1. The set of business questions that could be asked is infinite.
2. The subset of business questions that could be answered by our current toolset set is very large, but ultimately finite.
3. The subset of business questions that, when answered, could be valuable, is a smaller, finite set.
4. This subset is volatile.
5. The current analytical paradigm is ill equipped to grapple with statements 1 through 4.
Unpack these statements:
1. The set of business questions that could be asked is infinite.
Consider every single combination of characters and numbers that can be packed into quotation marks, ended with a question mark. Consider writing all of these down. You'd end up with millions of slips of paper with questions like:
"jdajfkdjak jdkjla dkeieuo kalkdjj?"
And
"Who are the most valuable customers"
And
"What caused customers who purchased the Samuel Jackson Snakes on a Plane commemorative plate who also happened to visit the discount page on the Snakes on a Plane microsite to convert?"
And
"What were married mothers, living in Colorado, with two children under 5 and four children from another marriage older than 12, who twitter, do between the hours of 2pm EST and 3pm EST on the Second Tuesday of every month ending in 'Y', for the past four years, accurate to within +/- 4%, 19/20?"
Everybody who reads this blog should have been asked questions like those above at one point or another in their career. There are certain questions that make you wince and ask "why the hell would you want to know that?"
The point remains that the set of questions that could be asked, that are human readable, is infinite. I'd go so far as to argue that it's the size of the Library of Smith, but I lack that skillset that allows me to write elegant proofs in the manner that people who write elegant proofs expect to see them. But I'm not going to withdraw the assertion. If you can find a counter-example, I'll accept it.
2. The subset of business questions that could be answered by our current toolset set is very large, but ultimately finite.
"jdajfkdjak jdkjla dkeieuo kalkdjj?"
- Don't know what you're asking.
"Who are the most valuable customers"
-That would require access to the customer database and a Novo Segmentation. If you have both the access and the tool (a spreadsheet for n<65000) style="font-weight: bold;">3. The subset of business questions that, when answered, could be valuable, is a smaller, finite set.
If I were to plot all of these question on a plane - on a surface, with the X-axis being some arbitrary descriptor of the questions, and the Y-axis being another arbitrary descriptor, and plot the potential business value minus the business cost of knowing the answer to that question for a given company n, this is what I'm expecting to see:
It's lumpy.
There are a whole bunch of questions that, if you keep on throwing resources at trying to answer them, you're not not going to derive any business value at all from them. They're like black holes. Only that they're not actually a hole in the plane. But they might as well be.
There are a few questions that, when answered efficiently, will yield very nice returns.
There are other questions that are more or less flat. There's still lump in there, just at much smaller scale.
Novo has pointed out that good marketers know the 12 to 15 questions that yield the highest returns for businesses: and I agree. I'll point out that an experienced marketer will be able to evaluate the subset of millions questions and say, "wtf, that has no value".
4. This subset is volatile.
Knowing the most valuable customers for sheet music in 1922 was far more valuable than it was in 2006.
Factor in that business conditions change, economies change, CMO's and CEO's churn.
The question surface is volatile.
5. The current analytical paradigm is ill equipped to grapple with statements 1 through 4.
A paradigm is a way of looking at the world that is loaded with bias, definitions, jargon, tools, methods and prejudices.
The present analytical paradigm is heavily based around tools - the most common question you get these days is "You know?". It heavily based around definitions that most analysts, even those working for the vendors, don't fully understand or can't explain. Or when they do explain them, they do so incorrectly.
The problem is that the paradigm is product centric. It's not business question centric. It denies that there is there is a surface outside what is offered, and worse, denies that the set of business questions shifts over time.
And analysts ourselves are part of the problem.
Instead of asking "what is the subset of questions we need answered" up front, we're jumping to what CAN be answered by a single tool versus another tool. Worse, certain analysts are receiving what essentially adds up to payola by aligning with specific vendor technologies. The more complex and technically difficult the product: so much the better.
And there it is.
What say you? Are you part of the solution or part of the problem?
What does the Philosoraptor think?
1. The set of business questions that could be asked is infinite.
2. The subset of business questions that could be answered by our current toolset set is very large, but ultimately finite.
3. The subset of business questions that, when answered, could be valuable, is a smaller, finite set.
4. This subset is volatile.
5. The current analytical paradigm is ill equipped to grapple with statements 1 through 4.
Unpack these statements:
1. The set of business questions that could be asked is infinite.
Consider every single combination of characters and numbers that can be packed into quotation marks, ended with a question mark. Consider writing all of these down. You'd end up with millions of slips of paper with questions like:
"jdajfkdjak jdkjla dkeieuo kalkdjj?"
And
"Who are the most valuable customers"
And
"What caused customers who purchased the Samuel Jackson Snakes on a Plane commemorative plate who also happened to visit the discount page on the Snakes on a Plane microsite to convert?"
And
"What were married mothers, living in Colorado, with two children under 5 and four children from another marriage older than 12, who twitter, do between the hours of 2pm EST and 3pm EST on the Second Tuesday of every month ending in 'Y', for the past four years, accurate to within +/- 4%, 19/20?"
Everybody who reads this blog should have been asked questions like those above at one point or another in their career. There are certain questions that make you wince and ask "why the hell would you want to know that?"
The point remains that the set of questions that could be asked, that are human readable, is infinite. I'd go so far as to argue that it's the size of the Library of Smith, but I lack that skillset that allows me to write elegant proofs in the manner that people who write elegant proofs expect to see them. But I'm not going to withdraw the assertion. If you can find a counter-example, I'll accept it.
2. The subset of business questions that could be answered by our current toolset set is very large, but ultimately finite.
"jdajfkdjak jdkjla dkeieuo kalkdjj?"
- Don't know what you're asking.
"Who are the most valuable customers"
-That would require access to the customer database and a Novo Segmentation. If you have both the access and the tool (a spreadsheet for n<65000) style="font-weight: bold;">3. The subset of business questions that, when answered, could be valuable, is a smaller, finite set.
If I were to plot all of these question on a plane - on a surface, with the X-axis being some arbitrary descriptor of the questions, and the Y-axis being another arbitrary descriptor, and plot the potential business value minus the business cost of knowing the answer to that question for a given company n, this is what I'm expecting to see:
It's lumpy.
There are a whole bunch of questions that, if you keep on throwing resources at trying to answer them, you're not not going to derive any business value at all from them. They're like black holes. Only that they're not actually a hole in the plane. But they might as well be.
There are a few questions that, when answered efficiently, will yield very nice returns.
There are other questions that are more or less flat. There's still lump in there, just at much smaller scale.
Novo has pointed out that good marketers know the 12 to 15 questions that yield the highest returns for businesses: and I agree. I'll point out that an experienced marketer will be able to evaluate the subset of millions questions and say, "wtf, that has no value".
4. This subset is volatile.
Knowing the most valuable customers for sheet music in 1922 was far more valuable than it was in 2006.
Factor in that business conditions change, economies change, CMO's and CEO's churn.
The question surface is volatile.
5. The current analytical paradigm is ill equipped to grapple with statements 1 through 4.
A paradigm is a way of looking at the world that is loaded with bias, definitions, jargon, tools, methods and prejudices.
The present analytical paradigm is heavily based around tools - the most common question you get these days is "You know
The problem is that the paradigm is product centric. It's not business question centric. It denies that there is there is a surface outside what is offered, and worse, denies that the set of business questions shifts over time.
And analysts ourselves are part of the problem.
Instead of asking "what is the subset of questions we need answered" up front, we're jumping to what CAN be answered by a single tool versus another tool. Worse, certain analysts are receiving what essentially adds up to payola by aligning with specific vendor technologies. The more complex and technically difficult the product: so much the better.
And there it is.
What say you? Are you part of the solution or part of the problem?
What does the Philosoraptor think?
Saturday, May 9, 2009
Organizational Design Patterns for Web Analytics
In my previous post, I argued that Web Analytics was not easy because of complexity, much of it caused by people. Things can get lost in translation when translating data into actionable insight and actionable insight into action.
Let's turn to the solution, something that Jacques Warren, fellow tweeter (and #wa guru), has termed "Organizational Engineering".
What follows is a laundry list of the elements, considerations, and biases that should feed a successful web analytics organizational design pattern.
1. It all starts with a great web analyst, a few things a great web analyst does or understands:
a) Takes the site map, goes through the site, and understands it.
b) If no site map exists (which is common), then that web analyst produces one themselves, in so doing, they will understand the site.
c) With site map in hand, uses path analysis to understand where the bulk of the traffic goes.
d) A great web analyst DOCUMENTS everything. What was done. Past reports. Past recommendations. Past successes. Past excuses. Indexes. A great web analyst is like an elephant - they never forget.
e) Understands that the tool is not the end all and be all.
f) Knows that the measure of success is not the number of reports produced, it's the amount of actionable insight produced.
g) Is vendor agnostic. Sees the tool for what it really is. A means - not an end.
h) 80/20 rule. Spends 80% of the web analytics budget on quality head count and 20% on technology. If they can't afford both an enterprise package and quality head count, goes for a free web analytics package until such time sufficient value and case studies have been derived to increase the size of the pie.
i) Knows they can't answer everything with sufficient certainty to make a decision on one input alone, all the time.
j) Leads with actionability and value. Do this, get X% lift, get Y% in incremental, costs Z, get A in profit. (Plus or Minus a confidence interval B)
2. Starting Small Versus Going Big
a) The advantages of starting small include: lower risk, higher odds of success, gaining of momentum, ability to easily reach again if one fails, ability to learn and re-apply knowledge of failures.
b) The advantages of going big include: bigger payoff and much greater credibility if something is executed, greater chance of a gaining support for a better design pattern from the get-go.
The decision of which way to go depends on the character of both the analyst and the company
3. Winning Support
a) Gaining the support of both middle and upper management for anything is vital. Not everybody really matters, but every little bit of support doesn't hurt.
b) Building a bridge to IT. IT typically owns the technical infrastructure of the website. They have your tags, and can break them. You need their support to get your tests and insights implemented.
c) Building a bridge to Finance. Jim Novo is absolutely bang on about this.
d) Building a bridge within Marketing. Web Analytics typically (not always), lives within Marketing. If a web analyst is there, chances are somebody in that department sees the value of you being there. Make sure that your manager is onside.
e) A web analyst without any support from management will initially fail to get anything executed. If this situation degenerates into a trend, the web analyst will ultimately stop generating actionable insight, suffer from skill decay, and eventually, defect to another company or department.
4. Communication Cadence
a) Automate dashboarding. Nobody ever made their company any money with a dashboard. A dashboard says what happened in the past. A dashboard has yet to tell me what is likely to happen in the future.
b) Bi-weekly or Monthly spotlights. Use powerpoint or video (the medium is the message). Remind them what you told them last time, what's been done about it since, what you found this time, options for exploiting the opportunity.
c) The web analyst is indispensable when considering any online marketing campaign.
d) Report generation cadence should not be so high as to preclude the analyst from doing their real job: adding value, intelligence, and making the company more money.
5. Prioritization
a) If nobody is owning it, own it.
b) Assist the person who owns it.
6. Anticipate the satisfaction/disatisfaction curve
a) The set of business questions that could be asked of an analyst is infinite.
b) The set of business questions that a web analytics tool can answer, at any given time, is finite.
c) Expectations of what the analyst is able to answer will inflate as the analyst delivers successfully.
d) The amount of incremental satisfaction with the tool will decrease as time passes.
e) The argument for supplemental tools that match queries that could not be answered will become more compelling over time, so long as these have been documented.
f) The request for more advanced tools should be as proactive as positive, in anticipation of the successor sets of business questions.
g) The business questions should be compelling enough to the people who matter to justify the increased budget.
h) Eventually, the web analyst will have to learn statistics once the "Hour a Day" questions have been exhausted.
And that's it.
It's a start.
If we can get together and agree on the general characteristics of a good organizational design pattern for web analytics, the adherents to that design pattern should be in a better place.
Let's turn to the solution, something that Jacques Warren, fellow tweeter (and #wa guru), has termed "Organizational Engineering".
What follows is a laundry list of the elements, considerations, and biases that should feed a successful web analytics organizational design pattern.
1. It all starts with a great web analyst, a few things a great web analyst does or understands:
a) Takes the site map, goes through the site, and understands it.
b) If no site map exists (which is common), then that web analyst produces one themselves, in so doing, they will understand the site.
c) With site map in hand, uses path analysis to understand where the bulk of the traffic goes.
d) A great web analyst DOCUMENTS everything. What was done. Past reports. Past recommendations. Past successes. Past excuses. Indexes. A great web analyst is like an elephant - they never forget.
e) Understands that the tool is not the end all and be all.
f) Knows that the measure of success is not the number of reports produced, it's the amount of actionable insight produced.
g) Is vendor agnostic. Sees the tool for what it really is. A means - not an end.
h) 80/20 rule. Spends 80% of the web analytics budget on quality head count and 20% on technology. If they can't afford both an enterprise package and quality head count, goes for a free web analytics package until such time sufficient value and case studies have been derived to increase the size of the pie.
i) Knows they can't answer everything with sufficient certainty to make a decision on one input alone, all the time.
j) Leads with actionability and value. Do this, get X% lift, get Y% in incremental, costs Z, get A in profit. (Plus or Minus a confidence interval B)
2. Starting Small Versus Going Big
a) The advantages of starting small include: lower risk, higher odds of success, gaining of momentum, ability to easily reach again if one fails, ability to learn and re-apply knowledge of failures.
b) The advantages of going big include: bigger payoff and much greater credibility if something is executed, greater chance of a gaining support for a better design pattern from the get-go.
The decision of which way to go depends on the character of both the analyst and the company
3. Winning Support
a) Gaining the support of both middle and upper management for anything is vital. Not everybody really matters, but every little bit of support doesn't hurt.
b) Building a bridge to IT. IT typically owns the technical infrastructure of the website. They have your tags, and can break them. You need their support to get your tests and insights implemented.
c) Building a bridge to Finance. Jim Novo is absolutely bang on about this.
d) Building a bridge within Marketing. Web Analytics typically (not always), lives within Marketing. If a web analyst is there, chances are somebody in that department sees the value of you being there. Make sure that your manager is onside.
e) A web analyst without any support from management will initially fail to get anything executed. If this situation degenerates into a trend, the web analyst will ultimately stop generating actionable insight, suffer from skill decay, and eventually, defect to another company or department.
4. Communication Cadence
a) Automate dashboarding. Nobody ever made their company any money with a dashboard. A dashboard says what happened in the past. A dashboard has yet to tell me what is likely to happen in the future.
b) Bi-weekly or Monthly spotlights. Use powerpoint or video (the medium is the message). Remind them what you told them last time, what's been done about it since, what you found this time, options for exploiting the opportunity.
c) The web analyst is indispensable when considering any online marketing campaign.
d) Report generation cadence should not be so high as to preclude the analyst from doing their real job: adding value, intelligence, and making the company more money.
5. Prioritization
a) If nobody is owning it, own it.
b) Assist the person who owns it.
6. Anticipate the satisfaction/disatisfaction curve
a) The set of business questions that could be asked of an analyst is infinite.
b) The set of business questions that a web analytics tool can answer, at any given time, is finite.
c) Expectations of what the analyst is able to answer will inflate as the analyst delivers successfully.
d) The amount of incremental satisfaction with the tool will decrease as time passes.
e) The argument for supplemental tools that match queries that could not be answered will become more compelling over time, so long as these have been documented.
f) The request for more advanced tools should be as proactive as positive, in anticipation of the successor sets of business questions.
g) The business questions should be compelling enough to the people who matter to justify the increased budget.
h) Eventually, the web analyst will have to learn statistics once the "Hour a Day" questions have been exhausted.
And that's it.
It's a start.
If we can get together and agree on the general characteristics of a good organizational design pattern for web analytics, the adherents to that design pattern should be in a better place.
Thursday, May 7, 2009
If Web Analytics is so Easy, Why is it so hard to get anything done with it?
If Web Analytics is so easy, why then is it so hard to get something done with them?
Start with physical technology - the machines and software:
1) The accuracy of the data is in question (See the case of Peterson v. Unique Visitors / Cookies)
2) Web Analytics tools are incapable of automating actionable insight
3) Web Analytics tools report hindsight. They typically do not forecast or perform what-if analyses
4) Some Web Analytics tools require very careful calibration to report anything of value
Solve the above 4 issues with social technology - the rules, processes, culture, norms, hierarchies and networks that people, tribes, communities, departments, companies and associations have developed over centuries:
1) The accuracy of web analytics will always be in question - there's a legal limit to how accurate we can get. The primary goal has always been to improve accuracy, however, we have a branch of science that enables us to estimate the risk associated with making a decision based on a data feed whose accuracy is in question. Statistics, for most, is not 'easy'. But assume that it is.
2) Machines do not produce actionable insight. They produce the data outputs that humans use to extract insight. There are very simple analyses that people can do if they have a book that shows them, step by step, how to do it. For most subsets of business questions though, the process of extracting insight out of data, and putting the data in context, is not easy. But assume that it is.
3) Web analytics vendors are not incented to invest in the processing cycles necessary to forecast data into the future. Data is typically extracted, transformed, and loaded into a machine that is capable of produce predictive outputs. This process is not easy, especially since the extraction and transformation processes are hard. But assume that it is.
4) Setting up a web analytics tool is easy to do - poorly. Dump the tags in and go! gogogogogo. However, most web analytics tools require the technitian to know the entire set of potential business questions that every user group might want an answer to. Otherwise the tags don't track it and it's gone! Forever. This process of gathering requirements and translating them into requirements is not as easy as dumping the tags in and going. But assume that this is all easy.
Assume we've reached a point where we can answer all the business questions necessary and we're in a place to extract a killer insight.
What now?
If you're also the developer, IA, creative and project manager of a website, it's no problem. You just go about getting your Google Optimizer tags on and your creative on and away you go. Awesome, you've just spent an hour a day and got a 900% lift in ROI. You're the man now, dog.
But what if you're in a company with an IT department. That happens to have an IA. Creative. Business stakeholders.
What if your company lacks the physical technology necessary to get those Optimizer tags in place?
What if the social technology of your company is so rooted in the pre-web-analytics-is-easy revolution that the processes and institutions necessary to make actionable insight actioned simply doesn't exist?
The answer becomes: build those processes and institutions, right? Lead. Build them. Argue for the value. Show them the value!
1. Do this
2. PROFIT!!! (1% boost in conversion = 5 million dollars in revenue. Oh yeah!)
And the building of institutions is easy, because inertia - the tendency of a body at rest to stay at rest, doesn't really apply to human hierarchies or networks. And humans instantly see the value that the web analyst is trying to bring and will willingly remove all obstacles.
To be fair - there are many very easy things you can derive from Google Analytics on your blog. You can tweak alot, and you don't necessarily need to extract the data and forecast things into the future. If all that knowledge is in your head, then you're optimizing just fine. The Analyst-Technologist can improve a website very easily. Web analytics, for them, is easy.
It's easy because they don't have to confront human complexity. Problems 1 to 4 are abstracted right out of existence. The user is doing some pretty simple analyses, some pretty good hindsight, and doing some inductive reasoning. The user trusts the users own judgement. There's nothing social about it.
If you're in a rather large organization though - the type of organization where a 1% bump in conversion results in a 5 million increase in revenue - the effort is totally well worth it. No question, there's loads of value in analytics.
Deriving an insight and following it through to execution is not as easy as we've all been led to believe. And yes, I can clearly say that we've all been led to believe that it is so easy. To be sure, as the industry matures and more of the process of optimization is automated, it'll become easier. (Keep it up guys!)
It's only by confronting the realities and building the bridges, processes, culture and institutions necessary that we'll be able to make optimization as routine as spaceflight. That we'll be able to realize the Optimization dream.
Web analytics, as a single person with all the power, all the authority - that's easy.
Web analytics, in the context of an organization, with distributed authority and all the design patterns that go along with it - that's not easy.
Now get off my lawn.
Start with physical technology - the machines and software:
1) The accuracy of the data is in question (See the case of Peterson v. Unique Visitors / Cookies)
2) Web Analytics tools are incapable of automating actionable insight
3) Web Analytics tools report hindsight. They typically do not forecast or perform what-if analyses
4) Some Web Analytics tools require very careful calibration to report anything of value
Solve the above 4 issues with social technology - the rules, processes, culture, norms, hierarchies and networks that people, tribes, communities, departments, companies and associations have developed over centuries:
1) The accuracy of web analytics will always be in question - there's a legal limit to how accurate we can get. The primary goal has always been to improve accuracy, however, we have a branch of science that enables us to estimate the risk associated with making a decision based on a data feed whose accuracy is in question. Statistics, for most, is not 'easy'. But assume that it is.
2) Machines do not produce actionable insight. They produce the data outputs that humans use to extract insight. There are very simple analyses that people can do if they have a book that shows them, step by step, how to do it. For most subsets of business questions though, the process of extracting insight out of data, and putting the data in context, is not easy. But assume that it is.
3) Web analytics vendors are not incented to invest in the processing cycles necessary to forecast data into the future. Data is typically extracted, transformed, and loaded into a machine that is capable of produce predictive outputs. This process is not easy, especially since the extraction and transformation processes are hard. But assume that it is.
4) Setting up a web analytics tool is easy to do - poorly. Dump the tags in and go! gogogogogo. However, most web analytics tools require the technitian to know the entire set of potential business questions that every user group might want an answer to. Otherwise the tags don't track it and it's gone! Forever. This process of gathering requirements and translating them into requirements is not as easy as dumping the tags in and going. But assume that this is all easy.
Assume we've reached a point where we can answer all the business questions necessary and we're in a place to extract a killer insight.
What now?
If you're also the developer, IA, creative and project manager of a website, it's no problem. You just go about getting your Google Optimizer tags on and your creative on and away you go. Awesome, you've just spent an hour a day and got a 900% lift in ROI. You're the man now, dog.
But what if you're in a company with an IT department. That happens to have an IA. Creative. Business stakeholders.
What if your company lacks the physical technology necessary to get those Optimizer tags in place?
What if the social technology of your company is so rooted in the pre-web-analytics-is-easy revolution that the processes and institutions necessary to make actionable insight actioned simply doesn't exist?
The answer becomes: build those processes and institutions, right? Lead. Build them. Argue for the value. Show them the value!
1. Do this
2. PROFIT!!! (1% boost in conversion = 5 million dollars in revenue. Oh yeah!)
And the building of institutions is easy, because inertia - the tendency of a body at rest to stay at rest, doesn't really apply to human hierarchies or networks. And humans instantly see the value that the web analyst is trying to bring and will willingly remove all obstacles.
To be fair - there are many very easy things you can derive from Google Analytics on your blog. You can tweak alot, and you don't necessarily need to extract the data and forecast things into the future. If all that knowledge is in your head, then you're optimizing just fine. The Analyst-Technologist can improve a website very easily. Web analytics, for them, is easy.
It's easy because they don't have to confront human complexity. Problems 1 to 4 are abstracted right out of existence. The user is doing some pretty simple analyses, some pretty good hindsight, and doing some inductive reasoning. The user trusts the users own judgement. There's nothing social about it.
If you're in a rather large organization though - the type of organization where a 1% bump in conversion results in a 5 million increase in revenue - the effort is totally well worth it. No question, there's loads of value in analytics.
Deriving an insight and following it through to execution is not as easy as we've all been led to believe. And yes, I can clearly say that we've all been led to believe that it is so easy. To be sure, as the industry matures and more of the process of optimization is automated, it'll become easier. (Keep it up guys!)
It's only by confronting the realities and building the bridges, processes, culture and institutions necessary that we'll be able to make optimization as routine as spaceflight. That we'll be able to realize the Optimization dream.
Web analytics, as a single person with all the power, all the authority - that's easy.
Web analytics, in the context of an organization, with distributed authority and all the design patterns that go along with it - that's not easy.
Now get off my lawn.
Monday, May 4, 2009
The ability to ask questions
Assume we had all the data we needed. Assume that all the systems were talking together. Assume that insights could be executed flawlessly. Assume that an analyst wasn't weighed down by reporting.
Assume we've reached the gates of paradise.
Would most analysts know what to do with it?
To be sure, we tend to build these systems with some idea of the business questions we want answered. These include:
"Who are my most profitable customers?"
"What are my most profitable customers like?"
"Where can I find new customers like them, cheaply?"
"Who are my least profitable customers?"
"What are my least profitable customers like?"
"How can I avoid attracting those customers, cheaply?"
In reality, the set of business questions might be far less powerful. Or more powerful. Or bigger. Or might include "How many people are coming to my website". Assume we could measure people for matter.
Assume an analyst could ask any question and get an answer to it that was sufficiently accurate to base a major business decision off it. Or a set of decisions for n customers.
Assume a can opener.
This is where the value of curiosity really starts to come in.
The ability to not only ask questions, but to define the set of questions that are most likely to drive value, and then to ask the questions after that question: that's the rare stuff.
I don't think we should take that skill for granted.
I figure that while we're building the Kingdom of Data Heaven, maybe we should keep in mind the people who are most likely to drive the greatest value out of it. For those who are curious enough to ask the big what if questions after the base questions.
Maybe we should be thinking about the people who are insatiably curious - who are obsessed with completing more of 'why' than asking more of 'what'.
Heaven isn't a static dashboard. Heaven is a sandbox.
Assume we've reached the gates of paradise.
Would most analysts know what to do with it?
To be sure, we tend to build these systems with some idea of the business questions we want answered. These include:
"Who are my most profitable customers?"
"What are my most profitable customers like?"
"Where can I find new customers like them, cheaply?"
"Who are my least profitable customers?"
"What are my least profitable customers like?"
"How can I avoid attracting those customers, cheaply?"
In reality, the set of business questions might be far less powerful. Or more powerful. Or bigger. Or might include "How many people are coming to my website". Assume we could measure people for matter.
Assume an analyst could ask any question and get an answer to it that was sufficiently accurate to base a major business decision off it. Or a set of decisions for n customers.
Assume a can opener.
This is where the value of curiosity really starts to come in.
The ability to not only ask questions, but to define the set of questions that are most likely to drive value, and then to ask the questions after that question: that's the rare stuff.
I don't think we should take that skill for granted.
I figure that while we're building the Kingdom of Data Heaven, maybe we should keep in mind the people who are most likely to drive the greatest value out of it. For those who are curious enough to ask the big what if questions after the base questions.
Maybe we should be thinking about the people who are insatiably curious - who are obsessed with completing more of 'why' than asking more of 'what'.
Heaven isn't a static dashboard. Heaven is a sandbox.
Subscribe to:
Posts (Atom)