The Toronto Analytics Ecosystem

Toronto is a very interesting place right now.

There's an energy here, roiling away at the lower level.

I'm hearing good things coming out of the data mining community. I'm hearing good things at MaRS, and a few interesting web analytics projects that are emerging around pub tables across the downtown core.

There are developers, statisticians, search professionals, ecommerce directors, data miners, entrepreuneurs, social scientists and hard scientists - working away on different implementations, on different levels, in different sectors. It's hard for me to know what's happening in the other analytical capitals - Hong Kong, London, San Fran, and the New York - Baltimore - Washington corridor.

But it's finally happening here.

Do you know a restaurant owner?

Do you know a restaurant owner?

I have an analytical optimization idea, and I need a case study to prove it.

The best thing you can ever do is to work on real data and measure real results.

If you could throw me an email: cberry.blog@gmail.com if you know one, that would help a lot.

Thanks!

Copy Analytics

An example of a copy analytics questions is: which call to action works better?

"Buy now!"

"Learn more..."

The answer, maddeningly, is: "it depends".

That's why we have a methodology for testing. It's the scientific method. Just because we have Google Website Optimizer that can, in theory, answer that question very easily, that doesn't mean that we throw away the scientific method altogether.

Just because it's so easy - that doesn't mean that we should be sloppy, either.

For instance, let's assume that you have direct ability to go in and edit your HTML with the Google Website Optimizer code. Assume that you can do this yourself, and you have the know-how. Also assume that you're disciplined enough to keep it simple, and you have a best guess at what you'd want to test.

If 100 visitors came to the site, and 3 clicked on "Learn More..." and 2 clicked on "Buy Now", that doesn't mean Learn More will result in a 6% conversion rate all the time, post test. (If 100 visitors came, 50 supposedly saw "Learn more" and 50 supposedly saw "Buy Now", 3 clicked on "Learn more", so 3/50 = 6/100 = 6%).

That's a sample size of 50. It's not enough.

How much is enough?

That depends on how accurate you want the result to be - and now we enter into statistics.

Thankfully, Google Website Optimizer has a lot of that math built in, so it's pretty good. However, you need to have a very high sample size to make the best decision.

If you have 5000 visits for each element in your test, you'll have a confidence interval of roughly plus or minus 1.4 percent, 2 SD. So, if you had "Buy Now" at 6.0%, and "Learn More" at 6.2%, you couldn't be all that confident that Learn More really is doing all that much better. It is, in absolute terms, doing better, but it's still really tight. The degree of uncertainty between the two is really tight. If "Learn More" was at 6.5% and "Buy Now" was at 5.0%, then under those circumstances, you could be fairly confident that by going with "Learn More", you'd realize a 1.5% lift. Frequently we're looking at smaller incremental lifts, meaning, sample sizes need to be much larger.

There's a lot more to copy analytics than just call to action.

The leading headline, lead sentence, second sentence, and choices of pictures are important. As is adeherence to various tones. For instance, the difference between:

"The AM-C Pro Remote Control features a prominent mute button."

versus

"So go on and shut up those annoying commercials with the jumbo sized mute button."

The second version of copy might resonate better with a target audience. Or it might not.

It depends.

Are you are Marketer, or are you really just all about Advertising?

Jim Novo asks: Are you are Marketer, or are you really just all about Advertising?

You should read it. Go on, I'll wait.

I'm a marketer.

Marketing involves examining the lifecycle and being really smart about it. There's so much hype over this white whale called "CRM", and it's to the point where I don't even want to call it CRM anymore. I'd much rather call it lifecycle management or lifecycle optimization or lifecycle hummingbird than call it CRM.

Novo's pioneering work around lifecycle management - doing it in a spreadsheet, was really inspiring. Most businesses don't have more than 65000 customers and 256 pieces of data on them. I took some of his techniques and applied them to SPSS syntax for more rows, columns, and ease of advanced analysis. The universality of the approach is just awesome.

I get excited when I read very elegant, well explained blog post on the topic.

So I'll pass the question along: are you a marketer or an advertiser?

The Productivity Trilemma 3: An Expansion on Hamel's Point

Dave Hamel, friend of the show, wrote:

"The problem arises when a company or product has no differentiating feature. If your product is producing websites, then you’re screwed. Sorry but you’re totally hosed because there are lots of places that produce websites. There is a library of software that can produce websites. There is an army of developers in India who will create exactly what they are told.

Many digital marketing agencies are just that, website producers. They may talk about being innovative, and cutting edge but they’re not. Innovation takes risk and “cutting edge” cuts both ways. They have a profit margin to meet or angry share-holders to deal with."

And he makes valid points.

It's all about profit. Economics and the LRAS curve teaches us that eventually, at some point, factories will be squeezed to where marginal profit equals marginal costs. The fixed costs are always lost. It's a very gruesome theory.

It's been drilled into us, for years: become a commodity, be prepared to make zero margin.

Maciek made the great point that people misread that as: get into management. Fight commodification with management!

We destroyed that false logic already. So now we enter into the question of how to differentiate, and how to differentiate in such a way that productivity gains are evident, and defend margin.

If we accept that innovation is the path to differentiation, then the inherent problems of fostering 'innovation' are not unique to digital agencies. There are core problems throughout the American and Canadian economies.

Everybody, in theory and on paper, wants 'innovation'. How many people really want to take the risks to actually achieve it? What is the social process through which 'innovation' happens? What are the design patterns available for 'innovation'.

Naturally, I'm expecting a deluge from the HBR people, who will drown us all in case studies. And while I am from the actual scientific comparative method, those case studies don't always fit in with anything that I'd consider 'science' at all. Ie. There's no scientific cohesion or understanding in the literature I've read out of that school.

I'm aware of the innovation skunk works, labs, that major corporations have. Get big brains away from the routine and politics of the broader organization, and let them come up with crazy ideas. Interesting model. It's a Quaternary Sector approach, isn't?

I'm aware of all sorts of case studies around enterprising managers. The balance between breakers and maintainers and how companies evolve.

I'm acutely aware of Maciek's sub-point that the big brains don't want to go to the major corporate ladder, they want to go to the hot startup.

Innovation is risky. But isn't that the crux? Without risk, there is no reward. There is no margin. If you want to avoid all risk - go produce a commodity.

How do we manage the risk of innovation? What are the best design patterns to foster a culture of innovation? I'm looking for actual criteria here. They can be drawn from the case study literature, but it needs to be put into a framework. An actual, coherent, framework - with an actual comparative method.

That's how we kill the LRAS curve and grow our way out of this hole. We kill it with fire.

The Productivity Trilemma 2: A Second Response to Maciek

In response to the post on The Productivity Trilemma, Maciek went replied with another, focusing on Productivity, Niches, and Aesthetes.

Maciek's first point - that most people associate gains in productivity as a deliberately cheapening of the inputs, resulting in a crappy, cheap output, is quite apt. Any trip to a big box retail outlet will confirm that link for you. And that sort of productivity gain is just fine if you're in a race to the bottom.

The implicit assumption running through Maciek's post, I'm happy to say, is a race to the top. How does one go about producing artisan products, of high quality, for niche markets, in a manner that is highly productive? This culminates in the passage:

Moving forward, we are transitioning to an age of niche products and services that can't hope to attain gigantic followings and easy economies of scale. Seth Godin would suggest that every tribe of 1000 customers is the key to another 1000, and so on, but what if a product only will ever appeal to a small number of people? What if all products worth having a knowledge economy salary for are niche products? What if we shrink that number even further?

I think you can shrink the number down even furthur.

Frequent readers of this blog can watch the struggle. The Economics of Analytics I, II, and III were all struggles to clearly enunciate why I think so many client-agent relationships struggle along, especially when it comes to Niche Services, or ultra Niche Services - services that may only have 10 clients in the world. For instance, I could invisage a service whose only customers would be major web analytics vendors - of which there are no more than 10 who are 'majors'. Those types of services could be incredibly niche.

What about mass market products?

Consider Starbucks. I think I read somewhere that customers can create 10 billion different combinations of products through their requests. (Or something like that). At Subway, you have a similar array of choices. Of course, both businesses start off with a base product - for Starbucks, it's just about making hot water taste like something. At Subway, it's about adding stuff on top of bread (and during the Atkins boom, low carb 'wraps'). These companies are, at least at the franchise level, 'productive' on some level. They're also Just In Time (JIT) in other respects. And they produce very niche products for their customers.

I'd argue that a lot of software development, a lot in software engineering, has been all about how to make a Subway sandwich without fail. Except the Bread is a framework of documentation. I know I'll get into trouble for comparing the two processes - but I think they're striving for the same thing. A repeatable process that results in a highly customized output without fail. Regardless of Subway's label of 'Sandwich Artist', they're not actually Artists. The Client is mostly in control of the entire process.

Indeed - it's very difficult to assemble a piece of software in the same way that a Sandwich is assembled. Imagine that you're the client, not the Artist, and you don't know what Onions taste like. But you see that purple, and you want it. The Artist says, 'alright', and puts them on. Then, when you're cramming that sub down your word hole, you taste your first onion, and proclaim, loudly, that this isn't what you ordered. The Artist, having just completed his degree in aviation security policy, is bright enough to quip that "It's exactly what you ordered, it just isn't what you expected".

So, to Maciek's point - when you exist in a sector where what you do by definition is not commodified, expectations will be skewed. Expectations are especially skewed when the agent doesn't know enough to set expectations, or to communicate clearly what the end result will, and will not, do.

This leads to client-agent innefficiencies, and ultimately, a high degree of waste.

Enter design patterns and competitive advantage through efficiency and brilliant client experience design?

Design Patterns in Organizational Behavior

Continuing the themes coming out of the last Web Analytics Wednesday in Toronto - I now turn to Design Patterns in Organizational Behavior.

A great director, Matt Milan, once gave a presentation on design patterns - it revolved around some World War 1 pilot whose name I can't remember. A design pattern is a fundamental pattern that repeats or can be used again. It's a chunklet, if you would. In Information Architecture, a design pattern might be what we describe now as a form field with a button to the right of it that reads "search". It's what we now know as a search box. While the size of the box and button might vary, the general pattern of a search functionality remains the same.

So it is with Organizational Behavior. Let's apply his concept to organizational behavior.

My take on organizational behavior is from the public policy side of things. It's a world with its own jargon - of Lock-In and Garbage Cans, Hollow Cores and Organized Anarchies. I suppose we all see the world in paradigms, in one way or another. I don't know anything about sociology.

Getting back to the now proverbial "can opener", if I may, what does a good design pattern for an analytics capacity look like? That, indeed, is the cheap question. It's worth nothing. The answer to that question is worth hundreds of millions of dollars if somebody could find the answer and execute against it.

Somewhere, at the intersection of how people are measured (or, if you would, judged), the relationships of power between people (cohort, direct report to report, competitor, faux-competitor, slave, and so on), policy (process, standards, rules, habit), culture, and training protocols (or put simply, 'learning') - there's a solution.

I can name a number of design patterns.

I call one of them "The Clusterfuck". A Clusterfuck is characterized as too many people involved in a project, with no clear, single owner, and ill defined boundaries around what each person is an expert on. A Clusterfuck typically arises when, for want of time, people are thrown at a problem. The time pressure contributes to stress, and as a result, people commonly stop talking to each other. Some people focus on what others judge to be the wrong details. Things get dropped. Of all the design patterns, I abhor the Clusterfuck.

The next design pattern, is the "Hollow Core". A Hollow Core is caused when a vacuum is created, either for want of definition, or deliberately. There is no leviathan (a decider), and each participant in restricted from filling that void (as we're told that nature abhors) by the link to each other. It's a wicked problem. Most people will agree that that there's a void. Nobody will agree who should fill it. Worse, nobody really has the authority to do so, and those who do have authority frequently can't see them. Hollow Cores can exist within organizations for decades, running around like little primordial black holes, sucking in alarm clocks and astronauts. They're insidious - and I fear growing up and being one of those people in authority who can't see them. Or won't admit that there is one.

Another design pattern is the Routinized Hierarchy. This is the bane of creative departments, but ironically, from what I've observed, agency creative departments seem to be the most routinized. Deliverables are well known, quantified, and it's the quality that varies widely. Relationships are well defined, and work is typically done on time, on spec. Nobody debates the difference between a 45x45 pixel, RGB, flattened PSD and 50x50 one - it's well known. I think they benefit from that. Decisions are made from the top - down. Maybe the grass really is green over there.

There are many design patterns in organizations. Understanding that you're in a Clusterfuck is great. You understand the problem you're confronted with. Being in a Hollow Core is another. Then you're supposed to know a solution for getting out of one. (You shatter the ring that's causing the Core. Or, my friend at the CBC would have somebody else shatter the ring. Sly kitty.)

I'm interested in hearing about any other organizational design patterns.

I won't and don't pretend that I have all the answers. (Nor would I announce it on a blog if I did). But let's hear them!