Metering and the false notion of overuse

If you're following a Canadian tech entrepreneur or scientist on Twitter, you might be noticing the #stopthemeter hashtag and statements questioning something called the #CRTC.

The CRTC is the regulatory body responsible for regulating radio, television, and Internet Service Providers in Canada.

On paper, Canada has six or seven major teleco's. These are divided up by region. Telus and Shaw compete in the West. Rogers and Bell compete in Ontario. Videotron and whoever competes in Quebec. There are regional variants and government monopolies in the smaller provinces.

Canada is peppered with duopolies - which are, in effect what economists might call natural monopolies.

A complicating factor is that Canada is a massive country with a very dispersed rural population. These duopolies are enabled by the government to use one portion of the base to subsidize the other. This isn't such a problem, though, it has the capacity to be a major irritant given the cleavage between urban Canada and rural Canada.

(What is a problem is when a member of that duopoly refuses to fund a rural expansion, capital is raised by a local ISP, and then a member of duopoly rushes in to undercut and destroy that local ISP. This problem isn't foreign to rural Canada - we have rich history regarding captive railway shipping. Same phenomenon, different century. I digressed.)

A sequence of regulatory decisions by the CRTC has allowed the major teleco's to lower the ceiling for bandwidth, and to charge extremely high fees for bandwidth over that limit. To put into perspective, what used to be unlimited bandwidth plans, introduced by the big teleco's to run upstart ISP's out of the market in the late 1990's, are now being replaced with 25GB max and several dollars per incremental GB plans. This is, by a factor of 10, more restrictive than what is enjoyed in the United States.

The argument eliminating from PR departments and the CRTC relates to the 'average consumer' - who uses well under 25GB. They portray high-bandwidth households as being overusers. Who needs all that bandwidth anyway? They tend to strongly suggest that overusers are complicit in piracy.

It's a gross oversimplification and I'm calling them out on it.

Consumers are demanding greater choice and control over the content they consume. And some companies have responded.

In recent years, Apple and Netflix have emerged as major content providers. Telco's have grown into providers of content themselves. Many high bandwidth users migrated from the major players, onto smaller ISP's that didn't have the caps. In effect, they want to defend their broadcast cable content businesses at the expense of paid content providers.

One might say that the major players have been responsive to consumer demand - so there is no need for Apple or Netflix. I'd argue that if the major telco's were successful in sating consumer demand, Apple and Netflix would not exist. Aren't we supposed to let the free market decide?

Moreover, fundamentally, captive shippers do not have access to the free market. This is the reason for regulation in the first place. Natural monopolies, those generated by exclusive access and high entry costs, are caustic to overall well being of the economy. Which brings me to my final point and thrust of my concern.

My concern is centered on Canada's analytics, tech development and startup industries. It's based on the effect that these punitive regulations will have on cloud computing and Canadian competitiveness.

Will college students be apt to create the next Facebook and bootstrap it? Will entrepreneurs, many of whom start out of their basement, be likely to transfer high quality video to their websites? What impact will such bandwidth requires have on telecommuting and skyping in this country? What of the incentives for Canadians to consume new media generated in Canada?

What will this do for talent retention in Canada?

What will this do to our international competitiveness?

I can forecast that students won't share data heavy apps if there's a huge cost to innovate. Entrepreneurs in cloud intensive applications won't bootstrap out of Toronto, Calgary and Vancouver. The sharing of high data applications by way of telecommuting will fall. Canadians will consume less video online. It will harm talent retention.

It will severely blunt our international competitiveness.

Canada, to retain middle power status, needs every single advantage we can get.

Metering sends us backwards, not forwards.

On Listening Platforms

The space is tremendously fragmented because social itself if fragmented, unstructured, and ill behaved.

Broadly, there's 'listening', which has its origins in the PR space, and then there's marketing performance, which has its origins in the analytics space. Although there are nearly 250 (+) 'listening' companies out there, none of them will have a solution that fits your unique set of circumstances, biases, and needs. You are simply not a PR person.

Web analysts entering into social should be prepared to confront fragmentation and complexity on a level that they have yet to experience. If you work in an enterprise with more than 150 people, you will rapidly reach a stage where you will not be able to keep pace with demand for basic reporting - little though insight generation. It's simply such a beast.

Solving this problem is among why I went to Syncapse.

ETL

ETL stands for Extract, Transform and Load. They're the three vital steps most analysts do before Analyze, Investigate and Storytell.

Most of the time, the ET'ing is done for us. You log into a tool and hit export. The Loading part, getting the data into a format where it can be statistically analyzed or presented in a culturally acceptable way, is longer. And it's where we spend too much time.

But not today. I'm unpacking a tricky T problem.

In an attempt to fully automate an algorithm further and unlock an area of possibility, involves a tricky operation of flattening lists of lists of lists, which, sadly for me, are also composed of lists. It's tricky. There are functions that do that. It's a matter of finding the right one and using it the right way. I've found plenty of functions that do it the wrong way. (oh yes).

Transformation can be especially difficult for an analyst that understands programming. The basic data structures, logic, and iteration functions are enough to get started. To really start solving somewhat specific problems involves a few additional skill sets, like research, planning, testing, and becoming incredibly familiar with extended libraries of functions. Usually, a solid knowledge of algorithms and complexity is required to solve the trickier problems. Thankfully I can run downstairs and keyword hunt with the real programmers.

And that's where I'm at this morning. Right at T.

.

Why convinient reasoning isn't insight generation

An insight is:

• New information
• Executable
  
• Causes action
• Profit results
  

With that piece of jargon unpacked, the next one is 'convenient reasoning'.

Convenient reasoning is:

• An existing heuristic, hunch, feeling, belief, or instinct
• The seeking of validation or evidence
  
• Evidence to the contrary or modifying the position will be rejected

Convenient reasoning differs from a hypothesis. A hypothesis is rejected if it's proven wrong.

No amount of evidence to the contrary will ever deter a convenient reasoner.

Building cases in support of a project, plan, or prospect is an incredibly important skill. Rallying persuasive evidence is a key part of that. A whole industry was built around the provision of convenient facts.

It's an essential skill.

Perhaps there would be fewer disasters if, during the research portion of building a case, evidence - if it's a valid insight - could actually alter the case being mad for the better.

The difference would be research for the sake of supporting a supposition, and genuine insight generation.