Digital Crows Nest

Hard data about piracy has always been hard to find. Traditional “Anti-piracy” companies have been known to make broad assertions from limited data sets in order to galvanize a sense of fear and urgency and boost their own sales. The logical flaw in anti-piracy is that if these businesses were truly capable of stopping piracy, they would be putting themselves out of jobs. Ending piracy is not in their best interest; and it also isn’t possible. Piracy has been around for a very long time and it is constantly evolving to overcome the efforts to stop it.

What the industry needs is a different approach. For over two years I’ve been tracking digital piracy for Magellan Media Consulting Partners as part of an ongoing study.  The preliminary results of the study to date have put many assumptions about how piracy is impacting the industry into question and has led to the conclusion that what publishers need is a better method of understanding  and dealing with the issues. I started Digital Crows Nest with the goal of giving me some space to investigate the philosophies and technologies behind piracy and the publishing industry. This blog is only a small part of the efforts I’ve been undertaking to quantify the data available. Behind the scenes there has also been the development of programs and systems to aid in the tracking of piracy.

As a result of our behind the scenes work, the tracking I’ve been doing for for the Magellan Media study has now become fully automated. The systems are currently tracking the piracy rates of ebooks across many of the major public bittorrent piracy sites. What this means is that we are capable of expanding our tracking into additional areas, allowing us to expand beyond the initial confines of the Magellan study (which focuses primarily on public p2p and bittorrents). It is an exciting time, but filled with plenty of other work to keep the ball rolling.

And none too soon. Publishers are stepping up their vigilance on piracy. Authors are calling piracy one of the biggest threats to the industry and the U.S. government just released a 65 page plan outlining their proposed plan to deal with piracy.  All of this despite numerous reports indicating that we don’t have any real idea as to what is actually going on. More data is needed, and as we continue to work towards more advanced systems, we hope also to be able to provide that data to aid publishers in the decisions that will help them effectively balance their digital assets.

The Crow’s Nest has been silent for the last month or so, primarily because behind the scenes we have been very, very busy. As such, expect some announcements within the next few days about the top secret project that we have been working on, which will be ready for launch very soon.

Macmillan. Amazon. Two large companies, two different perspectives. In a debate over how the future of the ebook industry should proceed, which handled their brand better? The argument raged hotly in the first week or two of the infamous showdown, but has since tapered of in a seemingly disgruntled consumer acceptance of the inevitable. Now, two months later, on the inaugural day of the new model, we have an opportunity to analyze the incident that started it all, and offer it up as an example of brand management.

In an attempt to quantify the reactions to the Amazon-Macmillan debate and how each company’s actions have been viewed by the public (and shaped their perception of the corporate brands), I conducted an informal survey over the last 2 months. Looking at forums and blogs that focused articles on the controversy (including- Amazon, the Macmillan blog, Dailymail, Reuters, Teleread, and around 20 others), I analyzed approximately 5000 forum comments and blog postings and derived a general breakdown of the discussions that took place.

Within the data set, we find the overall breakdown of consumer responses was 30% in favor of amazon, 7% for Macmillan, 48% given over to heated debate and 15% of general off-topic information. Looking specifically at those who voiced a definitive position for either amazon and Macmillan (filtering out multiple posts, additional posts clarifying or debating their positions, and off-topic conversations) we find that that 80% of consumers sided with amazon, while only 20% agreed with Macmillan.

The most often chosen arguments in favor of Amazon were:

1. Retailers should have right to set prices- the free market argument
2. Amazon is protecting the consumer and watching out for consumer needs
3. Ebooks are overpriced to begin with

The most often cited reasons posters sided with Macmillan were*:

1. It depends on the book- on certain books people don’t mind paying more
2. Macmillan is protecting author and publisher interests

*67% of posters who sided with Macmillan admitted they were themselves either authors or in the industry

Your average consumer generally doesn’t know or care about what occurs behind the scenes between retailers, distributors, or any other slice of production. By and large it is the end product they are concerned with, and their loyalty to author or series brands far outweighs the importance of retailers or publishers. But when Amazon dropped the buy buttons on Macmillan books it caused a ripple that alerted consumers to what otherwise would have been a behind the scenes negotiation. They pulled the consumer card, and Macmillan fell for it hook line and sinker.

Amazon stated their case and appealed directly to consumers. This is a positive brand move. You analyze what your consumer wants, you attempt to give it to them, and if you are forced to do otherwise, you make it clear what and why you are doing what you are doing and how it effects them. Communication is important, not just making statements, but listening and responding to the feedback. Customers expect Amazon to defend their rights to lower prices, and Amazon did so. And once their stance was made, they could capitulate, citing the desire to let the consumers decide for themselves.

John Sargent made a stand as well. He issued statements and addressed the authors and publishers. He defended his stance openly and while I give him credit for making the attempt, from a branding perspective he failed. While his justifications may be founded on sound reasoning, he fails because not once did he actually address the end-user or mention how this change might benefit the consumers. Indeed, he mentioned them twice without actually addressing them directly and only as a means to accomplish his goal of raising prices. The consideration of what the consumer has stated they would be willing to pay, or how consumers might react to the shift, barely seemed to register as a concern. As a consumer, what does this brand promise? A disregard for the needs and wants of the consumer? A hard-line value focused not on the end-user, but on the back-end of the industry?

Like many publishers, Macmillan has not traditionally put themselves out as a consumer brand, preferring to leave the issue of branding to authors and retailers and allowing them to reach out and respond to consumers. But the agency model changes that and puts the publishers into the public scope by dictating purchasing terms to the consumer. Publishers who follow this model need to recognize that whether they like it or not, consumers will react and it will be the publishers who bare the brunt of the criticism,

Will consumers pay a higher price for a brand they have voiced low confidence in? Will publishers focus on higher quality ebooks, less DRM, and other improvements to make up for the price differential and smooth the ruffled feathers of their consumers? The next few months will certainly be interesting for publishers, retailers, consumers and pirates (old and soon-to-be) and will hopefully serve as a demonstration of the importance of listening to and understanding the consumer base.

One of the first recorded acts of media piracy was perpetrated by an Abbot by the name of Columba in Scotland, around 560 C.E. Columba was well known in the region for visiting monasteries and copying their written works. He would then take the copies back to his own monastery where they were used to help spread the Catholic faith. While visiting a particular monastery in Ireland, he attempted to copy a book of psalms that belonged to the local Abbot, who, upon discovering Columba’s work, demanded that the copy be given to him. Columba refused to hand the copy over and the matter was taken to the local King, Diarmait, who handed down the ruling “To every Cow its calf, to every book its copy.” This statement refers to the rule of the time wherein a calf was the rightful property of whoever owned the cow that bore it, regardless of where the calf was found. The same logic was thus applied to the book, and that Columba had no right to the text since he did not own the original. Columba, however, refused to give up his copy and the battle of Cul Dreimhne ensued, pitting Columba and a group of rebels against the King. The battle ended with over 3000 deaths, including that of King Diarmait. Columba himself went on to become the patron saint of bookbinding and poetry. He is credited for transcribing over 300 books, many of which only survive today because of his willingness to copy those texts despite the letter of the law.

Which of course makes one wonder- should we applaud him for his accomplishments, or condemn him for his crimes? Is he a pirate or a patron saint? What Columba did was a labor of love. He did not copy these texts out of a desire for profit or malicious intent. Yet he clearly violated the dictates of the local law. He copied these texts because he wished to spread the word and the religious texts of his faith and yet in so doing sparked a conflict that cost thousands of lives.

Copyright holders maintain that piracy is going to damage the publishing industry, that the long term effects of piracy will outweigh the short term benefits of increased exposure for authors, and that ultimately, the act and art of writing will be severely curtailed, damaged beyond repair. But consider this: If you are lucky, the average lifespan of a book is 5-10 years. If you are really lucky, the average lifespan of an author after they’ve published the aforementioned book is 50 years, so if copyright extends beyond that by 70 years, the average copyright of that book is 120 years. What are the chances that 115 years after a book goes out of print, with legal copyright locking it away,  that it will still exist to fall into the public domain? What is the impact on art and intellectual advancement if these books are lost altogether?

Consider also that the digital conversion of books has been going on, perpetuated by pirates, for almost 30 years. Much as Columba’s transcriptions were the only means by which several hundred texts survived, many of the books that you can find through piracy sites are books and works that you can’t find anywhere else. So in the short term, piracy helps authors out of obscurity. In the long term, through the dedicated love and sharing of consumers who work outside the dictates of what is considered the word of law by conventional standards, it may be the only thing keeping them from disappearing altogether.

I own a kindle.

I never use it.

It isn’t that the kindle is bad. In fact, it is a great device for reading traditional books. And that is all you are really doing. Reading a digital copy of a print book. When I look at the current complement of available e-readers on the market, I find myself asking one simple question- Are digital readers promoting digital technologies or continuing to lock users and creators into the traditional book mindset?

The experience shouldn’t always be the same from one medium to the next. Just as adapting a book into a screenplay means making changes to build the story properly in a new medium, so should be the case with digital books. An ebook shouldn’t be just a scanned copy of the original. Anybody can do that (and everybody does- they’re called pirates). An ebook should be something new entirely. Just as someone who reads a book goes to see a movie adaptation of the book, so too should a reader want to pick up both the ebook and the original- because the user experience is fundamentally different. The question, of course, is how? Let’s look at an example.

A typical fantasy series such as Robert Jordan’s Wheel of Time or George R.R. Martin’s Song of Ice and Fire series contains chapter by chapter of different characters with different viewpoints, going through events at places simultaneously. With traditional books, the author is solely determining what perspectives you read, in what order, and how often you hear about what is going on with each character. I can’t even begin to tell you the number of times I’ve gotten to the end of a chapter that ended on a cliffhanger *suspense* and then by the time I got around to that character again, I’ve forgotten what it was I was waiting to see resolve. Digital can change that.

In the beginning of each of these books is a map that covers the area (generally a continent) where our heroes and villains and other prospective viewpoints reside. There are generally points of interest and sometimes some basic data such as routes covered, etc.

Imagine if this map were interactive.

You start with a timeline across the top. Let’s say the book begins with five characters at five locations each indicated on the map. As time moves forward (which the user can choose to do at any time by scrolling across the timeline) the characters move locations. You can read from a single character perspective and follow them across the country over the course of a year or jump around to different characters to see what they are doing at the same point in time.

Each location on the map would allow you to zoom in and get a history, possibly a sketch or illustration of the area, and even zoom back in time to witness events that occurred in the past (a smoking ruin at one point might have been caused by a character in a previous point along the timeline). When two characters meet, witness the scene from both perspectives side by side or individually.

Additional content can be added perpetually. You introduce a character briefly, and find that your audience loves them and wants to know more, suddenly you have a spin-off storyline derived right in the same world with all the same elements.

Of course, it also presents print possibilities as well. Print on demand versions of the book that follow the storyline according to how each user wants to read it. Or author’s choice versions (traditional print) that outline specific perspectives with an author’s narrative slant. Each book becomes a malleable platform. You could even distribute a single storyline for free as an introduction into the world you’ve created, getting them to interact in the world and enticing them to continue reading if they like what they see.

Given the proper platform, authors could also include aspects of sound and visual novels including background scenes, noise and interactions. Users could also share an experience with friends by choosing a section of storyline to send to friends to read. Don’t get me wrong though, I’m not advocating that we get rid of the reading aspect of our books, just give the reader a chance to interact with their books a bit more.

This is why the ipad catches my attention. It is capable of utilizing digital technologies in a way that current e-readers don’t bother to do. It leaves developers/authors/publishers a lot of room to create great content in a new medium. Users could purchase the latest George RR Martin App, which contains the storylines of several major characters for, let’s say, 9.99, and then download optional content as it is added (different, related stories, new perspectives, etc) for a dollar or two a piece, and they would be incorporated into the app.

The app would also add the ability to track where you’ve been, what you’ve already read, give you synopsis based only on materials that you’ve read (to avoid spoilers).

Naturally, it isn’t for everyone. Some authors may want more control over what their audience sees and when. Some may balk at the idea of producing content without having the entire story from all sides written, thus leaving them unable to make minor changes to fit the sequence of events as they write and develop. That’s cool, keep your silent films, I’m going to the talkies!

In a world where even Google is being held responsible for user generated content, one has to wonder what would happen if every website unwittingly hosted the bits of information that are required to download copywritten material.

Imagine a distribution system more insidious than spam. By taking the metadata generated by parsing an ebook using the bittorrent protocol, one can embed tiny bits of an ebook all over the Internet. Each piece of content is only a few bits of data, small enough to fit into a forum comment, for example, without being noticed. By using a single torrent file, users are able to gather all of the bits of data from thousands of sites across the Internet and combine those files to create a document they can download.

None of this data is traceable. Each bit of information is simply a string of numbers until it is decoded by the torrent source. And each data piece could be downloaded literally thousands of times before the data transfer would register as anything more noticeable than a page load.

The crux of this is that all the data exists on the Dark Web (the underlying structure of the Internet) and would be nearly impossible to track to its originating source. Likewise, it would be nearly impossible to track downloaders, as even legitimate visitors to your site would be inadvertently downloading these bits of data as part of the page load itself. Every website, every forum comment, every spam email, Facebook post or twitter tweet could be a potential source for disseminating this data.

Today, publishers are desperately trying to treat a symptom rather than dealing with the disease. Unfortunately many publishers seem to eschew piracy understanding and tracking in favor of more traditional methods of ‘anti-piracy’ including lawsuits and flooding p2p networks with corrupt data. One has to wonder what their reaction would be to the newest possibility of p2p- the peerless network.

Who in this new/emerging model are publishers going to sue? Who do we hold responsible for infringing content when even the publishers and copyright holders are now connected to the chain of piracy? Clearly, it is time to rethink the game plan.

As a technology and a philosophy, DRM only serves to hamper the ability of publishers to provide a quality product,  prevents the consumer from getting the best experience possible, and reduces the ability for authors to get the best return for their work. While many arguments have been made about the effectiveness of DRM, some key points must be made clear.

First of all, there is no such thing as Unbreakable DRM. For every programmer or company you pay to develop the technology to prevent your product from being used, there are a dozen programmers who will break your DRM for fun or any number of  motivations all their own. Just because DRM hasn’t been broken, doesn’t mean it can’t be. Usually, it either means that your content isn’t of interest to the piracy community, or there was an easier way to circumvent the DRM.

Which of course, brings us to the second point: you can’t DRM a print book. Harry Potter is probably the most infamous case of this, but in essence, even going so far as not making a book available as an ebook (the ultimate form of DRM) won’t stop it from being pirated. As long as there are fans who want a product and as long as publishers fail to provide that product, someone else will. Piracy isn’t about theft, it’s about the failure of companies to fulfill a consumer need and the consumer stepping forward to fulfill it themselves.

To consumers, DRM stands for “Digital Restrictions Management” or, more simply put, “Don’t Read Me”. It makes your product inferior, and why would anyone pay for an inferior product? Get rid of the notion of Digital Rights/Restrictions Management and into the mindset of providing a higher end user experience by utilizing the benefits of the digital realm. Managing your Digital Rights shouldn’t be about stopping people from reading your works, it should be about making sure that your assets are represented in the best way possible and ultimately ensuring that you are getting the biggest return for your investment.

There are two ways publishers can approach ebooks:

1) Stick to their guns and continue making static books and then importing them to “ebook editions” . A tactic which can be achieved just as easily by anyone with a scanner and some time (and offered to the public at no cost)

-or-

2) Embrace the capabilities of digital: Guaranteed high-quality content, instant downloads, no need to search through dubious sites to download content that might not even be what you are looking for, no risk of viruses or spyware, customized to your specific needs, with value added content and updates for hardcore fans.

Which would you pay for?

It’s a difficult question. Yet, is it any more difficult than measuring the impact of advertising? Or the impact of library loans? How do you tell whether it was a billboard or tv ad that sent a consumer to the nearest store to purchase the latest product? Is the little girl who got her favorite book at the library really a lost sale?  Today, we can track the clicks of the consumer and gain a much deeper understanding of  how effective advertising is or isn’t. While it isn’t foolproof, an ad viewed now may not be acted upon until much later, it is a start- one that advertisers have embraced wholeheartedly to help define the value of their ads.

Of course, free electronic book distribution isn’t advertising (or is it?). And certainly unregulated distribution raises issues because publishers don’t have direct control over what happens with their products. But, like advertising, the impact of piracy on sales isn’t a direct 1 to 1 correlation. Every book downloaded is not a lost sale. And for those that are lost sales the question is why? What drove the consumer to pirate rather than purchase? Are pirates a market segment that aren’t currently being served by traditional marketing techniques? Was it a failure of the publisher to meet a consumer need? Or was it an intentional and blatant refusal to pay for content? Publishers need to consider the motivations of their consumers- are they in it for free? for sampling? are they self-styled “ethical pirates” who already own copies and are acquiring DRM free versions for their personal use? In essence, we need to ask: Where and when does piracy occur, how often, under what circumstances?

Book Industry specific piracy research is hard to come by.  Attributor has released preliminary results from their study and I’ve been working with Magellan Media on an ongoing p2p based piracy study, but both studies admit they are limited in what they can currently accomplish, especially without support from publishers.

The question is, what will it take for publishers to get on board with industry accessible, in-depth piracy research?  What information are publishers interested in? Are they only concerned with being able to issue take down notices or implementing other anti-piracy measures to stop the spread of their content? If they can stop illegal downloads without resorting to lawsuits (such as by responding to consumer needs), isn’t that more beneficial for everyone? If there is a possibility of benefit, isn’t it worth investigating?

Earlier today, Attributor released a study investigating the degree of ebook  piracy occurring online. While the  figures initially seem staggering- an estimated 9 million pirated book copies downloaded, representing potential losses of almost $3 billion- a look beyond the executive summary quickly makes these findings less terrifying than the first glance suggests. Much of the information is based on estimates that leave a lot of room for speculation. They are only able to track download data from 4 of the 25 sites they monitored and calculated totals for the others based on  a percentage share of the number of takedown notices issued during July and December of 2009.

Rather than looking at projections made based on estimations, I would really like to focus on the hard data that Attributor has been able to acquire. There are only four sites that Attributor can verify the numbers for: Scribd, 4shared, wattpad and docstoc. Between those four sites they’ve discovered 3.2 million downloads of their 913 titles (averaging 3,500 downloads per title). I would point out that while the study was conducted over 90 days it appears to cover data representing several years. Each of the four sites that Attributor was able to derive hard download data from tracks their information on a cumulative basis. Which means that while Architect’s Drawings by Kendra Schank Smith had over 10,000 downloads on a single site, that is over the course of 2 years (it was first uploaded in December of 2007). With that in mind, I threw together a quick synopsis of the time tables involved, using the titles that Attributor lists on their blog which we can verify as being part of the study.

Now, I hesitate to draw any specific conclusions from the limited data set above but I’ll leave the data there for readers to pick over. In the meantime, I’ve contacted Attributor in hopes of getting a more comprehensive list of the titles they tracked. If possible, I’d like to see how many of the titles are reprints, backlist, frontlist, old editions, and the time lag between release and piracy. Hopefully there is a reasonably large subset of frontlist titles available so we can observe the trends from day one of release.