The Personal Data Eco-system

Post from 2009 reposted here to facilitate further discussion.

At the VRM workshop, we discussed the need for the concept of the Personal Data Store, what it would do in practice, and what that will ultimately enable.

Why we need such things – because individuals have a complex need to manage personal information over a lifetime, and the tools they have at their disposal today to do so are inadequate. Existing tools include the brain (which is good but does not have enough RAM, onboard storage, or an ethernet socket……thankfully), stand alone data stores (paper, spreadsheets, phones, which are good but not connected in secure ways that enable user-driven data aggregation and sharing), and supplier based data stores (which can be tactically good but are run under the supplier provided terms and conditions). NB Our current perception of ‘personal data stores’ is shaped by the good ones that are out their (e.g. my online bank, my online health vault); what we need is all of that functionality, and more – but working FOR ME.

What they will do/ enable – the term Personal Data Store is not an ideal term to describe a complex set of functions, but it is what it is until we get a better one (the analogy I’d use in more ways than one is the term ‘data warehouse’ – again a simplistic term that masks a lot of complex activity). A Personal Data Store can take two basic forms:

Operational Data Stores – that get things done, and only need store sufficient breadth and depth of data to fulfill the operation they are built for (e.g. pay a credit card bill, book a doctor’s appointment, order my groceries).

Analytical Data Stores – that underpin and enable decision making, and which typically need a more tightly defined, but much deeper data-set that includes data from a range of aspects of life rather than just that from one specific operation (e.g. plan a home move, buy a car, organise an overseas trip).

A sub-set of the individual’s overall data requirement will lie in both of the above, this being the data that then integrates decision-making and doing.

In both cases, the functionality required is to source, gather, manage, enhance and selectively disclose data (to presentation layers, interfaces or applications).

We also discussed ‘who has what data on you’ and introduced the following diagrams to explain current state and target state (post deployment of Volunteered Personal Information (VPI) tech and standards).

The key terms that require explanation are:

My Data – is the data that is undeniably within, and only within, the  domain of an individual. It’s defining characteristic is that it has demonstrably not been made available to any other party under a signed, binding agreement. This space has been increasingly encroached upon by technology and organisations in recent history (e.g. behavioural tracking tools like Phorm) and this encroachment will continue. Indeed a general comment can be made that ‘my data’ equates to privacy in the context of personal data; so the rise of the surveillance society and state is a direct assault on ‘My Data’. Management of ‘My Data’ can be run by the individual themselves, or outsourced to a ‘fourth party service’.

Your Data – is the data that is undeniably within the domain of an organisation; either private, public or third sector. Proxy views of this data may exist elsewhere but are only that. This data would include, for example, the organisations own master records of their product/ service range, their pricing, their costs, their sales outlets and channels. Customer-facing views of much of Your Data is made available for reproduction in the ‘Our Data’ intersect.

Our Data – is the data that is jointly accessible to both buyer and seller/ service provider, and also potentially to any other parties to an interaction, transaction or relationship. It is the data that is generated through engaging in interactions and transactions in and around a customer/ supplier relationship. Despite being ‘our’ data, it is probably technically owned, or at least provided under terms of service designed by the seller/ service provider; in practical terms this also means that the seller/ service provider dictates the formats in which this data exists/ is made available.

Their Data – is the data built/ owned/ sold by third party data aggregators, e.g. credit bureaux, marketing data providers in all their forms. It’s defining characteristic is that it is only available/ accessible by buying/ licensing it from the owner.

Everybody’s Data – is the public domain data, typically developed/ run by large, public sector(ish) entities including local government (electoral roll), Post Offices (postal address files), mapping bureau (GIS). Typically this data is accessible under contract, but the barriers to accessing these contracts are set low – although often not low enough that an individual can engage with them easily.

The Basic Identifier Set/ Bit in the Middle – this is the core personal identity data which, like it or not, exists largely in the public domain – most typically (but not exclusively) as a result of electoral rolls being made available publicly, and specifically to service providers who wish to build things from them. This characteristic is that which enables the whole personal eco-system and its impact on data privacy to exist, with the individual as the un-knowing ‘point of integration’ for data about them.

Propeller Current State

The ovals in the venn diagram represent the static state, i.e. where does data live at a point in time. The flow arrows show where data flows to and from in this eco-system; I use red to signify data flowing under terms and conditions NOT controlled by the individual data subject.

Flow 1 (My Data to Your Data, and My Data to Our Data) – Individuals provide data to organisations under terms and conditions set by the organisation, the individual being offered a ‘take it or leave it’ set of options. Some granularity is often offered around choices for onward data sharing and use, i.e. the ‘tick boxes’ we all know and which are one of the main bitsof legacy CRM that VRM will fix.

Flow 2 (Your Data to Your Data, including Our Data) – Organisations share data with other organisations, usually through a back-channel, i.e. the details of the sharing relationship are typically not known to the data subject.

Flow 3 (Your Data, including Our Data to Their Data) – Organisations share data with a specific type of other organisation, data aggregators, under terms and conditions that enable onward sale. Typically the sharer is paid for this data/ has a stake in the re-sale value.

Flow 4 (Everybody’s Data to Their Data) – Data Aggregators use public domain data sources to initiate and extend their commercial data assets.

The target state is shown below, a different scenario altogether – and one which I believe will unfold incrementally over the next ten years or so…..data attribute by data attribute, customer/ supplier management process by customer/ supplier management process, industry sector by industry sector. In this scenario, the individual and ‘My Data’ becomes the dominant source of many valuable data types (e.g. buying intentions, verified changes of circumstance), and in doing so eliminates vast amounts of guesswork and waste from existing customer/ citizen managment processes.

The key new capabilities required to enable this to happen are those being worked on in the User Driven and Volunteered Personal Information work groups at Kantara (one tech group, one policy/ commerce one), and elsewhere within and around Project VRM. The new capabilities will consist of:

– personal data store(s), both operational and analytical

– data and technical standards around the sharing of volunteered personal information

– volunteered personal information sharing agreements (i.e. contracts driven by the individual perspective, creative commons-like icons for VPI sharing scenarios)

– audit and compliance mechanics

Around those capabilities, we will need to build a compelling story that clearly articulates, in a shared lexicon (thanks to Craig Burton for reminding us of the importance of this – watch this space), the benefits of the approach – for both individuals and organisations.

The target state that will emerge once these capabilities begin to impact will include the 4 additional individual-driven information flowsover and above the current ones. The defining characteristic of these new flows is that the can only be initiated by the data subject themselves, and most will only occur when the receiving entity has ’signed’ the terms and conditions asserted by the individual/ data subject. The new flows are:

Flow 5 (My Data to Your Data (inc Our Data) – Individuals will share more high value, volunteered information with their existing and potential suppliers, eliminating guesswork and waste from many customer management processes. In turn, organisations will share their own expertise/ data with individuals, adding value to the relationship.

Flow 6 (Everybody’s Data to My Data) – With their new, more sophisticated personal information management tools, individuals will be able to take direct feeds from public domain sources for use on their own mashups and applications (e.g. crime maps covering where I live/ travel)

Flow 7 (My Data to (someone else’s) My Data) – An enhanced version of ‘peer to peer’ information sharing.

Flow 8 (My Data to Their Data) – The (currently) unlikely concept of the individual making their volunteered information available to/ through the data aggregators. Indeed we are already starting to see the plumbing for this new flow being put in place with the launch of the Acxiom Identity Card.

Propeller Target State

The implications of the above are enormous, my projection being that over time some 80% of customer management processes will be driven from ‘My Data’. I’m pretty confident about that, a) because we are already see-ing the beginning of the change in the current rush for ‘user generated content’ (VPI without the contract), and b) because the economics will stack up. Organisation need data to run their operations – they don’t really mind where it comes from. So, if a new source emerges that is richer, deeper, more accurate, less toxic – and all at lower cost than existing sources; then organisations will use this source.

It won’t happen overnight obviously; as mentioned above specific tools, processes and commercial approaches need to emerge before this information begins to flow – and even then the shift will be slow but steady, probably beginning with Buying Intention data as it is the most obvious entry point with enough impact to trigger the change. That said, the Mydex social enterprise already has a working proof of concept up and running showing much of the above working. A technical write up of the proof of concept build can be found here. And the market implications of this are explored in more detail in new research on the market value of VPI shortly to be published by Alan Mitchell at Ctrl-Shift.

The two hour session at the VRM workshop was barely enough to scratch the surface of the above issues, so the plan is to continue the dialogue and begin specifying the capabilities required in detail in the User Driven and Volunteered Personal Information (technology) workgroup at The Kantara Initiative. The workgroup charter can be found here. A parallel workgroup focused on business and policy aspects will also be launched in the next few weeks. Anyone wishing to get involved in the workgroup can sign up to the mailing list hereand we’ll get started with the work in the next couple of weeks.

Omie Update (version 0.2)

We’re overdue an update on the Omie Project…., so here goes.

To re-cap:

We at Customer Commons believe there is room/ need for a device that sits firmly on the side of the individual when it comes to their role as a customer or potential customer.
That can and will mean many things and iterations over time, but for now we’re focusing on getting a simple prototype up and running using existing freely available components that don’t lock us in to any specific avenues downstream.
Our role is demonstrate the art of the possible, catalyse the development project, and act to define what it means to ‘sit firmly on the side of the customer’.
For now, we’ve been working away behind the scenes, and now have a working prototype (Omie 0.2). But before getting into that, we should cover off the main questions that have come up around Omie since we first kicked off the project.

What defines an Omie?

At this stage we don’t propose to have a tight definition as the project could evolve in many directions; so our high level definition is that an Omie is ‘any physical device that Customer Commons licenses to use the name, and which therefore conforms to the ‘customer side’ requirements of Customer Commons.

Version 1.0 will be a ‘Customer Commons Omie’ branded white label Android tablet with specific modifications to the OS, an onboard Personal Cloud with related sync options, and a series of VRM/ Customer-related apps that leverage that Personal Cloud.

All components, wherever possible, will be open source and either built on open specs/ standards, or have created new ones. Our intention is not that Customer Commons becomes a hardware manufacturer and retailer; we see our role as being to catalyse a market in devices that enable people in their role of ‘customer’, and generate the win-wins that we believe this will produce. Anyone can then build an Omie, to the open specs and trust mechanisms.

What kind of apps can this first version run?

We see version 1 having 8 to 10 in-built apps that tackle different aspects of being a customer. The defining feature of all of these apps is that they all use the same Personal Cloud to underpin their data requirements rather than create their own internal database.

Beyond those initial apps, we have a long list of apps whose primary characteristic is that they could only run on a device over which the owner had full and transparent control.

We also envisage an Omie owner being able to load up any other technically compatible app to the device, subject to health warnings being presented around any areas that could breach the customer-side nature of the device.

How will this interact with my personal cloud?

As noted above, we will have one non-branded Personal Cloud in place to enable the prototyping work (on device and ‘in the cloud’), but we wish to work with existing or new Personal Cloud providers wishing to engage with the project to enable an Omie owner to sync their data to their branded Personal Clouds.

Where are we now with development?

We now have a version 0.2 prototype, some pics and details are below. We intend, at some point to run a Kickstarter or similar campaign to raise the funds required to bring a version 1.0 to market. As the project largely uses off the shelf components we see the amount required being around $300k. Meantime, the core team will keep nudging things forward.

How can I get involved?

We are aiming for a more public development path from version 0.3. We’re hoping to get the Omie web site up and running in the next few weeks, and will post details there.

Alternatively, if you want to speed things along, please donate to Customer Commons.

VERSION 0.2

Below are a few pics from our 0.2 prototype.

Home Screen – Showing a secure OS, a working, local Personal Cloud syncing to ‘the cloud’ for many and varied wider uses. This one shows the VRM related apps, there is another set of apps underway around Quantified Self.

Omie 0.2 Home Screen

My Suppliers – Just as a CRM system begins with a list of customers, a VRM device will encompass a list of ‘my suppliers’ (and ‘my stuff’).

Omie 0.2 My Suppliers

My Transactions – Another critical component, building my transaction history on my side.

Omie 0.2 Transactions

Intent Casting/ Stroller for Twins – Building out Doc’s classic use case, real time, locally expressed intention to buy made available as a standard stream of permissioned data. Right now there are about 50 online sellers ‘listening’ for these intent casts, able to respond, and doing business; and 3 CRM systems.

Omie 0.2 Intent Casting

So what have we learned in the build of version 0.2?

Firstly, that it feels really good to have a highly functional, local place for storing and using rich, deep personal information that is not dependent on anyone else or any service provider, and has no parts of it that are not substitutable.

Secondly, that without minimising the technical steps to take, the project is more about data management than anything else, and that we need to encourage a ‘race to the top’ in which organisations they deal with can make it easy for customers to move data backwards and forwards between the parties. Right now many organisations are stuck in a negative and defensive mind-set around receiving volunteered information from individuals, and very few are returning data to customers in modern, re-usable formats through automated means.

Lastly that the types of apps that emerge in this very different personal data eco-system are genuinely new functions not enabled by the current eco-system, and not just substitutes for those there already. For example, the ‘smart shopping cart’ in which a customer takes their requirements and preferences with them around the web is perfectly feasible when the device genuinely lives on the side of the customer.

Meet Omie: a truly personal mobile device

This is Omie: OMIE-blank-slate-pcloud-in-corner

She is, literally, a clean slate. And she is your clean slate. Not Apple’s. Not Google’s. Not some phone company’s.

She can be what you want her to be, do what you want her to do, run whatever apps you want her to run, and use data you alone collect and control.

Being a clean slate makes Omie very different.

On your iPhone and iPad you can run only what Apple lets you run, and you can get only from Apple’s own store. On an Android phone you have to run Google’s pre-loaded apps, which means somebody is already not only telling you what you must do, but is following you as well.

Omie uses Android, but bows to Google only in respect of its intention to create an open Linux-based OS for mobile devices.

So Omie is yours, alone. Fully private, by design, from the start.

At Omie’s heart is your data, in your own personal cloud — not Google’s cloud or Apple’s cloud or Amazon’s cloud or the cloud of any other silo’d service.

Think of your personal cloud as a place for your stuff. Right now most of the data you use in the online marketplace — what should be your stuff —  really isn’t. It’s out in clouds that aren’t yours: one for every Web site and service you deal with.

Consider your wallet — the one in your pocket or purse. That’s your wallet. Not Google’s or Paypal’s. Yet right now Google, Paypal and a dozen other companies think the wallet you carry online should be theirs. Wouldn’t it be better to carry all their wallets inside one that’s yours alone? Omie  is desgned to make that possible, simply because she is yours alone.

Consider your shopping cart. Today that’s not even imaginable, because eevery shopping cart you’ve ever seen belongs to a company. Amazon, Ebay, Etsy, Walmart and the rest of them all have their own shopping carts for you. Why shouldn’t you have your own shopping cart, where you can see all the stuff you’ve almost-bought from all those online stores? With Omie you can at least imagine that, because Omie is yours. And imagining is the first step toward making.

So: what apps would you like Omie to run? Once we get the first few nailed down, we’ll crowdsource funding for developing both Omie and her first apps, or at least the specs for them.

To make that easy, here are just two requirements:

  1. Each app must be a kind that can only run on a device that is the owner’s alone. It can’t be one that only a corporate platform-owner (such as Google or Apple) can provide.
  2. Each app must rely first and foremost on data in the owner’s personal cloud.

The box we need to think outside of is the one that starts with a company. Here we’re starting with you.

Omie should be an instrument of control — by you. That’s why we’re stepping forward with it. Our job at Customer Commons is to stand on the side of the customer. That means we want apps that work for the customer first, and not just the seller. We need something solid to hold at our end of the demand chain — rather than, once again, to hold a device that serves as the far end of the supply chain’s whip.

We’ll bring up Omie at IIW. If you’re one of the 250 people here, come to the Omie session and let’s talk about where to go with the project. If you’re not here, put your thoughts and requests below.

Enhanced by Zemanta