“The road of excess,” said the English poet William Blake, “leads to the palace of wisdom.” But as many rock-n-roll artistes have found out – like Doors frontman and Blake fan Jim Morrison – it can also get diverted via the ‘room of emergency’ or permanently gridlocked by the ‘defibrillator of doom’. Of course, William Blake wasn’t really talking about that kind of excess, and for the purposes of this article neither will I; yet the key uncertainty remains – can too much of anything ever be a good thing?

Take big data, for example. As Peter Kronstrøm, Head of the Copenhagen Institute for Futures Studies LATAM, explained to the audience at CONAREC 2014 last month, our attitude towards big data is often like teenage attitudes towards sex: we talk about it all the time, but we don’t really know what it is. This is because there is ambivalence about the concept of big data itself, and the way this ambivalence is handled now will have a profound impact on the relationship between providers and consumers in the future.

The case for big data is more than compelling: it’s intuitive. By tracking and extrapolating from the decisions I make, and from the decisions made by people like me, companies can offer individualized products and services that are specifically tailored to my needs. I don’t even have to spend time defining or explaining my preferences: this happens automatically as I consume.

At a higher level, the potential of big data is breathtaking. Huge, complex, real-time data collections enable us to map marketing trends and deliver on consumer needs as they evolve. They can help us to effectively predict and act on resource and environmental crises. Big data on genetics could even speed up the shift towards predictive healthcare, enabling us to live longer and better regardless of our health backgrounds.

So why the ambivalence? For most it simply comes down to the perceived risk of having personal data collected by private entities that may use it for purposes that are prejudicial to our interests. What if I am refused a loan to see my daughter through the last year of her university course because, even though I am perfectly healthy, there is a history of heart disease in my family?

Who, or perhaps better ‘what’, has become the subject of these decisions? At the Institute we believe that the notion of having a ‘digital self’ – a quasi-autonomous digital doppelgänger born of all our purchasing decisions, medical records and Google searches – will become as engrained in this century as the idea of having a ‘subconscious’ was in the last.

While the war against these binary beasts is already being waged – with new smartphone apps and devices that ‘guarantee’ anonymity – many futurists deplore the fact that technology is not advancing quickly enough to realise their potential. Just 0.5 per cent of the world’s data is analyzed today, and while there will be over 5 GB of information for every human by 2020, large quantities of useful yet untagged and unstructured data will remain doomed to oblivion.

The project to tame big data will be the defining undertaking in the field of data technology over the next two decades. Developments in open data – where datasets must be made available for anyone to use – are already enhancing the transparency and usability of stored information. At the same time, pioneering projects based on the principles of openness are beginning to secure public acceptance. The UK government’s Midata Initiative (midatalab.org.uk), for example, encourages companies to share the data they hold on their customers with their customers.

Putting the power of data back into the hands of consumers should be a strategic imperative for any company or brand that wants to secure trust and preference in the future. Investing in analytics and sharing insights will add previously unimaginable value for customers, empowering them to make choices, save money, and even manage their day-to-day life more efficiently.

Maybe then we can stop fighting against our digital selves – and start working with them.

(Portuguese translation appears in the November 2014 edition of Consumidor Moderno Magazine, Brasil)