One of the major changes is the fact that there has been a shift from permission based to non-permission based. In the past it was mainly possible to collect data after having permission. We gathered information via interviews, surveys and questionnaires. By using these techniques the respondents did agree to participate and so there was permission. Maybe the only exception was observation and mystery shopping. Today the majority of data is gathered by logging in on social media site (LinkedIn, Facebook, Instagram, YouTube, Snapchat, etc.) and finding all sorts of personal data, figures and facts. But also applications like Street View and on-line communities will reveal a lot of data.

Compared to 20 years ago this data comes in much more forms than just figures. Data in 2017 includes images, video, gestures, emotions, locations and movements, which will tell us much more than just the figure. I am amazed to see all the personal information that people put on-line and on top of that seeing the surprised reactions of the same people and asking: how do you know that? Well, you put this info on social media. Oh really? I can’t remember.
The grey zone is the loyalty card. Although by signing up you give a kind of permission to use the data, you don’t know who will receive that data and what they will do with that. They only thing you see is a lot of personalized ads and invitations.
With the Internet of Things it will even go a couple of steps further. There will be so much more information available generated by kitchen appliances, cars, sports equipment, home-automation and personal devices. The majority of this IoT data will be used without a firm and articulated permission. If these devices start exchanging data in a smart way and taking intelligence decisions than we have even a more sophisticated level of information. What I mean is that if you take your last soft drink form the fridge, the fridge could order new drinks, but your outlook calendar will tell the fridge that you are traveling for the next 3 weeks, so, no need to order it now. In the near future ambient intelligence will be a fact. You arrive in Paris and “the system” will suggest via your smart-phone to book your favorite restaurant for tonight. This is even more data that can be used for insights and foresights.
Hence, there will be so much data available, that collecting data for the sake of collecting data does not make sense any longer. Selectivity is the key word. Social media monitoring needs to shift from “hearing and not understanding the important details”, to social intelligence: “listening and understanding the meaning, the context”.
As data collection will be cheaper, faster, more detailed and dynamic, we need to be more selective. The question is not any-longer: “WHAT to collect?” but “WHY to collect?” To answer this question, needs analysis is the key starting point to deliver valuable “must have” intelligence and foresights. When getting a Key Intelligence Question from a stakeholder, do we ask the right questions to understand the objective or the assumption? Do we know the need behind the need, is the problem the problem? The deeper reason of the Key Intelligence Question? This means that more attention needs to be paid to the relevance of data. Will the collected data help to answer the WHY question? The consequence of this new WHY question is that research companies need to implement new ways of working. They should use inspirational and the less obvious data sources to deliver an understanding to their stakeholders, not just the data. It would not surprise me if a new type of companies will start offering the new normal data research and position themselves as partner helping to answer the WHY question instead of being just a data provider.
On top of that the accessibility of data will allow us to do more trial and error as new form of research. It is easier to experiment and see if we can connect the dots with dissimilar data sets in completely different ways to reveal less obvious conclusions. Conclusion is that will help us to become for forward looking, more predictive.
The companies that will deliver tools to cope with massive data, like data monitoring and sophisticated analysis, are in the right position and will have a bright future. Intelligence professionals that do proper needs analysis will add value that gives them a seat at the decision making table. And this leads to a closing question: Is data-collection still the right thing to do. Do we need to collect data or is just having access to data enough and so: the new normal?