Category: data availability

How humans intervene in Internet’s workings

“There has been a shift in our thinking,” said Scott Huffman, an engineering director in charge of search quality at Google. “A part of our resources are now more human curated.”

Brain in a computer
Which is really the brain of internet: humans or chips?

This is an except of an interesting paper, Computer Algorithms Rely Increasingly on Human Helpers. Where you discover that humans are increasingly being used to increase the relevance of some choices where algorithms struggle.

According to the paper, there would be an increasing trend in putting humans at the core of certain processes to enhance them. Is that a temporary fix until new algorithms appear that would be better or is it a durable trend? Actually there seems to be two types of human intervention:

  • those that do systematic data churning like the Facebook and Google Indian offices that administer part of the services – they will be replaced someday by a machine (like we had mentioned in our post on Google Maps, there are armies of people behind many of the internet services we are using daily, that link or correct information that machines cannot link or understand properly.)
  • those that work to improve the machine (algorithms), which are those mainly described in the paper and which number will remain steady or increase in time.

Internet is an ever complex machine that needs engineers and technicians to improve the tools. When it comes to the borders of human nature and psychology, humans will remain for long at the core of the design of the machine.

Of course, the internet algorithms are still designed by humans. And as requirements become finer and finer, edging to psychology, humans are still indispensable, with an ever higher level of knowledge. And humans will remain so. Machines are not yet fully in charge!

Share

How we can really identify important information

We still identify important information through personal contacts (or even, get the information through these contacts) – and this will continue. Faceless social networks will not replace the amount of information filtering through personal trust. Internet will not replace personal networks built over time and patience.

social word of mouth in action
Nothing will replace personal networks as a source of worthy information

While our information environment has grown to the point of being inaudible, we try to filter it, and we do that by leveraging on our network. We look at those links and materials shared by our known contacts with much more interest and focus. We ask our relations to help us in finding the right answer to our needs.

Some marketers are trying to leverage this observation into a technique, like Andy Sernowitz and “word of mouth marketing“.

Fourth Revolution’s social networks allow us to draw much quicker and easier on our community of relations, and get to the information efficiently and remotely. Yet it does not mean that we would believe social networks members we would not know personally, or with which we would not have developed a relationship through consistent exchange over time.

Do you leverage enough your network community to filter out important information or to get informed of things that are important to you? What could you do to make it more effective?

Share

Why it is ever more difficult to identify actionable information

The Fourth Revolution comes with a new challenge for most of us: we have much more information to our fingertips or in our phone than ever before – and potentially, too much information, a noise that hides what might really be important.

We have maps, public transportation schedules, the latest news and rumors from all over the world in our pockets. We can apprehend the world with a detail never achieved before. Yet the same old problem still lingers: how can we really identify what is important for us and for what we intend to achieve?

difficult to hear from the background noiseWhat is really difficult is to extract the useful information. Data might be available but because connections don’t happen, it is not transformed in useful information (for example after 9-11, it was ascertained that federal agencies in the US collectively had enough data to understand what was being prepared, but failed to connect the pieces). Even more annoying, because there is ever more data available, there are ever more spurious correlations – and the possibility of wrong decisions and innocent people being impacted.

We all reach the limits of what we can physically receive, process and transmit (whereas in the past that was only limited to a few individuals at the top of organizations).

While some believe that thanks to new technology, we are better informed in our decisions, it might not be true. Because the background noise intensity has grown so much, it has become harder to discern the important information. It also sometimes give the wrong impression of being able to understand what is happening.

How are you discerning important information, the information at the basis of your action?

Share

How Big Data Will not Help our Understanding of Complexity

It is not possible anymore nowadays to open a serious newspaper or a financial investment document without reading about the great prospects of “Big Data”: analyzing the troves of data progressively acquired by organizations when we surf, shop, spend money, etc., to create Value. And companies have been setup which raise money on the markets to exploit data like other exploit mines and oil wells.

Curve errors/ data size
The number of spurious correlations increase with the size of the data. Be careful when you read about discoveries from “Big Data”!

Be careful, says Taleb in his book Antifragile, lots of data also means lots of spurious correlations. The argument is detailed in the book and in this Wired Article “Beware the Big Errors of ‘Big Data’“, from which we reproduce the curve on the right (it is also in the book).

To translate the point in conventional language, the bigger the size of the data and the number of potential variables considered, the higher the probability that bullshit is produced when it comes to the identification of possible trends. From there to consider that all this “Big Data” trend is just a vast hoax, there is a step we won’t take (I don’t invest in this “Big Data” thing unless it is very focused application). However, this potentially shows that a high proportion of the ‘discoveries’ that people do analyzing “Big Data” will be spurious. Be careful next time you read about one of these new ‘discoveries’.

A further challenge: is it really possible to understand complexity through more advanced data analysis? Following Taleb, we can have high doubts about that. In particular because all data analysis tools will never consider anything than conventional statistical approaches, and will never consider those discontinuities and benefits from volatility which makes real life what it is: interesting!

Share

Must Read: “AntiFragile” by Nassim Taleb

The latest book of Nassim Taleb, Antifragile, is an absolute must-read. It puts a lot of common wisdom on its head and provides an interesting picture of the mistakes of the society surrounding us.

Antifragile by TalebThe book is thick and takes some effort to read through, but it’s worthwhile to take the time. Taleb is a very unconventional thinker; his approach to complexity and predictability is absolutely brilliant.

What is Antifragile? According to Taleb it is the property to thrive in situations that are highly uncertain and volatile.

Most of the things we produce are fragile. The Industrial Age civilization, through its manufacturing standardization and search for efficiency, tends to be fragile. And actually it would seem, according to Taleb, that modern civilization is much more fragile than before. Look at how single freak events like 9/11 or other terrorist attempts fundamentally change the life of travelers and how we are ever more deeply intolerant to unexpected events.

Fundamental mistakes of our scientific approaches are denounced in the book, in particular our custom to observe averages when volatility might be an even more influential parameter; and the limits of conventional financial statistical analysis.

In summary, take the time to read AntiFragile. It gives also an insight on how our world might become when we overcome some psychological hurdles from the Industrial Age. Several posts will be posted here that will inspire themselves from the book in the next few weeks.

Share

How essential it is to check information sources on internet

Mid-February, the web went viral with a study that showed that there were not more than 19 links between any page of the internet (examples here, here and here (in French)). This epidemics followed a paper published on 18 February 2013 by the Royal Society.

world wide web graphics
The worldwide web galaxy is smaller than you think – only 19 links to get anywhere!

The thing is, when you read the paper (link to the original paper of the Royal Society here), you realize that this discovery was made in… 1999. Is it still valid? Nobody knows, actually. It is quite possible actually that today there are even less links needed thanks to the Googles and other search engines, plus all the social tools.

Some publications did correct their initial post (like the Smithsonian here), but most did not, in particular the news outlets from where most of the public gets its information.

There are two interesting lessons to take from this event:

  • On the negative side, what you read on the internet, in particular what is viral, might have been severely distorted by transmittal, editing, search for sensationalism, etc
  • On the positive side, thanks to the internet, one can access the original source easily – I was able to find it in a very few minutes thanks to the links embedded in the published pages and Google! (and actually it took only 2 or 3 clicks, not 19) (think what would have happened if you had read such a news in a newspaper 50 years ago: would you have ever been able to go back to the original sources within less than a few weeks?).

It is easier now than ever to post and disseminate trash or inaccuracies, but it is also easier than ever to check – or at least get various opinions on a subject. Do we teach ourselves and our children enough to enhance their critical skills and how to search by themselves for diverse opinions?

Note – for those who are interested, reading the original short article (by Barabási in the Royal Society publication) is quite instructive on the topology of the internet.

Share

If a Team could not Solve a Problem, the Person with the Information you Need was not Invited

This quote is from a presentation by Matt Fourie, a consultant specialized in problem-solving and thinking tools, at the last Project Management Institute regional conference in Singapore.

Empty chair in a meeting
Who in your your organization knows about the issue?

What he means is that throughout his decades of experience as a consultant helping organizations solve problems, he has found that in more than 95% of the cases, the solution and the understanding of the problem was known by someone in the organization. But that this knowledge was not mobilized effectively. This very much connects with my experience too – often by listening to the front line people, the answer is obvious.

Usual issues that impede proper problem solving are:

  • problem-solving meetings involve managers that do not know about the details of the operation
  • there is a disconnect between management and operators
  • people jump to conclusion (and action) without analyzing the data

It is where all the initiatives such as Total Quality management are really useful, as they seek to mobilize effectively all the organization members’ knowledge. Today with internal social networks we have a further opportunity to leverage that knowledge.

Next time when you face an issue in your organization, rather than looking for external answers, the first question you should ask is: “who would know the answer in the organization, who should I include in the discussion?”. Do that today! You’ll see the difference.

Share

Monitoring flu trends in real time… thanks to Google!

Google is known for weird projects that make sense. Here is a great project – Flutrends. This tool give instantaneous measurement of the flu epidemics level based… on the number of searches related to ‘flu’ on Google! This turns out to be quite an efficient way of crowdsourcing the information.

Here is an example of a historical graph for France

(more info and more countries covered on Google Flutrends – the same tool also covers Dengue)

What are the troves of data that are available but not exploited by those companies like Google (or at least, not given out to the public?)

As usual I am aghast at the power of the Fourth Revolution. Do you know other similar examples? Please comment!

Share

Video of the quarter: Don Tapscott on the Collaborative, Open world

Revise your knowledge of the Fourth Revolution by watching this entertaining video by Don Tapscott that covers most of the basics (Don Tapscott is the author of Wikinomics: How Mass Collaboration Changes Everything and Macrowikinomics: New Solutions for a Connected Planet). It even speaks of the transition between Agricultural, Industrial and Collaborative Age!

I like how he considers the younger generation to be ‘natives’ of the collaborative Age while he is only an explorer!

Share

How the Fourth Revolution transforms our view of Earth

Working in the field of large, complex projects, I can’t see a project file nowadays without great pictures coming from Google Maps adorning the file: pictures of the site, of the present facilities…

Industrial Facility seen from Google Satellite
Industrial Facility seen from Google Maps

And indeed it is so easy from any computer to have precise satellite pictures of any place on Earth that we just don’t think any more about the miracle that is… ten years ago only, this privilege was reserved to government authorities and large corporations who could pay to get the picture taken by an aircraft or a satellite.

And today in developed countries most roads in large cities are also visible through Google’s StreetView.

This unprecedented democratization of earth imagery has far reaching consequences. It changes fundamentally the concept of ‘public space’ (which can now be observed from anywhere on Earth) but also the concept of ‘private space’ (you can’t hide what you have in your garden from your neighbor any more!). It changes fundamentally how we perceive the physical space around us. This transformation is as strong as when we saw the blue planet Earth for the first time from space, floating in emptiness, in the 1960’s.

Some people argue that because only a limited number of actors manage this data (Google being the most prominent), we are exposed to possible manipulation of the data for commercial purposes, showing us only what they want to show us (see the paper “The Dark Side of Commercial Mapping”). We are not so pessimistic, because there will always be competitors and the crowd will denounce abusing behavior. But certainly our view of the geography that surrounds us has been transformed and this has already changed our decision making when it comes to property or project management.

Today, our perception of our immediate and distant geography has been transformed. Like our new vision of Earth transformed our mindset in the 1960’s, how will our action taking relative to geography, in particular in the field of ecology, change?

Share

The best digital maps are created by humans and crowd-sourced

Following the outcry about the poor quality of the new Apple mapping application that came out with the iPhone 5, let’s investigate how the best electronic maps around have been created.

Google Digital MapNo surprises, there’s a lot of human work, sweat and crowd-sourcing behind them. This excellent paper on “How Google Builds its Maps and What It Means for the Future of Everything” describes how an army of people (most of it probably in India) have reviewed all the grueling details of the Google map database, helped lately by Google Street View cars to achieve the appropriate accuracy.

But Google Maps also allow users to give their input to correct inaccuracies (even wondered at the small clickable “Report a Problem” at the bottom right that allows you to give input directly to Google?). So, a significant additional accuracy is given by crowd-sourcing.

Digital maps aggregate an incredible amount of geographical information on several layers, giving insights into our world in a way that had never been possible before – and thus changing our view of it. Even if Google maps was only created in 2005 (!), taking into account the effort that has gone into it, you can’t just improvise an alternate version. Catch-up of the current mapping data will take significant amount of effort. That effort will be done by competitors because this data is so strategic commercially, but only those will be successful that have significant capability to mobilize people to code the data – and to mobilize the crowd to ensure constant accuracy even in case of changes. Google has an advance over everybody else, and this huge investment they made will be one key to their future success.

Share

Why the Fourth Revolution is the Era of the Exponential, and How this Changes Everything

The Fourth Revolution is the era of the exponential – whereas the Industrial Age was the era of linearity. And that changes everything in the way we live our life:

Moore's Law 1971-2011
Moore’s Law 1971-2011 (from Wikipedia)
  • the complexity of the products we use every day increases exponentially. For example the Moore’s law states that microprocessors density on chips doubles every 2 years; and that’s the case for many other products we use every day without realizing it;
  • Successful companies and services grow exponentially, soon dwarfing existing players (the revenue of Apple was multiplied by 11 in 10 years… not to mention the even more exponential growth of the Facebooks of the world)
In the Industrial Age, things were more linear. It was easier to extrapolate the future from the past. Of course a factor is that things go faster today so that it is easier to watch exponential change in action. Yet the Moore’s law rate did not change in the past 40 years or so. Microprocessors’s density still double in the same number of years. So speed of change is not the discriminant. The fact that things grow visibly exponentially and have higher ceilings than before makes the Fourth Revolution different.

 

story of rice on chessboard
Are you sure you want to get to the end of the chessboard?

The problem is that we are not geared to feel intuitively the power of the exponential. It is very difficult to seize how fast it can grow. Do you remember the tale of the wise man that told the King who wanted to thank him: “only put a grain of rice on the first square of a chess board, then on the next square put two, then on the next square put four, then double for each square until the end of the chess board…” The King never realized that at the end of the 64 squares the quantity of rice needed would vastly exceed his available supply – and the world’s supply and even more!

This explains why so many people today have difficulty understanding what happens in the world. In their linear Industrial Age mindset, they can’t grab how the exponential is changing our lives faster and deeper than ever before.

Are you ready for a world full of exponential change?

Thanks to Mitch Joel and his post on “The Era of Exponential Marketing” – a specific area where most people also don’t realize we are in for exponential growth of product sales- for the inspiration.

The great picture of the rice on the checkboard is by Paul Starkey on Flickr. 

 

Share