Stress Detectors and Big Data

Over recent years, computing systems have included more and more sensors. However, I’ve long joked that every computer contains a “stress detector” that kicks into operation when the user is stressed. Printers, of course, have a double sensitivity sensor. As a result of this technology, whenever you have a really important or hard deadline, your computer starts to play up. Yesterday, in the light of my coming vacation, the stress detector that is in our network routers decided to kick in. In the end, we got the network working properly again, but I’m really not sure, at the end, what was causing the problem. Just what I need when I’m going away!

There may be some reality to the concept. Under stress, we may make more mistakes or change established patterns of behaviour that have previously ‘worked around’ issues on PCs. Younger readers may see Windows as a relatively stable environment for computing, but many of the early versions of the operating system were incredibly sensitive and easy to break. A lot of the problems came from device drivers that would break the operating system. I remember systems where you didn’t dare move the mouse too fast in case Windows crashed!

A lot of this was sorted out when internet connectivity became widespread and Microsoft was able to set up a system where it heard about all the crashes that users had and could start to identify patterns in those crashes. Arguably, the (relative) stability of Windows these days is a result of this ‘big data’ analysis process.

This week’s issue includes a report from the Digital Signage Summit that covered, among other topics, a talk by Vizualize, a company that uses analytics to track the people entering stores, how they move around the store and even where they are looking. That’s not terribly new, but what surprised me is the speed and accuracy of these analytics these days. The company claims to be able to accurately measure the people entering one of the most busy building entrances in the world, which sees two million visitors per day. It’s one of the entrances to a station on the MTR subway in Hong Kong. The company also said that the total video volume in US retail is around two million hours per minute (compare this to the 100 hours per minute that gets uploaded to YouTube!).

Such huge data could really change the ability of corporations, and governments, of course, to understand who is doing what, where and when. Vizualize points out that it has developed software that can reduce the images of people on the screen to just silhouettes, so that individuals are not obvious (although with age, height, estimated weight, clothing and other data from mobile communications, it may not be too difficult to tie this back to individuals). The company also said that it can analyse the video in real time, so there is no need to store the video itself – once you have the data you want from it, storing it may have no purpose.

I have not been particularly concerned about privacy issues, but part of the reason for this is that I live in a relatively open society with a democratic and (at least occasionally) accountable government and with (relatively) little corruption. It’s reasonably easy to live in the UK and believe that “the government doesn’t mean you harm”. If I lived in some parts of the world, where I was less sure about this, I think I would rapidly get much more concerned about this kind of topic. Certainly, among concerns (and I have a lot) about Brexit is that the UK will lose some influence in this kind of topic from Germany, where there is much more concern over issues of privacy.

Anyway, although I will be taking advantage of my freedom to travel in the EU for my holidays, ‘normal service’ will continue on the newsletter, although we are already seeing something of a slowing down of news as we move into the summer.