Blog

Can an algorithm read our feelings, so what does "affective computing? mean?

• 2020-10-20
• Maciej Szczerba
• Understand technology

We have managed to translate such sensual feelings as: colour, sound, image (static and video) into the binary code, which is a mathematical notation system similar to the decimal system we use on a daily basis. Can a machine read human feelings? 

Rosalind Picard, Professor of Massachusetts Insititute of Technology (MIT), created the term "affective computing" by publishing a book with this title in 1995. This is an area of science that explores how computers can recognise human feelings in order to improve interaction between humans and computers. 

The outstanding futurologist Yuval Noah Harari states in his book "21 Lessons for the 21st Century” wrote: feelings are not a mythical phenomenon, human emotions are the result of a biochemical process. They are therefore something physical. It seems that in the not so distant future the AI algorithm will be able to analyse the biometric data provided by sensors located inside and outside the human body. Such an application would, over time, be able to categorise people according to their personality profile and to recognise changes in their mood.

In the future, according to Harari, we can forecast applications that can evaluate the influence of music or film on our emotions and suggest, for example, music that is able to calm us down in a specific stress situation, taking into account that the application understands our personality.

What happens if politicians start to use the application described above? Will they be able to create electoral materials tailored individually to our personality profile? What happens if, in an authoritarian country such as China or Russia, the application will be able to tell the authoritarian government what kind of emotions are triggered by the propaganda addressed to a particular citizen, for example by reading facial expressions ?

What will happen if the technology for recognising emotions and matching the relevant content is used by large corporations? Suppose a large clothing company can scientifically manipulate the emotions of its customers (e.g. in materials distributed through a mailing list), it intentionally causes a feeling of sadness, so that the customer wants to improve his mood with shopping ?

Will the era of free will and free choice come to an end?

How far are we from this state of affairs in terms of technology?

Below is a link to an interesting interview with Rosalind Picard

https://www.youtube.com/watch?v=kq0VO1FqE6I

Founded by Picard, Affectiva (www.affectiva.com) has developed a technology that is able to measure a person's pulse from a video image only, without any sendory device. The Affdex technology offered by Affectiva allows the facial expressions of certain human emotions, such as surprise, joy and concentration, to be read from the face. The software is used by Mars and the CBS television network, among others.

According to Picard, on the one hand, we are still limited in what we can read from a human face, on the other hand, it is much more than most of us are aware of. When we use only an ordinary camera in our laptop or smartphone, we can see if the human pulse accelerates or if breathing starts to be irregular, we can see changes in the colour of the skin, which the human being is not able to notice, while the camera is able to. "All the time the application does not know for sure how you really feel, but based on your body movements, even minimal, the application is able to predict how you feel emotionally with a very high probability".

Anyway, check out the very cool link to the demo below on how Affectiva products work:

https://demo.mr.affectiva.com/

Here are some interesting examples of "affective somputing" from recent years, collected at TechCruch.

Humanyze (www.humanyze.com)- software that monitors the interaction of employees and their influence on each other, used among others by the Bank of America. Humanyze offers intelligent "badges" that include bluetooth, a microphone, an accelerometer to collect data: how workers moved during the day, with whom they interacted, what tone of voice they had, whether they spoke or listened more, and many other types of interpersonal interaction in the workplace. Intelligent badge generates about 4gb of data per day, which is sent to cloud applications and categorised into 40 types of data that affect the 6 main measures such as engagement. Humanyze stipulates that the employer sees only aggregated data, the individual data is visible only to the person being measured.

RealEyes (www.realeyes.com)- A British start-up that analyses a person's emotional response to a video through a camera, even as short cavity as 6 seconds. It also offers a predictive analysis tool providing guidance on the effectiveness of the video message.

cogitocorp.com - analyses in real time the tone of a person's voice and suggests how to modulate the voice for use in customer service.

EQ Radio - a research project that recognizes emotions with radio waves

At TechCrucnch Disrupt in December 2019 in Berlin, I personally met the founders of a very interesting start-up retorio.com. It is a video interview tool combined with facial expression analysis, which allows to recognize the sincerity of the candidates' answers, and thus their personality profile and their fit into the organizational culture from a "soft" angle.

However, we can look at "affective computing" from the other side of the coin - a learning application that recognizes human emotions can, for example, keep a lonely person company, adjusting emotionally to their needs, creating a perfectly matched avatar-companion. Something like a literary character in a book that you can't get away from.

According to Roz Picard, applications of such Apple Siri based on the tone of voice are very likely to recognise real suicidal tendencies and distinguish them from suicidal messages when a person, for example, with depression, tries to attract attention only to himself. Recognition of emotions by AI can therefore help save lives.