Tired? Angry? Your car knows how you feel


March 24, 2014

PSA Peugeot Citroen has teamed up with EPFL to develop an emotion detection system designed to recognize signs of irritation and fatigue in a driver’s facial expressions (Photo: EPFL/j. Caillet)

PSA Peugeot Citroen has teamed up with EPFL to develop an emotion detection system designed to recognize signs of irritation and fatigue in a driver’s facial expressions (Photo: EPFL/j. Caillet)

Image Gallery (3 images)

Ever experienced road rage? Someone cuts you off while you’re trying to merge and next thing you know you’re tailgating them like a NASCAR driver at Fontana trying to get a slingshot off the bank. Then they hit the brakes … "screech-crash-bang" … there goes your platinum rating with the insurance company. What if an on-board emotion detection system could tell that you were getting annoyed and intervene? PSA Peugeot Citroen has teamed up with the Swiss Federal Institute of Technology (EPFL) to develop an emotion detection system designed to recognize signs of irritation and fatigue in a driver’s facial expressions.

Technology and software for reading facial expressions has been used widely in applications from treating autism and depression to market research and brand development. Now researchers at EPFL’s Signal Processing 5 Laboratory (LTS5), who specialize in facial detection, monitoring and analysis, are exploring applications for the motor industry.

Irritation has been identified as a key factor in drivers becoming aggressive. Facial expression reading software can identify seven universal emotions: fear, anger, joy, sadness, disgust, surprise, and suspicion. And though everyone displays their irritation behind the wheel differently, lead researchers at LTS5, Hua Gao and Anil Yüce, concentrated on two of these universal emotions, anger and disgust, as these have been accepted as being similar to irritation.

Helping your car to get to know you better

Firstly, the system was given a series of control inputs, still images of the subjects in both an office setting and in real life situations, such as behind the wheel, so that it could learn to read the emotions under review. Next, using an infrared camera mounted behind the steering wheel, video was taken and later processed using facial monitoring algorithms.

"The rapidity with which the comparison between filmed images and thus detection could be carried out depended on the analysis methods used," said Hua Gao of EPFL. "But overall, the system worked well and irritation could be accurately detected in the majority of cases."

Difficulties in detection of irritation were put down to the complexity and variety of expressions shown. Everyone displays irritation slightly differently. Pursed lips and a scowl, or maybe open mouth and eyes bulging.

"When the test failed, it was usually because this state is very variable from individual to individual," Hua Gao continued. "This is where the difficulty will always lie, given the diversity of how we express anger."

The team also tested a fatigue detector designed to measure the percentage of eyelid closure. Future plans for the project also include using voice recognition or lip reading technology, and reading other driver states, such as distraction.

The next step for EPFL is testing the system in real-time using a more advanced facial monitoring algorithm. However, EPFL and PSA Peugeot Citroen have not as yet revealed how this technology would be applied.

Could we, for example, hear a calming voice saying, "I’m not sure you should be doing that, Dave?" or better still, a proximity sensor and speed limiter that kicks in to ensure you can’t tailgate no matter how much you want to, or maybe a friendly wake-up call if you’re starting to doze.

Source: EPFL


I hate when people see research and call it pointless a pointless waste of time but when I drive in unfamiliar metro areas I still have trouble checking my blind spots and it's 2014.

Instead of automotive companies working to better inform me of where cars around me are positioned they are investing in trying to figure out what kind of mood I am in?

Another example is I hit a speed trap a month ago at 1 AM when the speed dropped from 55 to 40 because 55 is dangerous for that stretch of road in heavy traffic which was far from present at the time of my ticket. My GPS knows both my speed and the posted speed limit but doesn't have a feature that beeps to warn me when I am ~12+ MPH over the speed limit?

That feature would be really easy to implement and has practical use today but why do that when they could invest money into figuring out what kind of day I am having?

If the car decides I am in a bad mood what does it do about it?


I Could see how a system like this could bite people on the arse. If you have an accident and the same camera that detects your mood also records, the investigators would reliably correlate the driver's preoccupation at time of incident.

Constructively, why can't this system detect when a driver is : 1. falling asleep 2. on the cell phone 3. reaching over or not looking at the road

Regarding the change of speed thing, I think some modern vehicle GPS product do offer speed zoning as a subscription feature. But road works and short notice impromptu speed restrictions due to whatever are not vigorously updated on the GPS database, which is why cops love to target these areas so much.


I'm with Diachi. There's far better things to be researching.

How about this: Personal responsibility and self control. I don't need a nanny state or a nanny car! Be an adult; driver control thyself!


Better idea, why don't we just all switch to self driving cars and make driver emotion an obsolete idea. This would be especially great for taxis, which may be my main cause of "emotional status change" in this city.

Charles Bosse
Post a Comment

Login with your Gizmag account:

Related Articles
Looking for something? Search our articles