3d Heatmap

My eye-data extraction program needed some upgrading and bug fixing.  What it does is read a series of datafiles produced by SMI Vision’s IviewX eye tracking software and displays the gaze data.   It scans each selected trial on each selected participant and counts the number of gaze points on each pixel.  Next, it smooths those counts by adding in the counts on neighboring pixels using a normal-distribution falloff function.  I typically use a radius of 20 pixels and a standard deviation of 6.  These values can now be modified by the user in the program by way of textboxes and an Update button.

Next, those smoothed are converted to a heat map.  The maximum count is set to 1 and every count thereafter is set to a percentage of that maximum (p).

The last step is to map these values to colors.  My old system was a hack, and I honestly don’t recall how I did it, but it did involve a bunch of “if” statements.  But my new one is somewhat more elegant.  Each color is defined by the standard combination of Red, Green, and Blue values.   Each of these values is determined by the function exp( -(p – center)^2 / (2*spread^2) ).   The spread for each is set at .21 and the centers vary.  The center for Blue is set at 0.01, green at .5, and red at 1.  These values and function produce curves for each component as follow. 

image

Thus, when the percentage is very high, the pixels are mostly red.  As the percentage decreases it fades to orange and then yellow at around a value of .76.  From there, it fades to green at .5, and turns cyan at .25.  Below that, the points are predominately blue.  I also modulate the alpha such that as they turn blue, they are faded out so that areas which receive very little attention do not show brightly.   Alpha is determined by the function 1-(1-p)^15.   It generally produces decent heatmaps as below.

image

That one is made from a study on latent inhibition using the learning game where people are looking at the screen when a Red sensor is being pre-exposed, collapsed across trials. You can see that the majority of the points are distributed over the sensor area (red, yellow, and green shades) with the remainder scattered about over the screen.  What I have added is the ability to light the heatmap.   

Each count is treated as a height.  Look at the square below and imagine each number is a height at a pixel location.  Every pixel, such as the “1” is surrounded by pixels 2-8 as in the square below.

2 3 4

5 1 6

7 8 9

Imagine a line from 1 to 2, 2  to 3 and back to 1 and you have a triangle.  Do that for all points surrounding 1 and you have 8 triangles. The distances in X and Y are simply the pixel coordinates, but each pixel also has a height variable associated with it, creating a set of triangles oriented in 3d space. I take the 3d direction from “1” to the other two corners of each triangle and cross those directions to get a normal.  Then I average all 8 normals to get a normal for point 1 as if it was a vertex.  I do that for all visible points and use those normals for lighting.  The lighting is a simple diffuse lighting model (intensity being the dot product of the normal with the direction of the pixel to the light).  Its not strictly a diffuse model because I do provide an ambient source of .08 intensity, and allow for the light to interact slightly with back-facing sides (.1 * –(dot(normal, direction_to_light)).

When the “With lighting” button is checked, the scene is lit as if there was a light at the mouse cursor.

image

Here the light is near the top left corner.  As the image shows, the lighting really turns the heatmap into a terrain with bumps and valleys that reveal details about the gaze patterns that are not evident in the heatmap coloring alone.   By holding the right mouse button down and moving it, you can move the light in real time.   The video below shows that movement using another data set.  In addition to the lighting, the video also briefly shows the effects of changing the radius and standard deviation of the smoothing functions.  Presently all the work is done on the CPU, so it’s a little herky-jerky.  Someday I may move the heavy lifting over the the GPU & make it silky.

video of 3d heatmap
Advertisements
This entry was posted in Eye Tracking, Programming in general. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s