Introduction
- Blueberry is a bio-sensing computer that is attached to an eyeglass, which collects biological signals to track physical and mental states.1,2
- We analyzed five key metrics from Blueberry: HbO (oxyhemoglobin), HbR (deoxyhemoglobin), flowActivity (a representation of changes in blood flow over a 10–20 second period), heartRate (beats per minute), and flowIntensity (a composite representation of blood flow changes and cardiovascular indicators such as heart rate).
- The data came from a session where a person read The History of the Future: Oculus, Facebook, and the Revolution That Swept Virtual Reality by Blake J. Harris and took notes, with their notes, a CSV file, and a screen recording provided.
- We used Google Colab, a software that allows us to run Python and work together, and a few libraries like pandas and Matplotlib to visualize the data.
Discussion
Smoothed Optical Data From Reading and Note-Taking Session
Looking at this graph, we can infer many things. For example, when looking at the HbO levels throughout the 12 minutes, you can see several sharp peaks and dips. This represents changes in oxyhemoglobin concentration due to brain activity, which is most likely due to taking notes and processing information. Oddly, when the person first starts taking notes at 43 seconds, HbO drops almost immediately after. This is weird because, as we just stated, taking notes requires brain power, increasing brain activity as a result. If you zoom in more closely, you can clearly see that HbO drops at 48 seconds, which indicates a muscular response event. More interestingly, if you take a look at around 51 seconds, there is a sharp peak showing that the muscular response event actually caused an increase in HbO, showing that the person probably noticed something important in the reading.
- HbR mirrors HbO, showing an inverse relationship. When oxyhemoglobin increases, deoxyhemoglobin decreases, which is typical in brain activity.3
- The flowActivity, heartRate, and flowIntensity data all reflect the mental engagement of the reader. For example, a spike in flowActivity can show active comprehension, while flat regions indicate passive reading.
- Since flowIntensity looks a lot like flowActivity, and it is a composite of both heartRate and flowActivity, we can assume that flowActivity dominates.
After a few days, we received another drive folder, this time having a screen recording and a CSV file of a person playing a pattern recognition game.
Smoothed Optical Data From Pattern Game Session
After taking a quick glance at this graph, we immediately noticed that it was different from the reading and note-taking session one. HbO was really jagged and trended upwards instead of being relatively steady with a few peaks and dips. This makes sense because pattern games require you to think more. The first big peak occurred around 1 minute 9 seconds, which was when the game was explaining what the person was just doing and what the overall course is about. This could suggest that the person was reflecting on the patterns they had just created. Additionally, the course summary may have caused the person to try to connect what they had just done with the overall objectives of the course.
At 5 minutes 36 seconds, when the person is forced to make a purchase decision, both HbO and flowIntensity spike. This is likely due to the fact that the person was evaluating whether or not they should make the purchase, which triggered their neurons. Additionally, that might’ve stressed them out a little, explaining the cardiovascular response.
Surprisingly, HbO reached one of its highest points when the person was at the level selection screen, which occurs at 9 minutes 36 seconds. We think this could’ve been due to two factors we previously discussed: reflection and decision-making. The person might’ve been asking themselves whether or not they should move on or give their brain a chance to take in everything that just happened. It could’ve also been due to anticipation; the person could’ve been expecting a harder level, so they were getting mentally prepared for it. We later found out that it was a muscular response artifact.
Nonetheless, all of these responses could’ve simply occurred because the brain had to reorient itself to the change of scenery, the new page.
The graph below is especially interesting because it shows fluctuations and growth over time. Let’s examine two of the biggest peaks. The first notable peak is in the first part of the session and spans from approximately 1:30 to 2:22. The second major peak comes later in the second part of the session, spanning from around 11:45 to 12:22. The first peak lasts longer, however, the second peak has a higher baseline. This could suggest that while the first peak represents a longer period of increased activity, the second peak represents a period of stronger and more intense increased activity. Before, after, and during both peaks the person was working to solve puzzles, but during the first peak there was a screen that prompted the person to create an account. The break from the game could’ve potentially relaxed the person, which serves as an explanation as to why the first peak’s baseline was lower.
This bar chart shows the highest and lowest peaks of brain activity during the reading and note-taking session. The left side of the chart represents when the person first started taking notes, which makes sense since they just finished reading a page that didn't require much understanding. The right side of the chart, on the other hand, indicates a point of deep comprehension as the person spent a lot of time reading that page.
Similarly, when analyzing the pattern game session, a different set of cognitive patterns emerges.
Like the bar chart at the top, this chart also shows the lowest and highest points of brain activity during the pattern game session. The left side of the bar chart shows where the person first started the pattern game and had to identify the shape added to each block. The right side of the bar chart represents the question where the person playing the game had to count the number of blocks in the last shape. The HbO levels are clearly higher on the right side, which makes sense because counting the blocks requires more cognitive effort than just looking at answer choices and matching the shape that was added.
These are some mock images of what the app would show and track.
See when you’re most focused
Track HbO and other brain signals during study or homework sessions to discover your peak focus windows. Plan your study and homework sessions accordingly.
Train for high-pressure points
Identify which moments (deuce, break points, tiebreaks) drive peaks in stress or focus. Use those insights to help build your practice plan.
Your brain data, made useful
Identify what times your brain is the most active and productive through a dashboard displaying weekly trends, and use that to help build your daily schedule.
Note: We used ChatGPT to provide the rough structure of the code and also the conversion of the units (milliseconds to minutes with intervals in between). We also used it to generate some of the images.