Tag Archives: face recognition

Ghosts in the machine

Posted on by | Leave a reply

Well, not literal ghosts, but blank spots. It seems we may be facing our first serious bandwidth issues with 28 cameras installed and plenty of summer visitors. Whatever the reason, we’re getting hiccups in our getalongs – cameras are randomly freezing for a few seconds to several minutes each, losing connection with the system, and generally not behaving correctly.

Today, for example, we were collecting images of ourselves from both the video cameras and a still digital camera for comparison of performance for facial recognition. As Harrison, Mark, and Diana moved from right to left along our touch tanks, only one of three close-up “interaction” cameras that they stopped at actually picked them up. It’s not a case of them actually moving elsewhere, because we see them on the overhead “establishment” cameras. It’s not a case of the cameras not recording due to motion sensing issues (we think), because in one of the two missing shots, there was a family interacting with the touch tank for a few minutes before the staff trio came up behind them.

This morning I also discovered a lot of footage missing from today’s feeds, from cameras that I swear I saw on earlier. I’ve been sitting at the monitoring station pulling clips for upcoming presentations and for the facial recognition testing, and I see the latest footage of some of the octopus tank cameras showing as dimly lit 5 a.m. footage. It’s not a problem with synchronization, either (I think): the corresponding bar image on the viewer that shows a simple map of recording times across multiple cameras shows blanks for those times, when I was watching families on them earlier today. However, when I look at it now, hours later, there don’t seem to be nearly as big of gaps as I saw this morning, meaning this mornings viewing while recording might have just delayed playback for some of the recent-but-not-most-immediately-recent footage at that time, but the system cached it and caught up later.

Is it because some of the cameras are network-powered and some are plugged in? Is it because the motion sensitivity is light-sensitive, wherein some cameras that have too much light have a harder time sensing motion, or the motion sensitivity is based on depth-of-field and the action we want is too far afield? Maybe it’s a combination of trying to view footage while it’s being recorded and bandwidth issues and motion-sensitivity issues, but it ain’t pretty.

Color me surprised!

Posted on by | Leave a reply

If you’ve been following our blog, you know the lab has wondered and worried and crossed fingers about the ability of facial recognition not only to track faces, but also eventually to give us clues to visitors’ emotions and attitudes. The recognition and tracking of individuals looks to be promising with the new system, getting up to about 90% accuracy, with good profiles for race and age (incidentally, the cost, including time invested in the old system we abandoned, is about the same with this new system). However, we don’t have any idea whether we’ll get any automated data on emotions, despite the relative similarity of expression of these emotions on human faces.

But I ran across this very cool technology that may help us in our quest: glasses that sense changes in oxygen levels in blood under the skin and can sense emotional states. The glasses amplify what primates have been doing for years, namely sensing embarrassment from flushed redder skin, or fear in greener-tinted skin than normal. Research from Mark Changizi at my alma mater, Caltech, on the evolution of color vision to allow us to do just that sort of emotion sensing has led to the glasses. Currently, they’re being tested for medical applications, helping doctors sense anemia, anger, and fear, but if the glasses are adapted for “real-world” use, such as in decrypting a poker player’s blank stare, it seems to me that the filters could be added to our camera setups or software systems to help automate this sort of emotion detection.

Really, it would be one more weapon in the arsenal of the data war we’re trying to fight. Just as Earth and ocean scientists have made leaps in understanding from being able to use satellites to sample the whole Earth virtually every day instead of taking ship-based or buoy-based measurements far apart in space and time, so do we hope to make leaps and bounds in understanding how visitors learn. If we can get our technology to automate data collection and vastly improve the spatial and temporal resolution of our data, hopefully we’ll move into our own satellite era.

Thanks to GOOD magazine and PSFK for the tips.

The End of the Road?

Posted on by | Leave a reply

We heard recently that our developer contractors have decided they have to abandon their efforts to make the first facial recognition system they investigated work. It was a tough call; they had put a lot of effort into it, thinking many times if they could just tweak this and alter that, they would get better performance than 60%. Alas, they finally decided it was not going to happen, at least without a ridiculous amount of further effort for the eventual reward. So, they are taking a different tack, starting over, almost, though they have lots of lessons learned from the first go-round.

I think this indecision about when it makes sense to try and fix the leaking ship vs. abandon ship and find another is a great parallel with exhibit development. Sometimes, you have a great idea that you try with visitors, and it flops. You get some good data, though, and see a way you can try it again. You make your changes. It flops again, though maybe not quite as spectacularly. Just enough better to give you hope. And so on … until you have to decide to cut bait and either redesign something for that task entirely or, if you’re working with a larger exhibition, find another piece to satisfy whatever learning or other goals you had in mind for the failed piece.

In either situation, it’s pretty heartbreaking to let go of all that investment. When I first started working in prototyping, this happened to our team designing the Making Models exhibition at the Museum of Science, Boston. As an intern, I hadn’t invested anything in the failed prototype, but I could see the struggle in the rest of the team, and it made such an impression that I recall it all these years later. Ultimately, the final exhibit looks rather different from what I remember, but its success is also a testament to the power of letting go. Hopefully, we’ll eventually experience that success with our facial recognition setups!

Test mount

Posted on by | Leave a reply

 

Ladies and gentlemen, I present for your consideration an example of our signature rapid prototyping process. The handyman’s secret weapon gets a lot of use around here, and I even had a roll of Gorilla Tape on my wrist in case of emergencies.  Fortunately, it didn’t come to that.

The angles necessary for good face detection and recognition (up to about 15 degrees from straight-on) require careful consideration of camera placement.  The necessary process of checking angles and lighting isn’t always pretty, but I, for one, find the above image beautiful.

Making way for new tech

Posted on by | Leave a reply
Left to right: Harrison, Bill and Laura discuss wave tank placement and accessibility.

With new tools and exhibits on the way, we’ve had plenty to keep us busy.  We’ve come up with a new wave tank layout.  We’ve been working with our new Open Exhibits Kinect system.  We’ve tested the limits of face-recognition demo software.  We’ve laughed.  We’ve cried.  We’ve waved our arms around in closets.

Mark tries out the Kinect interface from Open Exhibits

For a brief overview of the research camera placement process (boldly undertaken by McKenzie), take a look at this video.

 

Exhibit of Learning (+10 Wisdom)

Posted on by | 3 Replies

Mark and Katie identified a useful model for data collection using the face-recognition system. That model is Dungeons & Dragons. Visitors come with goals in mind, often in groups, and they take on a variety of roles within those groups. D&D and similar role-playing games provide a ready set of rules and procedures for modeling this kind of situation.

In practice, this means the Visitor Center exhibit space will be divided into a grid, with the system recording data based on proximity, direction and attributes of agents (visitors, volunteers and staff) and the grid squares themselves.

For example, the cabinet of whale sculptures inside the front door would occupy a row of “artifact” squares on the grid. Visitor interactions would be recorded accordingly. Interactions with the exhibit would update each visitor’s individual profile to reflect engagement and potential learning. To use only-slightly-more D&D terms, spending time at the whale exhibit would add modifiers to, say, the visitor’s “Biology” and “Ocean Literacy” attributes. The same goes for volunteers and staff members, taking into account their known familiarity with certain material.

Mark and Katie have drafted what is essentially a dungeon map of the VC, complete with actual D&D miniatures. Staff members will even have character sheets, of a sort, to represent their knowledge in specific areas—this being a factor in their interactions with visitors.

In a visitor group scenario Mark walked me through today, the part of the touch pool volunteer was played by what I believe was a cleric of some sort. Mark has happily taken to the role of Dungeon Master.

This all forms a basic framework, but the amount and specificity of data we can collect and analyze in this way is staggering. We can also refer back to the original video to check for specific social interactions and other factors the face-recognition software might have missed.

In other news, Ursula will be released Friday morning. Here’s the word from Jordan:

“Wednesday afternoon the HMSC Husbandry team decided that now is the best time to release Ursula back into the wild. Our octopus from British Columbia is as feisty as ever and we feel that she is in good health. Because of this, we will be preparing Ursula for her journey back to the ocean Friday, December 23, 2011. We invite you to attend starting at 8:30 a.m. in the Visitor Center. We will then proceed to release her off Yaquina’s South Jetty about 9 .a.m.”