we're in week 2 of putting together our vr simulation. it's becoming more manageable as time goes on. i tried wit.ai and dialogflow for speech recognition before i settled with microsoft's speech sdk for unity.
the basic idea is that i need to be able to do things in-game based on the user saying a correct phrase. the first step was to get voice recognition into a unity project.
with the speech sdk i was finally able to use a script that would start when i pressed a button, listen to what i say through my microphone, and output it to a text object in the game. i got this working with a physics-based button as well as with a floating button that you would see in most vr games.
the next step was to see if i could modify connect this script to another, and to modify something when a specific phrase is spoken. i also changed the language that would be listened for.
i added "yes" to the message returned by the speech sdk when i spoke the phrase "tambien necesito lechuga y pan." then i had it print through another script on another object in the scene, which is what i assume will need to happen in a game like this anytime we interact with another npc. so i'm getting closer to what i need.
my next step is to try this out on placeholders for npcs that will be in our game and see if i can get more complex things to happen. eventually this will mean that once a correct phrase is spoken the scene will continue. or if an incorrect phrase is spoken, the npc and ui will indicate that the player needs to try again.
i had a really discouraging first week of my group capstone, but it ended in success. that's also the reason why my blog post is being turned in late. i was so absorbed by what i was trying to do that i forgot we had to write blogs for the class. good thing we drop our 3 lowest posts!
our project is a virtual reality simulation, and my team decided on a simple language learning rpg-style game where you can learn basic words and phrases in spanish (for now). well, in order to make this as immersive as possible, we want users to be able to say the words and phrases instead of just having options float in front of you and you select the right one.
my job is to integrate the speech recognition into unity so it can go into our game. i spent days going over our options for software that could potentially do the job. the recommended software for oculus was a software called wit.ai, but their documentation was not up-to-date, the software itself was a pain to get working, and when it did work, it didn't work well.
the next option was google cloud's dialogflow, and i could not get it working at all. their documentation was much better than wit.ai, but either something was missing or i was copying the instructions poorly because i could not get it to even authenticate my credentials, and we just don't have time to play around with the basics of a program.
the last option i tried was microsoft's recently updated speech services. the authentication was a breeze, but it did not have any guides for how to use it in a vr environment so i had to watch a bunch of unity videos to understand how to use it (never done game dev before yay). then once i knew how to do some basics, i messed around with the sample script microsoft gave to try and see if i could get voice recognition in a sample scene. i am not kidding when i say that i spent a whole day making a button in a vr scene that would work with a voice recognition script that would show the text of what i spoke into a microphone. at 10pm i finally succeeded. go me.
now that we know it's possible, i move on to making it work for what we need. i have to see if i can make certain phrases do things to the game, or set off another script i guess. i'm hoping that's not too hard. we will see.
i think data is only going to become more and more valuable as time goes on, not just for things like marketing, but for decision-making in general. it interests me a lot, but i don't know that much about it right now.
i play a first-person shooter called valorant made by riot games, and they have an api that you can pull data from. this includes stats on each game you play, which can be useful to understand your gameplay and how you can improve, but it also includes data on player positions at certain times of the game.
a company called runitback gathers all this data from the api and presents it to paying customers in a way that helps them understand typical gameplay in professional games.
this tweet, for example, is a heatmap they created and shared during a professional event that happened in september of last year. in it, you can see where every defender died, as well as the most common locations a defender will die. they also made heatmaps of player locations in the first 15 or 20 seconds of each round, which shows valuable information about how spread out teams like to be, as well as how aggressive each team likes to be so that analysts for each team can be better prepared for their opponents.
that's just one example that caught my interest. i look forward to learning more about how data can be used for strategic purposes in other areas.
hello, my name is angel tapia. i'm in my second-to-last quarter at oregon state university, majoring in computer science. i didn't realize that i could take my capstone class before the last quarter until a friend told me and i decided that it might be a good idea to do so since i wasn't so thrilled with one of my classes. it was 3 days into the week when i dropped that class for my capstone class so i'm starting behind.
on the bright side, i likely have a group and an idea of what we're going to be working on so i'm excited because my other class (mobile development) seems fun as well.
my typical interests in the field of cs relate to data and web development. my most recent project i have worked on outside of classes was a website i made to present my grandfather's artwork using aws services. before that i was working on a small javascript program that i used to learn how to interact with riot's api for their game valorant. now i'm curious how both my courses will go since i'm not familiar with anything i'm going to be doing.
our top choices for projects in capstone are all related to vr and ar development, which is new to me. but i have been a consumer of both for such a long time that i figured it was a good opportunity to see what it's like to work on them. the top two that i would personally like to work on are a simulation project or an arcade game ar recreation. they both seem like they have a lot of creative freedom and would be good for learning. maybe i will learn enough so that i can make something interesting in my spare time.
