I gave a talk for co-workers about the magic tools CDT developers use, how we use them, and where I think we’re heading. I tried to use this blog post as my notes, opening links in new browser tabs as if they were slides. Now I’m fleshing the sections out a little, so you can just read this post by itself (please feel free to swing by and ask me questions about any of this!). The idea is that it starts off pretty tame, but gets progressively … crazy scifi.
10 hardware topics in 60 minutes:
- basic gear
- audio
- video
- studio
- depth sensing webcams
- 360
- scanning
- nick H.
- spendy weird stuff
- too cool
_________________________________________________________
1. Basic Gear
* We have cabinets full of endless cables, adapters and card readers, weird mice, styluses for ipads, etc. just ask.
* We have many laptops (macbook air. a VR-ready-ish beast?), ipads (4+? including a large), etc. You can rent laptops from the front desk, but in a pinch you might ask us? (we probably have the cords/droids you’re looking for)
* OSU’s 3D printing service #3dprints
OSU Valley library offers this (and several other departments around campus have 3D printers). You pay for the material used, and it usually takes less than a week. We’ve looked into making: mic holders, light switch covers, pipettes for distance students, lab kit materials (soil “ped” chunks included in SOIL 466 lab kit), etc. You can upload your own .STL files to have them printed. And you can download thousands of those sorts of files here: thingiverse. Note: We can help prepare 3D print files (if you find something cool, and want help preparing it for print: just ask)
FUTURE:
+ 3D printing will get crazy in the coming years (print fabric and flesh. Mix in transparent and conductive materials while printing – note that some campus departments are looking to get a $150K+ system, the HP Jet Fusion. I went to a talk they gave a couple month ago. Exciting powder printing tricks!)
_________________________________________________________
2. Audio #audio
* We have two Olympus LS-10 recorders. They are lame, and we never use them any more. But they’re portable, stereo, save to .wav file, and become a usb drive when connected. HAPPY TO LEND OUT (not getting much use).
* We multiple Blue “Nessie” mics floating around (and a “yeti“?). They have an easy room noise cancellation switch. HAPPY TO LEND OUT (not getting much use)
* We have 3+ audio studios. Sarah, nick, and zach manage them. Equipment: Audio Technica AT2020 USB condenser microphones, mic stands (boom and desktop), an acoustic shield (only in one room). The room’s wall foam isn’t so much “sound-proofing” as it is reverb-deadening/sound-diffusing. Various laptops running Camtasia, Presenter, Audacity, Logic, Audio Hijack, etc. Rumors of plans to upgrade the stylus screens available this year (Wacom Cintiq? MS Surface?).
Note: We have several musicians on staff, eager to record custom songs (note zach’s jingle on recent phone message)
FUTURE:
+ Transcription is possible with machine learning. Nick and Warren looking into it. (maybe we’ll use OSU cs dept tools?)
+ Virtual lectures from instructor voice samples (just type it in). Adobe demo’d VoCo in 2016. Not sure why it hasn’t come out (Tech exists. Legal fears?) Just needs to trickle down to tools we can find.
_________________________________________________________
3. Video #video
* Tons of Equipment: motion control arm (robot) offers small repeatable moves and turns. Steadicam (heavy vest with camera mount), teleprompter (you slot an ipad, then control speed wirelessly by phone).
* Working to provide more training and support so faculty can create simple media objects themselves (personal creation = more personality). (we used to give out FlipCams until they were bought out and discontinued (custom faculty forum 2010 designs)
FUTURE:
+ Higher budget look without increasing the budget. With more team members and time for pre and post-production, we’ll produce near television standards. A class field trip will be shot and edited like a show, with aerials, music, motion graphics, etc.
+ More interactive video. In the next few years we may have some interactive element to all the videos we produce.
+ More mixed reality (see below). Teachers lecturing in VR with any virtual objects they want, as if inside the whiteboard.
_________________________________________________________
4. Studio
Here is a rough point cloud scan of the studio (you can measure exact distances in the app that made this).
* An entire wall is greenscreen (random tests), we can roll in the Learning Glass equipment (random samples), and we’ve done mixed reality (test from R&D week, VR test).
* Current studio is Student Media Services (SMS) property, but they let us schedule use (would love to knock out the wall to their old dark room and make it into a control room, but they prefer to use it for box storage). We just got reasonable rates to access the smaller KBVRtv studio in Student Experience Center (SEC) daily. Maybe someday we’ll seek access to New Media Communication’s Visualization lab in Snell? It was a TV studio 5+ years ago, but now it’s filled with 4+ VR setups (good place to practice).
* We’ve handled live streams through WebEx and youtube.
FUTURE:
+ Live streaming will become more broadly accepted (Look to Twitch streaming of video games and VR. Lectures will be streamed from inside VR. Students will record their streams as well (so, we’ll offer virtual studio control for every user)+ Add the ability to interact with high end apps through simple buttons over the video feed, and students can “run” cutting edge software without needing a high end computer (Amazon Web Services also offers a form of this?).
_________________________________________________________
5. Depth sensing webcams
* We have several logitech and other HD options in office with mics built in. We can use them to drive animation from your face. Zach has done some good work in Adobe Character Animator. (I’ve done some bad work?) (note: FaceRig is 3D version of same concept)
* Kinect ($149. larger range: 6 to 12 feet. discontinued). We brought in personal devices for tests and hacks.
* Intel RealSense VF0800 ($129. shorter range: 1 to 3 feet)
FUTURE:
+ I’m suprised phones aren’t already used to replace mouse and webcams, wirelessly. (sigh)
+ Teachers will use their cell phone (or web cam) to record lecture into a 3D avatar. We can lip sync and edit performance (in Maya, MotionBuilder). Note the new iPhone x has kinect front facing cam built in (demo ad)
_________________________________________________________
6. 360 videos and photos #360video
* cell phones can do it (google cardboard camera app has you move around. Add-on gizmos slip fisheye lenses over your camera.)
* We have old rico theta S ($299): Low quality image. Can live stream for half hour, but tends to overheat. (james and warren test)
* We have rylo ($499): Image stabilizing. Cut out and export a normal video from the the sphere. Tap to track something as it moves around sphere. Export split screen with two videos from same sphere (ex: one of speaker, other of slides. Either side can move to show audience member when they ask question). 16gb SD card holds half hour total. Device spits out a clip every 7 minutes, so you have to put them back together later. {sorry i don’t have examples up yet. coming soon! it’s frickin’ great!}
FUTURE:
…hard to say. unlike other topics in this blog, there are too many products here for us to track and evaluate! (hopefully a standard emerges before the industry crashes?)
+ Maybe someday we’ll have stereoscopic HD at consumer level (in everyone’s cellphones?). Right now the good stuff is 10’s of thousands of dollars.
+ We’ve also seen research into recording every focal range at once through prisms (“hyperspectral“), so you can decide where to focus later. Hopefully this will get folded in.
+ fun note: The Wachowski’s already built crazy camera for filmmaking (in Jupiter Rising), so they could control camera moves later (they were doing expensive helicopter stunts, during magic hour, over chicago).
+ I have lots of ideas to hack 360 audio/visual for lecture tricks. Turn away from main lecturer to see a TA explaining the same concept in different way, or with different examples (but you only hear what you’re looking at). Place 3D assets into 360 video streams of lectures/events (represent other distant viewers, and where they are looking). I’ve recorded a few of CDT’s “Cool Project” talks (which we do each semester), and hope to turn them into concept demos.
_________________________________________________________
7. SCANNING #3Dscanning
*Cell phones can do “photogrammetry” (make 3D stuff from overlapping 2D pictures). Lots of apps make it simple: 123D, ReCap, ms Photosynth, etc. Agisoft Photoscan ($30 for educational license) can produce high end results with 80+ still photos (and 3+ hours of processing). Tons of better stuff coming with ARkit and ARcore (realtime experiment). Good way to do full body scans.
* Tango (Lenovo Phab2Pro $600. Discontinued): This is a phone we bought to quickly scan and measure real spaces (think “location scouting”). Gets point clouds quickly, which are also useful ref for modeling. (diagon-alley/dagon-allie scan sample). Note: Tango technology is no longer being pursued, instead it has been wrapped into the ARCore push.
* Artec Spider ($25k): High end scanner that is portable. nick’s demo video from recent Portland Event. Good way to do head or hand. (currently on loan to fish lab people).
FUTURE:
. we will likely work to scan all the archived materials hidden boxes around campus.
+ we could all have 3D avatars pretty easy.
+ everyone will be able to scan with their phone. GDC Note: 3D objects will become the indie standard for quick and cheap assets. potentially huge threat to all clipart/stock photo services? (So, custom crafted objects, and stylized cleanup, will be what you expect to pay for)
_________________________________________________________
8. Nick Harper
Nick is a huge hardware resource himself. Very skilled at making custom solutions.
* built turn table (3D printed surface and base. assembled motor and power) used at Portland event. Built temperature gauge in cubicles.
* 3D printed animal skulls from his 3D scans. Only person who knows how to use our high end scanner.
* Built macro lens robot for photogrammetry in his spare time.
FUTURE:
+ Sand Pit display ($1k?) Nick and warren offered to build this. Just needs projector and kinect (which we have? just need $50 worth of sand?). Uses free linux software. (there is reluctance to have sand dust in the office. OSU has one out at coast?)
+ we also offered to build a learning glass for less than a grand. teacher in another department has already done this?
_________________________________________________________
9. SPENDY WEIRD STUFF #interesting
* Wacom Cintiq ($2k?) (video overview)
We have 4 or more of these already. Simulate any art tool on screen. The company is based up in Portland/Vancouver area.
FUTURE:
Microsoft maybe killed them with windows10 stylus meddling? Maybe we’ll switch to MS Surface tablets or iPads, and wacom will die?
* OptiTrack Expression ($8k when we got it. now more like $80k?): We got thise facial mocap system with 7 custom cameras back in 2010.
Kineseology department just bought a modern system in January. (note: OptiTrack is here in Corvallis, was my previous job) (360 video of demo at their HQ)
FUTURE:
. Hoping to get Motive (full body software):
+ expand our ability to animate humans quickly and accurately in Maya, Unity3D, etc.
+ larger VR usage volumes (can replace Oculus Rift’s crummy web cam system).
_________________________________________________________
10. TOO COOL (for school). (Check the slack channels for many examples of these)
* Augmented Reality #AR
We’ve made phone and ipad apps over past 3 years, but no course has jumped in yet. Core benefits: proper scale (better than a picture in a book), measuring real world spaces, spatial memory.
FUTURE:
. VR undersells in it’s advertising, but AR oversells. Really suffers from a tiny field-of-view problem. Some people in office are more hyped for AR than VR, but i strongly disagree.
+ This will go back to headsets (within 5 years? MS Hololens was a dev demo (discontinued, basically). Google glass is sneaking back for hands free corporate training. Magic leap is too spendy (and maybe shady?), but offers light field trickery (basically it layers two displays, so you can better simulate different depths of focus. feels more real). ).
* Virtual Reality #VR
We have Oculus, vive, gear VR, windows MR, cardboard, etc. This should basically be it’s own talk.
We are always up for a demo. Wireless “inside out” tracking is now hotness (imagine 2 or more kinect cams on device. It doesn’t need to be tracked, because it can tell where it is in realtime, by scanning the environment around you)
FUTURE:
+ Initial hotness was “presence” (eliminating travel. field trips). Now it’s “embodiment” (kinetic learning. muscle memory). Next is “social” (a visiting professor can appear in class, while still in Dubai. See the work of Portland’s Kent Bye)
+ Bubbling now: walking through multiple rooms. eye tracking. hip and feet input. Haptic feedback. tons of things happening right now.
+ better connection to existing social networks (teens require constant access to facebook, twitter, tumblr, twitch, etc. They feel it’s rude to cut off access to their phone by strapping on headset)
+ notable problems with culture in VR (personal space, identity, harassment, privacy, etc.). Not clear who’s solving it (facebook? uh oh.)+ have to try it:
“oh, it works” (it’s so fast. really surrounds you) becomes “oh, you’re in the room with me!” Which will become “oh, i can attend conferences in my cubicle” (mention getting coffee while Oculus keynote continued in headphones. Blending layers of reality)+ watch ready player one. (outward facing screen on back of your goggles/cell phone? IT’S GOTTA HAPPEN!) [edit: Having now seen RPO, i’d like to retract this endorsement of the movie for future thinking. It’s much more a story of nostalgia and escapism. oh well]
+ some think robots (arms) will be added to VR experience, since it can place itself in any area you try to touch or move (and simulate various textures). But there some debate about whether virtual can ever REALLY cross threshold. (recent Research Department talk had interesting notes on our connection to real objects. Knowing an object is permanent seems kind of like we knowing when a movie’s special effects are practical). I prefer to think of VR as form of “structured dreaming” we can use to teach and reflect (but not aware of anyone approaching it that way. yet).
But, who knows, maybe get ready for ROBOTS to be the next cool tech used in Ecampus.
The End
p.s. Here are some of the newsletters I receive, in case you’d also like to get interesting tech news delivered to your inbox:
Level Up Report (21st century learning),
VR digest,
OSU Engineering Momentum,
Gamasutra Newsletter,
Portland TechFlash,
Intel Developer Zone Newsletter,
etc.
p.p.s. Please feel free to leave feedback. I’ve heard from a couple people that 10 topics was too broad or overwhelming (like it might have been easier to digest if it this split into 3 talks, about: audio, video, and crazy gizmos). (others noted they weren’t clear what AR was, and I found it hard to explain that the tech just puts virtual objects in the world you’re looking at through a device. it doesn’t necessarily know what the object is. it doesn’t necessarily have any ability to tell what is being seen in the real world) Thoughts?
