END OF LIFE NOTICE

Support for Intel® RealSense™ Extension for Scratch has ended and the plug-in is no longer available for download.

If you have a camera and the plug-in already installed you may still use it as long as you like.

Thank you very much for your support.

Welcome Scratchers

Intel® RealSense™ Extension for Scratch introduces new and amazing capabilities - all made simple with just a few Scratch blocks

Scratch is a project of the Lifelong Kindergarten group of the MIT Media Lab. Learn more

Hand tracking. Gesture recognition. Face tracking. Facial expressions. Voice control.

Setting Up

Download Plug-In

Download and install Intel RealSense Experience for Web Browsers

Get a Camera

Make sure you have an Intel RealSense Camera (F200 or SR300)

Go to ScratchX

Open the extension in ScratchX.org

Privacy Notice: Even though Intel RealSense Extension for Scratch uses a camera, no images are captured, stored or sent to the cloud.

Everything you need to know

Try out different capabilities by switching the green flag between the blocks

Full Hand

Tracks the entire set of 22 joints of the hand

Ping Pong

A simple hand tracking game

Guess the Colors

Advanced voice recognition

Avatar

Advanced sample - Face and Hands working together

Simple Face

Track the major landmarks of the face

Rock Paper Scissors

An intermediate level hand gestures game

Simple Hand

Shows a hand skeleton that tracks the major joints of the hand

Full Face

Track the entire set of face landmarks

Voice Bug

Simple voice control game

Samples

Scratch Blocks

Face

Returns true if face is visible

Returns the position of the selected face landmark in selected axis

Returns true if the selected face expression is detected

Returns the rotation of the head between 0 to 180

Hands

Returns true if the selected hand is visible

Returns the position of the selected joint in selected hand in selected axis

Returns true if the selected gesture is detected on selected hand

Returns the foldedness amount of the selected finger of selected hand

Voice

Returns true if the user said the word written in the box

Returns true if the user spoke (actual speech, not just loudness)

Returns the last recognized word the user said

If you are new to Scratch, you might want to check out this link first

Getting Started With Scratch ›

Tracking the index finger

Choose Cat1 from the Sprite library
Drag the green flag block and connect a forever block under it
From Motion, drag set x to and set y to blocks under the forever block
Go to More Blocks and drag a hand position block into each set block
Set the position axis of each hand hand position block to the corresponding axis of the set block
Make sure both hand position blocks are following the same hand and same joint
Press the green flag
Present your hand to the camera, palm facing forward, fingers facing upwards, and move your hand around, the cat should be following your hand

Rotating with the head

Choose Cat1 from the Sprite library
Drag the green flag block and connect a forever block under it
From Motion, drag Point in direction block under the forever block
Go to More Blocks and drag the rotation of head block
Set the rotation axis to Roll
Press the green flag
Tilt your head left and right in front of the camera to see the cat rotate accordingly

Change the size with a V Sign gesture

Choose Cat1 from the Sprite library
Drag the green flag block and connect a forever block under it
From Control, drag an if/else block under the forever block
Go to More Blocks and drag the gesture block as the condition of the if
Go to looks and drag a set size to blocks under the if and the else
Set the set size block under if to 200%, keep the other one at 100%
Press the green flag
Present your hand to the camera and make a "V sign" with your index and middle fingers. the cat should change size when you perform the gesture, and get back to it's original size when you open your palm

Control a sprite with voice commands

Getting Started

User Experience (UX)

Choose the Beetle from the Sprite library
Drag the green flag block and connect a forever block under it
From Control, drag an if block under the forever block
Go to More Blocks and drag the User Said block as the condition of the if
Write the word left in the string box
Go to motion and drag a turn left 15 degrees block under the if
Set the degrees to 90
Press the green flag
Say Left and watch the Beetle turn left.

Making the most out of Intel RealSense technology

Like any other technology, Intel RealSense experiences have their sweet spots and limitations.

The 3D camera has a range and a field of view. When developing an application, you need to be aware of these, among other things.

To see how to get the most out of Intel RealSense technology, check out our UX design guide:

Intel RealSense SDK User Experience Guide ›