Gesture-Controlled Light Using Quarky Using ML

Gesture-Controlled Light Using Quarky is an interactive project that uses Artificial Intelligence and Machine Learning to control an output through hand gestures. The project uses a webcam to recognize two hand gestures: ON and OFF.

A trained Hand Pose Classifier model identifies the user's hand gesture in real time. Based on the recognized gesture, PictoBlox sends instructions to Quarky. When the ON gesture is detected, Quarky displays a selected LED matrix pattern. When the OFF gesture is detected, the LED matrix is cleared.

This project demonstrates how Machine Learning and robotics can be combined to create a touch-free control system.

2. Working Principle

The project works through the following steps:

1. The webcam captures the user's hand gesture.
2. The trained Machine Learning model analyzes the hand pose.
3. The model classifies the gesture as ON or OFF.
4. PictoBlox checks the identified class continuously.
5. If ON is detected, Quarky displays an LED matrix pattern.
6. If OFF is detected, Quarky clears its LED matrix.
7. If no trained gesture is recognized, Quarky displays an instruction message.

Components Required

Component Quantity

Quarky board 1

Computer or laptop 1

Webcam 1

PictoBlox software 1

Machine Learning Environment 1

Note: One Quarky board is used for real-time testing. The provided coding screenshot shows the built-in Quarky LED matrix as the output. An external bulb or relay connection is not shown in the supplied circuit or code.

1. Software and Extensions

The project is developed using:

1. PictoBlox: Used for block-based programming and hardware control.
2. Machine Learning Environment: Used to create and train the hand-pose recognition model.
3. Hand Pose Classifier: Used to classify the ON and OFF hand gestures.

The Hand Pose Classifier recognizes different hand poses using hand-position data points.

First, create a Machine Learning model that can recognize two different hand gestures.

Step 1.1: Open the Machine Learning Environment

1. Open PictoBlox on your computer.
2. Open the Machine Learning Environment.
3. Select the Hand Pose Classifier.
4. Create a new model.

Step 1.2: Create Two Classes

Create the following two classes:

1. ON: Represents the hand gesture used to switch the output ON.
2. OFF: Represents the hand gesture used to switch the output OFF.

Step 1.3: Collect Hand Pose Samples

Use the webcam option to collect hand-pose samples for each class.

The model used in this project contains:

Gesture Class Samples

ON 142

OFF 133

Total 275

Keep your hand clearly visible while collecting samples. Capture the gestures from slightly different positions and angles to help the model recognize them reliably.

After collecting the samples:

1. Click on Train Model.
2. Wait for the training process to complete.
3. Observe the accuracy graph.
4. Check the training status.

In this project, the training screen displays:

1. Training Completed
2. Model Accuracy: Good

The trained model is now ready for testing.

Test the Trained Model

Before using the model in the coding environment, test it using the webcam.

1. Open the Testing section.
2. Select the Webcam option.
3. Show the ON gesture in front of the camera.
4. Check whether the model identifies it as ON.
5. Show the OFF gesture.
6. Check whether the model identifies it as OFF.

Make sure both gestures are recognized correctly before proceeding.

Add the Machine Learning Environment Extension

Now open the PictoBlox block-coding workspace.

1. Open the Extensions section.
2. Find Machine Learning Environment.
3. Click to add the extension.
4. The Machine Learning blocks will become available in the coding workspace.

These blocks allow the sprite to analyze webcam images and identify the trained hand-pose classes.

The complete program follows this logic:

Step 3.1: Start the Program

1. When Green Flag Clicked starts the program.
2. Forever continuously repeats the instructions.

Step 3.2: Enable the Webcam

- Turn ON video on stage with 0 transparency displays the webcam feed on the stage.

Step 3.3: Analyze the Image

- Analyse image from web camera captures and analyzes the hand pose.

Step 3.4: Open Recognition Window

- Open recognition window opens the gesture recognition interface.

Step 3.5: Check the ON Gesture

The first IF condition checks whether the identified class is ON.

If ON is recognized:

- The Display matrix as block displays the selected LED pattern on Quarky.

Step 3.6: Check the OFF Gesture

If the ON condition is false, the program checks whether the identified class is OFF.

If OFF is recognized:

- The Clear screen block clears the Quarky LED matrix.

Step 3.7: Handle Other Gestures

If neither ON nor OFF is recognized:

- Quarky displays the message:

"Place hand in front of Camera to control Light"

- The message is displayed at medium speed.

Follow these steps to test the project:

1. Connect one Quarky board to your computer.
2. Open the project in PictoBlox.
3. Allow webcam access.
4. Click the Green Flag.
5. Place your hand in front of the webcam.
6. Show the trained ON gesture and observe the LED matrix.
7. Show the trained OFF gesture and observe the LED matrix clearing.
8. Show an unrecognized gesture and check the instruction message.

Repeat the process to verify that the system works consistently.

Problem Possible Solution

Gesture is not recognized Collect more samples and retrain the model.

ON is recognized as OFF Check the samples stored in both classes.

Webcam is not working Check camera permissions and camera selection.

Quarky does not respond Check the board connection and PictoBlox communication.

Instruction message keeps appearing Keep your hand visible and show a trained gesture.

This project can be used as an educational demonstration of touch-free control, Machine Learning, computer vision, and human-computer interaction. The same gesture-recognition concept can be extended to smart-home control, classroom automation demonstrations, assistive interfaces, and other interactive control systems by connecting suitable output hardware.

Gesture-Controlled Light Using Quarky demonstrates how Machine Learning can be used to recognize hand gestures and control an output in real time. By training an ON/OFF Hand Pose Classifier and integrating it with PictoBlox and Quarky, users can explore the practical applications of Artificial Intelligence, robotics, and physical computing.

The project provides a beginner-friendly introduction to gesture recognition and touch-free interaction.

Please Find the project Link Below:

Open Project