Application Customization Using AI

Application Customization Using AI is an interactive project developed using PictoBlox and Artificial Intelligence. It features a voice-controlled virtual assistant named Mike that listens to user commands, recognizes spoken instructions, and performs different activities. By integrating speech recognition, text-to-speech, weather data, and music features, the assistant can tell the current time, narrate stories, ask riddles, provide weather updates, and play piano music. The project demonstrates how multiple AI and software extensions can be combined to create a personalized and interactive application.

1. .Computer or laptop.
2. Microphone for voice input.
3. Speaker or headphones for audio output.
4. Internet connection.
5. PictoBlox software.
6. Weather Data extension.
7. Artificial Intelligence extension.
8. Text to Speech extension.
9. Music extension.

The project contains two sprites named Mike and Mike2.

Both sprites have five costumes: Thinking_Emoji, Talking_Emoji, Giddy_emoji, Sleeping_Emoji, and Smiling_face.

The original project file contains programming scripts and custom functions for both sprites. The main assistant interaction is explained below.

Open PictoBlox and add the following extensions:

Weather Data: Retrieves weather information based on geographical coordinates, including temperature, wind speed, and humidity.

Artificial Intelligence: Provides speech recognition functionality to understand spoken user commands.

Text to Speech: Converts written text into spoken responses so that the assistant can communicate with the user.

Music: Allows the assistant to play musical notes using different instruments, including Piano.

Start by programming the main sprite, Mike.

When the Green Flag is clicked, perform the following operations:

1. Switch the costume to Talking_Emoji.
2. Speak: "Hello, You can call me Mike. I like to sleep, you can call my name to wake me up."
3. Switch the costume to Sleeping_Emoji.
4. Call the custom block Wake_Up_Mike!.
5. Switch the costume to Smiling_face.
6. Speak: "Yes?"

After the introduction, use a Forever loop to continuously listen for user commands.

Create a custom block named Wake_Up_Mike!.

This function allows the user to activate the assistant by saying its name.

Follow these steps:

1. Set the noise removal threshold to 30%.
2. Recognize speech for 5 seconds in English (United States).
3. Store the recognized speech in the Speech_Reconisation_Result variable.
4. Add a Repeat Until block with the condition that the recognized result equals Mike.
5. Inside the loop, make Mike speak: "I didn't hear you clearly, please say it again."
6. Recognize speech again for 5 seconds.
7. Update the speech recognition result.

The loop continues until the user says "Mike".

Inside the Forever loop, add the following blocks:

1. Set noise removal threshold to 30%.
2. Recognize speech for 5 seconds in English (United States).
3. Set Speech_Reconisation_Result to the speech recognition result.
4. Use conditional statements to check the recognized words.
5. Call the corresponding custom function based on the user's command.

The assistant supports the following commands:

1. "Tell me the time" calls Report Time.
2. "Tell me a story" calls Tell a story.
3. "Tell me a riddle" calls Play a riddle.
4. "Tell me the weather" calls Report_Weather.
5. "Play some music" or "Play the piano" calls Play piano.

The script checks different variations of some phrases to recognize similar voice commands. If no matching command is found, Mike speaks:

"I didn't hear you clearly, please say it again."

Create a custom block named Report Time.

This function retrieves the current hour and minute using PictoBlox date and time blocks.

Follow these steps:

1. Retrieve the current hour.
2. Retrieve the current minute.
3. Join the hour and minute values with suitable text.
4. Use the Text to Speech block to announce the current time.

When the user asks Mike for the time, the assistant retrieves the current time and speaks it aloud.

Create a custom block named Report_Weather.

This function uses the Weather Data extension to retrieve weather information.

The original project uses the following configured geographical coordinates:

1. Latitude: 1.434641
2. Longitude: 103.837683

Follow these steps:

1. Use the Weather Data block to retrieve weather information for the configured coordinates.
2. Retrieve the current weather condition.
3. Retrieve the temperature in Celsius.
4. Retrieve the wind speed in metres per second.
5. Retrieve the humidity percentage.
6. Use Text to Speech blocks to convert the retrieved information into spoken responses.

When the user asks Mike about the weather, the assistant provides the available weather details through voice output.

7.1 Create the Storytelling Function

Create a custom block named Tell a story.

This function allows Mike to narrate different stories using Text to Speech.

Follow these steps:

1. Generate a random number between 1 and 3.
2. Store the generated number in the Tell a story variable.
3. Speak: "Okay. Here is the story prepared for you."
4. Use IF and ELSE conditions to select a story.
5. Narrate the selected story using Text to Speech blocks.

The project includes three stories.

Story 1 – The Fox and the Grapes

A hungry fox sees a bunch of grapes hanging high on a wall. He tries repeatedly to reach them but fails. Finally, he gives up and walks away, saying that the grapes were probably sour.

Story 2 – The Proud Rose

A beautiful rose growing in a desert feels proud of its appearance and makes fun of a nearby cactus. When summer arrives and water becomes scarce, the rose learns the importance of kindness and friendship.

Story 3 – The Wolf and the Sheep

A wolf asks a sheep for water and tries to convince it to bring him food. Their conversation forms an interactive story narrated by Mike.

7.2 Create the Riddle Function

Create a custom block named Play a riddle.

This function selects a random riddle and speaks both the question and its answer.

Follow these steps:

1. Generate a random number between 1 and 5.
2. Store the number in the Play a riddle variable.
3. Use conditional statements to select the corresponding riddle.
4. Speak the question.
5. Wait for one second.
6. Speak the answer.

The project contains the following five riddles:

Riddle 1: What has to be broken before you can use it?

Answer: An egg.

Riddle 2: I'm tall when I'm young, and I'm short when I'm old. What am I?

Answer: A candle.

Riddle 3: What month of the year has 28 days?

Answer: All of them!

Riddle 4: What question can you never answer yes to?

Answer: Are you asleep yet?

Riddle 5: What can't talk but will reply when spoken to?

Answer: An echo.

7.3 Create the Piano Function

Create a custom block named Play piano.

This function uses the Music extension to play a predefined sequence of piano notes.

Follow these steps:

1. Make Mike speak: "Sure."
2. Set the instrument to Piano.
3. Add the required musical note blocks.
4. Set the duration of each note using the beats value.
5. Arrange the notes in the required sequence to create a melody.

The original project uses different note durations, including 0.125, 0.25, and 0.50 beats.

When the user says "Play the piano", Mike executes this custom function and plays the programmed musical sequence.

7.4 Create the Command List

Create a list named What_Can_Mike_Do.

This list displays the available activities that the assistant can perform.

When the Green Flag is clicked:

1. Show the What_Can_Mike_Do list.
2. Delete all existing items from the list.
3. Add the following five commands:
4. Tell the time
5. Tell a story
6. Tell a riddle
7. Tell the weather
8. Play the piano

The list helps users understand which voice commands are supported by the assistant.

7.5 Configure the Wi-Fi Connection

The project also includes a Wi-Fi connection script.

When the Green Flag is clicked:

1. Set Speech_Reconisation_Result to 0.
2. Use the Wi-Fi extension block to connect to the configured wireless network.

An active internet connection is required for online features such as weather information and supported AI services.

Security note: Do not include your personal Wi-Fi password in publicly shared project documentation.

7.6 Configure the Mike2 Sprite

The original .sb3 project contains a second sprite named Mike2. The project file also confirms that Mike2 has its own programming scripts and custom functions.

The custom functions present in Mike2 include:

1. Wake_Up_Mike!
2. Tell a story
3. Report Time
4. Play a riddle
5. Play piano
6. Report_Weather

Keep the scripts assigned to their respective sprites when editing the project. Both sprites are part of the original project, so avoid deleting either sprite when loading or modifying the application.

7.7 Test the Application

Follow these steps to test the completed project:

1. Open PictoBlox on your computer.
2. Load the Application Customization Using AI project.
3. Check that all four extensions are available.
4. Connect the computer to the internet.
5. Allow PictoBlox to access the microphone.
6. Ensure the speakers are working.
7. Click the Green Flag.
8. Listen to Mike's introduction.
9. Say "Mike" to activate the assistant.
10. Try saying "Tell me the time."
11. Try saying "Tell me a story."
12. Try saying "Tell me a riddle."
13. Try saying "Tell me the weather."
14. Try saying "Play the piano."
15. Verify that Mike performs the corresponding activity for each command.

7.8 Troubleshooting

Mike does not recognize voice commands: Check microphone permissions, reduce background noise, and speak clearly.

Mike does not wake up: Say "Mike" clearly within the speech recognition period.

Weather information is unavailable: Check the internet connection and configured geographical coordinates.

Mike does not speak: Check the speaker volume and Text to Speech functionality.

Piano does not play: Confirm that the Music extension is enabled and the instrument is set to Piano.

Incorrect command is recognized: Speak clearly and use one of the supported voice commands.

This project can be used as an educational model for AI-powered virtual assistants, voice-controlled applications, smart home interfaces, interactive learning systems, and human-computer interaction. It helps students understand speech recognition, text-to-speech technology, weather data integration, and automated responses. The same concept can be extended to develop personalized digital assistants, educational applications, voice-operated services, and smart classroom solutions.

Application Customization Using AI demonstrates how Artificial Intelligence can be integrated into an interactive virtual assistant using PictoBlox. By combining speech recognition, text-to-speech conversion, weather information, storytelling, riddles, and music playback, the project creates a personalized assistant capable of responding to different user commands. It provides a practical introduction to AI-powered application development and helps learners understand how multiple software extensions can work together to build intelligent and interactive applications.