BillBot: Smart Billing Assistant AI-Based Object Detection and Automatic Billing Using PictoBlox.

by rishi775 in Design > Software

24 Views, 0 Favorites, 0 Comments

BillBot: Smart Billing Assistant AI-Based Object Detection and Automatic Billing Using PictoBlox.

BillBot is an educational smart-billing prototype made in PictoBlox. It uses webcam-based object recognition to identify configured product classes, announces their prices, and adds the prices to a running total. Tobi acts as the virtual cashier. A red circular button resets the bill, and a green circular button triggers the final bill announcement.

Supplies

  1. Computer or laptop with PictoBlox installed
  2. Working webcam
  3. Speakers or headphones for speech output
  4. The object-recognition extension used by the supplied project
  5. Tobi, cashier-counter, red-button, and green-button sprites

— Add the Sprites

Add or retain the four visible project elements: Tobi, the cashier counter, Sprite1 (red circle), and Sprite2 (green circle). Arrange them on the Stage so the cashier and buttons are easy to see.

Check the Recognition Extension and Model Labels

The screenshots show an “open recognition window” block and conditions that check classes from the webcam. They do not show the extension name, model-training workflow, or model configuration. In PictoBlox, open the same recognition extension used by the project and verify that its model can return every exact label listed in the price table below. Do not assume that the model automatically knows these products: the labels must exist in the configured model.

  1. Confirm the recognition window opens when the green flag is clicked.
  2. Check that the model’s labels match the script exactly, including “water bottle” and the lowercase product names.
  3. Confirm the configured “Class 13” label exists and is intended to trigger the welcome message.
  4. If a label differs, update the dropdown condition or the model labels so they match.


Create the Total Variable

Create a variable named total and make it available to the scripts that update or announce the bill. The screenshot shows total being set to 0 at startup and changed when products are recognized.

— Start Recognition and Initialize the Bill

Script A: when green flag clicked → open recognition window → forever [product-recognition conditions]

Script B: when green flag clicked → set total to 0

The first script opens the recognition window and continuously checks for objects. The second initializes the total. Keep the reset block outside the product-detection loop.

Welcome the Customer

if [web camera identifies Class 13] then

speak “hi welcome to our store, please move away from the camera while showing your items”

wait 1 seconds

This welcome condition is visible in the first screenshot. Confirm that “Class 13” is the intended trigger label in your recognition model.

— Add Product-recognition Conditions

For each product, use an if condition inside the forever loop. When the webcam identifies the class, use say to display the price, speak to announce the price, change total by the price, and wait for the configured duration.

Recognition class

Price displayed

Change total by

Wait visible

book

₹30

30

1 second

pen

₹20

20

1 second

scissors

₹20

20

1 second

sharpener

₹5

5

1 second

pencil

₹5

5

1 second

eraser

₹5

5

1 second

paper

₹2

2

1 second

watch

₹50

50

10 seconds

glue

₹25

25

10 seconds

scale

₹30

30

1 second

water bottle

₹40

40

1 second

Note: The paper condition in the screenshot displays ₹2 and adds 2 to total, but the speech block says “5 rupees.” Correct the speech text to “2 rupees.”

— Announce the Final Bill

when I receive message1

say total

speak total

wait 1 seconds

speak “thank you for coming to our store, hope you have a nice day”

When message1 is received, Tobi displays the current total, speaks it, waits one second, and speaks the thank-you message.


Reset the Total

when this sprite clicked → set total to 0

The supplied screenshot has set total to 0 inside a forever loop. This is not recommended: it can repeatedly overwrite the total while the loop runs. Put set total to 0 directly beneath the click event so the total resets once per click.

Trigger the Final Bill

when this sprite clicked → broadcast message1

The green button broadcasts message1. Tobi receives this message and runs the final-bill announcement script.

End.

. Conclusion

BillBot: Smart Billing Assistant demonstrates how webcam-based object recognition, speech output, variables, and sprite events can be combined in PictoBlox to build an interactive billing prototype. Tobi recognizes configured product classes and updates the running total, while the red and green buttons reset the bill and trigger the final announcement. With duplicate-detection protection and thorough model testing, the project can be extended into a more robust educational checkout demonstration.