BillBot: Smart Billing Assistant AI-Based Object Detection and Automatic Billing Using PictoBlox.
by rishi775 in Design > Software
24 Views, 0 Favorites, 0 Comments
BillBot: Smart Billing Assistant AI-Based Object Detection and Automatic Billing Using PictoBlox.
BillBot is an educational smart-billing prototype made in PictoBlox. It uses webcam-based object recognition to identify configured product classes, announces their prices, and adds the prices to a running total. Tobi acts as the virtual cashier. A red circular button resets the bill, and a green circular button triggers the final bill announcement.
Supplies
- Computer or laptop with PictoBlox installed
- Working webcam
- Speakers or headphones for speech output
- The object-recognition extension used by the supplied project
- Tobi, cashier-counter, red-button, and green-button sprites
— Add the Sprites
Add or retain the four visible project elements: Tobi, the cashier counter, Sprite1 (red circle), and Sprite2 (green circle). Arrange them on the Stage so the cashier and buttons are easy to see.
Check the Recognition Extension and Model Labels
The screenshots show an “open recognition window” block and conditions that check classes from the webcam. They do not show the extension name, model-training workflow, or model configuration. In PictoBlox, open the same recognition extension used by the project and verify that its model can return every exact label listed in the price table below. Do not assume that the model automatically knows these products: the labels must exist in the configured model.
- Confirm the recognition window opens when the green flag is clicked.
- Check that the model’s labels match the script exactly, including “water bottle” and the lowercase product names.
- Confirm the configured “Class 13” label exists and is intended to trigger the welcome message.
- If a label differs, update the dropdown condition or the model labels so they match.
Create the Total Variable
Create a variable named total and make it available to the scripts that update or announce the bill. The screenshot shows total being set to 0 at startup and changed when products are recognized.
— Start Recognition and Initialize the Bill
Script A: when green flag clicked → open recognition window → forever [product-recognition conditions]
Script B: when green flag clicked → set total to 0
The first script opens the recognition window and continuously checks for objects. The second initializes the total. Keep the reset block outside the product-detection loop.
Welcome the Customer
if [web camera identifies Class 13] then
speak “hi welcome to our store, please move away from the camera while showing your items”
wait 1 seconds
This welcome condition is visible in the first screenshot. Confirm that “Class 13” is the intended trigger label in your recognition model.
— Add Product-recognition Conditions
For each product, use an if condition inside the forever loop. When the webcam identifies the class, use say to display the price, speak to announce the price, change total by the price, and wait for the configured duration.
Recognition class
Price displayed
Change total by
Wait visible
book
₹30
30
1 second
pen
₹20
20
1 second
scissors
₹20
20
1 second
sharpener
₹5
5
1 second
pencil
₹5
5
1 second
eraser
₹5
5
1 second
paper
₹2
2
1 second
watch
₹50
50
10 seconds
glue
₹25
25
10 seconds
scale
₹30
30
1 second
water bottle
₹40
40
1 second
Note: The paper condition in the screenshot displays ₹2 and adds 2 to total, but the speech block says “5 rupees.” Correct the speech text to “2 rupees.”
— Announce the Final Bill
when I receive message1
say total
speak total
wait 1 seconds
speak “thank you for coming to our store, hope you have a nice day”
When message1 is received, Tobi displays the current total, speaks it, waits one second, and speaks the thank-you message.
Reset the Total
when this sprite clicked → set total to 0
The supplied screenshot has set total to 0 inside a forever loop. This is not recommended: it can repeatedly overwrite the total while the loop runs. Put set total to 0 directly beneath the click event so the total resets once per click.
Trigger the Final Bill
when this sprite clicked → broadcast message1
The green button broadcasts message1. Tobi receives this message and runs the final-bill announcement script.
End.
. Conclusion
BillBot: Smart Billing Assistant demonstrates how webcam-based object recognition, speech output, variables, and sprite events can be combined in PictoBlox to build an interactive billing prototype. Tobi recognizes configured product classes and updates the running total, while the red and green buttons reset the bill and trigger the final announcement. With duplicate-detection protection and thorough model testing, the project can be extended into a more robust educational checkout demonstration.