MECH4003 Mobile Robotics Assignment 3 – A vision-based sign detection system Demonstration to be held during Tutorial time in Week 14 (26/10/2022 and 27/10/2022 for Penrith students, and 28/10/2022 for Parramatta students): It is an Individual project

MECH4003 Mobile Robotics

Assignment 3 – A vision-based sign detection system

Demonstration to be held during Tutorial time in Week 14

(26/10/2022 and 27/10/2022 for Penrith students, and 28/10/2022 for Parramatta students):

It is an Individual project. Any misconduct will be investigated and treated according to university policy!

In mobile robotics, recognising road signs and react to these signs are important in controlling the robots. In this

project, each student is required to write an image processing program in MATLAB to detect a series of patterns.

Based on the detected patters’ shapes or colours, a statement is printed on the screen to describe the ‘action’ a robot

should take.

For example, the program will be able to recognise the directions of an arrow and decide what to do:

• When an image such as that in Figure 1 (a) is shown, the program will indicate “turn left”

• When an image such as that in Figure 1 (b) is shown, the program will indicate “turn right”

• When an image such as that in Figure 1 (c) is shown, the program will prompt “speed up”

• When an image such as that in Figure 1 (c) is shown, the program will prompt “slowdown”

• When the camera detects no arrow, the program will prompt “Stop!”

(a) (b) (c) (d)

Figure 1. Examples of signs to be recognised: a) left arrow, b) right arrow, c) up arrow, d) down arrow

Students can also write the program to recognise different shape or colours and print the ‘action statement’ on the

screen.

The basic functions the program should achieve are:

1. Periodically take images from the mobile phone of webcam

2. Processing the image and print out the “action statement” on the screen

3. Recognise at least 5 sets of patterns and print out their associated ‘actions screen’ on the screen. These

patterns can be shape or colour based

At the completion of the project, a demonstration will be held for students to show how their programs are working.

Students are required to use their mobile phones to capture images using MATLAB mobile App and MATLAB

program on their computers. The program will process the images and state what the ‘action’ associated with these

patterns.

Hints:

This is an “open-ended project”. Students can have different solutions. In general, you will need to have a set of signs

for the camera to detect. For example, you can have a large arrow printed out. When you rotate the arrow to different

directions the program will be able to determine the which direction the arrow is pointing and print out a statement.

In MATLAB Image Processing Toolbox, the ‘regionprops’ is a function that can obtain useful information of the

image, for example, the orientation of the object, the size of the object, and the extrema points of the object, etc. You

can find out more about this function in MATLAB help file.

Assessment:

This project forms a part of the assessment of the unit. Students will be working individually. Students will be

assessed based on the performance of their program in the final demonstration – demonstrations will be held in your

tutorial session in Week 14.

There is no written report to submit – however, submit your codes to vUWS by 23:59 on 28/10/2022 else you

will not receive marks! We will check the similarities of the program!

The overall marking scheme is:

• A working program that takes images from webcam or mobile devices continuously/periodically – 3/10

• Recognise at least 5 patterns – 6/10

• “Action statements” are clearly visible on the screen – 1/10