Description

1. Environment & Components

Coding Environment Tested on: ROG Zephyrus M16 Intel Core i7-11800H 16GB RAM Windows 10 21H2 WSL2: Ubuntu 20.04.5 LTS Board: Pico4ML Sensors: build-in LCD (ST7735) build-in camera (hm01b0) LSM6DS33 Gryo IMU and Accel sensor Pico4ML Pico4ML is using RP2040 for the microprocessor, with an LCD screen drived by ST7735. Basically we are refering to the code to drive the camera and LCD display. LSM6DS33 6-DoF Accel + Gyro IMU The sensor is basically dirved by i2c.

2. Developing Overview

To start with, we are planning to implement a simple flappy bird game on the C platform, with one button to manipulate the flying of the bird. To achieve our plan, we first implemented the flappy bird game with microPython on the RP2040 chip. With the external LCD screen, we can successfully play this game with the python code. Then, the next step was to write up drives to transform the code into C and make it able to operate on the PICO4ML broad.

3. Challenge We Ran into

One major problem we met during our development was the fps frequency. In the first attempt that we launch the program, we ran the chip at its normal frequency. However, the calculation speed does not provide optimal performance for the game, it runs pretty slowly and the movement is obviously discrete. Thus, we overclocked the chip to improve its performance. Finally, the fps has been stabled to 15 frames per second, which seems much more smooth for a visual game.

4. Reflection on Our Design

The biggest mistake that we made in our project is that we didn’t realize that the PICO4ML broad has its own IMU sensor embedded on the chip; Thus, it is totally unnecessary for us to implement an external drive for an additional chip.

5. Advantage

The advantage of our project is that we have to implement a control-free strategy, that the player doesn’t need to use any button to play the game; all he or she need to do is to flip over the board, then we can start the game and play! In addition, the game has great completion, which means that the game has its way to attract people to continue playing: who does not want to control his or her cars to rush on a busy street?

6. PIO module

We are basically using pio to drive the i2c, just like what we do in Lab2b. This is a more reliable and faster way to estabilish communication between sensors and MCU. We modified the code in Lab 2b, by configuring the registers within the IMU chips, we successfully received the serial data in the test program on QT2040 broad; then we transplanted the function onto our PICO4ML broad. When we fully grasp how i2c works, we can use pio to simulate an identical i2c function.

7. Future Improvements

As a final project embryo, there is a lot we could improve to make the game more playable. For example, as we mentioned in the challenges sections, we can use external resources and pictures to replace the code-generated shape, to make the game more pretty. Also, we could implement more functions to the game. As an embedded program, we need also replace the communication protocol between LCD and the chip. Currently we are using GPIO to drive the spi protocol, which seems not fast enough and requires more operational effort; later we can also switch it to PIO ports, which could be faster and this would be beneficial for the performance of the program.

2. Developing Overview

To start with, we are planning to implement a simple flappy bird game on the C platform, with one button to manipulate the flying of the bird. To achieve our plan, we first implemented the flappy bird game with microPython on the RP2040 chip. With the external LCD screen, we can successfully play this game with the python code.

flappybird

Then, the next step was to write up drives to transform the code into C and make it able to operate on the PICO4ML broad. However, during the midpoint check-over with the professor, he suggested that our program should be more fun, and more complex with more concepts implanted. Then, we noticed the camera and screen of the PICO4ML and decided to design a game, which could use the scene captured by the camera as the background.

We first created code to successfully drive the camera to make it work. With this code, what is captured by the camera can be reflected on the screen on the frequency of 15 frames per second. During the tryout of the code, we get a cool idea: Why not we make a race game that makes more sense with a real-world background, rather than the flappy bird game? After all, a classic flappy bird game does not need a background. Thus, we overturn our design of the flappy bird and decided to implement the car racing game on the basis of the camera monitor code.

We then figured out the mechanism of how the picture was shown on the screen, and we used the drawing tool to generate multiple blocks on top of the picture buffer every time the picture is refreshed. The car of the player is filled with red color, and we made the blocking cars on the road, and to make it like a real racing car game, we made the pavement markings on the road to imply the lanes of the road.

game1

After we set up the layout of the game, we started to implement the game mechanism and the control logic. We made the barrier car move towards the player at a fixed scale, and once they hit the car ( which is judged by the collision algorithm), the game would end and automatically restart. To make a ramping difficulty, we set up the speed to be accelerated from 2 units to 7 units, as the playing time goes by. After that, we achieved the movement of the player-owned car by driving an external IMU sensor.

We began driving the IMU sensor first on the QT2040 broad. As it could output the data of the gyroscope and accelerometer on the serial output terminal, we have successfully made it work. Then we started to transplant it on PICO4ML. By implementing operations based on the data output of the X, Y, and Z axis of the accelerometer and the gyroscope, we can then move the player car to the left, right, up, or down. Then, for the restart option of the game, we designed that the flipping-over — turning the whole sensor upside down to invert the Z axis— would make the game restart. At this point, our viable version of the AR race car game is finished and available for use.

sensor

Before the Final Demo Day, we checked off your final delivery with the professor. Thanks to his advice, we refined our control algorithm by using data from both the accelerometer and gyroscope to achieve smoother control and interaction of the car; Also, we changed our action of the restart, from the vertical flip-over to the horizontal one. Thus, our instruction would make more sense to users and become intuitive for them.