A Sandscape installation driven by the number of passers-by aims to remind us how our subtle impacts can drastically change our delicate planet earth.
The sandscape rotates a small degree every time the camera detects a pedestrian passes by (video shown on the upper-left corner), and eventually inversely rotates upside down when it hits a large number of passers-by.
I am a believer of global warming, maybe because I was a huge fan of natural documentary series who would always cried for the animals suffered from that. Therefore, I want to do something to raise people's awareness of global warming.
The idea came from the installation in my room — a Sandscape, which mimicked the landscape changes of our planet earth. The stunning thing was, it created a destroyed beauty when it turned upside down. It occurred to me that I could use this to emphasize how small things can add up to the impact of our planet earth which eventually lead to a drastic interruption.
To demonstrate this idea in a more dynamic way, I tried to use the population as the input, and the rotating of the Sandscape as the output to show subtle as well as drastic changes. More specifically, people passed by the installation would be the trigger of subtle rotations of the Sandscape, which meant, the computer would send a message to micro-controller each time it detected a person passed by to trigger a small rotation. And as passers-by accumulated to a certain amount, the computer sent a different message to trigger a different and large rotation of the Sandscape.
After constructing my idea, I split my project to 3 parts ([number] is the sequence of doing things).
Found an api or anything that could calculate the cumulative amount of people passing by. (which is under the part of "object detection" in Computer Vision. 
Figure out the serial communication between computer and Arduino. (depends on the api of passenger detection I used). 
Modify the api and set the serial communication for micro-controller. 
Bought a Sandscape/or made one. Calculated its weight and estimated the center of gravity.
1/4 inch wood sheet to hang the Sandscape. 
After that, figured out how to build a foundation of Sandscape to let it hang on the wood sheet. 
Physical Computing Parts
Chose a stepper motor that was powerful enough to drive the sandscape upside down with a easy driver.
Wrote a test script to figure out how to control the stepper motor to rotate different degree. 
Add serial communication part. 
Object Detection Counting API
After browsing the branches of computer vision, I targeted the keyword as "object detection" to search for the api I wanted. After finding more than 30 repos, I finally found an api that specifically counted the pedestrians rather than just detecting the person. See Codes HERE (Thanks bro).
I git the repo and found that the example only provided a video output with a video input. So I had to modify two things: (1) changed the input as webcam instead of video; (2) rendered each of the frame and displayed so that we can watch the changes in real-time. The modified codes can be seen here: Main Code & API Code.
Since the object counting api used python, I need to find the way how Arduino communicated with Python. Of course it was easy to target the package "Pyserial" as my tool. So I made a test code to learn the communication between Arduino and python, and it worked.
I bought one in Micheals', which helped to save lots of time.
After validating the feasibility of programming, I brought my idea confidently to Danny Rozin. He listened to my idea and suggested that, the connection between people passed by and the changes of the Sandscape installation was kind of loose. He suggested to add some elements that showed the changes in a more obvious way.Those were very useful advice. I began to think about how to convey this idea, the impact on earth in a more persuasive way.
In the mean time, I was stuck in finding a stepper motor that was powerful enough. The motor I bought just couldn't lift up the heavy Sandscape in a fast speed. But it could rotate in a small degree cause slow speed allowed larger power.
So I began to modify the design of the installation, trying to convey better and require less force to rotate the Sandscape.
At one moment, it happened to me that the mechanism inside the automatic watch can be a good way to show the small forces that passengers create. And the better thing was, if I reverse the input force of the escapement system, I could store the forces of small rotations in the spring, which helped to rotate the Sandscape inversely.
I began to find the watch escapement mechanism online and thankfully there was an amazing blueprint (see HERE). It just required some adjustments to let it accept the input force from the balance spring side instead of the main spring side. I redesigned a gear in Blender and 3D-printed the components we need so as to assembly them.
I used a rubber cord to connect the gear on the watch system and our sandscape. In that way, the forces from the small rotations of sandscape can be accumulated on the main spring of the system.
I made a dismountable foundation for the Sandscape. I 3D-printed a component (lower left picture below) that was inserted in the foundation so that it could hang the Sandscape vertically on the stepper motor. For now, the most part of fabrication had been done. I began to combine the people counting algorithm and Arduino codes together to see if it worked.
The test code (file3) showed that message could be transmitted to the Arduino. However, the number of the stepper motor rotated was unmatched with the number of people that were detected. I checked the code twice and realized that, if the message was sent when the motor executed its rotation task, those messages would be lost. And it happened a lot when a large amount of people were detected in a short time.
To solve the problem, there were at least two ways I could think of. The first one was that I could pop in the number of detected people in a queue one by one, and pop out in a certain slower speed to allow the rotation tasks being finished. The second one was that, keeping the time of loop (in Arduino) constant so that it seldom missed the message that computer sent.
I tried the first one and it worked (see final code). And later I found an example written by the designer of the easy drive that was similar with my second solution. I was really inspired by that thought, so I tried it as well. It turned out that the motor required a relatively long delay (1 microsecond) to let it work, which made fast message transmission (9600Hz) impossible. Compared two of solutions, I finally used the first one and here was the demo.
The Kits I used:
- NEMA 14 bipolar stepper motor
- Power supply that provides constant voltage (> 9V)
- Sparkfun easy driver
- Arduino Nano 33 IoT
Arduino Code Here
Cornfield Chase -- Hans Zimmer