Robots @ SIT

Items required for this project:

• A computer with Windows 10 / Mac OS [M1 Chip Devices not tested yet; use at your own risk]
• Visual Studio Code (use this link to download)
• EV3 Brick (with 6 AA Batteries)
• MicroSD Card (Minimum capacity of 4GB and a maximum capacity of 32GB)
• MicroSD Card Reader (if your computer does not have an SD card slot)
• Mini USB cable (included with Lego Mindstorms Set)
• A MicroUSB Flasher Tool (use this link to download)

Follow the instruction of this link to setup your development environment.

Step 1: Assembling of MindCub3r robot

The robot MindCub3r can be assembled with the following guide from David Gilday - MindCub3r Assembly Guide using EV3 Home Set

Step 2: Setting up SD card and Python

In order to utilize and program the robot in Python, MicroPython is needed. MicroPython is Python for Microcontrollers, In this case, MicroPython for EV3 is needed. The MicroPython image and guide for EV3 can be seen in the following link - MicroPython image and flashing to SD card guide

Detail Steps

1. Download the EV3 MicroPython microSD card image from the link mentioned above in the Introduction. There is no need to unzip the file
2. Download and install a microSD card flashing tool. In our case, we used balenaEtcher
3. Insert the microSD card into your computer or card reader. In our case, a 16GB SD card is used
4. Launch the flashing tool and install the downloaded EV3 card image into the microSD card. Once installed, eject the SD card
5. Ensure that the EV3 Brick is turned off. Important: Attach a tape to the back of your SD card to easily remove the card from the EV3 brick. Insert the SD card into the EV3 brick
6. Turn on the EV3 brick and wait for it to boot up until it reaches the menu screen

Step 3: Connect EV3 Brick to the Internet via USB cable

Internet connection is needed to update the software as well as to install the dependencies and Python code. The guide can be found here, Internet Connection via USB Guide

Detail Steps (Windows)

1. Using the mini-USB cable that came with the EV3, connect the the EV3 brick to your computer. The computer will take a minute to install the EV3 driver.
2. Open Devices and Printers in the Control Panel under Hardware and Sound.  (Or just type Devices and Printers in the start menu.) The computer will detect the EV3 as Remote NDIS Compatible Device.
3. Right-click the Remote NDIS Compatible Device and select Network Settings. This takes you to the Network and Sharing Center.
4. In the Network and Sharing Center, make a note of which connection is your Internet connection (Access type: Internet) and then click on Change adapter settings.
5. There will be a connection that says Remote NDIS Compatible Device. This is the EV3. Let’s rename it so it is easy to identify. Right-click the adapter and select Rename. In our case, we renamed it to EV3.
6. To share our Internet connection with the EV3, double-click the Internet network connection that you noted earlier (not the one we just renamed, but the one with the Access type: Internet). This will show the connection status.
7. If you have more than one other connection, you will want to verify that this connection is indeed the Internet connection. Then, click on the Properties button.
8. In the window that opens, select the Sharing tab and check the box that says Allow other network users to connect through this computer’s Internet connection. If you have more than two network connections, it will ask for a “Home networking connection”. If you see this, select your EV3’s network adapter from the list (this will be called EV3 if you renamed it earlier in the guide).
9. Click OK when you are done and close the status window as well. Windows will automatically reconfigure your other network connections.
10. Now, we need to do some setting up on the EV3. On your EV3 Brick, go to Networking and select Manage connections…, then select the Wired connection.
11. Check the box for Connect automatically. This way you don’t have to connect manually after you reboot. Go to IPV4, select Change, cycle through the list to select DHCP. Then select Connect. State should change to Online to indicate that your EV3 is connected to the Internet.

Notes

• Connecting to the Internet is hit or miss, you may have to keep resetting the network shared connection by turning off and on the Allow other network users to connect through this computer's Internet connection at your computer, and reconnecting the wired connection at your EV3 Brick
• How to check if EV3 Brick is connected to the internet: On the EV3 Brick, go to Networking > Manage connections > Wired > IPV4, in the display, Gateway should contain an IP address. If it does not, your EV3 Brick is not connected to the internet. Additionally, you should be able to ping the EV3 Brick from your computer.

Step 4: Connect to EV3 terminal using Secure Shell (SSH)

Using a SSH client such as PuTTY to connect to the EV3 Brick to install additional software and run the program. In our case, PuTTY is used as our SSH client. PuTTY is an open source SSH and telnet client. PuTTY can be downloaded here, Download PuTTY. The steps to connect PuTTY to the EV3 Brick can be found here, SSH connection to EV3 Brick.

Detail Steps

1. Download a SSH client in your computer. In our case we use PuTTY. Once downloaded, run PuTTY
2. In the PuTTY Configuration window, type in ev3dev for the “host name”. Then click the Open button to connect
3. Once you are connected, type in the ev3dev username (robot) and the password (maker) if you haven’t changed it yet and then you should be logged in.

Notes

• The first time you connect, you'll get a warning about the new fingerprint. This is normal. Just click *Yes* to continue. You won't see this again unless you re-flash your SD card
• Default username: robot
• Default password: maker

Step 5: Download and installation of software

The Python program for the Rubik's cube solver that we will be using is here, Mindcuber Code Program. In that link, there is a guide to the installation and running of program. With the internet connection connected at the EV3 Brick and SSH terminal established, use the SSH terminal to perform the below steps.

Detail Steps

Installing kociemba

kociemba is a two-phase algorithm for solving of 3x3x3 Rubik's cube, created by Herbert Kociemba. The algorithm is briefly explained here, Herbert Kociemba's Algorithm This algorithm is then ported to a Python package with two equivalent implementations in C and Python. The library that we are using is from here, dwalton76's kociemba python library, which was originally created by muodov. Firstly, update of the operating system and installation of the dependencies of kociemba is performed before installing kociemba.

In your SSH terminal, perform the below steps:

1. sudo apt-get update
2. sudo apt-get install build-essential libffi-dev
3. cd ~/
4. git clone https://github.com/dwalton76/kociemba.git
5. cd ~/kociemba/kociemba/ckociemba/
6. make
7. sudo make install

Installing rubiks-color-resolver

This library is found here, dwalton76 rubiks-color-resolver python library. The purpose of this library is to resolve the detected Red, Green, Blue (RGB) values to the six cube colors of Rubik's cube. This library analyzes all RGB values and assign one of the six colors of the cube to each square. It then uses the Traveling Salesman algorithm to sort the colors. It is intended to work with kociemba. Firstly, pip for Python 3 will be installed, also known as pip3. Using pip3, rubiks-color-resolver package is then installed from its git repository.

In your SSH terminal, perform the below steps:

1. sudo apt-get install python3-pip
2. sudo pip3 install git+https://github.com/dwalton76/rubiks-color-resolver.git

Installing MINDCUB3R program

The program is mentioned at the download and installation of software section. There are two different code programs, namely calibrate_white.py and mindcuber.py. The purpose of the calibrate_white.py is to allow the robot to measure and calibrate its color sensor by measuring the RGB values of the white center of the Rubik's cube. It is a good idea to run calibrate_white.py whenever you move to a room with different lighting. The mindcuber.py is the main program where it controls the robot, scans the colors of the 6 different faces of the Rubik's cube, performs algorithms to solve the Rubik's cube, and controls the robot to turn the Rubik's cube to the solved state. Firstly, the MINDCUB3R repository is cloned from Github. Then a command to kociemba will be made to initialize the kociemba program.

In your SSH terminal, perform the below steps:

1. cd ~/
2. git clone https://github.com/ev3dev/ev3dev-lang-python-demo.git
3. cd ~/ev3dev-lang-python-demo/robots/MINDCUB3R/
4. kociemba DRLUUBFBRBLURRLRUBLRDDFDLFUFUFFDBRDUBRUFLLFDDBFLUBLRBD

The fourth step is essentially to run the kociemba program by giving a series of string representation of the colors of each of the 54 squares of a 3x3x3 cube. The letters represents different moves which is known as the Rubik's Cube Notation. An explanation of the notation can be found here, Rubik's Cube Notation. With the first run of the kociemba program, it takes some time to process, ~30 seconds, where it build a series of tables that caches to the filesystem. After the first run, the kociemba program will then run faster.

Step 6: Running of MINDCUB3R program

calibrate_white.py

As mentioned previously, this program is to calibrate the robot's color sensor by measuring the RGB values of the white center of the Rubik's cube. It is advised to run this program if you move to a room with different lighting. It would be more accurate if a piece of white paper is attached on the white center of the Rubik's cube for this calibration section. This is because the white center of the Rubik's cube contains other colors which can affect the reading of the color sensor.

To run this program, perform the following steps in your SSH terminal:

1. cd ~/ev3dev-lang-python-demo/robots/MINDCUB3R/

2. ./calibrate_white.py

mindcuber.py

As mentioned previously, this is the main program where it controls the robot to compute and solve the Rubik's cube. It takes some time for it to start as well as the computation of the solution of the Rubik's cube. Before starting the program, if you have attached a white paper on the white center of the Rubik's cube for the calibrate_white.py, remove the paper as this affects the flipping of the cube.

To run this program, perform the following steps in your SSH terminal:

1. cd ~/ev3dev-lang-python-demo/robots/MINDCUB3R/
2. ./mindcuber.py

Do note that the MINDCUB3R drains battery quickly and is not perfect as some issues can arise. Some issues detected were:

• Inaccurate color reading by the color sensor
  • Red & Orange (Similar colors)
  • Blue & Green (Similar colors)
• Bad lighting, where it can affect the color sensor
• Robot flipper not robust enough to flip the Rubik's cube, causing the cube to remain at the same position affecting the rest of the process

Changing the speed of flipper in mindcuber.py

It was discovered that the default program sometimes fail due to the flipper not able to flip the Rubik's cube. An adjustment to the flipper speed was made and it shows an improvement to the flipping of the cube. The robot was tested three times and it successfully flips the Rubik's cube til completion of every trial.

To change the speed of the flipper, perform the following steps in your SSH terminal:

1. cd ~/ev3dev-lang-python-demo/robots/MINDCUB3R/
2. nano mindcuber.py
3. Press and hold the "down" key on your keyboard til you see two variables, flip_speed = 300 and flip_speed_push = 400. Change the value of both to flip_speed = 700 and flip_speed_push = 800
4. Once changed, press "CTRL-X" on your keyboard to exit
5. It will issue a confirmation message on whether you want to save your changes. Press "y" on your keyboard to save the changes
6. You can proceed and run the program as described at the mindcuber.py section

Explanation of calibrate_white.py code

The calibrate_white.py code can be found here, calibrate_white.py code.

With the imported MindCuber class, the program uses its functions to perform following tasks:

• Runs wait_for_cube_insert, where it uses the robot's infrared sensor to detect if the Rubik's cube is present on the robot's Rubik's cube holder. The infrared sensor detects based on the distance of the cube. Once the Rubik's cube is detected, it uses the robot's flipper arm to position the Rubik's cube for the color sensor to scan and moves the flipper arm away afterwards.
• Adjusts the color sensor arm to the middle square of the Rubik's cube. It then calls the calibrate_white() method of the ColorSensor library from ev3dev as seen from here, calibrate_white() method. This method determines the maximum RGB raw values of the white square of the Rubik's cube and assigns the RGB values to red_max, green_max, and blue_max variables, which helps to scale future RGB raw values.
• Writes the red_max, green_max and blue_max variables in a txt file which will be used by the main program. The program then removes the color sensor arm back to the original position, and shuts down the robot.

Explanation of mindcuber.py code

The mindcuber.py code can be found here, mindcuber.py code.

With the start of this program, a MindCuber() class object is created that initializes various variables and methods. Python class and instance variables are present in the MindCuber() class. These are object-oriented programming concepts which you can read it in more details in this link, Python Class Variables vs. Instance Variables.

After initializing the variables and methods, the program perform following tasks:

• Checks whether the Rubik's cube is present on the turntable by using the infrared sensor distance detection. Once the Rubik's cube is detected, it uses the flipper to position the cube in the expected position.
• The program scans the 6 faces of the Rubik's cube utilizing the various motors and the color sensor. Upon every square, the RGB value detected by the color sensor is recorded and saved in the order of the scan_order list. The program then uses the imported RubiksColorSolverGenericmethod and input the scanned information. With the scanned information, the program utilizes kociemba and algorithms to derive the solution of the Rubik's cube.
• With the solution data, the program then resolves the Rubik's cube. Utilizing the flipper and turntable, the Rubik's cube is then sorted to the derived solution. Once completed, the robot is then shutdown which stops the various motors and sensors. The Rubik's cube has now been solved by the robot!

Below is the explanation of each class and variable in this program. 

MindCuber() Python Class Variables

MindCuber() Python Class Instance Variables & Methods