## 13.6 Identify aruco code block
This case uses the eye_in_hand mode and the camera to identify and locate the unique aruco code of OpenCV and obtain its spatial position coordinates through opencv and some of its related dependencies. Set a set of related actions for the manipulator and put the identified aruco blocks into the bucket. In the following chapters, the code implementation process of the whole case will be introduced in detail.
#### **一、Robot camera assembly**
* As the case of the manipulator is based on the *eye_in_hand* mode, the camera is required to be combined with the manipulator, so the front end of the head of the manipulator needs to be replaced with the front end with a camera with a screwdriver.
* In order to ensure the normal operation of the program, the six axes of the manipulator need to be calibrated when replacing the front end of the head of the manipulator. Align the six axis zero adjustment position, as shown in the figure below,Modify the *tools.py* of the *mycobot_ai* package and run it according to the following instructions. Make sure that the camera on the robot arm is as shown above after the program initializes the robot arm. Otherwise, please terminate the program and refit the camera.
```python
from pymycobot.mycobot import MyCobot
# The name of manipulator equipment shall be consistent with the actual name of manipulator equipment.
port = "/dev/ttyUSB0"
mc = MyCobot(port)
# Release the robot arm
# mc.release_all_servos()
# Calibrate the six axes of the manipulator
mc.set_servo_calibration(6)
```
#### **二、Case reproduction**
The above video operation can realize the *demo* of object block recognition and capture. Next, we will describe the operation process in the video in words:
1. Go to the *mycobot_ai* package in the *mycobot-ros* workspace through the file manager.
2. Right click to open the terminal.
3. Give permission to operate the manipulator, enter `sudo chmod 777 /dev/ttyU` and use the *tab* key to fill in the name of the manipulator equipment.
4. If the device name is not `/dev/ttyUSB0`, you need to change the *port* value in the *vision. Launch* file.
5. Enter `roslaunch launch/vision.launch` to open the *vision. Launch* file, which contains some core libraries and dependencies of *ROS*.
6. Create a *marker* in the *rviz* graphical interface and name it *cube*.
7. Type `ctrl+shift+t` in the command terminal to open another command window under the same directory.
8. Enter ` Python script / detect_ obj_ Color. Py ` open the color recognition program to realize color recognition and capture.
> If you don't know how to modify *port* value and create *marker*, please refer to：[ROS building block model](../2-preparation/3-build_cube.md)
**Matters needing attention**
1. Because the camera needs to be assembled by itself, the offset of the camera relative to the suction pump will be different. If the grabbing effect is not ideal, you can modify the offset by yourself.
#### **三、Code explanation**
* Recognizing *aruco* according to *opencv* will obtain a translation coordinate value *tvec* of *aruco* code relative to the camera and a rotation coordinate value *RVEC* relative to the camera.
Here we use its *tvec* value to obtain the coordinate value of the *aruco* object block relative to the camera.*pump_y*，*pump_x*.
```python
# Convert picture to gray scale
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Detection of aruco blocks
corners, ids, rejectImaPoint = cv.aruco.detectMarkers(
gray, self.aruco_dict, parameters=self.aruco_params
)
if len(corners) > 0:
if ids is not None:
# Get a series of information about aruco
ret = cv.aruco.estimatePoseSingleMarkers(
corners, 0.03, self.camera_matrix, self.dist_coeffs
)
# Get rotation and translation coordinate values
(rvec, tvec) = (ret[0], ret[1])
(rvec - tvec).any()
xyz = tvec[0, 0, :]
"""
Calculate the coordinate value of aruco block relative to suction pump;
pump_ y，pump_ X represents the total offset of the camera relative to the suction pump and the error occurred during recognition；
You can modify pump_ y，pump_ The X offset corrects the grab effect;
"""
xyz = [round(xyz[0]*1000+pump_y, 2), round(xyz[1]*1000+pump_x, 2), round(xyz[2]*1000, 2)]
for i in range(rvec.shape[0]):
# Draw aruco in the picture
cv.aruco.drawDetectedMarkers(img, corners)
cv.aruco.drawAxis(
img,
self.camera_matrix,
self.dist_coeffs,
rvec[i, :, :],
tvec[i, :, :],
0.03,
)
```
> The tutorial only explains the key and difficult points of the program. Please check the source code for specific implementation.
* As the case of the manipulator is based on the *eye_in_hand* mode, the camera is required to be combined with the manipulator, so the front end of the head of the manipulator needs to be replaced with the front end with a camera with a screwdriver.
* In order to ensure the normal operation of the program, the six axes of the manipulator need to be calibrated when replacing the front end of the head of the manipulator. Align the six axis zero adjustment position, as shown in the figure below,Modify the *tools.py* of the *mycobot_ai* package and run it according to the following instructions. Make sure that the camera on the robot arm is as shown above after the program initializes the robot arm. Otherwise, please terminate the program and refit the camera.
```python
from pymycobot.mycobot import MyCobot
# The name of manipulator equipment shall be consistent with the actual name of manipulator equipment.
port = "/dev/ttyUSB0"
mc = MyCobot(port)
# Release the robot arm
# mc.release_all_servos()
# Calibrate the six axes of the manipulator
mc.set_servo_calibration(6)
```
#### **二、Case reproduction**
The above video operation can realize the *demo* of object block recognition and capture. Next, we will describe the operation process in the video in words:
1. Go to the *mycobot_ai* package in the *mycobot-ros* workspace through the file manager.
2. Right click to open the terminal.
3. Give permission to operate the manipulator, enter `sudo chmod 777 /dev/ttyU` and use the *tab* key to fill in the name of the manipulator equipment.
4. If the device name is not `/dev/ttyUSB0`, you need to change the *port* value in the *vision. Launch* file.
5. Enter `roslaunch launch/vision.launch` to open the *vision. Launch* file, which contains some core libraries and dependencies of *ROS*.
6. Create a *marker* in the *rviz* graphical interface and name it *cube*.
7. Type `ctrl+shift+t` in the command terminal to open another command window under the same directory.
8. Enter ` Python script / detect_ obj_ Color. Py ` open the color recognition program to realize color recognition and capture.
> If you don't know how to modify *port* value and create *marker*, please refer to：[ROS building block model](../2-preparation/3-build_cube.md)
**Matters needing attention**
1. Because the camera needs to be assembled by itself, the offset of the camera relative to the suction pump will be different. If the grabbing effect is not ideal, you can modify the offset by yourself.
#### **三、Code explanation**
* Recognizing *aruco* according to *opencv* will obtain a translation coordinate value *tvec* of *aruco* code relative to the camera and a rotation coordinate value *RVEC* relative to the camera.
Here we use its *tvec* value to obtain the coordinate value of the *aruco* object block relative to the camera.*pump_y*，*pump_x*.
```python
# Convert picture to gray scale
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Detection of aruco blocks
corners, ids, rejectImaPoint = cv.aruco.detectMarkers(
gray, self.aruco_dict, parameters=self.aruco_params
)
if len(corners) > 0:
if ids is not None:
# Get a series of information about aruco
ret = cv.aruco.estimatePoseSingleMarkers(
corners, 0.03, self.camera_matrix, self.dist_coeffs
)
# Get rotation and translation coordinate values
(rvec, tvec) = (ret[0], ret[1])
(rvec - tvec).any()
xyz = tvec[0, 0, :]
"""
Calculate the coordinate value of aruco block relative to suction pump;
pump_ y，pump_ X represents the total offset of the camera relative to the suction pump and the error occurred during recognition；
You can modify pump_ y，pump_ The X offset corrects the grab effect;
"""
xyz = [round(xyz[0]*1000+pump_y, 2), round(xyz[1]*1000+pump_x, 2), round(xyz[2]*1000, 2)]
for i in range(rvec.shape[0]):
# Draw aruco in the picture
cv.aruco.drawDetectedMarkers(img, corners)
cv.aruco.drawAxis(
img,
self.camera_matrix,
self.dist_coeffs,
rvec[i, :, :],
tvec[i, :, :],
0.03,
)
```
> The tutorial only explains the key and difficult points of the program. Please check the source code for specific implementation.