Integration Guide Yaskawa BPS 1.12

Integration guide for Yaskawa Robots with Bin Picking Studio version BPS 1.12.

1. Robot Controller Setup

1.1 IP Configuration

Start the Robot Controller in a Maintenance Mode by holding the Main Menu button while turning the main switch on. You should hear a short beep during the boot. The Maintenance Mode screen is shown in the figure below:

Switch the Security Level from Editing to Management Mode. Browse to System → Security

And select Management Mode from the list:

The password for accessing Management Mode is 9999 9999 9999 9999 (16 x 9 without spaces):

Note

the three keys in the top status bar - this indicates that - Management Mode is Active.

To configure the IP address of the Robot Controller, browse to System → Setup → Option Functions and select LAN Interface Setting to enter the Network Configuration pane:

Select Manual Setting for LAN2 Port and amend the IP Address to meet your network requirements:

Note

It is recommended to plug the Ethernet cable into the LAN2 port.

Note

Subnet Mask 255.255.255.0 equals 24 bit subnet mask representation. See this table for more combinations: https://dnsmadeeasy.com/support/subnet

An example of matching Network configuration on Vision Controller side

• Vision Controller IPv4 Address: 192.168.1.1 / 24

• Robot Controller IPv4 Address: 192.168.1.2 / 24 (as configured in previous step)

1.2 Enabling MotoPlus functionality

If MotoPlus functionality has not been already enabled on Robot Controller, browse to System → Setup → Option Functions

Switch MOTOPLUS Func State from NOT USED to USED.

The MotoPlus APL icon should now become visible on the left pane.

Enter the MotoPlus Func. Settings:

Enable the MotoPlus Autostart option:

1.3 Loading of Motoplus Binary (.out)

The Photoneo Yaskawa Interface consists of

• MotoPlus binary - PhotoneoModule_BPS_1.12.0.out

• INFORM files - Request INFORM Jobs + Program INFORM Job Templates

Based on the type of your robot controller, use the .out file from the appropriate folder (DX100, DX200, FS100, YRC1000) and make sure it is available at the root directory of USB Flash drive.

Motoplus binary is loaded using following steps:

Select MOTOPLUS Apl. → Load (User Application)

Select the PhotoneoModule_BPS_1.12.0.out file and confirm Load.

Note

Only single .out Photoneo binary can be uploaded at the same time. Check File List to make sure there is no other Photoneo MotoPlus binary uploaded.

The Maintenance Mode section of the configuration process is now complete.

Restart the Robot Controller back into Normal Mode.

Note

If you see the following error message after restarting the controller back to regular mode don’t panic. We will get rid of this error by configuring the user coordinate frame in section 1.7. Just reset the error and follow the steps below.

1.4 Loading of INFORM Jobs

Make sure all Inform jobs are available at the root directory of the USB flash drive. Browse to Ex. Memory → Load and Edit → Select All to select all available JOBS from INFORM folder as is shown in the figure below:

Hit Enter to start the transfer:

All JOB files should now be available in the Robot Controller.

1.5 Loading of VARNAME.dat

In order to populate names for variables used by Photoneo, VARNAME.dat system file is provided together with INFORM jobs. Loading this file will erase all existing names and populate ones used by Photoneo.

Make sure VARNAME.dat is available at the root directory of the USB drive. Go to Ex. Memory → Load

Select System Data

Select VARIABLE NAME → VARNAME.DAT file from the list, hit Enter button and confirm Loading.

After executing this operation, all variable comments will be erased and names of variables used by Photoneo will be populated (BYTE, INTEGER, POSITION).

1.6 Set IP address of Vision Controller on the Robot Side

As a next step we need to let the Yaskawa Robot Controller know what is the IP address of Vision Controller as configured on the Bin Picking Studio Network page. First 4 integer variables are used for this purpose:

IP Address configured in first 4 Integer variables must match Vision Controller IP address as defined on BPS Network page:

To confirm that Robot Controller is physically connected to Vision Controller and Network is configured properly use Ping feature from System Info → Network Utility:

Type IP address of Vision Controller to Host field and hit Execute button:

If a connection exists, Ping will return a “OK” message on all 4 attempts. You can also use Test Connection button on Photoneo Vision Controller Network page to Ping Yaskawa Robot from Photoneo Vision Controller

Based on the result of Test Connection (ping command) Robot Available or Robot Unavailable is returned

If the robot is unavailable, please check cabling, network settings on both sides and try again, make sure ethernet cables are connected to proper ports and if there is a switch used, it is not blocking the connection.

1.7 USER COORDINATE setup

The World origin frame for Bin Picking Studio sits at the intersection of J1 and bottom of the robot base (see image below). Yaskawa World/Base origin is located above this point - it is the intersection of J1 and J2 axis plane (See image below). In order to make the bin picking system and calibration working properly, the distance between these two origins needs to be compensated on the Yaskawa side.

Compensation is achieved by defining USER COORDINATE 1 with negative Z offset.

Choose user frame - press Robot → User Coordinate

Select first frame and press Display → Coordinate Data to enter

Set the Z coordinate value according to the Base Frame Offset Table below. For the purpose of writing this manual, we are using Yaskawa GP8 so the Z offset needs to be set to -330mm.

This configuration will “push” down Yaskawa World origin to match Photoneo World origin.

Base Frame Offset Table for currently available Yaskawa robot models:

Base Frame Offset Table for Yaskawa robot models - units[mm]	Base Frame Offset Table for Yaskawa robot models - units[mm]	Base Frame Offset Table for Yaskawa robot models - units[mm]	Base Frame Offset Table for Yaskawa robot models - units[mm]
GP12	-450	HC10DT-A	-275
GP180	-650	HC10DT-B	-275
GP180-120	-650	HC20DTP	-380
GP20	-505	MA2010	-505
GP25	-505	MH12	-450
GP25-12	-505	MH24-10	-505
GP35L	-540	MH24	-505
GP4	-330	MH3BM	-350
GP50	-540	MH3F	-290
GP7	-330	MH50-35	-540
GP70L	-540	MH50II-K00	-540
GP8	-330	MH5LSII	-330
GP88	-540	MH5SII	-330
HC10	-275	MotoMINI	-171
		SP180H	-650

Note

When a custom robot provided by Photoneo support is used and its Z offset value is not available in the offset table, check original robot documentation for Z value. Usually this number is clearly visible from robot working envelope drawing.

1.8 TOOL Frame

TOOL frame - Photoneo Bin Picking Studio reads Yaskawa Flange pose during calibration and hand eye scanning requests. By default TOOL 63 (the last one) is used under the assumption that it is all zeros (Ideal for Photoneo). Nothing needs to be changed on the TOOL side for successful calibration as long as Tool 63 is zeroed.

Note

Tools are only visible if Setup → Teaching Conditions → Tool No Switch is Set To Permit.

1.9 State Server + Visualization of Robot Pose

State Server is a MotoPlus backend routine that enables Photoneo Bin Picking Studio to read current Joint Poses + Cartesian Tool Position from Yaskawa robot. While joint poses are used for robot visualization purposes, Cartesian TCP data are essential for calibration as well as all Hand Eye scan requests. Current implementation sends data to Vision Controller with frequency 30Hz and is available on port 11004 by default.

If the MotoPlus backend was loaded properly and User Coordinate Frame 1 was set as described in chapter 1.7, Moto Plus state server is started automatically during the boot of the robot.

State Server can be useful for preliminary configuration checks for example visualizing current robot + gripper state on Environment Page as shown below using Robot Module Motion Mode.

If State client is connected it means that current Joint and Tool data are being streamed from Robot to Bin Picking Studio at 30 Hz rate. Visualization of robot pose on Environment Page as well as Calibration should work now.

You can visually verify whether correct tool pose is being reported if you start calibration and switch from Texture to Verification Tab and enable visualization of Tool Pose (robot controller) in the Axis menu. The frame marker should be centered on the flange with the Z axis pointing down and it should be identical to the Tool0 (robot model) frame.

2. Robot Module

Note

It is strongly recommended to read the Photoneo robotic API prior to this section.

2.1 Connection Procedures

Following 2 procedures are used for connection and disconnection of Motoplus backend from Vision Controller.

INFORM JOB	Description
PHO_CONNECT_TO_VC	INFORM Job used to notify MotoPlus backend to establish connection to Vision Controller
PHO_RECOVER	INFORM job used to notify MotoPlus backend to properly disconnect from Vision Controller. It also handles situations if disconnect happens during trajectory receive by allowing MotoPlus to read out all the data from Vision Controller before disconnecting.

2.2 Requests List

This section describes available API calls (Requests) supported by the Robot module. These procedures are intended for high-level control of the Bin Picking Studio application from INFORM program:

INFORM Job	Function	Input	Populates
PHO_BIN_LOCATOR	Placeholder for future Bin Localization Request	I005	I007
PHO_CALIB_ADD_POINT	Add Calibration Point Request	None	I007
PHO_CALIB_SAVE	Save Calibration Request at the end of Calibration	None	I007, I033, P009
PHO_CALIB_START	Start Calibration Request	I005, I006	I007
PHO_CALIB_STOP	Stop Calibration Request	None	I007
PHO_CHANGE_ENVI	Change Environment Request	I014	I007
PHO_CHANGE_BBOX	Change Bounding Box Request	I005, I036	I007
PHO_CHANGE_SOLUTION	Change Solution Request	I006	I007
PHO_INITIALIZE_VS	Initialize Vision System Request	I005, I008, I009	I007
PHO_PICK_FAILED_REQUEST	Request to Report Pick Failed back to Vision System	I005	I007
PHO_CAPTURE_REQUEST	Capture Scan Request - for meshing, doesn’t initiate localization	I005, I035*	I007
PHO_GET_STATUS_REQUEST	Get Vision Sys Status - Num of Localized, Pickable, Running/Finished	I005	I007, I016, I017, I018
PHO_GET_OBJECT_POSE	Request Cartesian Pose of the Object from Vision System instead of Joint Based Trajectory	I005	I007, P008
PHO_REUSE_SCAN_REQUEST	Reuse last captured scan for localization on different Vision System	I005	I007
PHO_GET_RUNNING_SOLUTION	Get Current Running solution	None	I007, I006
PHO_SCAN_REQUEST	Regular Scan Request - initiates localization	I005	None
PHO_START_SOLUTION	Start Solution Request	I006	I007
PHO_STOP_SOLUTION	Stop Solution Request	None	I007
PHO_TRAJ_REQUEST	Trajectory Request	I005	None
PHO_TRAJ_RECEIVE	Counterpart for Trajectory Request	None	I007, P010-P109
PHO_WAIT_FOR_SCAN	Counterpart for Scan and Capture Request	None	I007
PHO_COM_CHECK	Request for checking connection	None	I007

I005 Vision System ID, I006 Solution ID, I007 Error Data, I014 Environment ID, I016 Num of Localized, I017 Num of Pickable, I018 VS Running/Finished, I033 Calib Accuracy, I035 *Capture Gap Only used in Dynamic Meshing,I036 Bounding Box ID, P009 Calibration Result Pose, P008 Cartesian Object Pose, P010-P109 - Joint Trajectory Waypoints

2.3 INFORM Bin Picking Procedures

Following procedures are related to the actual pick execution. Gripper commands are empty by default and should be filled out by user with IO commands controlling gripper actions.

PHO_EXE_TRAJ_CNT/FINE holds the actual motion execution segments of MOVJ commands for every possible trajectory size from 1 to 100 waypoints. DO NOT EDIT!

PHO_PICK_OBJECT is the main bin picking procedure controlling execution of the entire picking sequence. DO NOT EDIT!

INFORM Job	Function
PHO_PICK_OBJECT	Main Bin Picking Procedure. DO NOT EDIT!
PHO_EXE_TRAJ_CNT	INFORM Job where the Joint Waypoint trajectory execution happens. It is a long job because there are IF THEN segments for each possible trajectory size from 1 to 100. The last waypoint of PHO_EXE_TRAJ_CNT has a NWAIT argument so the exact time of execution of gripper commands is not guaranteed after PHO_EXE_TRAJ_CNT.
PHO_EXE_TRAJ_FINE	INFORM Job where the Joint Waypoint trajectory execution happens. It is a long job because there are IF THEN segments for each possible trajectory size from 1 to 100. The last waypoint of PHO_EXE_TRAJ_FINE doesn’t have a NWAIT argument so gripper commands are executed after position is reached.
PHO_GRIPPER_ACTION	Internal procedure forwarding execution of gripper command to proper INFORM Jobs
PHO_GRIPPER_ATTACH	User defined procedure, empty by default, needs to be filled out by command making gripper attach the object. This procedure is called automatically during PHO_PICK_OBJECT execution if currently executed grasping method is configured to execute this method
PHO_GRIPPER_DETACH	User Defined procedure, empty by default, needs to be filled out by command making gripper detaching the object. This procedure is called automatically during PHO_PICK_OBJECT execution if currently executed grasping method is configured to execute this method
PHO_GRIPPER_USER_1	User Defined procedure, empty by default, can be used for custom operations during picking trajectory.This procedure is called automatically during PHO_PICK_OBJECT execution if currently executed grasping method is configured to execute this method
PHO_GRIPPER_USER_2	User Defined procedure, empty by default, can be used for custom operations during picking trajectory.This procedure is called automatically during PHO_PICK_OBJECT execution if currently executed grasping method is configured to execute this method
PHO_GRIPPER_USER_3	User Defined procedure, empty by default, can be used for custom operations during picking trajectory.This procedure is called automatically during PHO_PICK_OBJECT execution if currently executed grasping method is configured to execute this method

2.4 INFORM Examples List

There are several INFORM Example Templates available in BPS 1.12 module that demonstrate how to properly use requests listed in section 2.2 for various use cases:

INFORM Example	Description
PHO_MAIN_CALIBRATION PHO_MAIN_CALIBRATION_HAND_EYE	Template for Calibration procedure, first calibration of Vision System must be started and confirmed manually by the user on the Bin Picking side. Calibration procedure requires proper USER COORDINATE configuration as described in chapter 1.7. Teach all 9 poses required for calibration. If communication with the vision controller is established by PHO_CONNECT_TO_VC then PHO_CALIB_ADD_POINT request calls feed vision controller with current robot pose for each added point. If communication cannot be established, then it is possible to add robot poses manually on the vision side by typing values from the current robot pose from the pendant. General rule of thumb is that Calibration Error should be below 3mm. If it is more, then there is usually some systematic error (Robot not reporting proper pose, or calibration ball moved during transitioning between poses etc). If calibration error is below 3mm, save calibration on the Vision Controller side and quit calibration.
PHO_MAIN_AUTO_RECALIBRATION	After the first calibration is successful and automatic recalibration is allowed in Vision System settings, you can run automatic recalibration from the robot without even touching the vision system. Calibration ball or marker pattern must remain in the same spot as during initial calibration.This example adds PHO_CALIB_START, PHO_CALIB_SAVE and PHO_CALIB_STOP to the original PHO_MAIN_CALIBRATION example.
PHO_MAIN_BASIC	Very basic example for a statically mounted scanner where the user can see how to initialize a system, send scan request, request trajectory, receive trajectory as joint waypoints and execute paths. The provided example also demonstrates error handling - PHO_SCAN_REQUEST and PHO_TRAJ_REQUEST return error value to I007 - PHO_ERROR_DATA. Make sure you manually set Start and End Position prior running this INFORM job (Chapter 3.1) and fill out blank gripper commands (Chapter 3.2)
PHO_MAIN_BASIC_HAND_EYE	Same as PHO_MAIN_BASIC but for a Hand Eye mounted scanner. The only difference is that the scanning pose will be different than for the Static mount.
PHO_MAIN_MULTI_VS	Same as PHO_MAIN_BASIC but with switching between two Vision Systems. Integer variable I005 PHO_VS_ID is important in this case. Value in this variable controls which Vision System will be triggered or requested for trajectory. The first pick in the loop is executed using Vision System 1 and second pick in the loop is executed using Vision System 2
PHO_MAIN_CHANGE_ENVI	Same as PHO_MAIN_BASIC but demonstrates use of the PHO_CHANGE_ENV request. Integer variable I014 PHO_ENVI_ID is important here as it defines what environment state will be activated with the next PHO_CHANGE_ENVI request.
PHO_MAIN_CHANGE_SOL	Same as PHO_MAIN_BASIC but demonstrates use of all Solution Switching related requests. Integer variable I006 PHO_SOL_ID is important here as it defines what solution will be activated with the next PHO_START_SOLUTION or PHO_CHANGE_SOLUTION requests.
PHO_MAIN_GET_OBJECT_POSE	Same as PHO_MAIN_BASIC but instead of requesting trajectory, the robot is requesting the Cartesian Pose of the object. Useful in some situations like pick checks, slip sheet detection, simple picking tasks. Keep in mind that the system returns raw Cartesian Pose from localization with origin as defined in object STL. No gripping points or invariances are applied in this case. The result is stored to P008 - PHO_OBJ_POSE. There is no trajectory execution happening in this example, it is a simple navigation to Cartesian target.
PHO_MAIN_GET_STATUS	Same as PHO_MAIN_BASIC but demonstrates use of the PHO_GET_STATUS_REQUEST to monitor number of localized and ready to pick objects as well as the current state of vision system(Running/Finished). PHO_GET_STATUS_REQUEST can be used repeatedly in short intervals during localization. Integer variables I016 PHO_NUM_OF_LOC, I017 PHO_NUM_OF_PLAN, I018 PHO_VS_CUR_STATE are populated as a result of status request calls and are often used for more advanced decision making on when to start the pick procedure.
PHO_MAIN_MESHING_STATIC	An example of how to stitch multiple scans before initiating localization. It is useful for more complicated scenes or large objects. This example demonstrates a static approach with a robot stopping in each scanning position and triggering/capturing scan. State Server must be running with proper User coordinate configuration and B009 PHO_MESH_DYNAMIC needs to be set to 0. During the meshing process, each PHO_CAPTURE_REQUEST scan must be followed by PHO_WAIT_FOR_SCAN to finalize the request. When done with capturing, trigger regular PHO_SCAN_REQUEST to initiate localization - this request doesn’t trigger actual scan, it only initiates localization process. The total number of scans should be less than 10.
PHO_MAIN_MESHING_DYNAMIC	Similar to PHO_MAIN_MULTIVIEW_STATIC but in this case capturing happens while the robot is continuously moving. BPS integrates Photoneo Instant Meshing technology in combination with Parallel structured light technique provided by Motion Cam 3D. State Server must be running with proper User Coordinate configuration. Important Flags and Variables: B009 PHO_MESH_DYNAMIC, PHO_MESH_STARTED, I035 PHO_CAPTURE_GAP. This example demonstrates how to initiate a capturing procedure, allowing Motion Cam to trigger scans with mutual delays set by I035 PHO_CAPTURE_GAP while moving the robot through predefined waypoints. B009 - PHO_MESH_DYNAMIC must be set to TRUE to activate Dynamic Meshing mode, and I035 PHO_CAPTURE_GAP should be set to approximately 100-500 ms to throttle the scanning and not oversaturate system with scans. Robot position during first scan is essential for proper orientation of the final point cloud therefore robot motion is blocked by Photoneo interface until B010 - PHO_MESH_STARTED is set to 1 by Motoplus backend after first scan is received. Only the last PHO_CAPTURE_REQUEST in this dynamic case is followed by a PHO_WAIT_FOR_SCAN request. Regular PHO_SCAN_REQUEST must be sent at the end of capturing to initiate localization. This scan doesn’t flash any light, it only starts the localization process. The scanning trajectory should be smooth without large rotations and the scanned area needs to be in the field of view all the time to avoid losing track. The total number of scans should be less than 60.
PHO_MAIN_REUSE_SCAN	Same as PHO_MAIN_MULTI_VS but instead of sending regular scan requests, the new request called PHO_REUSE_SCAN_REQUEST is demonstrated here. This request is useful in situations where the scene hasn’t changed since the last scan was triggered but the user wants to repeat localization with a different configuration (searching for different objects, in different bounding boxes or with different settings). Scan for VS1 is regular and scan for VS2 is then reused from scan for VS1. Integer variable I005 PHO_VS_ID is used for reuse scans in the same fashion as for regular Vision System switching.
PHO_MAIN_CHANGE_BBOX	Same as PHO_MAIN_BASIC but demonstrates use of the PHO_CHANGE_BBOX request. Integer variable I005 PHO_VS_ID and I036 PHO_BBOX_ID are used to select the Vision System and the target Bounding Box ID. The example demonstrates switching between two bounding boxes during the picking loop.

2.5 BYTE Variables

This version of the BPS module uses following 10 Byte variables:

BYTE Var	Description	Set by
B000 PHO_CONNECT_VC	This flag is set from INFORM Job to notify MotoPlus backend to establish connection to Vision Controller	user
B001 PHO_CONNECTED	This flag is set internally by MotoPlus backend after connection to action server running on Bin Picking Studio side is established and reset after session is terminated	system
B002 PHO_NEW_REQ	This flag is set by request INFORM Job to notify MotoPlus backend that a new request has been initiated by the user. MotoPlus backend resets this flag after it gets processed on its side	user
B003 PHO_REQ_FINISHED	This flag is set by the MotoPlus backend to notify INFORM that a response for a recently submitted request has been received. The INFORM side resets this flag right after.	system
B004 PHO_PICK_ALLOWED	This flag is set by the MotoPlus backend at the end of PHO_TRAJ_RECEIVE to notify INFORM that trajectory data has been received properly and there is a green light for pick execution.	system
B005 PHO_OPER_READY	This flag is set by the MotoPlus backend to notify INFORM that the next segment of trajectory is ready for execution. Used only during PHO_PICK_OBJECT call	system
B006 PHO_OPER_FINISH	This flag is set by the INFORM during PHO_PICK_OBJECT to notify MotoPlus backend that the trajectory segment has been finalized and joint waypoints from the next segment can be copied to Position Variables.	system
B007 PHO_PICK_FINISH	This flag is set by the INFORM to notify MotoPlus backend that picking procedure has been completed so it can proceed to new request processing	system
B008 PHO_PICKING	This flag is set by the MotoPlus backend during the pick procedure and checked during the connection recovery routine.	system
B009 PHO_MESH_DYNAMIC	This flag activates Dynamic meshing mode. It must be set to TRUE before the first PHO_CAPTURE_REQUEST request and reset to FALSE right after reaching the final meshing waypoint to stop scanning sequence. Set to FALSE is using Capture in Static mode	user
B010 PHO_MESH_STARTED	This flag is set by MotoPlus backend to notify INFORM that first scan of the meshing sequence has arrived and predefined capture trajectory can be started on the INFORM side.	system

2.6 INTEGER Variables

This version of the BPS module uses following 51 Numeric Variables:

INT Variable	Description	Set by
I000 PHO_VC_IP_ADDR_1	First octet of IP address of Vision Controller	user
I001 PHO_VC_IP_ADDR_2	Second octet of IP address of Vision Controller	user
I002 PHO_VC_IP_ADDR_3	Third octet of IP address of Vision Controller	user
I003 PHO_VC_IP_ADDR_4	Fourth octet of IP address of Vision Controller	user
I004 PHO_REQ_TYPE_ID	Set by every request INFORM job to notify MotoPlus which request is being initiated	system
I005 PHO_VS_ID	Variable to control which Vision system ID is going to be used for the next request	user
I006 PHO_SOL_ID	Variable to control which Solution ID is going to be used for the next request	user
I007 PHO_ERROR_DATA	Populated as a result of request handling. See error table in API section for potential error codes. Set to 0 if the request was completed successfully.	system
I008 PHO_START_POS_ID	Variable used during PHO_INITIALIZE_VS request referencing which Position variable is going to be sent to Bin Picking Studio as a Starting Point for currently initialized Vision System	user
I009 PHO_END_POS_ID	Variable used during PHO_INITIALIZE_VS request referencing which Position variable is going to be sent to Bin Picking Studio as a Ending Point for currently initialized Vision System	user
I010 PHO_TRAJ_SIZE	Variable used during PHO_PICK_OBJECT for trajectory execution control	system
I011 PHO_NUM_OF_OPER	Variable used during PHO_PICK_OBJECT for pick execution control	system
I012 PHO_OPER_TYPE_ID	Variable used during PHO_PICK_OBJECT for pick execution control	system
I013 PHO_GRIP_ACTN_ID	Variable used during PHO_PICK_OBJECT for pick execution control	system
I014 PHO_ENVI_ID	Variable to control which Environment State is going to be used for next request	user
I015 PHO_TRAJ_MOVE_PL	Position Level for entire bin picking trajectory execution. Value range [0 - 8] 0 indicates no bending radius and 8 indicates the maximum bending radius	user
I016 PHO_NUM_OF_LOC	Number of already localized objects - gets populated by PHO_GET_STATUS_REQUEST	system
I017 PHO_NUM_OF_PLAN	Number of pickable objects - gets populated by PHO_GET_STATUS_REQUEST	system
I018 PHO_VS_CUR_STATE	Current Vision system status - gets populated by PHO_GET_STATUS_REQUEST	system
I019 PHO_RESERVED	Reserved for future use	system
I020 PHO_RESERVED	Reserved for future use	system
I021 PHO_TOOL_INV	Variable for Tool Invariance selected by the system for current pick	system
I022 PHO_GRIP_ID	Variable for Gripping Point ID selected by the system for current pick	system
I023 PHO_GRIP_INV	Variable for Gripping point Invariance selected by the system for current pick	system
I024 PHO_DIM_X	Variable for X dimension of the object currently being picked (only AI based solutions)	system
I025 PHO_DIM_Y	Variable for Y dimension of the object currently being picked (only AI based solutions)	system
I026 PHO_DIM_ROT_Z	Variable for Z rotation of the object currently being picked (only AI based solutions)	system
I027 PHO_NN_LABEL	Variable for NN Label (Neural Network object class label, populated for AI-based solutions)	system
I028 PHO_MAX_Z_HEIGHT	Variable for Max Z Height of the object currently being picked	system
I029 PHO_TILT	Variable for Tilt angle of the object currently being picked (milliradians)	system
I030 PHO_INFO_DATA_10	Reserved for future use - not populated by current info operations	system
I031 PHO_TOOL_ID	Robot Tool ID for PHO_GET_OBJECT_POSE request	user
I032 PHO_USER_COOR_ID	User Coordinate ID for PHO_GET_OBJECT_POSE_request	user
I033 PHO_CALIB_ERROR	Calibration Error from automatic recalibration procedure in micrometers	system
I034 PHO_RUNNING_SOL	Currently running Solution ID. Result of PHO_GET_RUNNING_SOLUTION request	system
I035 PHO_CAPTURE_GAP	Delay in milliseconds between consecutive scans in dynamic meshing mode	user
I036 PHO_BBOX_ID	Bounding Box ID	user
I037 PHO_RESERVED	Reserved for future use	system
I038 PHO_RESERVED	Reserved for future use	system
I039 PHO_RESERVED	Reserved for future use	system
I040 PHO_SPEED_CURR	Speed setting for currently executed segment	system
I041 PHO_SPEED_TRAJ_1	Speed setting for trajectory segment 1	user
I042 PHO_SPEED_TRAJ_2	Speed setting for trajectory segment 2	user
I043 PHO_SPEED_TRAJ_3	Speed setting for trajectory segment 3	user
I044 PHO_SPEED_TRAJ_4	Speed setting for trajectory segment 4	user
I045 PHO_SPEED_TRAJ_5	Speed setting for trajectory segment 5	user
I046 PHO_SPEED_TRAJ_6	Speed setting for trajectory segment 6	user
I047 PHO_SPEED_TRAJ_7	Speed setting for trajectory segment 7	user
I048 PHO_SPEED_TRAJ_8	Speed setting for trajectory segment 8	user
I049 PHO_SPEED_TRAJ_9	Speed setting for trajectory segment 9	user
I050 PHO_SPEED_TRAJ_10	Speed setting for trajectory segment 10	user

2.7 POSITION Variables

Most Position Variables used for Bin Picking Movements are in Pulse (Joint) Representation but there are some exceptions like P008 and P009 that remain in default Cartesian representation. Pulse Representation is mandatory for Start, End poses P000 to P007 + all Trajectory Waypoints P010 to P109 are also received as Pulse. Below is an example of Joint representation for P000 - start waypoint for VS1.

Note

If more than 4 pairs of position variables are needed for Vision System Initialization, use P110-P117

P000 - P007 are reserved for Start and End poses for Vision Systems 1-4. Start and End poses must be touched up before initializing the Vision System by PHO_INITIALIZE_VS request. PHO_MAIN templates reference these Position variables by IDs in I008 and I009 in the beginning of each PHO_MAIN example. (See image below).

Poses stored in these P variables are sent to Bin Picking Studio during the PHO_INITIALIZE_VS call and are used for bin picking trajectory calculations. Minimum configuration is one Start and one End pose. Multiple Vision Systems can share the same Start and End Pose.

P010 - P109 are reserved for Joint Waypoint execution of Trajectory segments. Basic bin picking trajectory consists of 5 major waypoints (Start, Approach, Grasp, Deapproach, End) and 4 segments connecting these major waypoints.Each segment can be up to 100 Joint Waypoints and needs to be loaded from the MotoPlus backend into Position Variables right before trajectory segment execution begins. So for basic trajectory, each Position Variable value can change 4 times during a single pick.

This version of the BPS module uses following 120 Position Variables:

Variable	Repre	Description	Set by
P000	PULSE	Variable reserved for VS1 Start Pose. Referenced by I008 START POS ID in PHO_MAIN templates	user
P001	PULSE	Variable reserved for VS1 End Pose. Referenced by I009 END POS ID in PHO_MAIN templates	user
…	PULSE	…	…
P006	PULSE	Variable reserved for VS4 Start Pose. Referenced by I008 START POS ID in PHO_MAIN templates	user
P007	PULSE	Variable reserved for VS4 End Pose. Referenced by I009 END POS ID in PHO_MAIN templates	user
P008	CART	Variable to store Cartesian Pose returned as a result of PHO_GET_OBJ_POSE. This is a raw localization pose with origin based on STL. No gripping points or invariances have any effect on this result.	system
P009	CART	Variable to store Calibration Pose from automatic recalibration invoked by PHO_CALIB_SAVE request. This pose can be useful for various checks or point reteaching especially in hand eye applications	system
P010	PULSE	Variable to store Joint Waypoints for Trajectory Segment received from Vision Controller	system
…	PULSE	…	system
P109	PULSE	Variable to store Joint Waypoints for Trajectory Segment received from Vision Controller	system
P110	PULSE	Additional variables reserved for Start and End Pose definitions	user
…	PULSE	Additional variables reserved for Start and End Pose definitions	user
P120	PULSE	Additional variables reserved for Start and End Pose definitions	user

3. Runtime

If the solution is configured, all Vision Systems are calibrated and all previous steps have been carried out, it is time to complete the final steps on the robot side and finally run the bin picking program from the robot.

3.1 Teaching Start and End Poses

Start and End poses must be touched up before running the main bin picking procedure. Start and End poses define the first and last bin picking trajectory waypoints and are transferred to Bin Picking Studio during initialization request. In this example we will define basic P000 and P001 above the center of the bin.

Click Variable → Switch Type to Position(ROBOT) and select P000.

In order to touch up the point, hit start Servo On Ready, jog robot to start position and hit Modify followed by Enter button to confirm new pose value. SLURBT values will be modified

Repeat the same for End Pose P001.

3.2 Gripper commands

Gripper INFORM jobs are empty by default and need to be configured by the user to execute proper gripper commands. What command gets executed in which phase of the picking operation is configured on the Grasping Methods Page in the Bin Picking Studio.

For example if Attach Procedure is defined at Grasp Waypoint as shown below,

then the user needs to fill out the blank PHO_GRIPPER_ATTACH.JBI in a way that when this JOB is automatically called during PHO_PICK_OBJECT.JBI execution, the gripper executes the correct operation.

It is also recommended to fill out the PHO_GRIP_DETACH.JBI and call this as a Start Waypoint procedure to ensure that gripper is deactivated at the beginning of picking operation.

3.3 Touch up local MOVJ positions

If you are using existing PHO_MAIN templates, scroll through the program and check for existing MOVJ commands (Scanning positions) and make sure the robot is in proper position during scanning (Out of scanning volume for statically mounted sensors or above the bin with hand eye carried mounts).

3.4 Runtime Prerequisites

Final pre deployment check before running bin picking interface from the robot side

Make sure that:

• Bin Picking solution is properly configured on the Vision Controller side

• Network Setup on Robot Side is completed and ping works in both directions

• All Vision Systems defined in solution are calibrated

• Start and End Pose for all Vision Systems touched up

• All local poses in MAIN INFORM Job you are about to run have been touched up properly

• Gripper procedures are prepared and working

3.5 Running MAIN INFORM Job

Deploy your solution. The Action Request Client (Robot) status on the Deployment page should be DISCONNECTED – from the Action Request Server, if the communication hasn’t been established yet.

Note

. It is strongly recommended to TEST START the application first before deploying to Auto.

Using Select button navigate to program you want to run, in this case PHO_MAIN_BASIC.JBI template:

Hold Interlock and Test Start keys to test start the program. If connection has been established properly you will see Action Request Client and Robot State Server status turn to CONNECTED. At this point the sensor should capture the first scan and localization should start localizing objects.

If there is a pickable object in the scene and the trajectory for the first object has been received by the robot controller, the robot should start moving towards the first object. You can check P010 and higher to see if these Position variables have been populated with joint data.

If everything looks fine, keep moving the robot towards the first target and check if the path is correct. At this point if the robot is too far from the object or pushes the object too deep, then make modifications on the Bin Picking Studio Tool Point configuration page.

If trajectories look fine, set up your own placing routine and slowly ramp up speed in Manual mode and switch to Auto mode eventually.

4. Error handling

If an error occurs during the execution of a request, the error value is stored in the variable I007 PHO_ERROR_DATA. All PHO_MAIN templates demonstrate basic error handling after PHO_SCAN_REQUEST and PHO_TRAJ_REQUEST but if more specific handling is needed, it can be implemented additionally. Error codes with their description and troubleshooting can be found here.

An example of Error Handling in TP after PHO_TRAJ_REQUEST request: