This data set contains sequences of image observations of an Epiroc Boomer E2 underground drilling rig during mock-up operations. The images are aligned to joint state readings of encoders attached to each joint of each of the two drilling booms.
The data set is meant to be used for visual state estimation, and in particular to test the transfer of visual state estimation modules between different lightning conditions. Data is collected under two test conditions: with background illumination and with on-board illumination. More details in the accompanying paper Visual State Estimation in Unseen Environments through Domain Adaptation and Metric Learning [PDF coming soon].
About the data
The data set contains images from a MultiSense S2 stereo camera, including grayscale appearance images and depth maps. In addition, a fisheye lens RGB camera is available, both in raw format, as well as calibrated undistorted images. For each image there is an associated text file with joint state information. Semantic segmentation labels are generated by projecting an articulated URDF model into the stereo camera view. Files are of the format:
image_depth_N: depth map png. Pixel values indicate depth in millimeters.
image_fish_raw_N: Raw image from the fisheye camera.
image_fish_undist_N: corresponding rectified fisheye image.
image_label_N: semantic segmentation labels image.
image_rgb_N: left stereo camera view.
image_right_rgb_N: right stereo camera image.
image_js_N.txt: corresponding joint state estimates from the encoders. 7 values per boom, white space separated, base to tip and left boom followed by right boom.