多路配准
多路配准(multiway registration)是把多块几何体对齐到全局空间的过程。通常,输入是一组几何体(例如点云或 RGBD 图像),输出是一组刚性变换 ,使得变换后的点云 在全局空间中对齐。
Open3D 通过位姿图优化(pose graph optimization)来实现多路配准,其后端实现了 [Choi2015] 提出的技术。
教程代码的第一部分从文件中读取三个点云,对它们进行下采样并一起可视化。它们处于未对齐状态。
def load_point_clouds(voxel_size=0.0): pcds = [] demo_icp_pcds = o3d.data.DemoICPPointClouds() for path in demo_icp_pcds.paths: pcd = o3d.io.read_point_cloud(path) pcd_down = pcd.voxel_down_sample(voxel_size=voxel_size) pcds.append(pcd_down) return pcdsvoxel_size = 0.02pcds_down = load_point_clouds(voxel_size)o3d.visualization.draw_geometries(pcds_down, zoom=0.3412, front=[0.4257, -0.2125, -0.8795], lookat=[2.6172, 2.0475, 1.532], up=[-0.0694, -0.9768, 0.2024])

输出:
[Open3D WARNING] GLFW Error: Failed to detect any supported platform[Open3D WARNING] GLFW initialized for headless rendering.位姿图(pose graph)有两个关键元素:节点(node)和边(edge)。一个节点是一块几何体 ,关联着一个把 变换到全局空间的位姿矩阵 。集合 就是要优化的未知变量。PoseGraph.nodes 是一个 PoseGraphNode 列表。我们把全局空间设为 所在的空间,因此 是单位矩阵。其余位姿矩阵通过累加相邻节点之间的变换来初始化。相邻节点通常有较大的重叠,可以用 Point-to-plane ICP 进行配准。
位姿图的边连接两块相互重叠的节点(几何体)。每条边包含一个变换矩阵 ,用于把源几何体 对齐到目标几何体 。本教程使用 Point-to-plane ICP 来估计该变换。在更复杂的情况下,这一两两配准(pairwise registration)问题应通过全局配准来求解。
[Choi2015] 观察到,两两配准容易出错:错误的两两对齐数量甚至可能超过正确对齐的数量。因此,他们将位姿图的边划分为两类。里程计边(odometry edges)连接时间上相近的相邻节点,ICP 这类局部配准算法可以可靠地对齐它们;回环闭合边(loop closure edges)连接任意非相邻的节点,其对应关系由全局配准找到,可靠性较低。在 Open3D 中,这两类边通过 PoseGraphEdge 初始化器中的 uncertain 参数来区分。
除了变换矩阵 之外,用户还可以为每条边设置信息矩阵 。如果使用函数 get_information_matrix_from_point_clouds 来设置 ,则该位姿图边上的损失近似等于两个节点之间对应点集的 RMSE,并带有一个线过程权重。详见 [Choi2015] 的公式 (3) 至 (9) 以及 Redwood 配准基准。
该脚本创建了一个包含 3 个节点和 3 条边的位姿图。在这些边中,有 2 条是里程计边(uncertain = False),1 条是回环闭合边(uncertain = True)。
def pairwise_registration(source, target): print("Apply point-to-plane ICP") icp_coarse = o3d.pipelines.registration.registration_icp( source, target, max_correspondence_distance_coarse, np.identity(4), o3d.pipelines.registration.TransformationEstimationPointToPlane()) icp_fine = o3d.pipelines.registration.registration_icp( source, target, max_correspondence_distance_fine, icp_coarse.transformation, o3d.pipelines.registration.TransformationEstimationPointToPlane()) transformation_icp = icp_fine.transformation information_icp = o3d.pipelines.registration.get_information_matrix_from_point_clouds( source, target, max_correspondence_distance_fine, icp_fine.transformation) return transformation_icp, information_icp
def full_registration(pcds, max_correspondence_distance_coarse, max_correspondence_distance_fine): pose_graph = o3d.pipelines.registration.PoseGraph() odometry = np.identity(4) pose_graph.nodes.append(o3d.pipelines.registration.PoseGraphNode(odometry)) n_pcds = len(pcds) for source_id in range(n_pcds): for target_id in range(source_id + 1, n_pcds): transformation_icp, information_icp = pairwise_registration( pcds[source_id], pcds[target_id]) print("Build o3d.pipelines.registration.PoseGraph") if target_id == source_id + 1: # odometry case odometry = np.dot(transformation_icp, odometry) pose_graph.nodes.append( o3d.pipelines.registration.PoseGraphNode( np.linalg.inv(odometry))) pose_graph.edges.append( o3d.pipelines.registration.PoseGraphEdge(source_id, target_id, transformation_icp, information_icp, uncertain=False)) else: # loop closure case pose_graph.edges.append( o3d.pipelines.registration.PoseGraphEdge(source_id, target_id, transformation_icp, information_icp, uncertain=True)) return pose_graphprint("Full registration ...")max_correspondence_distance_coarse = voxel_size * 15max_correspondence_distance_fine = voxel_size * 1.5with o3d.utility.VerbosityContextManager( o3d.utility.VerbosityLevel.Debug) as cm: pose_graph = full_registration(pcds_down, max_correspondence_distance_coarse, max_correspondence_distance_fine)输出:
Full registration ...Apply point-to-plane ICP[Open3D DEBUG] ICP Iteration #0: Fitness 0.6258, RMSE 0.1566[Open3D DEBUG] Residual : 1.85e-02 (# of elements : 17029)[Open3D DEBUG] ICP Iteration #1: Fitness 0.6873, RMSE 0.1427[Open3D DEBUG] Residual : 1.34e-02 (# of elements : 18703)[Open3D DEBUG] ICP Iteration #2: Fitness 0.7258, RMSE 0.1364[Open3D DEBUG] Residual : 1.11e-02 (# of elements : 19751)[Open3D DEBUG] ICP Iteration #3: Fitness 0.7514, RMSE 0.1330[Open3D DEBUG] Residual : 1.02e-02 (# of elements : 20446)[Open3D DEBUG] ICP Iteration #4: Fitness 0.7743, RMSE 0.1296[Open3D DEBUG] Residual : 9.36e-03 (# of elements : 21070)[Open3D DEBUG] ICP Iteration #5: Fitness 0.7881, RMSE 0.1214[Open3D DEBUG] Residual : 7.82e-03 (# of elements : 21444)[Open3D DEBUG] ICP Iteration #6: Fitness 0.7959, RMSE 0.1142[Open3D DEBUG] Residual : 6.81e-03 (# of elements : 21657)[Open3D DEBUG] ICP Iteration #7: Fitness 0.8024, RMSE 0.1127[Open3D DEBUG] Residual : 6.56e-03 (# of elements : 21833)[Open3D DEBUG] ICP Iteration #8: Fitness 0.8026, RMSE 0.1108[Open3D DEBUG] Residual : 6.34e-03 (# of elements : 21840)[Open3D DEBUG] ICP Iteration #9: Fitness 0.7959, RMSE 0.1050[Open3D DEBUG] Residual : 5.79e-03 (# of elements : 21658)[Open3D DEBUG] ICP Iteration #10: Fitness 0.7816, RMSE 0.0914[Open3D DEBUG] Residual : 4.30e-03 (# of elements : 21268)[Open3D DEBUG] ICP Iteration #11: Fitness 0.7676, RMSE 0.0758[Open3D DEBUG] Residual : 2.52e-03 (# of elements : 20887)[Open3D DEBUG] ICP Iteration #12: Fitness 0.7573, RMSE 0.0682[Open3D DEBUG] Residual : 1.73e-03 (# of elements : 20606)[Open3D DEBUG] ICP Iteration #13: Fitness 0.7525, RMSE 0.0656[Open3D DEBUG] Residual : 1.48e-03 (# of elements : 20475)[Open3D DEBUG] ICP Iteration #14: Fitness 0.7500, RMSE 0.0645[Open3D DEBUG] Residual : 1.40e-03 (# of elements : 20407)[Open3D DEBUG] ICP Iteration #15: Fitness 0.7489, RMSE 0.0639[Open3D DEBUG] Residual : 1.36e-03 (# of elements : 20377)[Open3D DEBUG] ICP Iteration #16: Fitness 0.7482, RMSE 0.0635[Open3D DEBUG] Residual : 1.32e-03 (# of elements : 20358)[Open3D DEBUG] ICP Iteration #17: Fitness 0.7477, RMSE 0.0632[Open3D DEBUG] Residual : 1.31e-03 (# of elements : 20347)[Open3D DEBUG] ICP Iteration #18: Fitness 0.7476, RMSE 0.0631[Open3D DEBUG] Residual : 1.30e-03 (# of elements : 20342)[Open3D DEBUG] ICP Iteration #19: Fitness 0.7475, RMSE 0.0631[Open3D DEBUG] Residual : 1.29e-03 (# of elements : 20341)[Open3D DEBUG] ICP Iteration #20: Fitness 0.7475, RMSE 0.0630[Open3D DEBUG] Residual : 1.29e-03 (# of elements : 20339)[Open3D DEBUG] ICP Iteration #21: Fitness 0.7473, RMSE 0.0629[Open3D DEBUG] Residual : 1.29e-03 (# of elements : 20335)[Open3D DEBUG] ICP Iteration #22: Fitness 0.7473, RMSE 0.0629[Open3D DEBUG] Residual : 1.28e-03 (# of elements : 20334)[Open3D DEBUG] ICP Iteration #23: Fitness 0.7472, RMSE 0.0628[Open3D DEBUG] Residual : 1.28e-03 (# of elements : 20331)[Open3D DEBUG] ICP Iteration #24: Fitness 0.7471, RMSE 0.0627[Open3D DEBUG] Residual : 1.28e-03 (# of elements : 20329)[Open3D DEBUG] ICP Iteration #25: Fitness 0.7470, RMSE 0.0627[Open3D DEBUG] Residual : 1.27e-03 (# of elements : 20326)[Open3D DEBUG] ICP Iteration #26: Fitness 0.7469, RMSE 0.0626[Open3D DEBUG] Residual : 1.27e-03 (# of elements : 20325)[Open3D DEBUG] ICP Iteration #27: Fitness 0.7469, RMSE 0.0626[Open3D DEBUG] Residual : 1.27e-03 (# of elements : 20324)[Open3D DEBUG] ICP Iteration #28: Fitness 0.7469, RMSE 0.0626[Open3D DEBUG] Residual : 1.26e-03 (# of elements : 20323)[Open3D DEBUG] ICP Iteration #29: Fitness 0.7469, RMSE 0.0626[Open3D DEBUG] Residual : 1.26e-03 (# of elements : 20323)[Open3D DEBUG] ICP Iteration #0: Fitness 0.5852, RMSE 0.0139[Open3D DEBUG] Residual : 1.17e-04 (# of elements : 15924)[Open3D DEBUG] ICP Iteration #1: Fitness 0.6369, RMSE 0.0121[Open3D DEBUG] Residual : 7.15e-05 (# of elements : 17332)[Open3D DEBUG] ICP Iteration #2: Fitness 0.6395, RMSE 0.0102[Open3D DEBUG] Residual : 3.27e-05 (# of elements : 17402)[Open3D DEBUG] ICP Iteration #3: Fitness 0.6390, RMSE 0.0101[Open3D DEBUG] Residual : 3.13e-05 (# of elements : 17388)[Open3D DEBUG] ICP Iteration #4: Fitness 0.6392, RMSE 0.0101[Open3D DEBUG] Residual : 3.13e-05 (# of elements : 17392)[Open3D DEBUG] ICP Iteration #5: Fitness 0.6391, RMSE 0.0101[Open3D DEBUG] Residual : 3.13e-05 (# of elements : 17390)Build o3d.pipelines.registration.PoseGraphApply point-to-plane ICP[Open3D DEBUG] ICP Iteration #0: Fitness 0.5669, RMSE 0.1655[Open3D DEBUG] Residual : 1.78e-02 (# of elements : 15427)[Open3D DEBUG] ICP Iteration #1: Fitness 0.6107, RMSE 0.1603[Open3D DEBUG] Residual : 1.72e-02 (# of elements : 16617)[Open3D DEBUG] ICP Iteration #2: Fitness 0.6706, RMSE 0.1484[Open3D DEBUG] Residual : 1.47e-02 (# of elements : 18249)[Open3D DEBUG] ICP Iteration #3: Fitness 0.7393, RMSE 0.1341[Open3D DEBUG] Residual : 9.62e-03 (# of elements : 20117)[Open3D DEBUG] ICP Iteration #4: Fitness 0.8061, RMSE 0.1313[Open3D DEBUG] Residual : 7.39e-03 (# of elements : 21934)[Open3D DEBUG] ICP Iteration #5: Fitness 0.8341, RMSE 0.1120[Open3D DEBUG] Residual : 5.13e-03 (# of elements : 22697)[Open3D DEBUG] ICP Iteration #6: Fitness 0.8784, RMSE 0.0993[Open3D DEBUG] Residual : 3.11e-03 (# of elements : 23901)[Open3D DEBUG] ICP Iteration #7: Fitness 0.9177, RMSE 0.0846[Open3D DEBUG] Residual : 1.47e-03 (# of elements : 24972)[Open3D DEBUG] ICP Iteration #8: Fitness 0.9376, RMSE 0.0795[Open3D DEBUG] Residual : 9.79e-04 (# of elements : 25512)[Open3D DEBUG] ICP Iteration #9: Fitness 0.9463, RMSE 0.0789[Open3D DEBUG] Residual : 9.53e-04 (# of elements : 25750)[Open3D DEBUG] ICP Iteration #10: Fitness 0.9491, RMSE 0.0788[Open3D DEBUG] Residual : 9.87e-04 (# of elements : 25827)[Open3D DEBUG] ICP Iteration #11: Fitness 0.9499, RMSE 0.0787[Open3D DEBUG] Residual : 1.00e-03 (# of elements : 25849)[Open3D DEBUG] ICP Iteration #12: Fitness 0.9503, RMSE 0.0787[Open3D DEBUG] Residual : 1.01e-03 (# of elements : 25858)[Open3D DEBUG] ICP Iteration #13: Fitness 0.9504, RMSE 0.0788[Open3D DEBUG] Residual : 1.01e-03 (# of elements : 25862)[Open3D DEBUG] ICP Iteration #14: Fitness 0.9505, RMSE 0.0788[Open3D DEBUG] Residual : 1.01e-03 (# of elements : 25863)[Open3D DEBUG] ICP Iteration #15: Fitness 0.9505, RMSE 0.0787[Open3D DEBUG] Residual : 1.01e-03 (# of elements : 25863)[Open3D DEBUG] ICP Iteration #16: Fitness 0.9505, RMSE 0.0788[Open3D DEBUG] Residual : 1.01e-03 (# of elements : 25863)[Open3D DEBUG] ICP Iteration #0: Fitness 0.6932, RMSE 0.0152[Open3D DEBUG] Residual : 1.42e-04 (# of elements : 18863)[Open3D DEBUG] ICP Iteration #1: Fitness 0.7057, RMSE 0.0106[Open3D DEBUG] Residual : 3.39e-05 (# of elements : 19203)[Open3D DEBUG] ICP Iteration #2: Fitness 0.7052, RMSE 0.0104[Open3D DEBUG] Residual : 3.06e-05 (# of elements : 19190)[Open3D DEBUG] ICP Iteration #3: Fitness 0.7054, RMSE 0.0104[Open3D DEBUG] Residual : 3.07e-05 (# of elements : 19195)[Open3D DEBUG] ICP Iteration #4: Fitness 0.7055, RMSE 0.0105[Open3D DEBUG] Residual : 3.07e-05 (# of elements : 19198)[Open3D DEBUG] ICP Iteration #5: Fitness 0.7055, RMSE 0.0105[Open3D DEBUG] Residual : 3.07e-05 (# of elements : 19197)Build o3d.pipelines.registration.PoseGraphApply point-to-plane ICP[Open3D DEBUG] ICP Iteration #0: Fitness 0.7980, RMSE 0.1231[Open3D DEBUG] Residual : 1.23e-02 (# of elements : 15272)[Open3D DEBUG] ICP Iteration #1: Fitness 0.9232, RMSE 0.1132[Open3D DEBUG] Residual : 9.84e-03 (# of elements : 17668)[Open3D DEBUG] ICP Iteration #2: Fitness 0.9638, RMSE 0.0957[Open3D DEBUG] Residual : 4.76e-03 (# of elements : 18445)[Open3D DEBUG] ICP Iteration #3: Fitness 0.9648, RMSE 0.0865[Open3D DEBUG] Residual : 2.63e-03 (# of elements : 18464)[Open3D DEBUG] ICP Iteration #4: Fitness 0.9576, RMSE 0.0784[Open3D DEBUG] Residual : 1.34e-03 (# of elements : 18326)[Open3D DEBUG] ICP Iteration #5: Fitness 0.9506, RMSE 0.0755[Open3D DEBUG] Residual : 8.08e-04 (# of elements : 18191)[Open3D DEBUG] ICP Iteration #6: Fitness 0.9446, RMSE 0.0746[Open3D DEBUG] Residual : 5.87e-04 (# of elements : 18077)[Open3D DEBUG] ICP Iteration #7: Fitness 0.9400, RMSE 0.0740[Open3D DEBUG] Residual : 4.83e-04 (# of elements : 17989)[Open3D DEBUG] ICP Iteration #8: Fitness 0.9381, RMSE 0.0739[Open3D DEBUG] Residual : 4.41e-04 (# of elements : 17953)[Open3D DEBUG] ICP Iteration #9: Fitness 0.9373, RMSE 0.0737[Open3D DEBUG] Residual : 4.30e-04 (# of elements : 17937)[Open3D DEBUG] ICP Iteration #10: Fitness 0.9372, RMSE 0.0738[Open3D DEBUG] Residual : 4.29e-04 (# of elements : 17936)[Open3D DEBUG] ICP Iteration #11: Fitness 0.9372, RMSE 0.0739[Open3D DEBUG] Residual : 4.29e-04 (# of elements : 17936)[Open3D DEBUG] ICP Iteration #0: Fitness 0.7654, RMSE 0.0129[Open3D DEBUG] Residual : 9.10e-05 (# of elements : 14648)[Open3D DEBUG] ICP Iteration #1: Fitness 0.7605, RMSE 0.0109[Open3D DEBUG] Residual : 4.28e-05 (# of elements : 14553)[Open3D DEBUG] ICP Iteration #2: Fitness 0.7594, RMSE 0.0108[Open3D DEBUG] Residual : 4.23e-05 (# of elements : 14533)[Open3D DEBUG] ICP Iteration #3: Fitness 0.7592, RMSE 0.0108[Open3D DEBUG] Residual : 4.24e-05 (# of elements : 14528)Build o3d.pipelines.registration.PoseGraphOpen3D 使用函数 global_optimization 来执行位姿图优化。可选择两种优化方法:GlobalOptimizationGaussNewton 或 GlobalOptimizationLevenbergMarquardt。推荐使用后者,因为它具有更好的收敛性质。GlobalOptimizationConvergenceCriteria 类可用于设置最大迭代次数及各种优化参数。
GlobalOptimizationOption 类定义了若干选项。max_correspondence_distance 决定对应关系的阈值;edge_prune_threshold 是剪除离群边的阈值;reference_node 是被视为全局空间的节点编号。
print("Optimizing PoseGraph ...")option = o3d.pipelines.registration.GlobalOptimizationOption( max_correspondence_distance=max_correspondence_distance_fine, edge_prune_threshold=0.25, reference_node=0)with o3d.utility.VerbosityContextManager( o3d.utility.VerbosityLevel.Debug) as cm: o3d.pipelines.registration.global_optimization( pose_graph, o3d.pipelines.registration.GlobalOptimizationLevenbergMarquardt(), o3d.pipelines.registration.GlobalOptimizationConvergenceCriteria(), option)输出:
Optimizing PoseGraph ...[Open3D DEBUG] Validating PoseGraph - finished.[Open3D DEBUG] [GlobalOptimizationLM] Optimizing PoseGraph having 3 nodes and 3 edges.[Open3D DEBUG] Line process weight : 15.334500[Open3D DEBUG] [Initial ] residual : 1.037615e+00, lambda : 2.960068e+00[Open3D DEBUG] [Iteration 00] residual : 2.034300e-01, valid edges : 1, time : 0.000 sec.[Open3D DEBUG] [Iteration 01] residual : 1.455356e-01, valid edges : 1, time : 0.000 sec.[Open3D DEBUG] Delta.norm() < 1.000000e-06 * (x.norm() + 1.000000e-06)[Open3D DEBUG] [GlobalOptimizationLM] total time : 0.000 sec.[Open3D DEBUG] [GlobalOptimizationLM] Optimizing PoseGraph having 3 nodes and 3 edges.[Open3D DEBUG] Line process weight : 15.334500[Open3D DEBUG] [Initial ] residual : 1.455332e-01, lambda : 3.051329e+00[Open3D DEBUG] Delta.norm() < 1.000000e-06 * (x.norm() + 1.000000e-06)[Open3D DEBUG] [GlobalOptimizationLM] total time : 0.000 sec.[Open3D DEBUG] CompensateReferencePoseGraphNode : reference : 0全局优化对位姿图执行两遍。第一遍在考虑所有边的情况下优化原始位姿图的位姿,并尽可能区分不确定边中的错误对齐;这些错误对齐具有较小的线过程权重,会在第一遍之后被剪除。第二遍在排除这些边的情况下运行,产生紧密的全局对齐。在本例中,所有边都被视为正确对齐,因此第二遍立即终止。
可视化优化结果
Section titled “可视化优化结果”变换后的点云被列举出来,并使用 draw_geometries 进行可视化。
print("Transform points and display")for point_id in range(len(pcds_down)): print(pose_graph.nodes[point_id].pose) pcds_down[point_id].transform(pose_graph.nodes[point_id].pose)o3d.visualization.draw_geometries(pcds_down, zoom=0.3412, front=[0.4257, -0.2125, -0.8795], lookat=[2.6172, 2.0475, 1.532], up=[-0.0694, -0.9768, 0.2024])输出:
Transform points and display[[ 1.00000000e+00 1.62736205e-19 0.00000000e+00 -8.67361738e-19] [-5.41042295e-20 1.00000000e+00 1.08420217e-19 -8.67361738e-19] [-1.08420217e-19 0.00000000e+00 1.00000000e+00 0.00000000e+00] [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]][[ 0.84016209 -0.14644296 0.52218973 0.34786264] [ 0.0061365 0.96535921 0.26085233 -0.39421642] [-0.54230065 -0.21595382 0.81195686 1.73016434] [ 0. 0. 0. 1. ]][[ 0.9627171 -0.07182071 0.26080179 0.37666506] [-0.00193493 0.96225765 0.27213318 -0.48961689] [-0.27050331 -0.2624919 0.92624293 1.29777557] [ 0. 0. 0. 1. ]][Open3D WARNING] GLFW initialized for headless rendering.合并为单个点云
Section titled “合并为单个点云”PointCloud 提供了一个便捷的 + 运算符,可以把两个点云合并为一个。下面的代码在合并之后使用 voxel_down_sample 对点进行均匀重采样。这是合并点云后推荐的后处理步骤,因为它能够缓解重复点或过度密集的点。
pcds = load_point_clouds(voxel_size)pcd_combined = o3d.geometry.PointCloud()for point_id in range(len(pcds)): pcds[point_id].transform(pose_graph.nodes[point_id].pose) pcd_combined += pcds[point_id]pcd_combined_down = pcd_combined.voxel_down_sample(voxel_size=voxel_size)o3d.io.write_point_cloud("multiway_registration.pcd", pcd_combined_down)o3d.visualization.draw_geometries([pcd_combined_down], zoom=0.3412, front=[0.4257, -0.2125, -0.8795], lookat=[2.6172, 2.0475, 1.532], up=[-0.0694, -0.9768, 0.2024])
输出:
[Open3D WARNING] GLFW initialized for headless rendering.