Accurate terrain mapping is of paramount importance for motion planning and safe navigation in unstructured terrain. LIDAR sensors provide a modality, in the form of a 3D point cloud, that can be used to estimate the elevation map of the surrounding environment. But, working with the 3D point cloud data turns out to be challenging. This is primarily due to the unstructured nature of the point clouds, relative sparsity of the data points, occlusions due to negative slopes and obstacles, and the high computational burden of traditional point cloud algorithms. We tackle these problems with the help of a learning-based, efficient data processing approach for vehicle-centric terrain reconstruction using a 3D LIDAR. The 3D LIDAR point cloud is projected on the ground plane, which is processed by a generative adversarial network (GAN) architecture in the form of an image to fill in the missing parts of the terrain heightmap. We train the GAN model on artificially generated datasets and show the method’s effectiveness by means of the reconstructed terrains.