In this work, we study the data collection problem using a data mule in millimeter Wave (mmWave) sensor net-works. The data mule aims to collect data (from a redundant set of sensors) within a minimum expected time, constrained by a lower bound on the delivery probability. Due to multiple factors of uncertainty, such as duty-cycling of sensors and mmWave link attenuation due to random blockage effects, the data mule faces a non-trivial challenge of how many times to probe a sensor (exploit) and which sensor to probe (explore), in order to meet the aforementioned goal. The aforementioned (exploit/explore) problem is modeled using a partially observable Markov decision process. As the direct method of solving the model characterized by a continuous state space is computationally intensive, we present an alternate method by investigating the structural properties of the model. To this end, a simple threshold-based myopic optimal data collection algorithm is shown to exist, and the closed-form expression for the threshold is derived. The effectiveness of the proposed optimal algorithm is validated against state-of-the-art learning-based protocols through extensive simulations.

Device-to-Device (D2D) communication helps in increasing the coverage range and throughput in millimeter Wave (mmWave) networks. The performance of the mmWave D2D communication mainly depends on selecting the best neighbor device through a process of Beamforming Training (BT). The process of BT involves pointing beams of multiple resolutions at different angles to select the best neighbor device in terms of link quality. While the sender exhaustively performs BT with all the neighbors in naive BT, the Reinforcement Learning (RL) based techniques, on the other hand, employ exploration/exploitation strategies to intelligently search for the best neighbor.The state-of-the-art RL algorithms typically yield sub-optimal performance, by trading-off computational tractability over optimality. Tractable optimal solving methods are therefore critical in improving the performance of neighbor selection through an intelligent BT process. To this end, this work studies the problem structure of the mmWave BT neighbor selection process formulated as a Multi-Armed Bandit (MAB) problem and provides a simple and scalable optimal method to select mmWave D2D neighbors with the objective of maximizing throughput performance. With extensive simulations, the efficacy of the proposed framework is demonstrated.