Probabilistic Constellation Shaping for OFDM ISAC Signals Under Temporal-Frequency Filtering

Journal article, 2026

Integrated sensing and communications (ISAC) is considered an innovative technology in sixth-generation (6G) wireless networks, where utilizing orthogonal frequency division multiplexing (OFDM) communication signals for sensing provides a cost-effective solution for implementing ISAC. However, the sensing performance of matched and mismatched filtering schemes can be significantly deteriorated due to the signaling randomness induced by finite-alphabet modulations with non-constant modulus, such as quadrature amplitude modulation (QAM) constellations. Therefore, improving sensing performance without significantly compromising communication capability (i.e., maintaining randomness), remains a challenging task. To that end, we propose a unified probabilistic constellation shaping (PCS) framework that is compatible with both matched and mismatched filtering schemes, by maximizing the communication rate while imposing constraints on mean square error (MSE) of sensing channel state information (CSI), power, and probability distribution. Specifically, the MSE of sensing CSI is leveraged to optimize sensing capability, which is illustrated to be a more comprehensive metric compared to the output SNR after filtering (SNRout) and integrated sidelobes ratio (ISLR). Additionally, the internal relationships among these three sensing metrics are explicitly analyzed. Building upon this, we further reveal that the normalized MSE can be interpreted as a penalty function version of the dynamic range, which is usually exploited to evaluate the behavior of delay-Doppler profiles. Finally, both simulations and field measurements validate the efficiency of proposed PCS approach in achieving a flexible S&C trade-off, as well as its credibility in enhancing 6G wireless transmission in real-world scenarios.