Constellation Shaping in Optical Communication Systems
Licentiate thesis, 2021

Exploiting the full-dimensional capacity of coherent optical communication systems is needed to overcome the increasing bandwidth demands of the future Internet. To achieve capacity, both coding and shaping gains are required, and they are, in principle, independent. Therefore it makes sense to study shaping and how it can be achieved in various dimensions and how various shaping schemes affect the whole performance in real systems. This thesis investigates the performance of constellation shaping methods including geometric shaping (GS) and probabilistic shaping (PS) in coherent fiber-optic systems.

To study GS, instead of considering machine learning approaches or optimization of irregular constellations in two dimensions, we have explored multidimensional lattice-based constellations. These constellations provide a regular structure with a fast and low-complexity encoding and decoding. In simulations, we show the possibility of transmitting and detecting constellation with a size of more than 10^{28} points which can be done without a look-up table to store the constellation points. Moreover, improved performance in terms of bit error rate, symbol error rate, and transmission reach are demonstrated over the linear additive white Gaussian noise as well as the nonlinear fiber channel compared to QAM formats.

Furthermore, we investigate the performance of PS in two separate scenarios, i.e., transmitter impairments and transmission over hybrid systems with on-off keying channels. In both cases, we find that while PS-QAM outperforms the uniform QAM in the linear regime, uniform QAM can achieve better performance at the optimum power in the presence of transmitter or channel nonlinearities.

multidimensional modulation format

probabilistic shaping

coherent receiver

optical communications

geometric shaping

constellation shaping

Kollektorn, Canyon, Kemivägen 9, department of Microtechnology and Nanoscience (MC2), Chalmers.
Opponent: Tobias A. Eriksson at Infinera, Stockholm.


Ali Mirani

Chalmers, Microtechnology and Nanoscience (MC2), Photonics

Low-complexity geometric shaping

Journal of Lightwave Technology,; Vol. 39(2021)p. 363-371

Journal article

Lattice-based geometric shaping

European Conference on Optical Communication, ECOC,; (2020)

Paper in proceedings

Comparison of uniform cross QAM and probabilistically shaped QAM formats under the impact of transmitter impairments

Proceedings of 45th European Conference on Optical Communication,; (2019)

Paper in proceedings

Performance of Probabilistic Shaping Coherent Channels in Hybrid Systems

International Conference on Transparent Optical Networks,; (2020)p. 1-3

Paper in proceedings

Coupled fiber optic channels

Swedish Research Council (VR), 2019-12-01 -- 2023-11-30.

Polarization-aware fiber optic transmisson

Swedish Research Council (VR), 2016-01-01 -- 2019-12-31.

Signal shaping in optical communications—Beyond the Gaussian channel

Swedish Research Council (VR), 2018-01-01 -- 2021-12-31.

Areas of Advance

Information and Communication Technology

Nanoscience and Nanotechnology (SO 2010-2017, EI 2018-)

Driving Forces

Sustainable development

Innovation and entrepreneurship

Subject Categories


Communication Systems

Signal Processing


Basic sciences


C3SE (Chalmers Centre for Computational Science and Engineering)

Technical report MC2 - Department of Microtechnology and Nanoscience, Chalmers University of Technology: 439


Chalmers University of Technology

Kollektorn, Canyon, Kemivägen 9, department of Microtechnology and Nanoscience (MC2), Chalmers.


Opponent: Tobias A. Eriksson at Infinera, Stockholm.

More information