This technical note presents proof of the convergence of the Alternating Direction Method of Multipliers (ADMM) for addressing the sharing problem when applied in conjunction with two algorithms: 1) the stochastic STEP-GP algorithm and 2) its variant named LGP, which includes adaptive uniform quantization. For the case using LGP, the coordinator can assign different quantization resolutions at each iteration and we assume that the number of bits that can be assigned is unrestricted and can go to infinity. This document describes and analyzes the two methods for integrating learning and uniform quantization into the ADMM to reduce its communication overhead and a general formulation of their communication decision method. The problems are formulated for a multi-agent setting.

The deep operator network (DeepONet) architecture is a promising approach for learning functional operators, that can represent dynamical systems described by ordinary or partial differential equations. However, it has two major limitations, namely its failures to account for initial conditions and to guarantee the temporal causality – a fundamental property of dynamical systems. This paper proposes a novel causal deep operator network (Causal-DeepONet) architecture for incorporating both the initial condition and the temporal causality into data-driven learning of dynamical systems, overcoming the limitations of the original DeepONet approach. This is achieved by adding an independent root network for the initial condition and independent branch networks conditioned, or switched on/off, by time-shifted step functions or sigmoid functions for expressing the temporal causality. The proposed architecture was evaluated and compared with two baseline deep neural network methods and the original DeepONet method on learning the thermal dynamics of a room in a building using real data. It was shown to not only achieve the best overall prediction accuracy but also enhance substantially the accuracy consistency in multistep predictions, which is crucial for predictive contro

Machine learning (ML) methods have been used to model complex dynamical systems, such as heating, ventilation, and air conditioning (HVAC) systems, to overcome the difficulty and high cost of modeling these systems using physical principles. However, ML-based methods often require large amounts of data, have poor generalization performance, and lack physical consistency. Physics-informed machine learning (PIML) has recently been introduced to overcome these drawbacks by incorporating physical laws into learning. There is, however, an unmet need to evaluate commonly used PIML methods to demonstrate their benefits and compare their performance in practical applications. In this comparative study, we evaluated various PIML methods and physical properties for modeling HVAC systems using real data. We considered physics-informed neural network methods and constrained Gaussian process methods, as well as physical properties that can be easily obtained in practice, such as smoothness, boundedness, and monotonicity. Our results showed the substantial benefits of PIML in improving model accuracy and data efficiency, and allowed us to compare the different PIML methods and physical properties to provide meaningful conclusions and recommendations for applying PIML in practice.

In networks consisting of agents communicating with a central coordinator and working together to solve a global optimization problem in a distributed manner, the agents are often required to solve private proximal minimization subproblems. Such a setting often requires a decomposition method to solve the global distributed problem, resulting in extensive communication overhead. In networks where communication is expensive, it is crucial to reduce the communication overhead of the distributed optimization scheme. Gaussian processes (GPs) are effective at learning the agents’ local proximal operators, thereby reducing the communication between the agents and the coordinator. We propose combining this learning method with adaptive uniform quantization for a hybrid approach that can achieve further communication reduction. In our approach, the GP algorithm is modified to account for the introduced quantization noise statistics due to data quantization. We further improve our approach by introducing an orthogonalization process to the quantizer’s input to address the inherent correlation of the input components. We also use dithering to ensure uncorrelation between the quantizer’s introduced noise and its input. We propose multiple measures to quantify the trade-off between the communication cost reduction and the optimization solution’s accuracy/optimality. Under such metrics, our proposed algorithms can achieve significant communication reduction for distributed optimization with acceptable accuracy, even at low quantization resolutions. This result is demonstrated by simulations of a distributed sharing problem with quadratic cost functions for the agents.

In distributed optimization schemes consisting of a group of agents connected to a central coordinator, the optimization algorithm often involves the agents solving private local proximal minimization sub-problems and exchanging data frequently with the coordinator to solve the global distributed problem. Such schemes usually cause excessive communication costs to the system, leading to the need for communication reduction for scenarios where communication is costly. The integration of Gaussian Processes (GPs) as a learning component to the Alternating Direction Method of Multipliers (ADMM hase been proven successful in learning each agent’s local proximal operators. Consequently, such a learning technique is effective to reduce the communication between the agents and the coordinator. A key element for the integration of GP in the ADMM algorithm is the mechanism upon which the coordinator decides when communication with an agent is required. The decision strategy used in this setting strongly affects the overall communication expenditure reduction and the ADMM’s algorithm performance. For that reason, we construct a general framework presenting a systematic querying mechanism as an optimization problem that balances the ideas ofmaximizinge the communication cost reduction while minimizing the prediction error. Motivated by such a framework, we propose different query strategies using the uncertainty measurement given by GP, to determine if the prediction is reliable enough to skip a communication round. We propose a joint query strategy following a simplification of the general framework that minimizesana L1 norm communication cost constraint by the trace of the joint uncertainty of the ADMM variables which is calculated using all agents’ prediction uncertainty. Additionally, we derive three different decision mechanisms (motivated by a confidence interval analysis) that make their decision by analyzing an uncertainty measurement for each agent individually. We integrate multiple measures to quantify the trade-off between the communication cost reduction and the optimization solution’s accuracy/optimality. The proposed methods can achieve significant communication reduction and good optimization solution accuracy for distributed optimization, as demonstrated by extensive simulations of a distributed sharing problem with quadratic cost functions for the agents.