Machine learning (ML) methods have been used to model complex dynamical systems, such as heating, ventilation, and air conditioning (HVAC) systems, to overcome the difficulty and high cost of modeling these systems using physical principles. However, ML-based methods often require large amounts of data, have poor generalization performance, and lack physical consistency. Physics-informed machine learning (PIML) has recently been introduced to overcome these drawbacks by incorporating physical laws into learning. There is, however, an unmet need to evaluate commonly used PIML methods to demonstrate their benefits and compare their performance in practical applications. In this comparative study, we evaluated various PIML methods and physical properties for modeling HVAC systems using real data. We considered physics-informed neural network methods and constrained Gaussian process methods, as well as physical properties that can be easily obtained in practice, such as smoothness, boundedness, and monotonicity. Our results showed the substantial benefits of PIML in improving model accuracy and data efficiency, and allowed us to compare the different PIML methods and physical properties to provide meaningful conclusions and recommendations for applying PIML in practice.