Cardiovascular diseases are the leading causes of death, and blood pressure (BP) monitoring is essential for prevention, diagnosis, assessment, and treatment. Photoplethysmography (PPG) is a low-cost opto-electronic technique for BP measurement that allows the acquisition of a modulated light signal highly correlated with BP. There are several reports of methods to estimate BP from PPG with impressive results; in this study, we demonstrate that the previous results are excessively optimistic because of their train/test split configuration. To manage this limitation, we considered intra- and inter-subject data arrangements and demonstrated how they affect the results of feature-based BP estimation algorithms (i.e., XGBoost, LightGBM, and CatBoost) and signal-based algorithms (i.e., Residual U-Net, ResNet-18, and ResNet-LSTM). Inter-subject configuration performance is inferior to intra-subject configuration performance, regardless of the model. We also showed that, using only demographic attributes (i.e., age, sex, weight, and subject index number), a regression model achieved results comparable to those obtained in an intra-subject scenario. Although limited to a public clinical database, our findings suggest that algorithms that use an intra-subject setting without a calibration strategy may be learning to identify patients and not predict BP.