Dear Editor,
Herein is my response to manuscript ID XXXXX, entitled “ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning” by Qianwen Wang and colleagues to the ACM Conference on Human Factors in Computing Systems.
The authors present an interactive visualization tool that supports machine learning experts in analyzing the automatic results and in refining the search space of AutoML. This research is well-suited for the remit of the conference.
General speaking, the hypothesis of Automatic Machine Learning is meaningful and foresight. The application of AutoML is beneficial for the workers at the Machine Learning field. It is helpful for the experts in the relevant field using less time, less effort than before to find a high-performance algorithm for their works.
I recommend weak accept, pending Case Study, Expert Interview, and User Study.
Basic reporting
Figures
- All the figures legible, relevant, and integrated into the text.
Data
- The supporting data included in the manuscript and in a relevant repository.
- The data are associated with their provenance.
- The presented data use a formal, accessible, shared and broadly applicable language for knowledge representation.
Experimental design
- The research was conducted the regular ways to prove the relevant questions, such as investigated experts in Machine Learning field, an organised case study with related experts, conducted a user study with 13 users.
Validity of the findings
- The conclusions supported by the results.
- The results contribute to the research field, positive feedbacks received from the investigation, case study and user study.
General comments
- The autors use appropriate structure.
- The language used appropriately for a scientific publication.
Abstract
- The abstract concise, it conveys the main research findings.
Introduction
- This covers the published literature sufficiently.
- It provides enough context to place the current research.
- The history of the research conveyed at all for historical significance.
- It finishes with a paragraph summarising the relevance of the current research.
Materials and methods
- The methods are clear and easy to follow.
- The methods are replicated.
- The source materials and data openly available, and is appropriate justification provided.
Results
- The results presented in a coherent fashion.
- The results reported in a way that is supported by the data.
Discussion
- The user studies are limited because of only 13 participants.
- In the expert interview and user study, the testing results are all good performance, which is means overfitting.
Conclusions
- The conclusions supported by the results.
- They concise and written in an impactful way.
Additional comments
- The authors didn't explain where were the figures come from, how did they conduct the experiment and what kind of data to support their results.
Grades
- [Significance] 5
- [Novelty] 3
- [Soundness] 5
- [Evaluation] 4
- [Clarity] 6
- [Confidence] 5
Congratulations to the authors on a great piece of work, and I look forward to seeing their research.
Sincerely,
Tian Wang