As students don't start from scratch, it is interesting to create a discussion based on their prior knowledge. To do so, interaction between the teacher and the students is needed. An audience response system is an efficient way to get that \cite{Draper_2004}, but there are also interactions between the students themselves which improves students understanding \cite{Smith_2009}.
Peer instruction is conducted through gamification: the use of remote controls in class lets students test their own understanding of a concept and compare it to others \cite{mazur_peer_2014}. It also let the librarian apply a formative assessment on students prior knowledge.
But turning students into conscious incompetent students is not enough, because they are still dependant of the teacher. In order to bring them to the next level, conscious competent students (who know to do something, but still with effort), a one-shot lecture is not enough. To set the path to autonomy for students, a few beacons need to show them how to navigate alone after the course into the right direction.
Context
The Swiss Federal Institute of Technology (EPFL) library is dedicated to the 10,770 students registered this year, from Bachelor to PhD degree. In 2015, 5,205 Bachelor students, 2,646 Master students and 2,077 PhD students were registered. Students are enrolled in different majors (Architecture, Life Sciences, Mathematics, Physics, Civil Engineering, Chemistry, Management and Computer Sciences) to become engineers. More than 125 nationalities are present on campus and EPFL has 6,173 foreign students (57%).
At EPFL, Information Literacy, like all transferable skills, is not a credited course, even though students are required to know how to cite their sources in documents they write. Librarians are invited by teachers to help students on this task. Embedding a librarian into a course is an efficient way to help students at the right moment, when they need to know how to share their sources, what to put into their bibliography, how to cite the data they reused, the differences between citing an article and reusing a graph or an image, etc. in the document the teacher ask them to write.
Since 2013, teacher's requests are highly motivated by avoiding plagiarism into their class. Even if librarians provide 1-to-1 training, only few Bachelor-Master (BAMA) students contact the library for a personal training. So a one-shot lecture into a teacher's course represents the first and only meeting with the library training staff for a majority of BAMA students. To present both challenges regarding both citation and solutions to avoid plagiarism, librarians have one-shot presentation (usually 45 minutes). Considering class sizes ranging from 10 to 150 students, its a big challenge.
We created a quiz, based on previous presentations and former experience of the training team. At the beginning of the presentation, clickers are distributed to students in order for them to be directly involved. As soon as they answer to the first question, the results appears and is subject to discussion. Then, following the same template, the quiz as well as the finding's interpretation continue. The course consists into a question-results-discussion training. We ask to teachers and assistants to stay in class with us. They used to participate, adding remarks, mentioning the course guidelines and even doing the quiz with the students.
Method & Data
Data were collected from February 2013 to December 2017 using Turning Technologies' clickers. During these 21 sessions, 815 students answered a quiz of 12 questions about citation (see quiz in the appendix). The first aim of the quiz is to discuss with the students. Therefore, most of the question are trickier than they expect. After giving an incorrect or incomplete answer to a question, students, who thought they know, are a little bit frustrated and ask for an explanation. This is the starting point of the discussion.
In this MCQ, 4 questions have multiple answers allowed (MAA: Q1, Q2, Q10 & Q12), 7 questions have only 1 correct answer, true or false, (1CA: Q3, Q5, Q6, Q7, Q8, Q9 & Q8/9) and 2 questions have only 1 answer accepted (1AA: Q4 & Q11), even if both answers are correct (cf. tricky questions mentionned before) .
Some comments and additional information provided in this paper come from the discussions conducted in class, but the data presented here are only based on the students' answers to the quiz, not to the discussion. Data collected during the 21 sessions were cleaned and published on Zenodo (
10.5281/zenodo.290155).
Analysis
Only 32 students (4%) gave the 3 correct answers to Q1 (Why should you cite when you write a report?). 240 students (29%) gave an incomplete correct answer (one or two correct answers without a wrong one). 105 students (13%) only chose one or both wrong answers.
Students' answers to MAA Q1: Why should you cite? (n=815)ANSWER | NUMBER (RATE) |
attribute to the author | 524 (64%) |
reuse work | 276 (34%) |
help the reader | 367 (45%) |
write a longer document | 69 (8%) |
avoid plagiarism | 474 (58%) |
didn't answer | 67 (8%) |
86 students (11%) gave the 3 correct answers to Q2 (How to insert a section borrowed from another document?). 670 students (82%) gave an incomplete correct answer (one or two correct answers without the wrong one). 11 students (1%) only gave the wrong answer.
Students answers to MAA Q2: How to insert a section borrowed from another document? (n=815)ANSWER | NUMBER (RATE) |
italic | 330 (40%) |
bold | 51 (6%) |
quotation marks | 627 (77%) |
separate paragraph | 221 (27%) |
didn't answer | 94 (12%) |
86 students (11%) chose both answers to Q10 (You consult an article [1] in which a section from another article [2] (that you haven’t read) is cited. You would like to insert this section from article [2] in your report. Which source do you cite?) which is the best answer. 396 students (49%) gave a correct answer (even if there were a better one). 241 students (30%) only gave the wrong answer.
Students answers to MAA Q10: Do you cite the article where you read the quote or the original one? (n=815)ANSWER | NUMBER (RATE) |
both | 86 (11%) |
read | 310 (38%) |
original | 241 (30%) |
didn't answer | 178 (22%) |
91 students (12%) chose both references to Q12 (Which reference is correct? - two answers are provided) which is the best answer. As no answer was wrong, all 575 responding students (76%) were right. 182 students (24%) were wrong because they didn't answer.
Students answers to MAA Q12: Which reference is correct? (n=757)ANSWER | NUMBER (RATE) |
both | 91 (12%) |
A | 347 (46%) |
B | 137 (18%) |
didn't answer | 182 (24%) |
Table 5 and 6 show that there is no significant difference between one level of study and another and not between sections regarding the answers to the 1CA questions. Only 9 students scored 100% on all 1CA questions. As no answer was wrong, all the 575 responding students (76%) were right. 42 students (6%) scored 0%. 15 (2%) of them didn't answer any question.
Overall number of correct answers given by students to all 1CA questions by level (BA2: 2nd year Bachelor, BA3: 3rd year Bachelor, MA: Master)CORRECT ANSWERS | OVERALL | BA2 | BA3 | MA |
| (n=696) | (n=28) | (n=354) | (n=314) |
6 correct answers | 9 (1%) | 0 | 6 (2%) | 3 (1%) |
5 correct answers | 52 (7%) | 0 | 31 (9%) | 21 (7%) |
4 correct answers | 159 (21%) | 9 (32%) | 85 (24%) | 62 (20%) |
3 correct answers | 225 (30%) | 11 (39%) | 102 (29%) | 100 (32%) |
2 correct answers | 172 (23%) | 7 (25%) | 73 (21%) | 69 (22%) |
1 correct answer | 95 (13%) | 1 (4%) | 38 (11%) | 42 (13%) |
no correct answer | 42 (6%) | 0 | 19 (5%) | 17 (5%) |
Overall number of correct answers given by students to all 1CA questions (n=696) by section [CH: chemistry, ESC: school of criminal justice, GM, mechanical engineering, IF: financial engineering, SIE: environmental engineering, SV: life sciences]CORRECT ANSWERS | CH (n=67) | ESC (n=192) | GM (n=269) | IF (n=16) | SIE (n=105) | SV (n=47) |
6 correct answers | 0 | 1 (1%) | 3 (1%) | 0 | 3 (3%) | 2 (4%) |
5 correct answers | 2 (3%) | 8 (4%) | 23 (9%) | 2 (13%) | 13 (12%) | 4 (9%) |
4 correct answers | 14 (21%) | 42 (22%) | 42 (24%) | 0 | 26 (25%) | 10 (21%) |
3 correct answers | 23 (34%) | 61 (32%) | 61 (28%) | 8 (50%) | 30 (29%) | 17 (36%) |
2 correct answers | 14 (21%) | 50 (26%) | 50 (21%) | 5 (31%) | 16 (15%) | 8 (17%) |
1 correct answer | 12 (18%) | 22 (11%) | 32 (12%) | 1 (6%) | 10 (10%) | 4 (9%) |
no correct answer | 2 (3%) | 8 (4%) | 17 (6%) | 0 | 7 (7%) | 2 (4%) |
But there are differences between the 1CA questions. Q3, Q5 and Q7 get a majority of correct answers. Q6, Q8 and Q9 (as well as Q8/9) get a very few correct answers.
Correct answers rate by 1CA questionANSWERS | Q3 | Q5 | Q7 | Q6 | Q8 | Q9 | Q8/9 |
| (n=815) | (n=696) | (n=815) | (n=815) | (n=696) | (n=754) | (n=61) |
correct | 65% | 75% | 68% | 36% | 23% | 18% | 10% |
wrong | 23% | 12% | 17% | 49% | 60% | 63% | 77% |
didn't answer | 13% | 13% | 15% | 15% | 17% | 19% | 13% |
Discussion
In this discussion, we consider six students categories divided within three types of answers:
1. correct answers
- know the answer
- guess correctly
2. wrong answers
- think they know, but are wrong
3. no answer
- take too much time to answer
- doesn't know what to answer
- doesn't listen anymore
47% of the overall responding students gave the correct answer to 1CA questions, 38% gave the wrong one and 16% didn't answer.
MAA questions turned out to be complicated for students. If many of them gave at least one correct answer (Q1: 79%, Q2: 82%, Q10: 70%), very few of them gave all the correct answers (Q1: 4%, Q2: 11%, Q10: 11%). This shows that with a basic prior understanding of why and how to cite their sources, students struggle with getting for into details.
We didn't expect to see a difference between the students from various disciplines and it happened to be not contrasted indeed. But on the other hand, we have been surprised by getting no significant difference between the level of study. We thought Master students would give more correct answers compared to Bachelor students. The reason is probably that we meet the students for the first time and that plagiarism issues is a new subject for them. Bachelor 3 and Master students should not be expected to perform better at the quiz than other students from first or second year of Bachelor degree : it doesn't depend primarily on their level of study but on their former Information Literacy curriculum.
From the set of questions in our quiz, we distinguish theoretical questions from practical ones. If there is no difference for one class to another, there is in fact a huge difference from one question to another. Theoretical questions get a lot of correct answers (Q3: 65%, Q5: 75%, Q7: 68%), but practical questions about how practice get very few correct answers (Q6: 36%, Q8: 23%, Q9: 18%, Q8/9: 10%). This confirms that students have a basic understanding of the topic (they know they shouldn't plagiarise), but they don't know how to do concretly when they have to write an essay, a report or an article.
As the quiz let them realize that they don't know, turning them from "uncouscious incompetent" student into "conscious incompetent" student, they then ask for guidance to become "conscious competent". At that point, we give them some tips to be able to make up their mind correctly the next time they have to decide how to do when they struggle with a question regarding citation.
First, we repeat that they don't write a document for themselves, but for the reader. Every time, they face a question like "how should I..." or "what is the best way to..." regarding citation, they should think about what is the best for the reader.
We also put them into a context of being an author: for students, this position is a bit unusual. We want them to ask themselves, when I take a picture and reuse it, 'how do I cite myself?', to make them consider that their work is as original as another work done by another person (and vice versa). The idea behind it is to make students aware that scholarly communication is a part of scientific publications. The ethical questions are also discussed with teachers.
We also underline the limits sometimes very short between the dissemination of their work into the class and on the web. Some posters are put online, thesis are available to everybody, so student names are linked to these works. Therefore, these works really have to be compatible with the copyright rules of the environment where they are disseminated. The more their citation work is well done, the more students will be considered as competent persons. To make students responsible, we explain that mastering copyright issues will be useful for their careers. Even if they do not become researchers after their studies, they will have to deal with copyright issues into the publishing world.
At the end of the quiz, we recommend to the students to use our guide entitled Rational Bibliographic
\cite{bibliotheque_de_lepfl_rational_2015} and a website,
citation.epfl.ch, in order to support them with citation tasks. Our guide is centered on how to complete a reference. So that students can figure out what pieces of information are required for giving the reference of a book, a journal article, a PhD thesis, a report, a tweet or a patent. Many references from other document types are analyzed. We consider that the citation style is not the main point of the topic, and during the Q12 comments, we mention the existence of reference manager softwares (like Zotero) that students can download and use when writing their essay. As soon as a correct and complete reference is saved in the reference manager, the software does the job.
We also encourage students to contact the library for help. We answer to several questions by e-mail; few students ask for an individual training. Last year, some Bachelor and Master students joined us individually to Dr Zotero sessions (
http://library.epfl.ch/dr-zotero), help provided at the Rolex learning center, 1h per month, to use Zotero as a reference manager software.
Conclusion & Future Improvements
As our first goal is to have a discussion in class with the students we will not remove the tricky or unfair questions where they can't give a perfectly correct answer. We seek for some frustration to make them talk and to be able to check if some complicated aspect of citation are understood. We always have a very interesting discussion with them and we think it's the best way to help them learn to think by themselves.
That said, we could improve the quiz by letting students answer "I don't know" to distinguish those who think they know (but don't) from those who are aware that they do not know. It would be a relevant aspect to know for us and for themselves. Refining the types of answer, we could be able to detect the level of the students more precisely: we adapt the discussion depending on the consciouness of incompetent students. The quiz gives us basics to discussions, but the content of our discussions into the class is very flexible depending on the students. Our strategy is to give as content as students need or ask for it. The more they look interested, asking advanced questions or adding relevant remarks, the more we go further.
For us, one-shot sessions about citation are not enough to bring students a long way on the road to autonomy. Students needs to be evaluated specifically on citation parts, and to receive feedbacks from teachers and from librarians. We feel sometimes frustrated not being able to come back after a while, nor to add other activities later after the quiz when students are practicing what they learnt or when they are facing new problems. It is also very rare to assess our presentations. Feedbacks come from teachers, usually by e-mail after the course. But as we do not provide credited courses, we do not automatically receive these assessments. Considering that we are spending one hour with the students, we prefer to take more time for discussion than for evaluation.
As we know that most of the students still struggle with citing their sources, we would like to follow them throughout their whole curriculum to build a sustainable understanding in that field.
Acknowledgements
The authors would like to thank Caroline Salamin and Pascale Bouton, who collected the data from some sessions used in this paper. They also work on this topic with the authors for years.
Thanks to Béatrice Marselli who read and commented the article.
Appendix: Quiz questions
There are 3 versions of the French quiz. The original one contains 12 questions. A second one with 10 questions has been used in 2015 and 2016: question 5 was dropped and questions 8 & 9 were merged. Finally, a short version has been used once. It only contains questions 1, 2, 3, 6, 7, 9 & 10.
One version of the English quiz. It was translated from the French original version. Here is the list of the original questions in English.
1. Why should you cite when you write a report? (several answers may be correct)
A. Assign the ideas/words to their author
B. Reuse what was already done
C. Help the reader who wants to know more
D. Write a longer document
E. Avoid plagiarism
2. How to insert a section borrowed from another document? (several answers may be correct)
Highlight the borrowed section...
A. in italics
B. in bold
C. with quotation marks
D. in a separate paragraph
3. I have the right to copy-paste!
A. True
B. False
4. I have to cite a document even if it has not been published (yet).
A. True
B. False
5. I have to cite a document that I published previously.
A. True
B. False
6. I have the right to modify a quotation.
A. True
B. False
7. In the bibliography, I have to mention everything I have read to write my article/report.
A. True
B. False
8. I can reuse an image found on the web if I cite my source.
A. True
B. False
9. I have the right to reuse a graph found in an article/book if I cite my source.
A. True
B. False
10. You consult an article [1] in which a section from another article [2] (that you haven’t read) is cited. You would like to insert this section from article [2] in your report. Which source do you cite?
[1] Kim S et Dale BE 2004 Global potential bioethanol production from wasted crops and crop residues. Biomass & Bioenergy 26(4): 361-375
[2] Zaldivar J, Nielsen J, Olsson L Fuel 2001 Ethanol production from lignocellulose: a challenge for metabolic engineering and process integration. Applied Microbiology and Biotechnology 56 (1-2):17-34
11. To cite a webpage, I indicate the URL.
A. True
B. False
12. Which reference is correct?
A. MONTGOMERY, Douglas Carter, 2013. Design and analysis of experiments. 8e éd. Hoboken, N.J : John Wiley & Sons. ISBN 978-1-1180-9793-9.
B. 1. Montgomery, D. C. Design and analysis of experiments. (John Wiley & Sons, 2013).