Navigating the Gray Areas of AI and ChatGPT as a Psychology Student

AKA Can I use ChatGPT? Should I use ChatGPT? HELP!?

_{By Sylvia Gray}

The release of numerous artificial intelligence (AI) tools last year has the potential to substantially alter the way we work and study. While the debate rages about whether AI is a blessing or a curse [1, 2], many students are understandably curious and nervous. Can university students make use of these tools? How can a tool like ChatGPT help with studies? Are there any ethical concerns about using AI tools on assignments? Can ChatGPT do my assignments?! Let’s explore them together!

What AI tools are available for psychology students?

Although the news regarding AI tools has recently exploded due to ChatGPT, a variety of AI tools have existed for years. In this article, we’re going to explore the following (click the links to jump ahead, if you are interested in a specific app): Grammarly, ProWritingAid, Hemingway App, CiteThisForMe, ResearchRabbit, Elicit, and ChatGPT.

Writing Tools: Grammarly, ProWritingAid, and Hemingway App

Grammarly, ProWritingAid, and the Hemingway App are three AI-powered tools which focus on improving writing. Each tool typically provides more in-depth analysis than spell-checkers in word-processing programs.

Q. What are the differences between Grammarly, ProWritingAid, and Hemingway App? Grammarly’s focus (and strength) lies in correcting grammar and spelling, while also providing suggestions on word choice. ProWritingAid focuses more on improving style and tone. The Hemingway App focuses strongly on readability (aka ensuring the information can be understood by a target audience).

Q. What are the benefits of Grammarly, ProWritingAid, and the Hemingway App?

Improve the clarity of your writing by simplifying complex sentences, avoiding unclear phrases and ambiguity
Improve the conciseness of your writing by removing redundant words or phrases (though we’d be remiss if we didn’t point out our own Guide to Reducing Word Count, which has a list of words to search-and-destroy, as well as a checklist and specific tips relevant to psychology studies

Q. What are the risks or downsides of using Grammarly, ProWritingAid, or Hemingway App? Some academics prefer that students do not use the tools, fearing it may encourage them to develop reliance and stifle their learning. These beliefs are typically more common for the fields where learning outcomes include grammar, syntax, etc (e.g., degrees in English or Writing).

They lack context. These apps cannot incorporate the rules and context set by your unit and/or university. Thus, they can sometimes make incorrect suggestions such as replacing pronouns when they may be permitted or encouraged.
They prefer varied language; assignment assessors do not. Grammarly encourages authors to introduce varied language, to prevent repetition and a dry reading experience. However, that can lead to inappropriate suggestions that may cost marks in assessments (e.g., replacing the names of key variables with synonyms, or renaming a central theory).
They may oversimplify your arguments. While readability is important, remember the purpose of an assessment is to convey your understanding of a topic. Overuse of the Hemingway App may see an author simplify content to the extent that the reader (aka your marker) may not be able to determine that you do possess the required knowledge.

Q. How are universities responding to Grammarly, ProWritingAid, and the Hemingway App? Can I use these tools? Always check whether your university has policies about specific programs, as universities have differed in their responses. For example, University of Adelaide encourages students to use Grammarly; however, some universities and/or professors have banned the use of similar tools in their courses. As a general indicator, most Australian universities are at least accepting of students’ use of Grammarly and similar programs.

Q. How can I use Grammarly, ProWritingAid, and/or the Hemingway App whilst adhering to academic integrity policies?

Learn to write in the active voice
Check the formatting of your APA citations
Get real-time feedback on your grammar, spelling, and syntax
Learn how and where to reduce superfluous words or phrases whilst preserving your arguments

Q. Which one should I use (Grammarly, ProWritingAid, or the Hemingway App)?

Grammarly is the best option for a focus on grammar, spelling, and word choice.
If you’re aiming to improve your writing for a general, non-scientific audience and want to ensure it’s as easy to understand as possible, the Hemingway App is the best choice.
If you’re looking for support in improving sentence structure, word choice and tone, go with ProWritingAid.

Q. Does the Study Support team use Grammarly, ProWritingAid, or the Hemingway App? Yes, we certainly do! It depends on the purpose; however, I (Sylvia) have the Grammarly app synced with word processing and Google Docs. It’s a brilliant safeguard against accidental typos!

Reference management: CiteThisForMe

CiteThisForMe is a citation and reference manager. There is a wide range of citation and reference managers — it seems everyone has an opinion on them! Some programs, like EndNote or Mendeley, are recommended by universities (or even required for some assignments). Others are more user-friendly and popular with students (e.g., Zotero). However, the only reference manager we’re aware of that currently incorporates AI is CiteThisForMe. Aside from helping students store, organise, and cite various sources, CiteThisForMe claims to also extract data to create references.

Q. What are the benefits of using CiteThisForMe?

Create citations across a range of citation styles. This is especially helpful if you find your electives using a different citation style with which you may be less familiar, and perhaps less willing or interested in learning.
Create a reference list from the citations you’ve created. Provided you’ve done the work with citation creation, creating the reference list should be straightforward. However, only the paid version allows students to export this into a Word file.
Save your citations online (though only for seven days on the free version of the account).

Q. What are the risks or downsides of using CiteThisForMe?

Incorrect citations or reference lists and not knowing APA formatting guidelines well enough to spot mistakes. Too often we see students using reference managers and assuming the output is correct. However, that’s not always the case. Common mistakes we often see produced by reference managers include: using the incorrect citation style; writing titles in ALL CAPS; duplicating references; and incorrectly formatting first names and group names. While we were producing the article, we tested CiteThisForMe with a basic reference and another error it produced was including “Retrieved on…” which is not required for APA 7th edition.
We know it’s a chore to learn APA and we too would rather you spend your time learning the psychology-related concepts; however, learning the basics to at least be able to proofread the output of a reference manager is still recommended to overcome this risk of using a reference manager.
Reference managers don’t identify or flag missing information. These tools will simply ignore missing information and craft the reference/citation without it. While this is handy, if some information is truly unattainable or irrelevant, it can also be dangerous if the student doesn’t spot the missing data themselves. To overcome this, consider making a checklist of data required for each reference and ensuring all relevant information is input into the reference manager.
Auto-completed information is not always accurate or complete. While the AI auto-generation of missing data can certainly be helpful, it can produce incorrect information. To overcome this, double-check the data it has found and ensure it’s accurate. (Though honestly, in the time it takes to double-check the citation, it may be easier to simply produce it yourself.)

Q. How are universities responding to CiteThisForMe? Can I use CiteThisForMe as a psychology student? Responses to CiteThisForMe (and other reference managers) are mixed. Some academics believe students should complete all references manually, to learn APA guidelines. Others use reference managers themselves and encourage students to explore and find the option that suits them best. Some university assignments require students to submit an EndNote file to demonstrate they have completed the required tasks. As before, please double-check whether your university or course has a policy about reference managers.

Q. How can I use CiteThisForMe whilst adhering to academic integrity policies?

Store your references and citations online and have access regardless of which computer you’re using (note, you can only store for 7 days on the free version)
Organise references by assignment, topic, or any other category of your choice
Let AI generate the required information for the reference/citation (though remember to double-check it!)

Q. Does the Study Support team use CiteThisForMe or any other reference manager? No, we do not. For a few reasons:

We find it faster to generate the list ourselves, rather than initiate the AI process, double-check the data, correct the data, initiate the integration with our file, double-check the output, correct the output, and correct information in the reference manager (again).
We have experienced the sudden and total loss of work within reference management systems at time-critical moments (e.g., finalising a meta-analysis) and wouldn’t wish that absolutely hellish experience (e.g., starting from scratch mere hours before a deadline) on anyone.
We see too many errors in the works produced by the reference managers to feel comfortable and confident that they would improve our efficiency.

Research tools: ResearchRabbit and Elicit

ResearchRabbit helps students and academics locate papers and authors, identify new and relevant literature, and visualise how research papers and authors relate to other publications and authors. It creates a visual representation of the interactions between papers and authors, a wonderful way of exploring a specific corner of the research world.

Elicit calls itself an “AI Research Assistant”. Similar to other databases and search tools, it helps you locate relevant papers based on your keywords. However, it goes above-and-beyond other search tools by summarising content and extracting key information from papers.

Q. What are the benefits of using ResearchRabbit or Elicit?

Easily map relevant research as it relates to important papers, authors, and/or topics.
Discover related research that may not have been previously identified in traditional searches by keyword.
Keep an eye out for emerging research and/or researchers related to your topic of interest.

Q. What are the risks or downsides of using ResearchRabbit and/or Elicit?

Copying AI-generated content is plagiarism. As Elicit.org generates content, this naturally carries the risk of academic integrity concerns (e.g., plagiarism). To minimise this risk, ensure you do not ever copy/paste any content from Elicit into your own writing. It’s important to paraphrase and summarise content yourself.
Data may not be complete. It is unlikely that ResearchRabbit will have all published articles at all times. Thus, there is a chance of missing content if this is the only tool utilised for your search.
Data may not be accurate. It is also unlikely that all articles and authors in ResearchRabbit are entirely and accurately mapped to other papers and/or academics. This means there’s a risk of missing papers and/or authors that may be of interest if this is the only tool utilised. Additionally, the information generated by Elicit may be incorrect or complete. Always double-check.

Q. How are universities responding to ResearchRabbit and Elicit? Can I use ResearchRabbit and/or Elicit as a psychology student? These are relatively new tools; as such, we’re not yet aware of any university policies about the apps. However, as the tools are similar in purpose (i.e., locating research) to other tools academics often recommend (i.e., EBSCOhost and Scopus), we imagine they will be acceptable to most academics. However, as the AI in Elicit generates content, we can also imagine some academics may strongly discourage (or even ban) the use of Elicit to avoid academic integrity concerns. While ResearchRabbit does not generate any content, this tool is likely to be more widely accepted by academics; however, there is certainly the possibility that some may view this program as hindering the students’ learning of traditional methods of searching for research. Thus, we do suggest having a conversation with your university about these specific programs to ensure you’re on the right side of their academic integrity policies.

Q. How can I use ResearchRabbit and/or Elicit whilst adhering to academic integrity policies?

Brainstorm research questions, keywords, and relevant search terms.
Discover new terms that may be helpful in your search.
Discover researchers that may be pioneers or experts in your topic(s).
Review published literature in an accessible manner.
Fall down the ResearchRabbit-hole (har har) and get lost in the world of relevant research

Q. Does the Study Support team use ResearchRabbit or Elicit? We certainly do! Sylvia has used ResearchRabbit for previous work-related research. Elicit is a new tool we’re adding to our toolkit.

The game changer: ChatGPT

ChatGPT is an AI that has “learned” from online and published content to predict the answers to specific questions, based on natural language processing. In other words, it’s a chatbot that doesn’t have pre-programmed answers; it generates responses to each new question. We tend to think of it as autocomplete on steroids.

Q. Why all the hype about ChatGPT?

ChatGPT is seen as revolutionary, particularly when compared to other tools, as it draws on a large dataset and is able to adapt and create responses to unique, often very specific questions. Many people view this as a serious threat to Google and other search engines; in fact, since we started writing the article, Bing has incorporated ChatGPT into their search and Google has announced their own chatbot for searches.

Sensational headlines have also indicated that ChatGPT has passed law exams and effectively completed an MBA with a passing mark. Naturally, this made us curious. Could ChatGPT complete the assessments of Psychology students? Could it pass psychology exams?

We challenged ChatGPT to complete two undergraduate psychology assessment tasks and were quite surprised by the results. First, we took a research report assessment (appropriate for a second-year undergraduate) and converted it into a series of questions. We fed the prompts to ChatGPT in a single chat (so it could use the context from previous questions) and modified them when it was clear ChatGPT didn’t understand the task. Finally, we reviewed ChatGPT’s responses compared to the literature and rubric. While ChatGPT was able to provide some decent background information for the first paragraph of the Introduction and it gave some reasonable definitions for key terms and theories, ChatGPT could not provide relevant or sufficient critical thinking. When prompted to do so, it often went off track, which dramatically impaired the majority of the report. Compared to the rubric, we estimated that the submission might get a passing mark for enough sections that the student would overall Pass (50%-59%), but we struggled to see the assignment getting into the Credit range (60%).

Next, we put ChatGPT to the test – literally! We asked ChatGPT to complete a multiple-choice statistics and research methods exam which we estimated was appropriate for a third-year undergraduate student. We had higher hopes here because the options were more limited and we assumed the narrowing scope would keep ChatGPT on track. However, ChatGPT surprised us by completely failing the exam – earning only 30%. When it was prompted to provide rationale for the incorrect answers, it provided fairly convincing, albeit very wrong, arguments and it’s easy to see how a student might be led astray.

What are the benefits of using ChatGPT?

ChatGPT can be very helpful in the brainstorming and creative stages of any work.
Exploring different possible perspectives.
Playing with language and practising different ways of expressing oneself.

What are the risks or downsides of using ChatGPT?

1. ChatGPT is not a database of facts. It can be incorrect and the likelihood of inaccuracy increases with a topic’s complexity. It cannot replace your own research; remember that it’s really just a large auto-complete chatbot. When you ask a question, it’s closer to polling the internet for an answer than it is to searching academic literature.

2. The citations it provides are not always accurate or even real. ChatGPT is predictive and generative, meaning that it will absolutely make up citations based on prediction. For example (source):

David Smerdon provides a good overview of how this happens in a Twitter thread; however, the short version is that we typically see ChatGPT will use the most common author on the topic, alongside a common co-author, a real journal name the authors have previously published in, and then create a legitimate sounding false article title. Spooky, right?

We have heard that updates to ChatGPT will now prevent this from happening. ChatGPT should only use real citations. So I (Sylvia) decided to test it! I started by asking ChatGPT who is considered the father of positive psychology, which it correctly answered (Martin Seligman). Too easy, though! I pressed it for a citation and while the book it quoted was legitimate, the direct quote it provided did not exist in that book. When I challenged it, ChatGPT suggested it hasn’t verified the citation. So naturally, I asked it to provide a citation that it had verified. Then it provided the exact same quote and claimed it was on the same page, but in a different book. Of course, when I checked, no such quote existed in the second book either.

Always double-check any information from ChatGPT because it can seem very confident, yet be entirely wrong. (proof above, click to expand; chat occurred feb 8 2023)

The bottom line is: don’t rely on ChatGPT to provide real, accurate sources and don’t believe it if it says the information has been verified.

3. The information can be very biased. For examples, check out this Insider article and how a later article by CNBC shows how the new guardrails are easily bypassed. Ultimately, remember the APS Code of Ethics and APA guidelines; both remind us that it’s our responsibility to ensure any work we produce is free of bias.

4. ChatGPT doesn’t know about anything that happened after 2021. This includes the release of the DSM-5-TR.

5. You have a distinct writing voice – either by choice or evolution. An overnight, dramatic change in your writing style and tone can and will be identified by those marking your work. This is especially the case in fourth year and beyond, where fewer people evaluate the work (and thus repeatedly evaluate the same students and become very familiar with each person’s writing voice). Not only is it unethical and against academic integrity policies, but it’s also easy to spot. Again, don’t copy/paste anything from ChatGPT. Remember that copying and pasting from ChatGPT is effectively saying that you’re prepared to gamble your marks against expulsion and your future career.

6. Anything you enter into ChatGPT will be used to further its training. Consider what that means for the privacy and confidentiality of any information you input. This also has ethical ramifications if you’re including anything you haven’t personally created.

7. You have no control over the information once it’s been entered into ChatGPT. You cannot permanently delete prior conversations or questions; you can only remove them from your list of visible conversations. While there is no indication that the information is shared elsewhere, there is no guarantee of your privacy or confidentiality, nor is there a guarantee that unethical, illegal use won’t be shared with relevant authorities.

8. ChatGPT cannot conduct critical analysis. In fact, it fails to do so in pretty basic scenarios. For example, here are some very simple questions I asked (Feb 8, 2023) based on prompts discussed by Gary Marcus in conversation with Ezra Klein and replicating the findings of Jacob Ferus.

This is especially concerning if you wanted to rely on ChatGPT for critical thinking in your assessment tasks. Consider how many of your assessments require critical thinking. How central is critical thinking to your marks? In our experience, the vast majority of assessments require a substantial amount of critical thinking. ChatGPT really can’t help with that. In fact, we’ve personally witnessed ChatGPT fall into uncritical, unhelpful thought patterns that would only exacerbate student confusion.

9. ChatGPT cannot understand or accommodate the context of your studies (e.g., class content, how the unit fits with other units, the personal preferences and quirks of your teachers, the ‘hidden’ rubric, the required readings, learning outcomes, etc.). As such, it cannot adapt to that information. Further, it cannot effectively help compose reflection-based assignments, which are common, especially in fourth-year and beyond.

10. ChatpGPT often relies on the same template/format and only changes a few keywords. This is not only dry and boring, it means anyone copying the content will be picked up for plagiarism. Again, don’t do that! For example, I asked ChatGPT to compile two short cover letters for jobs that are very different to one another: a bricklayer and an astronaut. Not only are they very similar, they are also completely against the rules of practice for effective cover letter.

11. Your instructors likely already know about ChatGPT or will very soon and are developing strategies to detect ChatGPT.

12. Many, many people are using ChatGPT — including other students in your cohort, working on the same exact assignment. Even if the instructor hasn’t already copied the ChatGPT output into TurnItIn to pick up on plagiarism, any other student copying/pasting into their assignment will mean that your content will be detected as plagiarism. Just don’t do it.

13. The purpose of an assessment is to develop skills you will need in the future; if you take shortcuts, you’re making future assessments and tasks harder on yourself. I know, that sounds like a real “tutor” thing to say, right? But it’s true. Assessments are structured to challenge you and for you to learn through the process. Taking a shortcut means that future assessments will be more challenging.

Q. How are universities responding to ChatGPT? Can I use ChatGPT as a psychology student? Worldwide, ChatGPT has caused a stir amongst academics and responses have been at every end of the spectrum.

On one end, some education systems are banning the use of ChatGPT (e.g., NYC schools and French universities). One example statement from a rubric by Assistant Professor Chrissann Sparks Ruehle is: “Since writing, analytical, and critical thinking skills are part of the learning outcomes of this course, all writing assignments should be prepared by the student. Developing strong competencies in this area will prepare you for a competitive workplace. Therefore, AI-generated submissions are not permitted and will be treated as plagiarism.” (Chrissann provided this in a private Facebook group; permission was granted to include it in the article).

Some professors are devising strategies that will effectively make ChatGPT obsolete. For example, Montclair State University provides strategies for instructors that include redesigning assessments as:

a response to video or images that ChatGPT cannot access,
drawing on current events,
self-reflection of practical activities,
in-class activities or debates,
multimedia assignments such as videos, podcasts, or infographics,
unscripted, impromptu oral presentations,
adding marks for attendance and engagement,
frequent in-class quizzes or minor assessment tasks,
in-person exams, and/or
handwritten responses.

Beyond these suggestions, we have also heard instructors discussing the possibility of one-on-one interviews with students or oral defences of written work.

However, some universities allow the use of ChatGPT, so long as students disclose it (e.g., South Australian universities and University of Queensland). Others have said students are welcome to use ChatGPT and there’s no stipulation that they must disclose its use beyond the requirements of academic integrity policies (e.g., Deakin).

The takeaway is to please check your university’s policies. And keep periodically checking, because they are rapidly changing!

Q. How can I use ChatGPT whilst adhering to academic integrity policies?

Brainstorm search terms or keywords.
Experiment with different ways of expressing your ideas.
Ask ChatGPT to write a multiple-choice quiz as part of your exam preparation. Remember that it may incorporate content that isn’t relevant and it may exclude important content, but this can be a useful tool.
After answering your core learning objectives, ask ChatGPT to answer them and see if you’re on the right track. Again, keep in mind that it might miss important information, fabricate, and/or include irrelevant information.
Ask ChatGPT to paraphrase or summarise key content, especially if you’re struggling to understand something. Note, there does seem to be a character limit. When asked, ChatGPT claims it’s 4096; however, we have entered more without a problem. Your mileage may vary.
Ask ChatGPT to provide suggestions on how to improve your grammar, spelling, or word choice. However, we prefer Grammarly for this as the Grammarly apps allow this to be real-time, as you’re writing, which is better for learning.

Q. Does the Study Support team use ChatGPT? We certainly do. We have used the app to help broaden our perspectives (for example, asking ChatGPT “what are some counter-arguments to this statement?”), summarise and simplify tricky content, and even just have a laugh (for example, Sylvia and Renae have shared ChatGPT-written inspirational poems about each other).

TL;DR: Ultimately, the point of a written assessment is to demonstrate your understanding of the concepts, theories, and purpose of the tool. Thus, writing a lab report is to communicate to/with the scientific community, policy briefs are to communicate with ministers and government officials, articles aim to communicate with the general public, and so on. As a psychology student, you’re not writing an assignment to demonstrate you have a robust understanding of the English language and all its weird and wonderful quirks; thus, using tools to support your spelling and grammar is typically approved. However, universities are unlikely to support the use of tools that bypass the purpose of assessing your comprehension of materials, critical thinking, and synthesis of information. Do not use these tools to complete your assignments; however, you can use them to generate ideas and explore different perspectives. I’m assuming that you’re doing your degree because you want to be a competent professional. Becoming a competent professional does mean completing the assessment tasks, even (and especially) when they’re challenging.

What are your thoughts? How are you using ChatGPT? Which other AI tools are you using? Reach out – we’d love to hear more!

P.S. For a giggle, here’s ChatGPT’s take on the opening paragraph, in the styles of George R R Martin, Dr Seuss, and J K Rowling.

“The winds of change had blown in with the dawn of a new age, an age of artificial intelligence. The realm of academia was not immune to its effects, as scholars and students alike pondered the implications of these new tools. Some hailed them as a godsend, a means to unlock knowledge and unlock the secrets of the world. Others warned of the dangers they posed, of the potential for machines to outstrip the wisdom of men. But as the great houses of learning debated the merits and dangers of these new inventions, there were those who dared to look closer. To explore the ways in which these tools, such as ChatGPT, could aid in their studies. To consider the ethical quandaries that arose when man and machine were brought together. And so, as the snows of winter descended upon the halls of learning, the questions remained: could these tools be tamed, or would they bring about the downfall of all that we knew and held dear?”

“AI tools, oh what a treat! They’ve made learning quite a feat! Some say they’re a blessing, some say a curse, But students all want to know, of course! Can we use them in our schoolin’? Will ChatGPT make our grades coolin’? There’s questions to be answered and explored, So let’s not be a bore, let’s get on board!”

“The wizarding world of education had been shaken to its core with the introduction of Artificial Intelligence (AI) tools. The students and professors alike were in a state of awe and trepidation, as they grappled with the implications of this new form of magic. Some saw it as a powerful tool to unlock knowledge and improve their studies, while others feared its potential to replace the hard-won wisdom of the centuries. As they delved deeper into the mysteries of ChatGPT and other AI tools, questions abounded: could they be tamed and controlled, or would they lead to disastrous consequences? And what of the ethical considerations, when man and machine were brought together in the pursuit of knowledge? These were weighty matters, and the students and professors of Hogwarts School of Witchcraft and Wizardry were determined to uncover the truth, no matter the cost.”

References

Where online sources were relied upon, links have been embedded within the article. Beyond these, we relied on:

Silverstein, S. (1981). A Light in the Attic. Harper & Row Publishers.

Robot photo by Alex Knight on Unsplash

Disclaimer

While care is taken to ensure accuracy, Study Support by Gray Matters cannot guarantee the information expressed here is correct and recommends that students exercise their own skill and care with respect to its use. Information was collated and distributed in good faith and is derived from sources believed to be reliable and accurate at the time of publication. However, the information is provided solely on the basis that readers will be responsible for making their own assessment of the matters contained or discussed herein and readers are advised to verify all statements, information and advice.

Study Support by Gray Matters cannot and shall not be responsible or liable for any loss, damage or injury, however caused, arising from the use of or reliance on any information, data or advice (including incomplete, out-of-date, inaccurate or misleading information, data or advice) expressed or implied in, or coming from, this article.

Study Support by Gray Matters takes no responsibility for the accuracy, currency, reliability and correctness of any information included in the material provided by third parties nor for the accuracy, currency, reliability and correctness of links or references to information sources (including websites).

Changes in circumstances after the document is published may impact the accuracy of information. No assurance is given as to the accuracy of any representation, statement, information or advice contained after the publication.

TL;DR: We did our very best to ensure the data presented here is accurate. However, things change, and there are many moving parts. Please double-check the data before making decisions and know that we cannot be responsible for decisions made on this information alone.