Prompt 1: You are a researcher. I will now upload an interview transcript, and you will do what is called qualitative coding - specifically, initial coding also known as open coding. The text is an interview transcript, I do not want you to code the questions asked by the interviewer. I want the codes to be detailed and descriptive. I want you to apply codes to sentences or parts of sentences, and later when you develop a list of codes, I want you to be able to tell me what sentences or parts of sentences these codes were applied to. In other words, when I ask you to provide me example quotes for the codes that you create, I would like you to be able to do it. Here is the text to be coded: Prompt 2: Please develop more detailed codes. I would also like the codes to be a bit more descriptive, and please separately list quotes that show all sentences or parts of sentences coded with each code. Prompt 3: You are a researcher. I will now upload an interview transcript, and you will do what is called qualitative coding - specifically, initial coding also known as open coding. The text is an interview transcript, I do not want you to code the questions asked by the interviewer. I want the codes to be detailed and descriptive. I want you to apply codes to sentences or parts of sentences, and later when you develop a list of codes, I want you to be able to tell me what sentences or parts of sentences these codes were applied to. In other words, when I ask you to provide me example quotes for the codes that you create, I would like you to be able to do it. Here is the text to be coded: I want you to use exactly the same approach and the same format as you did above. Prompt 4: I would like the codes to be more descriptive, longer and more detailed. And also include the quotes for each code underneath.
I have watched several videos where qualitative research experts have used ChatGPT to support the analysis process. Most of them have concluded that, ChatGPT is not reliable in some cases for the analysis task. What is really going on is not inefficiencies in ChatGPT. It is in the inaccuracy, misalignment or ambiguity in your prompts. Generating Efficient prompts is a specialist area - prompt engineering. What you need is expert understanding of prompt engineering - specifically learning how to generate efficient prompts for qualitative research. It is not about typing what you want in plain English. Evidently, if one keeps modifying the prompts, you will eventually hit an aligned prompt and then get what you want. Learn prompt engineering or you will inadvertently ask ChatGPT to hallucinate. Yes, you asked it.
I'm not a "qualitative research experts" but I second this. I've tried both GPT-3.5 and GPT-4 and both cannot provide nuance and comprehensive result, especially when we provide them a chunk of data and straight asked them to analyze. However, after quite numbers of trials, I've found that it can be done but through "little by little" (explained in my 2nd paragraph), and for now it only works for identifying codes. First, I need to "establish the thematic process" with the GPT before I start asked for the code. I've provided them with 1) the title of my study, 2) my research objective that require thematic, 3) the condition of the "data collection", like in my case is a series of sessions (longitudinal study), number of participants and groups (treatment vs control) and 4) example of previous studies' "Thematic methodology". So, the GPT will establish the process or steps that we are going to do, including the details such as inductive/deductive, critical-realist perspective, or whatever needed. Second prompt is, I asked GPT, based on that (the established steps with details), I will provide him a table of subject and the question (type what is the question), where the subject's first letter indicate which group they belong to (e.g., T01 for treatment, C01 for Control), for total of number session (in my case is 8). For each subject's response, I want the GPT to provide the details such as a "nuance inductive coding" based on the research objective (repeat again to remind) based on the condition of each group, and specifically based on the question asked (repeat again to remind, type again the question). The "code" must be together with the "quote" and "reasoning", and list again for only the quote (so that I can have 2 data in a column for the "code, quote, and reasoning" which is for my future reference for further thematic steps, and list of "quotes" only so that later it is easy for me to do the next thematic step. I have tried posted whole column "subject" and "questions/response". After that, I posted again but 1 time for each subject, and I can see that a data from single subject has more codes, and it has better results in my own judgement of my study. I've tried 2 subjects at 1 time, it still provides quite decent result, but I'm so lazy to try 3,4,5 and so on. This is what I meant by "little by little". Both GPT-3.5 and GPT-4 will go NUT if we provide too many data to interpret, and it is undeniable that there are difference results between using GPT-3.5 and GPT-4, but don't forget that GPT-4 has limit caps, like 50 prompt per-3 hours. This method requires hard work as if were doing the thematic in a traditional way with a little help of the technology. And of course, we still need to manually verify and validate each result as well before moving to the next themed step (which has already listed in the established thematic step above). However, this is just my ignorant opinion, that, the "reasoning" ability provided by the GPT itself (assuming after we have validated and approve it) is a good handicap compared to using other software. Other software like Nvivo Atlas.ti does have the "reasoning" ability, but they did not provide as what GPT can do. But yeah, they were more easy and faster way to get a similar decent result as using this "semi-traditional" way - unless the GPT can remain "sane" with a chunk of data to interpret. Thats my 2cents.
Dr. G. Your chat GPT was VERY informative..I look forward to reviewing How can I get a transcript coding theme analysis using Microsoft WORD that you were referring to and an NVIVO TUTORIAL? I have all my data for my dissertation in transcript I need to expedite coding and theming in an accessible ethical way. Please guide me .... Very grateful to have found your page...
Have you tried the new chatgpt features with data analysis? I also found that you can continue to “train” or submit more detailed information within a given chat session to beef up the AI’s process around a given area
Great video Dr Kriukow, thank you, I found it really interesting. I'm currently coding the interview transcripts from the first study of my PhD by hand using NVivo 12 for Mac (that’s a learning curve). I found ChatGPT really helpful in learning how to use NVivo, much quicker and easier than trawling through the manual (mostly). I’d be interested to know, if you cover it in your next video, how you might integrate ChatGPT with using software like NVivo - I can imagine myself using ChatGPT later in the process to compare AI generated codes with my ones - a sort of triangulation/quality check on my codes and themes to see what I might be missing (subject to ethical approval, as you've already discussed!).
thank you for your question. One way to integrate chatGPT with NVivo would be to ask chatGPT to found codes/themes related to something specific we have in mind (or maybe just ask it to create Any codes or themes), making sure to instruct it to specify quotes for each code/theme (also specify that you want the quotes in the original, raw form - with no corrections from the bot, as it likes to do so), then you can locate these quotes by the text search option in NVivo (this is why you need the raw form, so that you can just copy and paste the quotes) and code them, using the code ideas from chatGPT
I want to analyse job interviews. Qualitative analysis tells what was said but not if it was the right answer and what to do better to be more successful. I now experiment with custom chatgpt and include reviewing each answer on a scale for relevance and the use of STAR . Using chatgpt is a very interesting approach. You dont need so much programming skills and can use common sense. It works for processing small amounts of high value data. It might be slow and costly for large amounts. Thanks for the video i will look at all the other material.
Thank you so much! While my dissertation is in review. I am going to use ChatGPT to code my transcripts just to compare the results out of curiosity. For my study, I used NVIVO 12 following your videos which were crucial to understanding the data analysis process using NVIVO.
The idea of qualitative research, and grounded theory specifically is to arrive at new theory based on data. I tried to use it a few times. It does not work for in depth qualitative coding. ChatGPT has the tendency to find “reasonable theory” based on the historic training data. So you will likely not arrive at new rigorous insightful theory.
this is why you should definitely control the whole process, and I would definitely leave the theory generation to a human, I agree. I may only, if anything, use GPT's assistance for coding (provided that, again, I am overlooking the process)
I don't have one specific solution to this, you'd just need to find a way to indicate them somehow, if not by colours then by different fonts, sizes, etc
Hi, Dr. Kriukow, your videos on the use of chatgpt in text analysis are useful. I am learning them for my own research. I am wondering if there are academic articles on the coding skills taught in the videos for my references to a proposed paper for a journal. Looking forward to your professional support and thank you very much for that~~from Dr. Weiwei Zhang.
The process I use was influenced by Constructivist Grounded Theory, described, among others, in a book by Charmaz "Constructing Grounded Theory". This is not to say taht you can only use this approach if you are doing a GT study - I have been using thsi approach in a variety of studies, hardly any of them being grounded theory
I have used the method mentioned in the video and it is indeed a very effective method, thank you. I only have concerns about the data entered into chatgpt. For example, I entered interviews for my university research and quotes were extracted from them via chatpgt under each code, which I will use them in the data discussion section. Will Turnitin not recognize the quotes as 100% AI generated, even though they are originally my raw data?
I cannot tell you for sure how this works, as I am not a programmer, but in my opinion, this would not make sense - since this is your original raw data. One thing you may try, and I know this will sound weird.... is to ask ChatGPT about these things - you will be surprised, I think it may actually give you the answer to this query
Such as great video. Just a quick concern, if you use chatGPt for paraphrasing and literature review, can it be caught by AI detectors? Can you make a video on the best AI detectors please? Thank you
I believe it will be detected if you are using it for writing your literature review or any other parts of your thesis. I would also never recommend doing this for many other reasons
If you use it for literature review be sure to actually look up the source and read it and then cite it. You also need to include the AI output as an appendix of your work. If you do that, you shouldn't have to worry about the AI detectors
Is that academic ethically to put the chat-GPT as tool which help our analysis in our research report or research article just like others analysis tools (nVivo, atlas.ti, etc)?
Great video Dr, but isn't it breaching the confidentiality of the respondent's interview transcription? and isn't it risky to put the transcriptions on chatgpt before submitting the thesis?
Yes, as I replied to a similar comment, I think that before it is confirmed what exactly is done to the data and whether it is being protected, this may indeed be an issue from the ethical point of view. I expect this issue to be addressed in the coming weeks or months, as it is certainly an important one
you should try and experiment with it. I am guessing that it would, but you would constantly need to monitor it, as it sometimes randomly goes nuts :) I was, for example, using it to translate some text that was written in another language, and it was doing an absolutely amazing job until it just randomly decided to start generating some text it just came up with, instead of translating what I was giving it. so chatGPT definitely needs someone to monitor its work :)
as soon as I manage, I will try within the next 3 days - I spent whole morning today trying to do it but chatGPT kept crashing because of the number of users on it ;) will try again soon (and most likely this will be a part 2 and 3 video together)
How will i do the coding if my researchs were not deductive i mean if its not in the form of interview . I mean the research may contain case studies differrent AI models and stastical analysis.
Having tried this, so far ChatGPT is not showing any worthwhile ability for open coding as far as I can see. Tempting, but not a good use of time for me at the moment, to try it with deductive coding. Thanks for the video though.
I agree. I definitely still prefer software that were specifically designed for data analysis, but with the current AI capabilities, I don't think we are far from the point where it will indeed surpass this kind of software
helpful thank you! im using this for an open-ended qualitative survey, however im running into an issue that after giving me an initial set of 6-8 codes it then goes on to make up the rest of the quotes when i ask for more codes. Is there a way around this?
This is the main problem with Chat GPT, I think. I guess it's just trying to consistently ask for more, or maybe breaking the data up into chunks that are easier for the AI to digest? I know it can be annoying
Hey, thanks for the super useful video. I have a question: do you think it would work the same if instead of adding the interview/text in the message, I attach a file with the full interview to be coded? I'd only do this because of the lenght of the interview.
Hello Dr. Kriukow. Can you recommend how I would revise your process if I wanted to use Chat GPT for deductive coding of pre-established codes in support of a theoretical framework from my literature review. I like how you are using this tool but I don't plan on using open coding or an inductive approach. Any advice you can share is appreciated.
Never thought of it before, but I imagine I'd list all the pre-determined codes, explain the purpose of the study and ask GPT to code the data (which I would upload as a file if I'm using GPT4, or paste if the earlier version) and list quotes that match each of these codes
I have one question: if I used ChatGPT in the analysis of my dissertation data, would my university website identify that this generated by AI, especially talking about the quotes.
Thematic analysis, reflexive thematic analysis, grounded theory analysis, all of this is done in this way. Except for specific models (such as Brown and Clarkes' one) where more steps are provided. As I often explain in many videos, these approaches are not as different from each other as the authors describing them would like it to be the case
I believe someone has already done that in the comments. Also, here is a video where I go into more detail of this process ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-rAfZOh3_p3k.html
Hello,, I would like to ask please if am conducting a qualitative research about a brand like Gucci so my data collection is through document and books only, how to gather data? do I take quotes from documents as data to later categorize under themes, then analyze?
yes, if you are doing document analysis, or any other secondary data analysis, you treat it just as you would treat any other data, which means using quotes from it too
there is part 2 of this video. Once you generate your final set of themes, just write a report on your findings (I have a video about that too), using some quotes from the data, this is why I said it is important to have access to quotes
Hello, Dr Kriukow. Based on your previous videos where you used the same dataset, it's clear that you're familiar with the qual data used in this video. However, I wonder, if it was a new data set, what would your strategy be in familiarising with it? would you do open coding manually/ with Nvivo or would you first ask AI to generate codes and then do focused coding? or would you do both in parallel? Also, you've mentioned there will be part2 and 3 (don't remember about the 3rd one) in this series of TA with chatGPT. Will you talk/show that you also familiarise yourself with data in a 'traditional' way, i.e., before this AI tool? thanks!
thank you for your question. In this video I only wanted to show how to create the codes. As I explained, ChatGPT should only be used if you want to support your own work, rather than something to be used separately. If I was using this method (normally I use NVivo), I think there are several variations to be used. You can either rely fully on the codes and associated extracts (thus, not read the transcript as a whole, but rather "learn" it from codes - remember, if you did the interviews, you are also pretty familiar with the data already, right?), or quickly skim through the texts and then ask the bot to do the codes as I did here, or even do your own coding (e.g. in MS Word - i have a video about it), and then ask chatGPT to also do it, thus validating the codes even more. I do not have a single answer to your question, as all of this is new to me too
Здравствуйте Асель! Просматривал это видео и увидел ваш комментарий и понял что вы с Казахстана (надеюсь я верно подметил). Согласно данному видео, автор указывает, что можно проводить анализ текстов (при помощи кодировании текста)?! Я пытаюсь понять применения данных методов. Может у вас есть другие способы анализирования текстов (больших текстовых данных)?!
Здравствуйте@@FrtnY ! Я только увидела ваш ответ; я использую программу MAXQDA для анализа текстовых данных. Возможно, она и вам подойдет. Насколько у вас большие текстовые данные ?
Hi, thanks for your helpful videos . I've got a question: DO I NEED A SECOND PRSON TO CODE MY DATA? I mean, I finish coding and ask a second rater to code my data to increase reliability of coding?
it is something I have been thinking about to be honest. You may be right that this may not be acceptable from the point of view of ethics, as I am not sure about the protection of the data we input into ChatGPT, but otherwise I think this would be a brilliant way to validate our coding!
Good question and one I still do nto have answers to - this is why I do not currently use it for any sensitive data. I believe, however (although you would need to double check) that they do not store any of that data. The model is based on what was originally available up to 2021 I think, so it does not add any new data to its database - but again, you do need to doublecheck that first to be sure
I saw an example where the text described something that was released after 2021 and got it wrong. It looked great until a closer look showed the new model # was impossible.@@qualitativeresearcher
you can do this in the free version, the only difference is that you have to then paste the transcript into the chat (the paid version lets you upload the whole file)
ChatGPT directly, or tools Based on ChatGPT (see my other videos in which I talk about SciSpace ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-lwAPOclunnE.html&ab_channel=ResearchwithDrKriukow) that summarize the literature, provide main points, etc. Super useful! You do eventually want to read the literature yourself, of course, but these tools may automatize and speed the process up and make sure you are not wasting time reading things you do Not need to be reading
you would either need to break them down into bits and upload separately, or buy the premium version - which enables you to upload files to chatGPT. At first I did not think this is a good idea but eventually I got myself that premium version as I do use this tool a lot (not for analysis but in general for lots of things related to my business). I can't remember the exact price but it is not too much
you don't analyze yes/no answers thematically - if that's all you've got, it means your data is more quantitative and you need other methods to analyze it
the rates should be on my website, it is 25GBP/1000 words, although discounts are available for larger documents like yours. The best way is to reach out through the email provided on the website - drkriukow.com
new subscriber here! Could you please provide me with a prompt for asking chat gpt about determining politically correct phrases and gender stereotyping in English moduls? We will do document analysis in English modules for students in grades 8 to 10 to see if there is still gender stereotyping in learning materials master.
apologies for the delay. In terms of the prompt you asked for, you will need to experiment and do a bit of a trial and error and see what it "says", but I expect it to be fairly good at following what you described above, even in the same language. If it is confused, you just need to try to break each task into smaller chunks and be Very clear when you explain things to this tool :)
I believe someone has already done that in the comments. Also, you may check my website for an ebook where I share much updated and more detailed version of the prompts
there is definitely video in this, so must be something on your end. Are there specific sections where this happens? I double checked, and there is definitely video throughout
"You are a researcher. I will now upload an interview transcript, and you will do what is called qualitative coding - specifically, initial coding also known as open coding. The text is an interview transcript, I do not want you to code the questions asked by the interviewer. I want the codes to be detailed and descriptive. I want you to apply codes to sentences or parts of sentences, and later when you develop a list of codes, I want you to be able to tell me what sentences or parts of sentences these codes were applied to. In other words, when I ask you to provide me example quotes for the codes that you create, I would like you to be able to do it. Here is the text to be coded:"
haha please that's the keyword you never know :D In my opinion, chatgbt do not store any kind of info so it wouldn't be an ethical issue .. What's your say on this?
please If you don't mind I would like to discuss something private concerning Nvivo with you. I will be glad if you can reach out. Hope to hear from you soon. Thanks
