Тёмный

How to Read from a text .txt file in Python! Pulling in data and filtering and modifying the info! 

LeMaster Tech
Подписаться 14 тыс.
Просмотров 89 тыс.
50% 1

Check out my personal channel for fun engineering content!
‪@peterlemaster8310‬
This video discusses the method for reading data into python from a text file. This includes step by step instructions for accessing the file from the same folder as the python program, then storing the data in a list, then filtering the special end line characters out and storing them in a modified list. There are numerous ways of filtering, processing and modifying the data read in and then you can perform any actions you could do with string variables with the data inside your python function once you learn how to read the data in!
Learn to write to text files here:
• How to Write to a text...
Love the content on this channel? Feel free to support me on Patreon!
/ lemastertech
BUY A COOL SHIRT WITH MY LOGO IF YOU WANT TO: lemastertech.myshopify.com
If you have any questions or comments or want to see something specific in the future let us know about it with a comment!
We have hundreds of hours of instructional content on the channel and create more every week so be sure to subscribe to stay up to speed! Thanks for checking out the video and good luck with your projects!
/ @lemastertech

Наука

Опубликовано:

 

22 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 90   
@AFuller2020
@AFuller2020 2 года назад
Great video, many thanks!
@lemastertech
@lemastertech 2 года назад
Glad you liked it! You’re welcome!!
@DGonz0-
@DGonz0- 2 года назад
awesomely useful thank you
@lemastertech
@lemastertech 2 года назад
Glad you found it useful!!
@nadew.02
@nadew.02 Год назад
I understood well. Thank you
@lemastertech
@lemastertech Год назад
Awesome you’re very welcome thank you for watching!
@OOM2M
@OOM2M 10 месяцев назад
Thanks, that's really helpful
@lemastertech
@lemastertech 10 месяцев назад
You’re welcome thanks for watching!
@habonimanaguyaxel591
@habonimanaguyaxel591 Год назад
bro you're the man kabisa thx
@lemastertech
@lemastertech Год назад
Thanks so much glad you liked it!! You’re welcome
@iotekhaneokwire
@iotekhaneokwire Год назад
Thank you. This was very easy to follow as a beginner.
@lemastertech
@lemastertech Год назад
You are very welcome, I’m glad you liked it and thanks for watching!
@philisa_ntlahla
@philisa_ntlahla 2 года назад
great video thank you so much
@lemastertech
@lemastertech 2 года назад
Glad you found it useful friend!
@nerina1741
@nerina1741 Год назад
Excelente capo, buen video abrazo
@lemastertech
@lemastertech Год назад
Me alegra que te guste, bienvenido al canal!
@jeff7838
@jeff7838 Год назад
@LeMaster Tech Hey I was wondering if I can use this to match text and account for it? Do you have a video of that, or should I use something totally different?
@lemastertech
@lemastertech Год назад
Hi, I don't think I have a short video on this concept exactly, but I do have a video on the channel where I show how to make a beatmaker/drum kit in python and as a part of that project I open a text file to load saved beats, and I go through how to match text to specific column ID names and then use the data inside the program! That would probably get you what you need! Hope that helps, goodluck!
@balachandrasaimc7643
@balachandrasaimc7643 Год назад
hello sir, while i'm trying to remove the blackslash, i'm getting this(',' ', ' ',) at the end of the first line. How can I remove this?
@lemastertech
@lemastertech Год назад
Hi, I’m sorry I don’t really understand your question - are you saying you get one entry into your list that is just one empty space?
@Paul-ly1pw
@Paul-ly1pw Год назад
Can you store a series of lists in a txt file to be able to use each list as the source for a Tkinter combobox? I have tried a json file but that doesn’t allow the append method.
@lemastertech
@lemastertech Год назад
So I am sure this could be done but I think probably it would be easiest to store the data just with a title starting each list, like have line 1 say “list 1” then each line after that be one value for that list, then once it ends the next line says “list 2” then in your python code you could check to see for line in file.readlines If characters 0:3 == “list” create a new list and store the following values until you reach the end or the next list! You want to try and do as little fancy work inside the text files as possible since they are really just plain text! Hope this helps!
@AdamWrigley
@AdamWrigley 2 года назад
What if they are numbers? I've got \t at the end of all of the numbers and this doesn't get rid of it
@lemastertech
@lemastertech 2 года назад
Hey! Good question, you can use a for loop that checks “for line in file” then if line(-1) == \t: line = line(0:-1)
@zwephyaewin6330
@zwephyaewin6330 Год назад
brother you said there is a way to from where ever the directory txt file ,a file path joint please help me with that one i dont know how to do that please brother
@lemastertech
@lemastertech Год назад
Look into import os and then use os.filepath.join! Lots of good documentation online 👍
@nurzulhafizyacob5118
@nurzulhafizyacob5118 2 года назад
Lets say i want to capture data for "12345", how to do that?
@lemastertech
@lemastertech 2 года назад
So if you want that data stored as an integer take the data and use int() or float() to convert it to numerical data! Otherwise it will come in as a string
@issamhammadi8828
@issamhammadi8828 3 месяца назад
for line in read: line = line.strip() #I think this line is missing if line not in modified: modified.append(line)
@lemastertech
@lemastertech 3 месяца назад
Thanks for your suggestion! I think it works without that but the data is less nicely formatted!
@hananali1171
@hananali1171 Год назад
Thank you for this useful video, I have a text file with headers before each column, how to skip those headers? Should be grateful if you point to some of your videos if you explain this case.
@lemastertech
@lemastertech Год назад
Hi I have a video where I make a drum kit/ beat maker project in python and in that video I show how to process data from text files in a lot of detail! Basically if the headers are the first line of text you can just make a copy of the data where you exclude line one, or if it’s at the beginning of each row you can just parse out a certain number of characters out of every row!!
@hananali1171
@hananali1171 Год назад
@@lemastertech Thank you very much
@genevievehigdon2232
@genevievehigdon2232 Год назад
Very useful, how would you extract specific data from the text. I have a text with alpha and numerical data. I want to read just the numbers, then sort, find the length and the median. I see many videos on how to read the file and work with it, but not how to extract specified data only.
@lemastertech
@lemastertech Год назад
Thats an interesting problem! You could make a list of valid characters as strings you want to check for that you consider valid numbers, including decimals or commas potentially depending on formatting, then only parse out items that are in that list, and depending how they’re separated by spaces or commas you would use that to check as a start/stop - good luck!
@allaboutsophia9205
@allaboutsophia9205 Год назад
for line in read: if line not in modified: modified.append(line.strip()) hi, this line of code doesn't work with removing the duplicate data in my file.
@lemastertech
@lemastertech Год назад
Hi, try print(modified) after each modified.append action in your code to see what the list is saving each time, if you are getting duplicates exactly, then you might having indentation issues!
@Yobachi2007
@Yobachi2007 Месяц назад
What software are you using for the editor, is that Visual Studio Code?
@lemastertech
@lemastertech Месяц назад
I do all my python is the pycharm IDE! It’s free from jetbrains
@Yobachi2007
@Yobachi2007 Месяц назад
@@lemastertech Thanks for the response.
@willishuang3660
@willishuang3660 Год назад
Am I missing something - on my computer this code does not work for stripping duplicates. On my computer I have to use IF LINE [:-1] NOT IN MODIFIED: to strip duplicates because it is otherwise searching for the line+ (and thus doesn't avoid duplicates.)
@lemastertech
@lemastertech Год назад
Hi, this may have to do with the IDE you are using since some of them have difference shortcuts that will/won't work for them. Are you using pycharm like I am in this video?
@theHalken
@theHalken Год назад
I thought so, the last line doesn't have a newline so it matches the stripped duplicate from earlier but otherwise it wouldn't match. Should probably not use different ways of removing trailing characters tho, use line.strip() to compare against the list full of stripped lines. Or use index to -1 you are using that in the append. Basically you want the if-statement to compare the object in its potential future form to the list you already have.
@R3dV0m1t
@R3dV0m1t 9 месяцев назад
For those that may see this comment in the future: If the code in the video does not work for stripping duplicates, use "if line.strip() not in modified:" as it checks for the stripped line. Otherwise, it is comparing a line such as 'word ' where the list has 'word' so it will still add it to the new list.
@greeshmasurendran5197
@greeshmasurendran5197 2 года назад
Hey, I am struggling with a large and long tabular .txt file, about 400 MB which basically is a wind dataset from the automatic weather station. I need to select the days where the wind speed exceeds 90 km/h. My file is sooo long that it contains data from 1998 to 2021. How to do that in python?
@lemastertech
@lemastertech 2 года назад
Hi! I would recommend you take a look at the chapter of This video about loading saved beats in. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-PxnksbihWrE.html You basically need to find what key you’re looking for, in this case I imagine there’s probably a title like “wind data” then grab the 3-4 characters after that, then convert it to a numerical value, then check if it’s greater than 90 and if it is you can store it to a new list or dataset! Goodluck that sounds like a cool project!
@greeshmasurendran5197
@greeshmasurendran5197 2 года назад
@@lemastertech All I have is a .text file, which contains the data in tabular form..it contains the weather station ID, wind speed, temperature, dew point temperature, wind speed in km per hour etc..in tabular form, from the year 1998 to 2021..each row corresponds to each day .so imagine how long it is..and that makes my Jupiter notebook hang sometimes..all I have to do is to find the days on which wind speed exceeds 90 km per hour, using python...can you please help me
@lemastertech
@lemastertech 2 года назад
@@greeshmasurendran5197 Hello. You need to watch the 'load saved beats' chapter near the end of this video. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-PxnksbihWrE.html It shows you how to take a complicated line of text out of a file and read only the data you need and convert it to a useable format. I can't write the code for you, but this won't be too complex, you just need to make sure you get your formatting right
@greeshmasurendran5197
@greeshmasurendran5197 2 года назад
@@lemastertech I'll try.. thank you :)
@lemastertech
@lemastertech 2 года назад
@@greeshmasurendran5197 You're welcome, good luck!! Sounds like a cool project :)
@exlr8in
@exlr8in 2 года назад
Hello, I've been hitting my head against the wall (i've tried str, json, pandas and ntf) for a while and I can't seem to arrive at my destination, hopefully I can get some help? I get a result from a Cisco command and i want to read (focus) four lines of the output and then do something if something is present. The output is this: ``` TDR test last run on: June 01 07:50:18 Interface Speed Local pair Pair length Remote pair Pair status --------- ----- ---------- ------------------ ----------- -------------------- Gi1/0/11 auto Pair A 0 +/- 5 meters N/A Open Pair B 0 +/- 5 meters N/A Open Pair C 0 +/- 5 meters N/A Open Pair D 0 +/- 5 meters N/A Open ``` The idea is, if I get to see Open or Short on the last column, then i want it otherwise ignore and go on to the next ..... etc. It is that one particular lines 5, 6, 7 and 8 that I want to examine and do something if I find what I am looking for. Thoughts? import pandas as pd pd.set_option("display.max_colwidth", None) with open("data.txt") as f: dataline = f.readlines() #print(dataline) df = pd.DataFrame(dataline) #print(df) #print(df.loc[4]) Open = "Open" Short = "Short" if not df.empty: #print(df) if Open in str(df): print(df) elif Short in str(df): print(df)
@lemastertech
@lemastertech 2 года назад
Hi my friend! I’m not exactly sure I understand the end goal but your code looks alright to me! Are you getting any detailed error reporting when you run the program?
@AboodyHandal
@AboodyHandal Год назад
can i type the modified variable with pyautogui ?
@lemastertech
@lemastertech Год назад
Hi, I am sure you can but to be honest with you I haven’t used pyautogui a lot and not for this specific application so I can’t tell you exactly how but I don’t see why you wouldn’t be able to!
@AboodyHandal
@AboodyHandal Год назад
@@lemastertech i cant do it ... if u can help pls help 🙂
@AmlizeagaGaming
@AmlizeagaGaming Год назад
Thanks, useful for my project
@lemastertech
@lemastertech Год назад
Glad you found it useful you are welcome!
@setproject-sz3ku
@setproject-sz3ku 4 месяца назад
Thanks for the video. It is very helpful for beginners. I would like to know how to increase the reading and writing speed if I have 10000 lines in my text file
@lemastertech
@lemastertech 4 месяца назад
Hi, this honestly could probably handle 10,000 lines still fairly quickly but Python is also a little less optimized for speed than if you were to do something similar in a C based language so if speed is your main focus you may want to look into that!
@setproject-sz3ku
@setproject-sz3ku 4 месяца назад
@@lemastertech Okay sir.... Thanks for your guidance
@tettehabraham3303
@tettehabraham3303 Год назад
How do I create a python folder
@lemastertech
@lemastertech Год назад
Do you mean create a new folder from a Python program or just create a folder to store python programs in?
@samarbid13
@samarbid13 Год назад
You didn't close your file !!! with open("file.txt", "r") as f:
@lemastertech
@lemastertech Год назад
Hi you are correct! The next video I made right after this is how to write to a text file and I start that video right away by saying “I should have said this in the previous video but you should always close your file” and then show how to do it! Good catch
@AnkitG12
@AnkitG12 2 года назад
Hey it's not working. File not found error:Error 2.please help.
@lemastertech
@lemastertech 2 года назад
Hi! That error occurs when you are trying to access a file that does not yet exist, so double checking your spelling of the file name, make sure the text file is in the right folder otherwise you'll need to update the full file path, and make sure you have your file name inside quotes like a string!
@AnkitG12
@AnkitG12 2 года назад
@@lemastertech Ok brother. Do have any idea about how to solve the 'Cannot connect to this network' issue in Microsoft??
@lemastertech
@lemastertech 2 года назад
@@AnkitG12 Hi, that does not sound like a python issue that sounds much more like a windows issue. Like something where your system doesn't have the right password to a wifi network or something?
@chirontonmoheep8027
@chirontonmoheep8027 2 года назад
I didn’t get how that IF statement is removing duplicates. Like the wording isn’t providing much intuition.
@lemastertech
@lemastertech 2 года назад
So the statement ‘if line not in modified’ is checking one line of text at a time and comparing it to the existing list of data names modified in this tutorial. If the new line of text is already in the list as one of the entries then the code will not add the line again. It only adds the texts if it is unique
@augiblutz2852
@augiblutz2852 Год назад
It might be a REAL GOOD IDEA to post the code
@lemastertech
@lemastertech Год назад
It’s like ten lines of code my friend! You can skip to the end and copy it down in a minute haha
@matoxetu
@matoxetu Год назад
ty
@lemastertech
@lemastertech Год назад
Glad it was useful you are welcome!
@Tech_Brigadier
@Tech_Brigadier 2 года назад
And plz make sure that you make a your discord server soon
@lemastertech
@lemastertech 2 года назад
Sounds good buddy, will do. Good luck with your channel too!
@rizalbayu595
@rizalbayu595 2 года назад
I cant see it so small
@lemastertech
@lemastertech 2 года назад
Are you watching in vertical orientation on a phone? It’s pretty easy to see horizontally or on a computer screen
@ghostbrothers4710
@ghostbrothers4710 Год назад
AttributeError: 'tuple' object has no attribute 'readlines' it just keeps saying this and i cant get it to work
@lemastertech
@lemastertech Год назад
Hi, this sounds like you might have forgotten to add the word open to the previous line where you read in your file? It needs to be file = open(“filename”, “r/w”) which will populate your variable file with the data from the text file. But if you forget the word open then it will make your file variable a tuple just storing two strings and then you’d get this error! Goodluck!
@ghostbrothers4710
@ghostbrothers4710 Год назад
@@lemastertech ya I figured it out last night sorry but thank you
@lemastertech
@lemastertech Год назад
@@ghostbrothers4710 Gotcha, good job, no worries!
@nihalsn6915
@nihalsn6915 Год назад
How to read the txt file from other folders? I wish to read the txt file here from given path: '/home/user/OIDv4_ToolKit/OID/Dataset/train/Table_Label/Table0.txt' This doesnot work file = open('/home/user/OIDv4_ToolKit/OID/Dataset/train/Table_Label/Table0.txt', 'r') read = file.readlines() My python Script is in: /home/user/catkin_ws/src/apptronik_core/05_perception/cv_ml/image_scraper/scripts/final_txt_coco.py I even tried os.path.join: txt_path = '/home/user/OIDv4_ToolKit/OID/Dataset/train/Table_Label' txt_new_path = os.path.join(txt_path, 'Table0.txt') file = open(txt_new_path, 'r') read = file.readlines() solution: remove .txt extension from the path
@lemastertech
@lemastertech Год назад
Great tip thanks for the info!
@Tech_Brigadier
@Tech_Brigadier 2 года назад
Hi you already know me i am also a content created like you and would like to take a conversation with you on discord
@lemastertech
@lemastertech 2 года назад
Hi Friend, I will look into this! I am going to be out of town for a few weeks, and maybe near the end of July I will take a look at setting up a discord server. I am pretty busy so I am not sure how much time I'd be able to spend on it, but I'll check it out!
@Tech_Brigadier
@Tech_Brigadier 2 года назад
@@lemastertech ok take your time
Далее
We will see who will do it better 😂
00:14
Просмотров 3,6 млн
#9 Basic Python Data Extraction from Text Files
20:13
If __name__ == "__main__" for Python Developers
8:47
Просмотров 388 тыс.
Scripting with Python - Modify a TXT file
19:00
Просмотров 50 тыс.
25 nooby Python habits you need to ditch
9:12
Просмотров 1,7 млн
3 PYTHON AUTOMATION PROJECTS FOR BEGINNERS
17:00
Просмотров 1,6 млн
Pydantic Tutorial • Solving Python's Biggest Problem
11:07
5 Useful F-String Tricks In Python
10:02
Просмотров 280 тыс.
$1 vs $100,000 Slow Motion Camera!
0:44
Просмотров 25 млн
Красиво, но телефон жаль
0:32
Просмотров 1,4 млн
APPLE дают это нам БЕСПЛАТНО!
1:01
Просмотров 674 тыс.