I'm delighted that it helped!

JupyterLab can handle folders, including sub-folders. So you can put documents, including notebooks, inside of those folders.

@@ReuvenLerner Yes, well, I mean "include" (=call/embed/encapsulate) an external/underlying file (e.g. .md, .tex, etc.) into a cell of a new document, which will be automatically uptated when the content of the underlying file changes ...

@@kisho2679 Oh, I don't think that's possible. (Maybe I'm wrong, though!) Instead, you'll probably want/need to write a bunch of code in a Python module and then import that module into your notebook.

Thanks so much! And yeah, there are a number of reasons why Python's strings are immutable. One is that immutable data is more efficient and less error prone. A second is that only immutable (well, hashable, but that's almost the same thing) values can be dict keys. So for us to use strings as dict keys, they basically have to be immutable.

I'm so delighted you enjoyed it!

@@ReuvenLerner I was wondering if you teach on any other platforms? Anywhere to look at your courses? Thanks

@@CHeRKeSSS00 Yup, check out LernerPython.com -- lots of courses there!

I'm glad you enjoyed it!

My impression is that it doesn't really speed things up, but rather cleans up the API in a lot of ways. That said, I wouldn't be surprised if they managed to find *some* ways to speed things up a bit more. The real Pandas speedup will come, I think, via PyArrow. It already reduces memory usage quite a lot, and for many things it's super speedy. The other things (e.g., joins) aren't quite there yet, but I have to assume that they will.

I've got a lot to do just on the Pandas front, but I do hope to eventually cover some of these other technologies, too. I tend to cover them a bit more at Bamboo Weekly (BambooWeekly.com/),.

@@ReuvenLerner alright

I wish that my 9th language were as good as your English! Python doesn't let you have more than one version in use at a time. However, you can have virtual environments (venvs) with different versions of the same package. In that way, one project can use NumPy 1.x, and another project (in another directory) can use NumPy 2.0.

@@ReuvenLerner thank you for the appreciation. I am very fascinated that i came across your channel and i added your playlist into my watch later videos, i would love to watch and learn more from them. thank you for contributing to the developer world in this age of chat prompts. it is more understandable. btw, did you take the developer's survey from stackoverflow which they silently pushed and didn't make a lot of noise about this year? I found the questionnaire was only based on A.I. NOTHING ELSE, kind of signaled that many in the Dev space say is the end of Stackoverflow since the inception of LLM's. what's your take on that? also, i assume it must be very late in the US, and for you to reply me this quickly, i want to thank you again for that. My name is Samuel, nice to know you, Mr. ___?!

Glad you enjoyed!

I'm so glad to hear it helped!

My pleasure! Thanks for letting me know.

You can remove one or more columns with df.drop. If you want to remove all rows in a particular range, then you will likely want to use a boolean index to indicate what you do or don't want, and then apply it to the data frame. There isn't room here to explain that, but look for my video about "boolean indexing made simple" that explains it more.

Oh, no! I'm sorry to hear that you're having these problems. It definitely takes time to get your environment working in a way that makes sense and is stable.

My pleasure!

Maybe -- it depends on the context and what you're using it for. Feel free to e-mail me (reuven@lerner.co.il) to discuss this further.

CSV files need to have the same number of columns in each row. You can sometimes get away with null values, if there are commas next to each other, but I don't believe that you can ever have variable-length lows.

Glad it helped!

Yes, the new version of Jupyter notebook looks a bit different. They have moved around the UI and functionality. It's all still in there, plus a lot of new stuff, but they changed it because people didn't move to JupyterLab and still wanted the functionality. You can expect Jupyter to look and act this way moving forward, though.

@@ReuvenLerner yes I found out it is version 7. I downgraded it to the previous 6 something version and now I can customize it with different themes too.

In Python, a class is an object. And like all objects, you can have a variable that refers to it. When we define a class with the "class" keyword, we are assigning a variable to refer to the class object. That variable is just like any other variable, and can have any name that you want. Traditionally, we name classes with CamelCase, but that is a convention, not a rule. However, I'm not sure if this answers your question -- I'm not sure what you meant by "the real world," if that refers to non-Python languages, or something else. I'm happy to answer any other questions you have!

I don't use very many extensions, if any - in part because I've had trouble with extensions in the past. Autocomplete just kinda works for me, when I press tab.

I don't know about a book, but I gave two webinars/courses, "Python objects for newbies," that are part of my subscription service at LernerPython.com . And you can check out my video here, which explains some things: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-ZD3kKK1_deQ.html

First: Thanks so much for your kind words! Second: I have discussed another book possibility with Manning, and SQL Workout was what we are thinking about. But to be honest, I need a bit of a break from writing books! That said, I'm hoping that when things stabilize with my subscription service (at LernerPython.com/), both in terms of more subscribers and also getting the back-end systems more robust, I'll have time to think about new book projects, as well as re-recording a bunch of my existing courses and creating new ones.

My pleasure; glad you enjoyed!

Excellent!

You return True from `__exit__` to ensure that the exception isn't re-raised.

Glad to hear it helped!

The future of Pandas is clearly pd.NA, and I should use it more! But in this particular case, it didn't make a difference: Using either np.nan or pd.NA will turn the dtype into floats. That's because the standard int type isn't nullable, meaning that it cannot handle pd.NA as anything other than a float. If, however, you were to set the dtype to be Int64 (note the capital), then using pd.NA would indeed do what you (and I) want.

NaN is a float. So if you want to have NaN in an int column, then the ints will need to change to floats. HOWEVER, if you create your series with a nullable type, then you can use pd.NA instead of np.nan, and you'll be all set. That's because pd.NA is compatible with a wide variety of types: In [12]: s = Series([10, 20, 30, 40, 50]) In [13]: s.loc[3] = pd.NA In [14]: s Out[14]: 0 10.0 1 20.0 2 30.0 3 NaN 4 50.0 dtype: float64 In [15]: s = Series([10, 20, 30, 40, 50], dtype='Int64') In [16]: s.loc[3] = pd.NA In [17]: s Out[17]: 0 10 1 20 2 30 3 <NA> 4 50 dtype: Int64

You can totally do this! Just use "diff" on the result of a "groupby". For example: df = DataFrame({'category': ['wheat', 'corn', 'rice', 'wheat', 'corn', 'rice', 'wheat', 'corn', 'rice'], 'price': [10, 8, 6, 11, 7, 5, 15, 9, 4]}) df.groupby('category')[['price']].diff() You'll get a new data frame back (thanks to the double square brackets around 'price'), showing the difference for each row from the previous occurrence of that category. However, if you want to know which category is which, you'll probably want to join it back to the original data frame: df.groupby('category')[['price']].diff().join(df, rsuffix='_df')

You bet!

Glad you enjoyed it!

Self is the new instance. Attributes set on self are the Python equivalent of setting instance variables. We thus take the parameter x (whose value was set by the caller) and assign it to self.x, keeping the value around. We then do the same thing with y.

I think the launcher is just a way to start new notebooks, consoles, etc. Each of those still has its own kernel.

Thanks for the kind words! Notice that in many (not all) of the cases in this video, I'm not actually modifying our data frame g. Rather, I'm running a method (e.g., unstack), and getting back a new data frame. This leaves g unchanged, which means that if I first run g.unstack(level=1) and then run g.unstack(level=0), I'm unstacking on the same data frame each time. It's not that after unstacking on level=1 we're left with only one level in the multi-index. Does that make sense?

@@ReuvenLerner Yeah I got it now :) that was super quick response, thanks!!

Glad it helped!

Kept using # with words without space. I thought something is wrong with my pc.

Glad you enjoyed it! The idea of "cells" that can contain either code or Markdown is special to Jupyter. So if you're using PyCharm's paid (professional) edition, then you can fire up a notebook and use Markdown there. But in a regular ol' PyCharm (or Python) file, you can't.

Thanks so much! I'm not a big fan of query, just because it introduces a totally new and different syntax -- it's sort of like embedding SQL inside of a program. That said, I admit that query can sometimes make things run faster, and can also look a bit clearer.

It's in the data set for my book, Pandas Workout, at files.lerner.co.il/pandas-workout-data.zip

I'm delighted to hear it!

Thanks so much for your kind words! Hoping to do more videos very soon...

Great to hear!

Glad it helped!

Thanks for your kind words! I have a (paid, recorded) course on design patterns, but I don't think that it includes the builder pattern. (It often depends on whether I have time to fit it in.) If not, I'll see if I can do something here on RU-vid.

@@ReuvenLerner Thank you! I'll be looking forward to that video. In the meantime, do you have a link to your paid course?

@@hashimnaushahi Absolutely -- the design patterns course is at store.lerner.co.il/design-patterns (which isn't explicitly mentioned yet, but is a free part of my course membership at LernerPython.com). Let me know if you have any further questions!

Then you shouldn't be combining them, in this way or any other way! My assumption for this video was that you have a data set broken up across a number of CSV files, each with the same column names and dtypes. You want to take these multiple files and turn them into a single data frame. Pandas provides us with the "pd.concat" method, which is good for such things, but the problem is how you read them into Pandas quickly and easily. If you have geological data, survey data, and market analysis, then *perhaps* they have some factors in common. But you don't want them in the same data frame. Rather, read each into its own data frame, and use "join" or "merge" to combine them.

@@ReuvenLerner ok.. thank you

That's not a Jupyter problem per se; it means that you have a syntax problem in your Python code. Double check your code; maybe there is a missing parenthesis, quote, comma, or period?

Thanks so much!