EVERYONE Needs to Learn a Little Bit of AWK! 

Gary I will never forgive you for teaching everyone my only impressive linux skill

"forget spreadsheets and excel" crowd: ooh? "use command line!!" crowd : oh...

Awk and grep were the heart of many scripts I've written over the years.

If you like using Awk to do stuff like this, there are a few other cool Unix tools you should have a look at. Split, join, uniq will allow you to do DB-like things on text files. Another is tr. Of course sed, cut, grep as well. I took a grad course in which we had to create software development tools only using Unix tools. That class was both illuminating and fascinating. Learned a lot that I still use to this day.

Used AWK in 70's and 80's. Had an accounting system written in AWK. Also, had an AWK to C compiler, for the real hardcore number crunchers.

awk is one of my favourite tools, not because I'm any good at using it, mind, but just because of its history. It's from 1977! The basic apps of our unix/linux ecosystem have a rich history.

Back when I was contracting for EDS one of the sys admins handed me a book on AWK and asked me if I could figure out a way to extract billing info from the processing logs. I was hooked. :-) One of the handy features was using strings as array subscripts, and having "sparse" arrays, where only the accessed elements existed. Eventually, I had most of my production automated with AWK scripts.

This is the only skill in my job and my previous job where people turn to no one other than me..:*( Getting sentimental here...

Awk is really amaz... Syntax error: Missing ending '}'

AWK? I thought I was the only dinosaur in this world which still uses AWK. Glad to know that I'm not the only one.

00:00 Intro 01:15 more show data in file 01:29 wc -l show number of lines in data 01:48 awk '{print}' prints file 02:24 awk '{print $0}' $0 prints every line, $1 prints first field ect. 03:10 awk '/gcc/ {print $1}' match gcc print file name 03:41 awk '/^w/ {print $1}' Lines start with a double-u 04:23 awk '/path/ {print $1,$2}' Lines start with path, include multiple lines 04:41 awk '/path/ {print $1,$2/1024}' can divide output

I've tried reading more than one awk intro and I've never made it very far. You've successfully taught me way more than all of them, thank you!

This is the best introduction to awk I have encountered.

I survived my PhD thanks to awk and sed!! Command line rules!

Man I've watched so many awk videos and this has been the only one that has said "this is why you do it" rather than just do the thing I don't know how people expect you to learn when they don't explain what you're doing

AWK isn’t a number manipulation tool. It’s a text processing tool that can do math. And so much more. Explore GAWK, the Gnu version. And use the tools you know, as best as you can to get the job done. And don’t stop learning.

It's 2:00AM and I'm watching Gary Explain awk... and it was amazing! Thank you, Mr. Simms!

Just want you to know you saved my ass with this video. Procrastinated on an assignment for my CS class and this really helped me understand some stuff I'd missed and get the assignment done in time. Thanks a ton!

Just thought I'd explain, that 'up arrow' is a caret or circumflex.

Nice video Gary. In case you (or anyone else watching) didn't know - you can simplify your "rounding up the file sizes" example using printf() and a format specifier. For example to print the file sizes to one decimal place you could use: {printf("%s %.1fKb ",$1, $2/1024)} Or: {printf("%s %.0fKb ",$1, $2/1024)} To recreate your exmaple & round to the nearest integer…

WHOA! As the class went on, My eyes only widened. Thank you Gary! Much love

Best introduction I've ever seen! I've always been kind of reluctant to learn, but knowing inside that I should do it... Thanks for the video!

Gary, I love these introductions to Linux/unix commands/software.

I really appreciate how you build up here. I've done most of this in my tinkering but this adds some context that's really helpful.

This is the best awk tutorial I've seen so far.. please make a video for SED

I used awk to do data analysis of a large database that output data as a .csv. The company hired an "analyst" (really, he just ran DB scripts) to put the data into Excel, format it and run a few math functions on it. It took 3 hours to complete their way. The awk way took less than 15 minutes.

Awk and sed one of the most useful and powerful text manipulation and formatting tools I ever learned to use.

I used awk in the early 1990’s as a developer. Not sure it is something that a non developer should really use. Good demo, I wrote complex code with Awk to parse flat files.

Used quite a bit of AWK in my 3rd year physics project. I had hundreds of experimental data files to process and it was a good choice.

I have used it to generate a useful database of user information from emails after concatenating the emails for processing. It was not hard to learn and ended up being a very useful multi-purpose tool in addition to its primary mission success. Thanks for an excellent video!

For small files there are more user-friendly tools, but awk really shines when you have some huge text file that you need to massage because it is just about as fast as you can get. Really important to know about in those cases, because you could easily be lead down an unnecessarily slow or expensive "big data" path.

I really love awk, it was one of the main tools I used for my thesis.

Easy to learn, too. I love awk! Thank you Gary!

I use awk at my job and I am always in awe of it. This video is a great little intro and the rounding logic was pretty neat too! Thanks Gary!

I have been using AWK for 20 years, it rocks!

AWK, Sed, Bash, TCL, GREP, Perl and Nvim are my command line friends. 😍

*GARY!!!* *Good Morning Professor!* *Good Morning Fellow Classmates!*

I uses awk/sed on a daily basis at work. I uses AWK primary to analyze excel(exported to csv) or other data files for audits. That is on Windows! In both MINGW64 or WSL2 Linux.

I've been just doing this from any programming language I was learning when I get to the read and write files section of the documentation. Nice to see it can be done directly on the command line.

Gary, awesome job giving me the basic understanding of awk. All my little failed projects have been revived since Your walk thru of the AWK!

Love me some AWK and have made plenty of use of it over the years.

Thanks Gary, that was simple and useful. I usually write small Python scripts for such data processing because I'm more fluent in it, but if it were CSVs or tab-delimited outputs (like from other shell programs) awk is just way simpler. Always wished I got some proper simple intro to it, now it's done :-) - perhaps you could make a follow-up episode or two? Thanks again!

Thanks. Brought back some great memories of data manipulation of huge point cloud datasets on SGIs. We had to do very similar things before piping data into the OpenGL 3D engine for visualisation purposes. Awk is very flexible and fast and still have many usecases in todays system administration tasks.

He has the smoothest advertising transitions I've come across. Great job! Great content, too!

Good old integer arithmetic, takes me back to when I was a lad.. Awk would have been very useful in a former life. Thank you very interesting.

A long time ago when someone was telling me how wonderful Excel was, I simply said "ed, perl, tbl, troff" as in edit your data using ed (actually, I never use ed), process it with Perl (I don't know awk), and finally format it with troff using the tbl preprocessor.

I love it! The best awk demo on RU-vid. More awk vids in the future???

Very glad this video popped up on my feed! I've been currently working with data using sed but after watching this i think awk it much more suited for me, especially knowing I can write my own functions that run faster than Bash can! Great video, thanks for the explanation!

I love awk for text formatting and, arguably informal reporting. Also admin scripts. Honestly, though, you can all this and more with PERL, which I recommend.

I remember using Awk for extracting a column from a command result(using something like {print $1}, but I didn’t know that it could do much more than that.

high quality video. perfect audio. the pace was perfect. the explanation and examples were perfect.

Wow, i've just stumbled upon your channel and i'm very very impressed with your teaching skills. Absolutely brilliant. Thank you soooo much for doing this!

In the early 90s: hey everyone, learn guis!! Today: hey let's go back to the command line!!!

Please can you make some more AWK videos Gary? I'm learning AWK at the moment spent a few days on it, its hard to learn but the rewards in knowing how to use it is worth the reward. This is a great video to get people into using it and seeing the power of it.

awk is useful as a very short/quick way to get at Nth field in a text file, especially as it treats consecutive delimiters as one. for example, fixed width input files. But where awk really shines is multi-levelled line delimited files, like old style config files etc., as you dont need to write loops and keep flags about which section of the input file you're in.

I learned a lot of AWK about 20 years ago - very useful

My first time hearing and knowing this language!

I've used awk to extract useful information from pdf documents. The problem was that the information was awkwardly (pun intended) split into several tables throughout the document so I had to first process each table to collect up all the pieces for each element I wanted to output. The solution I came up with was pdttotext + awk to do the processing. The few hours I spent on that awk script has paid off nicely since I've had to reprocess new versions of the same document several times over the years. The alternatives would have been: a) Manually copy paste all the information. One thing I've learned over the years is you *never* trust anything copy pasted by a human (least of all myself)! Also would have been extremely tedious (which adds to the chance to making a mistake), and I would've had to repeat it whenever a new version of the document came out. b) Find some pdf library for my favorite programming language to extract more structured data from the document. Couldn't quickly find anything that worked and I didn't want to start debugging pdf libraries.

Awesome video man!. This was my first contact with awk command. Very very good explanation!

This is the first I have heard of AWK. I am number crunching sports recreational sports handicapper, so AWK might be useful to me.

I won't go into the specifics but AWK holds a special place in my heart. I know that might sound a bit weird but it's true. Even though I've only ever had to use it in anger twice it was well worth learning just for them.

As a senior software engineer, I understand the advantages of what you’re demonstrating. However, the common person will never use this on a wide basis. But, your heart is in the right place.

Thanks for the info. Spreadsheets are more practical for me, and more practical for newbie experimentation, but AWK really bridges the gap between sheet logic and optimizing work. Plus, you get support from a lot more scripting capabilities. Great stuff!

This has to be the best clarification i've ever seen . Thanks a lot !

Databases! I was using SQL and databases / Dbase for a long time. Can't stand using spreadsheets as databases.

perl >> (sed, awk ). You can do all of sed and awk in perl (and there are even conversion scripts for it, called a2p and s2p) but not the other way.

Outstanding explaining skills!

However much you are being compensated for producing these videos, it is not enough. Thank you !!!

awk -F "/" {print ...} . for specifying the different seperator other than default whitespace

Let's become terminal ninjas ;) , AWK is very used in linux commands

My ex-colleagues used to hate me writing awk scripts! Brilliant little language. One happy use was to take the output from informix commands to detail table descriptions and create an output shell script to recreate the database for disaster recovery purposes.

AWK has been used to create a full parser/tokenizer, and other purposes that are arguably way more complex for its intended use. Using AWK you can also make advantage of pattern matching with regular expressions, and AWK has many other tools for text manipulation. But I think one of the most powerful aspects of AWK is using it as a complimentary Unix tool. Use it together with other Unix command-line utilities with pipelining, not everything has to be done in AWK. For example, you can use AWK to parse out formatted words from a complex text file, and now pipe this data to be processed by another utility.

Your way of explaining topice is very easy... Please make more videos on linux

Reminds me of my days on the Sparc 2!! :-D Those were the days. *sigh*

Thanks Gary for explaining awk in cmd

Thanks Garry!!. It's really interesting

Just a question from a newbie: what can I do with those informations? I come to your channel, via ColdFusion and the graphene battery! Thanks in advance

awk one liners are great for ad hoc queries and I use it for that, but as soon as you go to scripting surely perl is the way to go? Or if you don't already know perl, then maybe Python which is more friendly for beginners?

Thank you for the crystal clear demo !

Haven't understood awk before. Thanks, Gary!

I used to use GREP, AWK and SED in the 80's while porting a CAD program from on operating system to other. But nowdays I tend to use PERL and many times with excel. You can do many things with excel, but complex data manipulation tasks are much easier with perl. One of the best concepts in data manipulation with PERL and AWK are associative arrays.

Most young devs don’t even know about commands that aren’t cd, ls, mv, cp or docker.

Gary! I'm excited to find your channel. Pleasing. Subscribed!

Excellent! You explained awk very well!

Kay, maybe if - for god knows what reason - I'm writing a super complex bash script. Even then, probably not. I generally just sub in a proper scripting language for that.

I use AWK a lot. And while I'm a C++ dev, I'd still recomment Python as a replacement for excel sheets for quick calculations. AWK has severe limitations, which makes it a bit harder to use for anything more complex than basic arithmetics (or string manipulations, but even that is a bit difficult sometimes.)

Regex, SED, & AWK are awesome tools!

Great video! I have always used grep to search strings in linux and never bothered to figure out what awk did.. This was a great introduction - Just what I need so that next time I have a use case I will remember this and figure out how to do it in awk!

subscribed. dammit, I'm not a coder (much of) but you made that understandable even while talking blisteringly fast! kudos.

Every time I hear the word “Awk”, I reach for my Perl.

Great tutorial, subbed.. thank you!

Finally a concise awk tutorial! Thanks!

Idk, the idea of learning awk has been rattling around in the back of my head for a while, I just don't feel like it's worth the overhead when I could do all this just as easily in Python.

So does this transfer seamlessly to "console" in MacOSX (a UNIX command line)? And thanks for pointing all of us "awk-ward." Fred

Awesome tutorial. Thanks.

I'm actually enjoying these technical videos of late!

Unfortunately he covers only functions working on single lines. But he promotes use cases like summing and aggregating. Would have been nice to see at least one example of that.

(AWK)ward?

My favorite tool for making sense out of long log files.

OMG... after YEARS of programming I happen to learn such a trick to implement a round() function! ASTONISHING! Thanks! I used to use awk to only get a specific column, but that is such a nice tutorial on awk.