Dave tests the speed of Bash vs Powershell vs CMD, then tours the code for each to show how they differ. Seeing a problem you already understand how to solve in a language you do not yet know how to use is a great way to learn!
All the worlds a nail when you’ve only got a hammer! :) I’m totally guilty of some really arcane batch scripts back in the day. Thankfully I haven’t needed that in decades, nowadays I abuse sh and bash :)
I once wrote a prime-number generator in dBase III back in '89. I just left it running when I went home from work for the day. It was as optomised as I could make it, but it was anything but fast.
I'm impressed as well, but it was totally not unexpected - in fact, when this thing started, I already expected that there will most definitely be someone who will write an implementation in BrainFuck - I just checked, just in case, and there in fact is a BrainFuck implementation. I mean... Not to belittle the crazy men who write these things in CMD, Bash, etc. etc., but BF kinda always takes the lead on how insane it is to program any "thing X" on language not designed for it. I think Turing wouldn't be surprised - if such thing as microcomputer running something like CMD was introduced to him back in his time, I think he would have marveled how powerful this high-level language of CMD was :DDDD
The importance of this series is not the fairness in time comparison. The best thing I learned here is that sometimes you only have one tool near you, it could be fast or slow, but even that cmd simple tool can achieve what you need. Thanks and congratulations to all the contributors.
Was really happy to see PowerShell come out on top of this. In a different life, I used to be a hardcore Bash guy. Over a decade ago, I got tasked with taking over a system that was used to automate deployment of training environments that are used to train new employees as well as existing employees and customers on new products and technology. These training environments were to the complexity and scale of production environments, but each one would function as a sandbox for students taking that particular class. The infrastructure as well as the student stations were all built on top of baremetal Windows servers and this project and its scope would completely redefine my expectations of what a scripting language should be capable of. I will likely get some flack for this, but not in Bash's wildest dreams could it accomplish what I was able to accomplish in PowerShell. Some background of what the PowerShell script did... First, it would prompt the user to either select a date/time to deploy or deploy now. Afterwards, it would hit a SQL server and find a list of classes in a table and present that to the person running the script to select from. Once you selected a class to deploy, it would pull the list of available baremetal servers and if the deployment job was for 'now', it would run a join against 'in use' servers to get a list of systems that were currently free. The user would then select which server to deploy the class to, some of which required 3 baremetal servers per student. After selecting the servers, PowerShell would create a background task-scheduler job in Windows Task scheduler for each machine so that it could multitask. It would add the machines via their MAC address to a System Center server and make selections on the System Center server about what OS and featureset each system required. From there, it would connect to the server via an out-of-band (iLO, iDRAC, CIMC, etc) powershell module, feed it a configuration file that would wipe the system, then initiate a PXE boot which would cause the server to get a DHCP address, then connect to System Center to pull the OS image, drivers and config. At each phase, it would monitor the status of the baremetal server and wait until it reached the OS before installing Hyper-V, System Center, and whatever other roles and features it needed via a dynamically generated answer file. After Hyper-V was installed (as most were virtual machine deployments), it would automatically install and configure iSCSI initiator, connect to a third party storage array via its own PowerShell module to clone a template LUN for that class' storage and set Access Control (via IQN and IP) for the desired server to access the cloned LUN. There were also physical switch configs and firewalls that were modified as part of all of this as well. Afterwards iSCSI storage was accessible from the baremetal server, PowerShell would launch a class-specific PowerShell script from the iSCSI disk. This would import virtual machines, create virtual switches, setup NIC teaming, Storage Spaces Direct, Switch-Embedded Teaming or whatever else the class environment needed for that station. There were a number of possible failures you could run into as well; system off, ip not responding, etc that the students might cause from previous classes, so I had to accommodate those via the same script as well as log everything that was happening to a database. Finally, it would create randomized user accounts in Active Directory and grant them access to their respective server, then email the person who deployed the class the credentials for the user stations. In the end, this allowed people who had no technical background to start the process and have a production-scale lab environment for as many students as they needed within 10-15 minutes for any of hundreds of class. It was a metric ton of work and took nearly 3 months before I had the first class deployed and another 3 months to incorporate all the other classes and work out the quirks. This project required interfacing with everything from system center, exchange, active directory, the IPMI device in the server via a third party powershell module, the third party storage array's powershell module, among many, many others to get all of these dominoes to fall just right and man was it an endorphin rush watching everything happen without me so much as touching the system. All of this was 20k+ lines of native PowerShell 5 and it proved to me that Bash is inferior to PowerShell in nearly every way as I don't think I could've accomplished it at all in Bash. Today, we've largely moved onto templated nested VM environments, Ansible, and Python for class deployments. In addition, the vast majority of the equipment and software we have now uses Restful APIs, so much of this could now be accomplished with curl in Bash, but for years, this was the mainstay of our lab deployment infrastructure and it was awesome! Anyway, love your channel and been waiting for this. I'm hoping to see a Python vs PowerShell one in the future as I've become quite comfortable with both of them.
@@gerardmarquinarubio9492 I got anxiety scripting it. Ended up pseudo-coding the entire thing initially though, broke down each piece of the puzzle and it went surprisingly smoothly. Honestly the most problems I had was with System Center, the PXE Server and the answer files and our DHCP server needing to support both UEFI and BIOS installs. It wasn't (and still isn't) very well-documented getting UEFI and BIOS working in harmony in a PXE environment and it requires configuration in DHCP to identify the host as BIOS or UEFI, separate images and separate answer files.
It is always a thing of joy to see stuff being done just to prove that it could be done, by something whose creator never even remotely had that task in mind when it was created and carried out by people that truly know how something works. No matter how slow or messy the result appears, it is always a thing of beauty because it exists.
The kill command is a built-in command in bash so that is probably why they are using it as no process needs to be forked. I believe the Korn shell, or at least some variants of it, has built-in math functions as that would be the primary reason why bash was relatively slow. Kudo's to the guy that tried this in CMD, I wouldn't have even considered that. After I started programming on big iron back in the late 70's, I left programable calculators but that would be an interesting comparison. My last one was the old HP-29C, as long as your program fit within something like 99 instructions you could do some neat things. Wish I still had one, and that it worked (I expect battery and continuous memory would age away).
Even if it isn't, OS caching should take care of it if being run multiple times, the date command has to run the formatting code after getting the unix timestamp from the kernel, the kill command is just asking the kernel to lookup that process ID.
A quick check reveals that ksh has at least following mathematical functions: abs acos acosh asin asinh atan atan2 atanh cbrt ceil copysign cos cosh erf erfc exp exp2 expm1 fabs fpclassify fdim finite floor fma fmax fmin fmod hypot ilogb int isfinite sinf isnan isnormal issubnormal issubordered iszero j0 j1 jn lgamma log log10 log2 logb nearbyint nextafter nexttoward pow remainder rint round scanb signbit sin sinh sqrt tan tanh tgamma trunc y0 y1 yn Also, it supports floating point numbers, but that's not useful for this purpose... But proper sqrt returns floating point number, so you would want to truncate it to integer for purpose of checking if a number is a prime or not. So, hmm, ksh implementation should definitely beat bash - someone has to do one, if someone hasn't already ;p
@@ovalteen4404 Yes, but ksh has floating point math support and a whole load of arithmetic functions, including sqrt, so I would think that ksh implementation should be able to leave the bash implementation eating dust :)
When I originally read this, I thought it was smart - not because "no process needs to be forked", but because the rules are that you aren't supposed to use calls to external commands, but rely on only what's part of the language - which is why calling `bc` for faster math for example would be a no-no. The use of sleep and kill for timing is a really smart idea, as both are bash-commands. The full ingeniousness really dawned on me right now as I realized that not only calling built-in 'kill' is faster and doesn't violate the rules, but it dawned on me that if bash didn't implement it's own kill (and here's also the reason for _why_ it does implement it - I mean, it doesn't implement commands that already exist by default on every Unix/Linux installation): ```Kill is a shell builtin for two reasons: it allows job IDs to be used instead of process IDs, and allows processes to be killed if the limit on processes that you can create is reached.``` *Calling /usr/bin/kill wouldn't even work!* Because `read` is also a bash built-in, it doesn't have a PID! The value it uses is the job ID of `read` and /usr/bin/kill would have no clue about it, just as it won't be listed by `ps` either. ...however... The implementation in the video is older though - it uses `sleep`, which breaks the rules despite of using `kill` to check if it's running. I checked the current implementation, and it uses this instead: `read -rt "$RUNTIME_SEC"
I had been looking forward to the continuation of this series for so long. I just finished running the github files on my 2020 M1 Mac Mini (like while this premiere happened).
...AND having the entire .net framework bolted onto the thing....and being able to pass output as an object instead of raw text--makes input/output chaining a helluva lot simpler.....
@@artstrutzenberg7197 and at least partial support for bash commands, which makes it a lot easier to operate when jumping back and forth between windows and linux machines frequently
@@romanpul the fun one? Did some .net 6 service installs on a Linux machine. the install scripts ended up being a combination of powershell calling various Linux commands :D
I remember back in the 80s somebody made a set of batch files for CMD that could do basic math -- it consisted of hundreds of files named 1.bat, 2.bat, 3.bat, etc, in subdirs named plus, minus, etc with sub-subdirs named 1.bat, 2.bat, etc etc. At least, I think that's how it worked. It was one of the kludgiest things I've ever seen, but it did actually work (so to speak).
COLOR.BAT is easier to understand on how this is made for as the 1.bat ...and we can use the choice command to press only the number 1 (checking errorlevel in revers order) to jump forward to the call COLOR.BAT command, so we do not need to write the full name of the batch file to call.
Oh dear god :D Btw, there's an example of doing object oriented programming with DOS batch files (which automatically also means that it's usable with CMD as well) - it involves some really kludgy stuff that involves creating sub-directorys and batch files at run time and then executing them when required... It actually doesn't need manually creating a set of files like that, the implementation is much smarter, but also extremely kludgy :D
It blows my mind that a prime siv in CMD can be written, never mind that it actually works. They could have saved some time with that square root calculation by saving the previous value and checking if it needed to be incrimented from there.
I can't wait to find out that CMD is the slowest and still write another emulator in it anyways Also, it's immediately clear to me that this guy isn't an experienced batch programmer but someone who happens to know a bit about OOPs that found ss64 and tried to transliterate some existing code. Awfully warped implementation there, I'll have to take a stab at it myself. There IS a FOR command that operates over a range - FOR /L IN (0,1,16) for example. EXIT is also a very slow command to process, so "GOTO :EOF" is usually used in place of EXIT/B Oh yeah, and whitespace is a big deal in batch script performance. Removing that can sometimes increase speed huge amounts and I'm sure in such a calculations heavy situation it would matter a lot.
bang on with the batch & oop experience. i would've used the for loop, but from my previous experience, it was much slower (didn't test it though in this instance, premature optimisation maybe xd). never heard of the `exit /b` Vs `goto :eof` perf difference, i just preferred `exit /b` due to how u can fit it inside macros (set variables that are expanded on the top level) interesting tip as well on whitespace being a perf point, coming from whitespace agnostic languages i'd never thought that could be a problem. a minifier for batch perhaps? finally, do reel me in if you ever decide to reimplement, i'd love to catch the ride (::
I was so happy to see this posted - I thought for sure this series had been dropped. Now I feel like I need to make a ksh version, since that's what I happen to work in on the daily...
PowerShell is surprisingly powerful with a lot of useful extensions. For handling JSON, for example. Not so popular as UNIX shells and tools, though. Working in plain CMD shell for anything complicated seems like an exercise in masochism to me.
Hey Dave! As you know I tinkered around with the prime Sieve recently and wrote a program from what I understood. Thanks for mentioning the Bit Array in this video, I picked up on the way the index of the array is used to represent the value of its position, Thanks! That will probably become very useful in many ways. It is very similar to the way I have used an array of characters to convert between binary, octal, decimal, and hexadecimal: [ '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F' ]. Now rather than an array of characters I understand the concept of using a Bit Array in the same way. THANK YOU!!
I put assembler mnemonics or mashine code in form of hexadecimal values(0-FF) inside of batch files to create executable with a little help from Debug. But it is also possible to use visible ASCII character as binary mashinecode to redirect the bytes into a new executable only with a pipe operator(ECHO some hyroglyph>file.exe), but not all values are visible character. I saw a bash script with an elf file header and a routine for linux too.
Dave I love how you acknowledge MS has made less-great stuff precisely because they have always been zealously backwards-compatible. I should own up here; I've been an MS hater for the longest time, but I've relaxed on it because 1. you and 2. I really ought to be professionally compatible with what's used in practice. Thanks Dave; thanks for being the reasonable middle man that brought me over. Also: zsh ftw!
Over a period of years I did oceans of C shell and quite a lot of Bash. However, latterely I needed to do some in-depth script for Windows automation and had to take the plunge into Powershell. Sooooo glad I did, cos it not only does everything the others can do, but (not being an agglomeration of stuff from different people at different times like the *nix commands set is) it is far more intuitive to use and far, far better documented. I've not done anything this mathematical, but for the things I have done, I have found it's performance to be very impressive.
As a Microsoft Admin, I use powershell almost daily, and like it a lot (even if I’m not doing anything that advanced with it) There’s so many things in m365, exchange, and windows that cannot be done in the GUI and require powershell
Now that I am thinking about it, it seems a lot of people who went into programming got their starts on C64s and other 8bit systems coding in BASIC. We had a PC in 1984/1985 when I was 2 or 3 years old, it's no wonder I never learned to code as a kid (when I may have had the patience and energy to do so). Sure I found HTML (the one thing I sorta self-taught myself in high school) and Turing (an educational language originally developed at the University of Toronto and was part of the computer science curriculum when I was in high school) somewhat cool, especially the air hockey game I made, but I never enjoyed the process. I remember going to my friends place and seeing their C64 and watching them type in a program, or some convoluted (compared to MS-DOS) command to load and execute a program from tape or disk. To me as a kid that was primitive. Today, as an adult, I wish my friend still had his C64 and would trust me to mess around with the hardware as I'd rather be soldering than coding lol.
@@stevekarvo6299 the GUI is just using the same APIs and cmdlets that are available in powershell, but have to be programmed by Microsoft, and they just knock out the basics in the gui. A couple good examples: in m365, if you want a new user to be in all the same distribution lists as an existing user, you need to do them one at a time in the GUI, but in powershell you can pull all the groups of the existing user and use a ForEach loop to add the new user to all of them in about 2 seconds. Another example is properties of an object, there’s hundreds of attributes stored about any object in m365, AD, etc. In AD you can use ADSIedit to access all the attributes directly, but if they put everything into m365, the gui would be huge and clunky. With powershell you can easily access and manipulate those hidden attributes.
I smacked the like button (AND subbed) again before even watching JUST for your work on NT. But, I commented to say this: As a fellow content creator, I love that you added some of the bad takes at the end. :P I keep intending to do that too, but I'm always so burnt out by the time I'm done a video, I just want to export it and go to bed.
The "new" operator in powershell means its referencing a .Net framework function, so powershell basically can do whatever you do inside a C# project, as long as you stick to .Net framework
Well that was quite a race! I tried to picture what each of those languages would look like if they were vehicles on the road, and well I have never seen a unicycler being chased by scooter following a golf cart but there you go! BTW I bought your book! and I'm through the 2nd chapter! Great words of thought there! I'm starting to wonder if I'm Autistic as well! That would explain a lot in my life.
I started with just the Bourne Shell years ago, toyed with CSH (hated it), then found bash and was enamored. I think I tinkered with zsh, don't remember what I thought but was very happy with bash for many years now since it's default on most Linux distros.
PowerShell is one of the biggest achievements to practical computing. It's clear concept, huge API, and broad availability makes it an extremely useful tool. It's a bit hard to learn, but it enables scripting where no script has run before.
i remember i wrote a "classroom control program" for a high school compsci class that was just a batch file that presented the teacher with a menu asking for their classroom number (the domain had a naming scheme where each pc in a classroom had a matching start to the domain name, eg. room 227 was something like HS227Cxx) to load the next portion, which would call their list into the next stage, which would ask for a computer number in the room, and ask if they wanted to shut them down, send a custom alert (shutdown window with custom message), view all the processes running, or kill a preset list of common games kids were running at the time. it would then go back to the "computer select" screen. i really love taking the longest way to do something
i've been doing powershell for i guess 3 years or so... it's almost just c# with different syntax with all those libraries available so it will have the edge in those respects. for you bash folk it's definitely worth a look at with PSCore available on the major OS's
BASH does have the ability to find programs in the current directory. It checks the PATH just like CMD does. If the PATH has "." as an item, that's where it will look. UNIX has long considered that to be a way to shoot yourself in the foot though, so it is not added by default.
That Dan brown dig earned you my upvote. I've been a member of the arcanum my entire career and made an industry transition and ended up in PowerShell word. I miss my silly little symbols and the Gregorian chants.
I found PowerShell to be perfect for writing DSLs (Domain-specific languages); also it is very easy to integrate PowerShell SDK into a .NET app - it will be able to execute those DSL scripts without PowerShell runtime installed system-wide.
Last script I wrote in CMD was a script that would take in a file name, pass it to a program as an argument, and save the result as the same file name with an extension added. Took me close to an hour to figure out...
There is a CMD compatible shell called TCC from JP Software. I’ve been using it for years because it provides colored file lists, pop up menus for command history, and dozens of functions and variables not available in CMD. It also has a For loop that can iterate over a numeric range. It would be interesting to see if the CMD implementation works as is using TCC, and then to revise it to take advantage of TCC extended features. I’d do it, but I’m currently deep in learning PowerShell and not ready to put that aside yet. In fact, for day to day work I’ve switched over from TCC to PowerShell now that I’ve configured it to give me everything I had in TCC. The only downside is that it takes over a second to load the profile, but I don’t need to launch a new shell that often so it’s not a significant drawback for my use case. One thing that this video taught me about PowerShell is that you can declare static variables. I had no idea, so thank you for that!
This video drives home the point that while shell scripting languages are powerful enough they're geared towards certain types of tasks and if you aren't doing those tasks you need to re-evaluate your decisions.
For Bash, date would require exec()ing an external command, while kill is a shell built-in, so there should be a measurable difference in runtime between the two.
Ten hours! I nearly swallowed my tongue; hilarious! We used to write stupidly huge batch files to gather data from several different machines; I can hardly read them today.
One of the contract jobs I had was to support a collection of batch programs that handled data being provided by vendors for a company. Most of it ran on a fairly early blade server that I don't think would have passed Y2K compliance. (different issue.) I ended up being hied into the company for my networking skills, and left after demonstrating that I didn't have the project management skills to handle a load of 80-100 network projects, but with the batch processes I ended up entirely re-writing the monitoring code in a few hours after accidentally deleting the earlier code, and learned a lot about re-entering batch scripts. and safely exiting. The best part was hearing through the grapevine that by my being contracted in, the support call volume for the collection of programs went from between 5 and 20 calls a day to 1-2 calls a week, and going down. In the first month. There was a reason they wanted to bring me on full time. I was also readily willing to admit that _at the time_ I was not ready to re-write the entire package in any of the compiled languages I knew about. It's possible I could now, but I don't need to, and since it was moving files about, I would probably re-write the entire system, including monitoring, in bash. Well, I might do it in python if I had to write something that would run on both windows and Linux or bsd. I could learn to write it in powershell, and it would probably be faster, but it would be part of a learning experience.
10 hours is on native cmd! all other impls ran inside a linux docker container, meaning you'd have to run docker, run wine, then run cmd *shudder*. it does work, but i never made it to the end to get a time with that config xd
I remember when ported a bunch of subversion commit hooks from BASH to CMD and I had to truncate whitespaces from a string in CMD... OMG was that less intuitive than I excepted it to be :P
Wow, that sieve in bash took almost as much time as compiling the Java JDK on Windows back in the day (30-45 minutes on Linux on the same machine, vs 10 hours on Windows and one and a half day to prepare the libraries).
I started off as a bash/python advocate, but at my current job they use powershell for scripting. Powershell is very underrated and powerful scripting tool. It has won me over. I'm glad this video confirmed that.
Interesting comparison and results, though I'd like to see a comparison with them all using external programs as well. For most general calculations, if I need them specifically at the terminal or in a shell script of some kind I use 'bc', but given the way it processes, I definitely wouldn't speed test with it if you value your time.
I enjoyed the video. It almost feels a little cheap that powershell won since it's using the .net framework for it's math. A few suggestions: It would be nice if there were links to the other drag race videos, as well as links to the related github repos in the description.
I really like this series, even though I haven't caught all the episodes. One thought I had was to see what changes when given a different task. The prime sieve is a great test of pure compute power, but it isn't what I would call an "every day" kind of task. I frequently have to search for duplicate files. I use a PS script that groups files of a certain extension by size (since only files of identical size can be duplicates) then hashes the files in each group to see if they truly are duplicates. There's a ton of directory traversal, file sorting, hashing... but it generally burns through some 16,000 files in just a couple seconds. I suspect this isn't an uncommon task. Would it be faster in a compiled language? Or is the bottleneck in the OS having to grab the files? Not sure how something like that cold be fairly tested though. Not to ramble, just wanted to toss out an idea for a "rematch" of sorts that might play to a different set of strengths.
Hi I’ve recently read a blog introducing paging tables and found it very fascinating. If you know how they work I’d love to know how they’re implemented in windows or a little backstory on them!
What I don't like about Powershell is that it depends on .NET, the system tools in my opinion should have as few dependencies as possible and be native..
Since Windows has shipped with some version of .NET out of the box for decades now, can it really be considered a separate dependency or not native? Bash has dependencies too, for example. libc being the big one.
@@guspaz And libc is needed by most C programs, and if they want they can just link it statically with bash. But that will increase the size of the executable. Readline library is also something bash needs, and as explained in the wiki : GNU Readline is a software library that provides line-editing and history capabilities for interactive programs with a command-line interface, such as Bash. It is currently maintained by Chet Ramey as part of the GNU Project.
@@xrafter You can also build .NET Core executables to be statically linked with the required components of the .NET Framework included in the EXE such that the user doesn't need it to be installed. But, as I said, Windows has had some version of .NET built-in as part of the base install since Windows XP, and PowerShell has been included in the Windows base install since Windows 7, so dependencies are meaningless when it comes as part of the OS.
yeah, the .NET dependency also drags in the C language relation, clib, as well as sdl. And for legacy reasons I would add VBA on top of the list, in both functional programing and with interpreted languages.
They 100% are, but disallowing it that means you are outlawing the standard library for the language. Personally I think using standard Unix tools should be fair game, but you don't want someone just writing the whole thing in an awk script and negating the whole point of the drag race.
@@noVicda Would be interesting to see the results of a bash version that makes full use of the standard "library"! Nothing totally cheating like writing out a version in C then compiling/running it to get the results, but doing things like awk for help with math functions. It makes me wonder if the overhead for calling all those processes would end up making it not as beneficial as it might first seem to be.
Would’ve been a really really nice lagniappe to compile the bash and present the results of the compile run. Also I bet a Canadian dollar that even the fastest PS w/o the built-in sqr function would be slower than bash.
I thought the comment above the sleep stuff would have explained the use of kill adequately, but I suppose it's not entirely obvious why it says subshell instead of subprocess. In order to compare with the date function the output needs to be captured so in effect using kill, which is a bash builtin, avoids the need to fork two processes each iteration. Forking processes is very very slow compared to calling a builtin. Of course Bash only makes it through one iteration in the test conditions, but for smaller sizes it will make a difference. As mentioned at the end calling out to awk, bc, or whatever to do sqrt would be cheating. Especially since awk itself is a language in the competition. I did try using a faster algorithm but it kind of became a wash since accessing variables in bash is quite slow. Given that sqrt isn't called often (I think only once per run?) it wasn't even worth trying to justify optimizing it. I call shenanigans for not using the inlined getBit version of the script though! :P
@@xrafter It doesn't. Profiling bash pointed to the actual variable name look up being where most of the time was being spent. It's been awhile so I could be misremembering, but I also don't think Bash does anything different storage wise when the -i flag is used, it's still a string, just a conversion is done when writing to the variable (thus normalizing the value). Also fun related fact associative arrays in bash are faster than indexed ones since associative arrays don't perform expression evaluation of the array index.
Your comment might have adequately conveyed it to your intellectual peers, but mortals like me sure didn't get it! What's the difference between a subshell and subprocess and why doesn't a subshell have a process? Is it a thread context in the same powershell.exe process, or a fibre, or just a context? You say it all like it's obvious because you already know it, but slow down and explain yourself to us noobs. I was kind enough to go out of my way to give you credit in the episode, so less snark and more explain would be great. If sleep is a builtin vs the /bin/sleep, no one has said that, so I don't get why any one of sleep, kill or date is cheaper than the other. Which one is the builtin?
@@DavesGarage Outside of the joke that was the very last sentence of the post no snark was intended, but rather was just admitting that I could have been clearer and trying to explain it. My bad if you took it that way. Let me start with the builtin question: sleep and date are not builtins, but kill is. The "help" command in bash will give you a complete list of builtins for your particular version of bash. Builtins avoid the need to fork/exec and are thus extremely cheap to call compared to a normal program. An easy exercise to see how much of a difference there is, time the execution of 1000 calls to /bin/echo vs the builtin echo. It's about 3 orders of magnitude on my system. A subshell is a particular type of subprocess. Specifically it's a term for when bash forks itself and executes two code paths in parallel, so they do indeed have a full process and independent copy of all variables. (To be clear bash does not do multithreading, just multiprocessing.) The most obvious case of this happening is when you capture a command's output with $(...cmd...) (or backticks). The code inside of the parenthesis are executed in a "subshell." There are also less obvious times a subshell is created is if you pipe the output of one command into another. The right hand side is technically executed in a subshell and you can even use braces to process that output over multiple commands. Now if one uses $(date) in a while loop we see that nominally a subshell gets created which in turn would create a process for /usr/bin/date. (In practice, bash will see that date is the last command for that shell and do tail call optimization to skip calling fork.) Ideally we want to measure only the performance of the code being benchmarked (i.e. runSieve), so we'd like to use a builtin command to avoid the expensive fork/exec. There are a few ways this could be accomplished, but using a background timing process (sleep in this case) and checking if it completed via the builtin kill command is probably the fewest lines of code way to do it. This pattern does three things: 1) sleep only needs to be spawned once for the entire execution vs $(date) which would need to be spawned every iteration. 2) We can move the process creation outside of the timed section of code saving a fraction of a second on even the first iteration. 3) Feels good to know that no external programs are used during the actual benchmark, it's pure bash between tStart and tRun. As an aside, it is technically possible to replace sleep with the read builtin. I didn't do this since doing it right would make it even more cryptic, still require spawn a process (a subshell), and is outside of the benchmarked code. If one wanted to get rid of all non-builtins date could probably be replaced by reading something in /proc like /proc/uptime, but then it wouldn't be portable. Hopefully that helps!
Its awesome to see this all in popular scripting languages. While I don't expect CMD to ever be speedy I wonder if it could be just 1 or 2 days if we optimized the SQRT function.
I didn't expect it to be speedy, not even able to come close to bash, but I didn't expect it to be *that* slow either! Considering what you could run with 4DOS and 4NT already back in the late 90's, while not very demanding, I suspect they would fare much better at running this (although 4DOS wouldn't really - it's a 16-bit DOS application running in real-mode). I'm currently wondering how fast it actually is on real Windows CMD.EXE - the implementation benchmark is designed to be run with Linux and uses Wine to run the CMD implementation. Wine has it's own version of CMD, which is possibly implemented very differently. I don't understand what in the CMD's scripting syntax could possibly make it so hard to interpret faster than the results say, I don't understand - and that makes me wonder if Windows CMD.EXE is as slow? The benchmarks are done under Linux using Wine, and if I'm right and the syntax itself doesn't cause this to expected performance, it would be completely reasonable to assume that the slow speed is because of inferior implementation from Wine's developers - it's not known from being slow, quite the opposite: Wine is reported to run some games better than Windows, though IIRC it's been reported specifically with Direct3D games (Direct3D being Windows only Wine has to actually translate D3D calls to OpenGL - but perhaps because Wine is coded by actual wizards, or more likely because D3D is actually inferior to OpenGL and shouldn't be used, it still seems to run just as good, or even better, than Windows), but overall the consensus is that it's not any worse - but then, CMD.EXE has no role with performance of actual Windows executables, the developers likely focused only on making it compatible and didn't want to waste more time than necessary on it. But I don't have a copy of Windows newer than Win2000 (and it's CMD isn't up to the job, it lacks features that were added much later) - and I wouldn't install it even on a VM just to satisfy my curiosity. I don't have friends who could simply send me a copy of CMD.EXE from their Windows install because they don't use Windows either, and I'm not going to ask for someone on IRC to send me a copy so I could test the real CMD.EXE from Windows with Wine. So I can only hope that maybe someone will see this, or my other post about this, and try the implementation on actual Windows CMD. Even the documentation of the implementation instructs to use Wine, and instructions for Windows guide to installing Ubuntu using WSL and installing Wine on it, but there's nothing preventing anyone from running the .bat file on Windows. Meanwhile I want to check if Take Command (formerly 4NT) can run it - JPSoft says it's compatible with CMD.EXE, but I don't know if they have been able to add all functionality of current CMD.EXE _and_ not have them clash with their own extensions to CMD batch syntax. But Take Command and 4NT also advertice performance increase with regular CMD scripts, so I want to see it's performance - unfortunately that still leaves me with no knowledge whether Windows CMD performs as badly as the Wine's CMD. I'd like to also rewrite the implementation for Take Command (and to lesser extent to 4DOS, even if I had to limit it to smaller sieve for memory limitations sake) that takes advantage of it's extended capabilities, and to see how well that compares. This idea is constantly nagging at me in my head, because I'm so often seeing retro videos/articles on DOS, and videos/articles on using CMD to do crazy things _not ever mentioning 4DOS/4NT/Take Command,_ almost as if I'm the only person on earth who ever even knew about them. It's my chance to remind people that "hey, this was/is a thing." :D
@@robsku1 The windows version runs much faster than the reported numbers. I get about 24hours to do the standard set. I'm using Win10 build 19045 on a i7-6820 @ 2.7GH. Not great speed but much better than what the benchmark on WINE is reporting. Truth in advertising I am using a version of the Batch scripts with several fixes made which I've not PR'd back yet.
Hey Dave, I have a question regarding Command Prompt... How modern command prompt accept environment variable although the original DOS doesn't? Thanks!
Bash and Powershell should by language design be faster than CMD. CMD does not support local variables, only environment variables which are stored in the process environment block. In Bash or Powershell you can define a local function variable. The internals can then be quite fast for updating values in the assigned memory/housekeeping. For CMD every update to a variable becomes a string "=" which must be then updated in the environment block via an OS call (SetEnvironmentVariable etc) and no matter how the OS decides to manage that env blob for the process (inplace insert/update string, update string at the end of all other strings) the work involved is orders of magnitude slower than updating a few bits in my local array variable. Take Command (since robsku1 brought it up) could be faster than CMD but slower than Powershell based on language definition. Take Command has very limited support for local variables - for example the SETARRAY command lets you define a simple array that would be handy for this Primes experiment. 4DOS only supported dos environmental variables so it would be handicapped by tons of string manipulation and have CMD like perf in theory.
I’ve been coding in PS for many years and can rip out some very functional code quick smart. I wrote a complete electricity generation and retail system in PS. Everything the CLR can do, you can invoke in PS. Integration with SQL Server is ridiculously simple. I am a great fan. I’ve coded in ksh and CMD batch, but these environments can’t hold a candle to PS.
I still consider BASH as a superior shell. I have never used math functions in my scripts - this was never ever needed. All UNIX-like OSes have the "bc" command (basic calculator) for mathematical stuff. The bc can compute many functions, including Bessel functions or trigonometric functions.
i would love to see a drag race between diffrent linux distros. stuff like pop os, manjaro, arch, ubuntu, and more. shure some of these are based off of one enother but i doubt this makes them the same
WRT to CMD -- it's not that the bear dances well. It's that it can dance at all! ;) If I want a real script, I use JavaScript otherwise CMD. Powershell is powerful but a learning curve WRT to my use cases hence my use of CMD.
Now we need to see the most complicated, difficult to understand version. Or maybe the strangest... Maybe 1 script with as many multi language dependencies as possible
How are results passed between PS cmdlets? In normal *nix or cmd there's the stdin and stdout char streams, but PS deals with objects and lists, what is the data encapsulation format and how is it passed?
