The Rise & Fall of LISP - Too Good For The Rest Of the World 

To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/GavinFreeborn/. The first 200 of you will get 20% off Brilliant’s annual premium subscription.

I used to be a compiler writer at AT&T research labs many years ago. I was a member of a small team that developed something called a "common runtime environment" which allowed us to mix code written in Lisp, Prolog, C with classes (an early version of C++), and a few experimental languages of our own. What we found was that Lisp was a write-only language. You could write nice, compact, even clever code, and it was great when you maintained that code yourself. However, when you handed that code over to somebody else to take over, it was far more difficult for them to pick up than with almost all the other languages. This was particularly true as the code based grew. Given that maintainability was paramount, very little production code ended up being written in Lisp. We saw plenty of folks agree it seemed like a great language in theory, but proved to be a maintenance headache. Having said that, Lisp and functional languages in general, did provide great inspiration for other languages to become side-effect-free and, perhaps more importantly, to improve their collection management.

Back then AI surely was a different field, how brave of them to actually consider Logic in the context of Artificial Intelligence and not just the training of virtual monkeys to type Sheakspeare (or Reddit, if we speak specifically of OpenAI).

Get on board with Clojure everyone!

Thanks for this video! Good overview.

Great work!

great work with this video 🚀

that was great. please make more videos about diff dialects and histories revolving around lisp. i'm fairly new to it and i find it absolutely fascinating! while it might not be something useful in a professional sense, i'm really enjoying learning lisp (some common lisp and emacs lisp) and i find the language structure very elegant. but i totally understand why it didn't take off to be honest. it doesn't map well to how CPUs work and there's inherent garbage collection which in many cases becomes a blocker. | I would love to have more deep dives on the origins and histories of diff dialects. great video

Great video!

I write Lisp every day for work. Targeting three separate language VMs. With strange aeons, even death may die. Probably work with a prolog-based data-store at some point too.

Great talk! Thank goodness that we have Clojure to carry the torch, even though there are only a few of us still paying attention to the land of parens.

omg bless you for this

I think it's important to recognize why Lisp was powerful, not why people *wanted* to believe it was powerful. There is one reason in my view: Homiconicity. The composability and modeling power homoiconicity gives you is unparalleled. The orthodox view has always been that macros and the cons cell were the secret superpowers, which misses the appeal entirely and is a big reason CL floundered. Most problems can be modeled to be solved with runtime functions, even building a DSL, and the best performance is achieved with abstractions on top of arrays. There is a reason that new Lisps like Clojure and Janet focus less on macros and ditch the cons cell: Composability, modeling power, productivity, those are what matter. From there, the way you get good performance is by backing that composability with high-performance data structures like array-lists and hash maps.

Back in the 80's we learned Lisp on IBM's Lisp/VM while at the University of Maryland. Recursion seemed so seamless in Lisp. I can't say why, later I felt that my work with Lisp helped me feel more comfortable writing with SQL queries.

I wrote tons of applications using autolisp (by Autodesk and used in autocad) in the late 80s and 90s. Also Wrote an NC (numerical controlled) software package for 5-axis machining, sold it for $1,500 per license, it did pretty well.

Thanx for interesting history aspect of LISP

Great video, I don't fear for the future of common lisp. It got some greate compiler (SBCL) and a fervent community. Sure it's not perfect, but so far I think it's the best imho. It's also nice to know that my code will still be able to run as long as there's an ansi common lisp compiler.

I have a hardcover book on NLP with Lisp. I've had it for a while but it's been winking at me furiously for the past couple of months so I'm looking into watching a couple of intro lisp videos and diving in. NLP is a big interest of mine and I'm looking to broaden my horizons.

I like your analysis about the rise and fall of Lisp. If you have time I would appreciate a video about the rise and fall of Prolog. Maybe it's similar to Lisp...

Thank you.

I love LISP. I got into LISP because of JESS rule engine and eventually created my own open source rule engine. The big roadblock for many developer is s-expression syntax. Over the years I've tried to teach it to other developers and 90% never get over the syntax. Parsing s-expression is elegant and beautiful compared to Java, CSharp and 95% of languages out there. I'm biased, so take it with a grain of salt. The real power of LISP is logic programming, which causes a steep learning curve. The downside is logic programming is hard and most programmers prefer imperative programs.

Question about the thumbnail - why does the line graph in your chart go from right to left? Time is usually shown from left to right. Or is that explained in the video?

As stress increases every language devolves to lisp

~ 2:00 - Is that working off an Unisys-1100 series?

This was a pretty good review of the situation. You mention the issue of algorithmic complexity with a language based heavily on linked lists. Another issue is the divergence between CPU and memory retrieval speeds, necesitating on-chip cacheing. That makes linked lists not just algorithmically bad, but bad in a contingent practical way. It also makes it much harder to reason about optimizing performance when using a LISP-like language, compared to a C-like language.

Thank you for that walkthrough many lessons on how a tech can miss a window! I Used LISP quite a bit in the 80's during college, and used it a bit on TI Explorer, on of the LISP machines of the time. As you described LISP was not a light weight language. And in the 80's the biggest killer for it was the Mac and PC. Experiements aside, it was not practical for any application development.. In that space, C then C++ fit much better. Fast forward to the internet, while JS is NOTHING like it, however, JSON sure made data sets very easy to construct and manipulate.

One of my first languages, after English, haven't used it since I was a kid though.. still, almost any programming I do now, I attempt to force the language into that structure..

It's worth mentioning here that some of the earliest symbolic mathematics programs were written in lisp. Macsyma (and its open source cousin Maxima), Axiom, and its forks openAxiom, and Fricas. Both Maxima and FriCAS are still being actively developed and are very powerful computer algebra tools.

I am here to fulfill the obligation of functional programmers in youtube comments with this PSA: Clojure is a good language.

Thank you for that; there is a lot of good information here. I think there are more reasons that Lisp "lost" than those mentioned. For example, (I am butchering this quote and I don't recall where I read it) it has been said that most systems written in a language like C will end up with a poorly-written Lisp implementation inside it eventually. This is an observation of the old, but as you know, the number of software developers has doubled every 5 years or so. So it is also an observation that most developers are not in a position to make.

"...the development past these simple prototypes was very slow and seemed to run into a wall very quickly" - unfortunately, this is a concise description of programming projects in general.

It's notable that, historically, there've been two languages that share a similar evolutionary trajectory: LISP and Smalltalk. Both languages are VM-based and image-based. Both are very dynamic and flexible. Both had special hardware to run them (in the case of Smalltalk, the Xerox Alto). Both needed expensive hardware (lots of RAM, fast processors, and in the case of Smalltalk, high-resolution displays). Both have experienced massive market fragmentation. Both have been the victims of Java hype. Both missed the mark with respect to the Internet. However, there are important differences, too. Smalltalk did become mainstream in the 1980s and 1990s. In fact, according to a 1995 IDC report, Smalltalk was the second most popular object-oriented language after C++ and ahead of Objective-C, Object Pascal, CLOS, and Eiffel. It was mainstream enough that IBM chose Smalltalk as the centerpiece of their VisualAge enterprise initiative. Today, Smalltalk is still used by thousands of enterprises around the globe supported by no fewer than three major commercial Smalltalk vendors: Instantiations, GemTalk Systems, and Cincom. I can't think of any major commercial LISP vendors unless you count Franz and LispWorks. Clojure (2007) and Pharo (2008) are modern adaptations of LISP and Smalltalk, respectively. Neither language is particularly popular - they don't rank in the Top 50 at TIOBE. For a while, Clojure did receive a lot of hype but now it's nowhere. What happened to it??? Interestingly, when LISP celebrated its 50th anniversary in 2008 (at OOPSLA '08), it was very low key. Last year's celebration of Smalltalk's 50th birthday was pretty awesome: levelup.gitconnected.com/mmxxii-year-long-celebrations-for-smalltalk-bed82541b7db.

Living that sweet closure fennel life. God bless

Loved this video. Also loved the honorary footnote to early day AI efforts, Symbolic AI in particular, and how they relate to the LISP community. Really interesting how tools and ideas propagate or enjoy adoption. Fascinating to think how the world could have looked different if just a few small changes were in effect. Anyways... Happily LISPing on (Emacs) but have to admit I write Python when it comes to practical ML stuff these days. 😅 Keep these vids coming. Read a request in the comments for more dialect explorations. I'd be curious about that as well. Would generally also enjoy this style of a documentary/history lesson into other languages. Great stuff this. 🎉

I am crazy enough to think a foundation to increase Lisp usage, marketing and viability would make a major difference in how much Lisp is used. Writing far faster code in Lisp than many of the most popular languages (looking at you Python) produce today is trivially easy in Lisp. And the reality is that programmer productivity and maintainable extendable code is WAY more important in the the overwhelming majority of programming done today than raw speed anyway. So I am really tired of this old chestnut. Anyone that knows anything at all about Common Lisp knows you don't do most everything is cons list. That is an absurd excuse. For how many decades has CL had Just in Time compilation, full arithmetic stack, tons of sophisticated data structures and the best OO system in existence? Java is modern COBOL and completely sucks in so many ways.

Can only do programming with Lisp. Also learned how to do formal mathematical proof in the sense of Curry-Howard btw.

When I started programming I was obsessed with game development, So I learned some java. After that I tried C++..... yeah(C++ wasn't for me at the time). And then I went to other languages, lisp is one of my favorite languages. But I really just use what I want and is required at the time. Great video.

Great video. Lisp isn't going anywhere. It will continue to evolve. I agree that future code maintainabilty is the main issue of why it isn't used on large projects, and rightfully so. But you can do anything with it. When you get the Lisp red pill of code is data and data is code it feels like your life changes. I tried learning programming in Java but didn't care for it. I downright hated it as well as OOP in general. It was an unneeded abstraction in most cases. Then I learned interpreted languages like Python, which made so much more sense. Then I discovered Lisp and learned it can be interpreted or compiled. And with compiling, variable declaration, and different data structures, you can make it as fast as C. There is a paper written on its speed capabilities. That blew me away. The history of Lisp is also fascinating as it was based on lambda calculus and so far ahead of its time. Programmers are concerned with wasting time and effort into technologies that fade. Lisp doesn't have that problem. Fortran, Cobol, and Basic have faded but Lisp is still with us and relevant with Clojure and Racket. That tells you something. You can use any data structure in Lisp, it's not just confined to linked lists. You don't have to use cons and cdr. It's AI capabilities will continue to be explored. It's still the ultimate AI language. It sounds cooler at conferences and cocktail parties to say you created a new programming language than to write a new, badass library in an old language. Imagine if a few devs created some revolutionary AI and machine learning libraries in Lisp. That would be huge. Also, some brilliant hacker could create a new web framework that uses Lisp on the backend and cause a massive resurgence like what Rails did for Ruby. You can port any library created in another language to Lisp by creating new bindings. Lisp is the most powerful and expressive language ever. But with great power comes great responsibility. We learned that from Peter Parker. We could get around the maintainablility thing by having more discipline with programmers to not create macros unless it's absolutely necessary. Don't use a macro when a function will do. Also create a macro library/index for the project so other devs can see them and not duplicate efforts. Code must be well documented for future maintenance. C works well because it's small and new devs can make sense of it.

It seems to me the reason Lisp isn't used is power in labour market. The writer of the program would have more leverage power and would command more demands in work force. Much easier to replace someone if need be when using the other popular languages today which makes it cheaper for companies. Barrier to entry at the start sounds like why it wasn't popular for homegrown programmers.

I'm tempted to learn LISP but every time I attempt to get conversant with it the syntax puts me off. I'm constantly looking for a strongly typed and expressive language, without a GC, which can be used for systems software development as well as 'higher level' software development - without strait jacketing the programmer - with perf close to C - with reasonable compile times and good C interop.

Well the concepts of lisp like meta programming and closures are the important part of IMO.

I just saw your video on learning lisp in 2022. I got enthusiastic about learning lisp. And immediately after that video i see this one. And I'm not enthusiastic any more - this video is like a dark cloud added to the previous silver lining. Should I bother learning lisp in 2023?

You convinced me to learn Lips! As an Emacs user, it was probably inevitable....

Java won in the enterprise because it's "portable" and had the weight of a company behind it, but also because it has a very strict syntax and is C-like enough that everyone who went to college in the US can understand/learn it will minimal effort, meaning companies who adopt it had a large hiring pool of potentially inexperienced programmers, as opposed to a much smaller pool of more experienced/well-versed programmers

Blink twice if you being held by LISP nerds. 😅

Make Lisp Great Again

You should have showed the XKCD cartoon about it.

What about AutoLISP used by AutoCAD?

I don't think those reasons are essential. The biggest reasons why Lisp is so unpopular are: difficulty of learning declarative programming, tiny ecosystem, and being a committee language.

15:58 caption says "the is still an active CL community" the should be there?

As I see it, Lisp has won. JavaScript is conceptually based on Self and Scheme. PHP, Python, Perl, and Ruby are also conceptually very close to Lisp - they're all practically Lisps with syntax. Many programming techniques were first developed in the Lisp world and were later introduced in other languages.Lexical scope comes to mind. Also Garbage Collection. Delimited Coroutines can be found some modern languages, Python calls them Generators, C# too, I think. And some modern compilers use Continuation Passing Style. The whole point of Lisp isn't that it has a certain syntax and semantics, it is that its syntax and semantics are programmable. That makes it easy to try out new programming ideas without having to invent a syntax and write a compiler for it. Even building a simple Lisp interpreter yourself is trivial. Building software with Lisp is also possible, as shown by Clojure. But almost nobody programs in Lisp because almost nobody programs in Lisp. It has become a self-fulfilling prophecy by now. And no, languages with a C-based syntax don't have fewer brackets, they just have more kinds and in different places. After you play around with Lisp for an afternoon, you don't even see the brackets anymore. You'll learn, in fact, that Lisp's bracket-based non-syntax is the key to easily writing macros and extend its syntax and semantics. Just give it a try already.

I remember reading papers on the lisp machine and its practicality, and well, after doing some calculations, it seemed that the pure lisp machine would need several ~6petabytes of RAM What we have now is an impure hack of lisp on imperative and impure system.

I don't think CL/Lisp in general's window has closed. While obviously not as old as Lisp, Ruby was around a LONG time before Rails made it explode in popularity. The right library to do something people want to do that makes them feel highly effective can get people on any bandwagon. The hard part is figuring out what that niche is that isn't already being served well by some language. Obviously the ML train has sailed and Python + R + Julia has it mostly wrapped up and only way you're breaking ground there is something SUPER revolutionary, which could include as easy to write but faster to iterate, but whatever.

2:04 *Beranek, not Birkin.

You can argue that Lisp has won -- Javascript is sugared Scheme which is lexically bound Lisp.

1,리스 언는 리스퍼 로 작 성 되어는 가, ,수치 표현 되어 문자 출력 값, 언어 출력 , 구조 화 되어나, 아니면 다시 작성 되어야 한다..2,./0,1로 리스퍼 기본 값으로 ,,라호와 숫자 입력 ,알파벳 , 문자 입력 , ....언어 입력 , 으로 구조화 체적 화 같아요 . 3,이것은 최소 비용 실행 값의 축구 , 4,정의로 부터 ,.5, 혼자의 생각 기록 해보아요

(Message ‘good ‘video)

I need to do chemical simulation programs and I was recommended LISP over C++ but god, why does LISP feel so heavy to learn.

I’ll still study thid

Well there's Clojure

I think the curse of lisp describes the problem really well, Lisp community is a disconnected community of lone wolves or lone tigers and this is why Lisp has fallen. However Lisp has raise and fall multiple times. What really has killed Lisp is ChatGPT.

LET"S FUCKING GOOOOOOOOOOOOOOOO Gavin. You getting ads for a company i already used (mainly due to ads tooj but is good).

beranek

You might be surprised to know that LispWorks is thriving very well.

I was exposed to LISP back in the 1980s in college. I wS very impressed by it's simplicity and power. Too bad the academics at the University of Chicago couldn't teach.

Kriz

How did it not fail, you explained why it failed isn't that failure?

You can do the same thing with a much more suitable syntax, and when a program actually leverages all these features, compilers generally struggle (to put it lightly.)

So all would be well if there was never an internet?

I think the only hope for mass adoption of any lisp language is to resurrect the lisp machine in 2 tiers: the prosumer and the enterprise

slug russell

11:00 Common Lisp has arrays.

Lisp makes it easy to modify code when the program is running and extend the language using macros. It is hard to write self modifying code, and it is hard to use macros to extend the language. Lisp makes it easy to write compilers for specialized domain-specific languages, but since most people don’t want to write compilers to extend the language to solve hard problems in a concise way. Lisp is not popular because most programmers don’t want to write compilers for new languages.

"If you look at the landscape today, something is wrong." - so true 🙂

Blink

❤

To me, lisp is the most versatile language. It’s too bad you have to read through all those parenthesis.

Lisp will be like learning Latin for english.

I shall always remember the Devil's DP Dictionary definition of LISP: Lots of Irritating and Spurious Parentheses.

The problem with lisp is that computers don't use lisp. They use ML. So if you want to learn how computers work, C is a natural choice, for a first language, or maybe Java if you want to obscure a few things. If you go beyond that you start separating people from the computer and that never ends well.

Having been a programmer of the 70s thru 90s I saw how C and it's ilk became popular because of it's (unnecessarily) arcane syntax so some programmers could play their game "you can't figure out what my code does, ha, ha, ha."

Let's be honest, the average programmer trying to learn list is still trying to match parenthesis(I'm kidding, calm down).

I don’t know if it’s because the lighting is all on you face and none in the back wall, but this video feels uncomfortably close to your face. It’s like a virtual close-talker. It made it difficult for me to focus on what you were saying. Hopefully that’s not too harsh, as I’m trying to be constructive.

That was a solid dose of copium.

you blink way to often bro

I watched this video thinking it was about the speech impediment.

You need more sleep.

In Short, Lisp died because of its syntax. The syntax lacks appeal, and syntax is the most highly visible component of a language. Syntax is a part of what markets a language, and LISPs marketing strategy was not progressive, to say the least. Nobody likes the syntax of Lisp. They just like the assumed benefits of it. Other people created other languages that *appealed* to them, and clearly, if LISPs syntax had appealed to them, they would have designed their language with that syntax. Well guess what, nobody else did! What does that tell you? Another thing to note, is that when LISP fans advocate for LISP syntax, visual, aesthetic and subjective appeal are not used as one of their arguments. Rather, it is the simplicity of the syntax's rules that is argued for. A language is *defined* by its syntax so prevalently in the human mind. If it weren't for the fact that language, as a practical tool, is meant to be read just as much as it is to be written, then this lack of appeal being such a prevalent factor could be more easily attributed to availability bias formed from the common negative experience of all those parenthesis. The truth may be that, as an industry, we have not thought through the psychology and pragmatics of the syntax of language, and how to optimize language for human psychology and for optimal programming outcomes. Languages really are a form of user interface for a reader and writer, and really deserve the approaches and perspectives of UI/UX design, psychological engineering, ergonomics, and human performance optimization. Human preferences for syntax really do comprise a highly psychological and emotional subject matter, and we know from psychology that humans are primarily emotion-driven. In psychology, it is well known that negative emotions carry 3 times the weight of positive emotions, and that in healthy relationships "for every one negative feeling or interaction between partners, there must be five positive feelings or interactions." Perhaps a similar relationship exists beteen humans and their "things".. We can extrapolate that when LISPs syntax is a regularly-occuring negative experience for newcomers, it pretty much keeps most people away. The movie was decent, as it was very focused on getting into details about LISP.

so lisp failed due to garbage decision makeing kk.

Lisp fell out of favor among developers because it did not offer what consumers wanted: performance. It doesn't matter that Lisp can do logic thinking elegantly if a badly written if-then-else C/C++ program can outperform it 10 to 1. Consumers don't see the elegance in the language. What they see is how fast they can get results. In terms of abstraction, Lisp is several abstraction layers away from the microprocessor as designed today. C/C++ is just one layer above assembly language. it offers better performance over a language which has to go through a thick layer of interpretation and processing to get essentially the same result. Even today, when AI has made a comeback, C/C++ still rules the performance arena and is the language of choice for programming the large language models underpinning ChatGPT and Bard. CUDA is based on C++ and will probably remain that way for some time because of inertia and traction. While Lisp may have a pioneering hand in the development of AI, it is Python which has taken away Lisp's mojo. Python works very well with C++ and Python is the preferred domain specific language for accessing the AI models. Will Lisp ever come back and take over Python's role in AI? Maybe. Lisp and similar functional languages are very good at implementing domain specific languages so Lisp could still be the gateway to AI. But the heart of AI will remain C and C++ for the near future. Performance still matters.

Lots of Incomprehensible Silly Parentheses - reason why LISP never quite made it

While I deeply appreciate your promotion of Lisp, this commentary is significantly inaccurate in many places.

10:30 lookups with an array are O(1) if you know the index no shit Sherlock every problem is solved in O(1) anyway if you already have the answer bruh

Lots of Interesting Silly Parenthesis, or NAMBLA. Of course it was not going anywhere. It really sucked, a lot. Bye bye LISP and Prolog, you wasted my time.