Compiler-Driven Development in Rust 

Days since last errata: 0

"(got all vaccines)" hopefully you got super-autism now!!!

8:56 variants is misspelled

Nobody notices your voice is slightly different.

It's actually the other way around, autism causes vaccines! Here's my video on autism btw: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-DhY3fu-YgHU.html

Funny story, that's *ALSO* half of what Lost Terminal is! (have you heard about this engine that burns Iron? Or this cool story from the early internet? or about this weird amateur radio trick? etc!)

​@@NoBoilerplate Great show! I've been enjoying it! Just wanted to let you know that the recap section on the website is two seasons out of date (well, three if you want to recap season 16 before it's released ;D).

... And the shampoo bottles

In the alternate timeline, where Seth knows Rust

There's already rust easter eggs in Lost Terminal.... 🕵️

RIGHT! Rust Doesn't have features that other languages dont, but it makes them coherant and MANDATORY!

i just picked up go and its definitely not

@@minnow1337 You can just as well choose JAVA or Kotlin or Swift. The main difference between those languages and C/C++/Rust is: you do not have to worry about memory management (with a Caveat when using languages like Swift or Objective-C which do reference counting instead of garbage collection). You can create data structures on the fly without feeling the cost to do so, but when measured vs. algorithms where developers do care about memory management (and Rust forces them to do so), you will find that your code performs worse by magnitudes: using more memory (by a lot) and more CPU time (by a lot). Which is even a cost factor when your code runs in the cloud.

if it compiels, it works!

My experience with Rust is that, if you spend time using the debugger at all, it is for finding flaws in your algorithm, whereas in C and C++ it is more often to find flaws in your data (i.e. memory) management or other pitfalls of the language (like why this goddammend template does not compile, but pesting me with totally inconclusive error messages instead?) . This not having to deal with the pitfalls of a programming language is what makes me fond of Rust, and enjoying to do programming in it as a hobby. I love things that get me focused to work on real problems, not tool issues. As I want to think on algorithms, not tools.

Thank you! This is why I can't *not* make these videos raving about this language. and it's #19th most popular in the world! YOU CAN GET PAID TO WRITE THIS!

You know there are lots of languages that already achieve the same? Rust is not discovering "CDD" and some could argue that is not the best at doing it too

@@jofla I honestly don't know of any. Do you have any examples?

I don't need Rust to be the best at it. I'd be writing Haskell or another ML language if I cared about 'best'. I want my language to be popular and powerful. Go is the former, Haskell is the latter, Rust is both. redmonk.com/rstephens/2024/03/08/top20-jan2024/

My assumption is that they assume compiled languages are all the same, which is reasonable if you've not tried an ML or Haskell style type system before!

I'm sorry it's taken so long! RSI is a hell of a thing - I'm on the road to recovery finally :-)

@@NoBoilerplate Nah dw about it! Whether it's you taking a break or you gifting us a video, all of us are glad you're here either way 😁 Loved the video btw! the TypeState pattern literally had me going all 🤩

I'm sorry it's taken so long! RSI is a hell of a thing - I'm on the road to recovery finally :-)

@@NoBoilerplate For better or worse, I stopped keeping up with code tubers since I have been on my own project (fantasy console) and writing for a novel. I have reached the point where my code feeds into my writing and vice versa lol. Plus, glad to see you recovered

@@NoBoilerplate Hopefully you have a more ergonomic setup. Not sure about you, but switching to a split keyboard with a vertical mouse in the middle of the two halves helped me a lot when I began developing wrist pains. Oh, and good wrist rest pads too.

many thanks for the tips! my stupid self-imposed 16-hour days gave me RSI even using a split kb and mouse! oops! breaks saved me!

Totally agree. For me, CDD is the whiteboard, TDD is the prototype, and functional testing gets you to production. But you must START with CDD 😀

@@NoBoilerplate Hmm I see, maybe I'm wrong but this video gave the impression CDD completely replaces the need for testing. I might be wrong! But then maybe testing is limited to logic errors and it's fully unnecessary to test anything else -- which is still amazing. Can you comment on this? 😄

That is why Rust makes writing tests so easy and executing them so accessible in all IDEs. Being a system programming language, where the ultimate goals are both correctness and performance, having a expressive test set is essential, as it allows you to verify whether your optimisations tried are sound or not in an instance. So I am more of a TDD guy. But I do not write any tests for a function until i believe it will work. Which it doesn't most of the time.... but fixing things in Rust is easy then, as every feedback of your toolchain is so conclusive.

Re-watch that video, the part that allows the graph-like thinking you hope for is the normalisation section. Think of it as tables in a database, not animal genetics. It took me a long time to unlearn OOP, python helped me, Haskell was revolutionary, Rust finished my path ☺️

@@NoBoilerplate Alright. By the way, when I talk about "graph-like structures", I'm not referring to inheritance, I know that it's bad. Rather, I'm talking about the ability to have different objects directly referenced by various others as many times as you want. In Rust, you can only have one owner of a (mutable) reference normally to ensure program validity, but it's very limiting. I'm trying to program a transportation network where it would be nice to have one object be owned multiple times, but solutions like Rc make the program look ugly and much more complex. I assume Rust is purposely designed this way to discourage the patterns I was using. To avoid this, I decided to assign items IDs and created connections between transportation sites using their IDs rather than direct references. Is this in line with what you were trying to communicate in your video? Thanks

I think you're on the right path, here's how to fix your complexity issue - abstraction. Use Rc for the plumbing of your code, but abstract that complexity through layers of your code. Imagine a web server like Actix or Rocket or Axum. You better believe they're using some GNARLY clever stuff behind the scenes, but I as the web developer never see that, because they've abstracted it away from me, as they damn well should. unsafe abstracts pointer access, String abstracts bytes - to program is to compose layers of abstraction. Refactor with that in mind, and you'll find your solution. Same as in any language, I bet! Good luck - come ask for help in my discord if you need it, there are far smarter folks than I there ☺️

Thank you! It's so hard to put these kinds of ideas into simple terms, if I had more time, the video would have been shorter :-D

Thank you, added to the errata pinned comment

when I must code js, I write typescript - the correct default 👌

I imagine Python - the #1 language in the world - will be around for a while! Especially if they use github.com/PyO3/pyo3

@@NoBoilerplate I hadn't heard of pyo3 actually, thanks for the info!

I really don't get the rate. Python is a good enough interpreted language. Rust is a compiled language. They are totally different things. I like both. I will not be making my plots in Rust for a single use application. I don't need to fight with the compiler for plotting some data and formatting it well. Single use scripts are much better in Python. Large projects can become an unmaintainable mess. That is why Rust is better. Stop being monogamous with your programming languages.

@@vidal9747 Absolutely! The comment was a joke, but Python is definitely better for throwaway scripts (and I often use it for that).

15-years of writing python here - it's the most popular language in the world for a REASON! There's lots of applications I would not use it for any more, but there's still plenty I *would*!

Quite right, CDD is the first step - quickly sketching out the core model of your code. One should RELUCTANTLY move on to TDD and so on only after exhausting CDD!

yoink! Just be sure to send folks my way 😉

@@NoBoilerplate but of course, also without stealing anything.. thanks for ur output :)

Pleasure, it's all public domain ☺️

what do you mean?

rust and languages like it are so good to use with ai tools like copilot because of the extra context they give to the LLM, and the extra confidence they give to me in inspecting what my 1000 clueless interns have written for their group project today 😀

Don't believe what people say, try it out for yourself. Here's how I recommend getting started: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-2hXNd6x9sZs.html

my pleasure!

Rust *has* first-class compile-time execution, in the macro system, here's my video on it: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-MWRPYBoCEaY.html And though I'm not familiar with exactly what you need, rust also has const generics for 7 years: practice.course.rs/generics-traits/const-generics.html

@@NoBoilerplate Rust macros are far from powerful enough. I could technically create a macro which generates all types for N-Dimensional GA. But then if multiple libraries use this macro, these libraries will be incompatible, since both generate their own multivector types. Besides that, I have to choose a dimension in advance. The number of multivector types for each dimension N is at least 2^N. It's technically possible, but it's not great. What's great about scopes is that there are macros, which return types and memoize their results. So if I have a multivector type as a macro (for example a quaternion MultiVector!), it would implicitly create the type definition only once at toplevel, and every usage of this macro will refer to the same type.

oh cool! First I heard of Scopes, very neat! The behaviour you describe (creating a type at top-level then every usage of the macro re-using the top-level type) certainly could be written as a simple rust macro.

@@NoBoilerplate I'm pretty sure it's not possible. I would have to manually call some macro a toplevel to create the type first, and then maybe have another macro to use this type.

I don't normally put links in the description (WAY to many! I put them in my original markdown script, linked on github). But you're right, I should make an exception for this one!

Haha I'm reading it right now and you're right, it's great!

Games are FULL of state machines that would be amazing modelled using the typestate pattern - read the cliffle article I linked to to see all of it's power!

I too think that there's a clear workflow: CDD, TDD, Functional Tests, Interactive Tests pure code -> impure code. CDD is the first step, for SURE. BTW, Rust can run arbitrary code at compile time, so you COULD if you want run unit-test-style code all ins your CDD workflow, check out my video on const: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-voRBS0r4EyI.html

interesting!

Yeah, smells similar, right? I considered expanding CDD to include compile-time TDD using const, but that's just TDD running at compile-time, there's nothing super compelling there. MODELLING your application using the type system is where the real power is!

WELL SPOTTED! Trafalgar Square is often given as the centre of London, I suppose I pulled those from the london wiki page!

That's why I run clippy super strict so it keeps me honest! rust-lang.github.io/rust-clippy/master/index.html#/wildcard_enum_match_arm

Quite right! PhantomData is mentioned in the linked Cliffle post - I stripped out as much as I could - traits, boxed state data, phantomdata etc - to make my demo as small as possible.

@@NoBoilerplate That's fair, a very informative video as always (:

"what the actual user wants" I think you're talking about BDD tests there, larger tests that are based on user stories? TDD is quite a pure, simple theory, at its heart is just "how to code without fooling yourself!".

@NoBoilerplate Well, user of the code or "actual" user are both very broad, and I meant it that way. User of the code is probably another dev or yourself in the future. TDD makes you write ergonomic interfaces because you'd be insane to think up a bad complicated interface for code that doesn't exist. You state what you want to do. And TDD tests can vary wildy in scope, depending on what you're working on. You might call them by different names like integration, acceptance or behaviour tests, or whatever, but at the core, you wrote a test first, made it pass and hopefully didn't forget to refactor, which counts as TDD in my book. If you enter the testcase from a function call or a call to a webapi for which you are writing a new endpoint doesn't matter, really. In fact, the latter case can be better because it's less coupled to the code you are/will be testing, which is super important to look out for.

Oh, the cliffle article I linked to has a few options for storing other information in the state - I just kept it very tiny for my example. Do read it!

To be clear, I said, "The compiler is faster than all these methods, but to be fair, and balanced, it's because it's doing less." I've written production lisp at a bank, I LOVE repl-driven development, but compiler-driven is SO much faster!

@@NoBoilerplate I'm not sure why you'd believe that given how slow Rust compiles, even when working incrementally. Every REPL that I know of can use the full width and breadth of the language, even down to importing modules, and thus iteration is nearly instantaneous because it's a simple matter of :w from the other window and the up arrow key followed by enter to rerun the import line again.

Garbage game for garbage people

@@NoBoilerplate game? I mean the programming language.

so perfect!

yeah! ugly and less semantic!

Thank you! I'm so glad to be back making videos again ☺️

Thank you! get well soon!

Thanks! But yeah. I've described this as "locking in" logical pathways, so that you can make a determination and conversion in one place which truncates the semantic space which is even possible to describe at later points. This approach is literally part of how I sleep at night!

Because popularity counts. I didn't talk about Rust before 2022 not because I didn't know about it, but because all new language have infinite potential when no-one's using them. Rust matters because it's in the top 20: redmonk.com/sogrady/2024/03/08/language-rankings-1-24/

@@NoBoilerplate I agree that popularity counts but in the specific case of mojo (and perhaps tons more) you do have compactibility with every python libary so it doesnt matter much, its like typescript: an optional superset that doesn't break anything.

i know this is an unpopular opinion, but my answer is because i *really* hate python's syntax

I've now reminded myself what mojo is. "python but native speed" isn't a feature for me. My whole career has been in python. 15 years. And I'm sick of it. Python's core compromise of a GC, coupled with the GIL, whitespace syntax, are no longer things I value. In short, the above core features make easy things easier, but hard things harder. I am no longer impressed by how small a hello world is, or how few characters I have to type to do simple things. Here's my video on why Rust is not a faster horse ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-4YU_r70yGjQ.html

@@NoBoilerplate tbh whitespace syntax is one of the biggest "it doesn't technically matter all that much but I hate it anyways" feature of basically any programming language. any language with it loses a lot of points in my mind

people who say that just repeat stuff they've heard, they've not actually compiled any rust 🤣

@@NoBoilerplate With C or Jai like compile times, CDD would actually be an enjoyable experience. Currently I'm using C++ for game dev and I don't want to endure compile times that are somehow even slower than Rust's. I've tried compiling ggez and bevy and noticed with my own set of eyes that they are way slower in compilation speed (and are also harder to use) than something like Raylib. I can see Rust being an ideal language for general purpose and server development, but not for the domain I'm interested in. "don't just assume what other people say" is not an argument and just ignores one of the few complaints I have about Rust.

LOVE Jeremy!

this is the way

LOVE idris. I feel like Rust is an idris I can get paid to write 😀

🔥

ugh, I'd LOVE that! Excitingly, it's absolutely possible to do so, someone just has to write a complex enough macro! (deptypes has the right idea: lib.rs/crates/deptypes)

@@NoBoilerplate My reply got deleted (probably because of links) but there is also an interested project - based on a paper - called RefinedRust!

@@NoBoilerplate There is also prusti which has pretty good tooling (which is a problem I usually have with most for research software)

oh cool! What makes me excited is that whatever is built can be written as simple libraries using the macro system!

​@@qexatKani would be quite close to that

RSI + Covid are a hell of a thing to get at once! Both getting better :-D

@@NoBoilerplate no way. you got covid after all the covid hype ended?

@@RenderingUser Looking the other way doesn't magically make the virus disappear. (That's kinda why i still put precaution till this day)

@@RenderingUser Yeah unfortunately the actual virus is unaware of its hype status and continues to do its thing despite how hard we're trying to pretend it isn't.

The good news, if my personal experience is anything to go by, the trend for plague viruses to get less deadly and more contagious certainly seems to be the case with the variant I got. I almost didn't notice it for the last 3 weeks, unless I did anything strenuous on that day (then I'd be strangely tired)

Rumours of my death have been greatly exaggerated :-P

Jeremy's a great guy!

This wasn’t a connection I was expecting, but I can’t say I’m disappointed! 😊

I mean, I'm much poorer (financially) because of how much I love his channel, but I'm rich in synthesisers 😀

@@NoBoilerplate haha, that's the spirit!

This further affirms my theory that every channel I watch on RU-vid is related in some way. I should make a data visualization for that, now that I think about it.

Thank you!

Thank you so much, I really appreciate you saying so ☺️

The compiler is so helpful!

:-D

No boilerplate

There's always way to improve things, certainly, but it's the best language for concisely expressing the typestate pattern that I have seen!

I could see having a derive macro that would automatically generate the states. You would define all the states and content using an enum, and it would generate from it. For the impls I don't know enough to tell how, but a simple attribute above a function to filter only some states could be great.

Yeah, a simple macro would do it. That'd be cool. My Light example is the absolute simplest I could boil it down to - using it with traits (as the Cliffle article shows) is the best way to do it, and there's a bit more boilerplate there. some work has begun already lib.rs/crates/typestate

How about Haskell, though? What I love about Haskell is that the compiler can actually ensure that the only valid type parameter you can apply to Light are On and Off. The equivalent of Light would be a compiler error.

@@kintrix007 In that case, you can't actually *make* Light because the new() method is only defined for Light I mean, you _can_ declare a variable/field of having that type. But you can't actually assign anything to it, making it practically useless.

Thank you so much! Regular breaks are the core of my recovery!

There's a killer feature for interpreted language - state preservation between code changes. The browser and how you use python for ML are two HUGE applications. But yes, if you don't need a long-running interpreter, it's time for Rust!

you gain so little? meh I think then you dont actually what you are talking about.

different languages are best for different applications. python is the second best for all applications ;)

@@red_roy What's n°1? Haskell?

@@red_roy you just contradicted yourself with your second sentence.

SUCH a comfortable way of living!

Absolutely right. I feel like the dream workflow is: CDD, TDD, Feature tests, E2E tests - KINDOF in that order!

OH I COULD HAVE MADE A WHOLE VIDEO ON IT! It's my fav pattern! I believe the answer is to use Rust's generics - this is what the trait system is for - many standard library functions can accept any kind of string (not just String or str or osstring etc) by specifying traits that parameters should adhere to, rather than concrete types.

Not quite. There are type systems and there are type systems. Certainly, you can do the basics of the typestate pattern in any language, and it's a good idea. But there are powerful parts of the ML-inspired Rust type system that lend themselves to compiler-driven development. Here's one of my videos talking about some of the magic that the Rust compiler can do, I'd love to know what you think: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-voRBS0r4EyI.html

Thank you! I'm a big fan too! If you can believe it :-D

You can enforce specific invariants, like non-empty Strings or even clamping values between min and max, with a wrapping type around the original one. I recommend looking into Rusts NewTypes.

You've got it!

Paley, you're gonna really love Rust, I promise you. Let me prefix this all with my video on how I recommend you learn quick: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-2hXNd6x9sZs.html I'll answer your points one by one: 1) The reason you don't think this is possible is you're not as familiar with statically-typed languages as dynamicly-typed, would be my guess? So there are 5 ways of doing this that come to mind: 1. Do runtime validation. `if name.len() > 0` - gross. 2. Use formal methods that use perfect modelling to prove that there are no execution paths where name is not an empty string. - esoteric and impractical. 3. Dependant types. Some languages can natively constrain types so that a string must be (say) between 1-128 characters long. 4. SOME languages (such as Rust) can use compile-time magic to do all of the above. But you're gonna be doing some magic, and most of the libraries are still alpha. (for example lib.rs/crates/deptypes%29 5. Use the "Parse don't validate" pattern (lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ don't be afraid of the haskell syntax in that article, it's simple and you'll get it even without knowing haskell. But do learn it!) 5 is the way most languages solve this. TLDR: 1. parse your input into a type that has no other ways of creating it other than being parsed. 2. use this type everywhere and you can safely assume it's got valid data inside. This is how Rust's strings are guarenteed to only contain valid utf-8 code points, there is no way of making them in the standard library that doesn't get parsed and valided at runtime, so you can assume the compile-time type is always valid. 2) A mercifully short answer: Yes, CDD just comes first. TDD later, normal testing after. 3) Don't let the bastards grind you down, my friend. 4) My Light example was stripped down to the very bones (to the point that rust developers are in my comments tellling me off!) To get what you desire, just read the linked Cliffle article, the final example he shows, the trait-based typestate example is incredible, and has everything you desire :-)

​@@NoBoilerplate Thanks a lot for your response! Sorry for the late reply, I got a bit sidetracked and forgot to get back to you. I really appreciate the advice, especially the part about not letting bad developers bring me down-that was spot on. I’ve already checked out the resources you mentioned, and they were really helpful. Thanks again for taking the time to answer my questions!

The reason I produce "Fast, technical videos" is that for accessibility, I don't put animations, music, sound effects, memes or other typical stuff in my scripts. So the only way to hold the audience's attention is to have a TIGHT, fast script. The pause button is there if you need it - and you may read all my scripts on github in the link in the description! :-)

One of the very few RU-vidrs I watch/read at normal speed... And absolutely the only one I'll ever drop into reduced speed for... The style of presentation encourages pausing and thinking. Thx for this.

I'm delighted, thank you! ☺️

let us know how it went

this is the way!

MISSION ACCOMPLISHED! So excited for you, here's my video on how I recommend learning it: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-2hXNd6x9sZs.html

LOVE Jeremy (though my bank account doesn't!)

Ah Scala, my first love. Can you believe they dropped XML literals from the base install? Short sighted that is. Rust has that feature as a macro! Love that. I talk about rust vs scala in this video, btw: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-voRBS0r4EyI.html

Thank you so much! Also, you should check out my adhd video 😅

My pleasure!

i mean, the first time sure. but the build caching is pretty damn good. so unless you build in release mode while testing - dont - its pretty fast the second, third, etc. time you compile some iterative changes.

Don't repeat stuff you've heard other people say - give it a try! Wouldn't it be amazing if what I'm saying was true? 😉 ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-2hXNd6x9sZs.html

I was just messing, I like rust, zig, and c

Woah, you have RSS for no boilerplate uploads? How would I set that up? Thanks :)

@@half_invisible I have a Nextcloud instance hosted on my HomeServer and the included RSS app can just take the link to a YT channel :) I believe most RSS aggregator are able to do it

I use miniflux for that! RSS is truly wonderful. Aaron Schwartz left us a wonderful gift and did not deserve what happened to him 💔

most rss readers can subscribe to youtube channels, RSS rules!

So exciting! Make sure to run `bacon clippy` while you code 😀

Thank you very much! Certainly, you can still use the pattern, not as elegantly at all :(

This is my workflow too - CDD, TDD, Feature Tests, Production, Forget about it for 4 years because it's done. I LOVE writing rust code so much for this exact reason.

Ooh, can concur on the "forget about it, because it will work.". One year ago I cobbled together an rss/atom feedreader server with a web frontend. Both in rust. It has been running flawlessly on my raspberry pi ever since, and I use it every day. I wouldn't thrust code I write in python or c++ to do the same 😄.

I love how you can just build stuff and then move ON! (I might have done a video on this topic!)

Thank you! Heh, my code gets better every time I make one of this videos!

Thank you! if you want HOURS MORE I'm half-way through season 16 of my audiofiction podcast, Lost Terminal: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-p3bDE9kszMc.html I'd love to know what you think!

With an advanced type system, you can use types not just to store some values, but express relationships between types, including behaviours that would otherwise have to be enforced in your run-time program (think of rust's enums: they *model* a requirement that they must be in exactly one state at a time) You can model your application on a whiteboard, or in UML or in dedicated prototyping tools, but in Rust, you can model a LOT of logic inside the type system!

love it!

I see it like we all get 1000 interns ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-glpR1MD1UoM.html

Ha! I think probably the opposite is true, I sure love a table! Listening back, I think I clearly said "analysts", and my script matches that 😀

Very good point! The example is as cut-down as I could possibly make it, so I swerved constraining the type by trait, which would solve your problem. It's not even much more code, but for a quick intro to typestate, I swerved it. Check out the Cliffle article, it's amazing!

Quite possibly, It's the absolute smallest example I could show. Maybe I should have gone with the classic traffic light? DO read the linked cliffle article, it's so powerful!

Great minds!

There's nothing worse than writing bullshit code, even if you're getting paid for it. You deserve more, annoy your colleagues with perfectly modelled code at work, too 😀

Using the type system, in any language, is 100% reliable, if it's not, it's a compiler bug. Rust's type system can encode far more information than other popular languages (such as, say, Java or Go), and so you can extend these 100% guarantees to much more of your code, leaving less requiring unit testing.

I actually love loads of other things more than Rust - but Rust allows me to FINISH my code and go and go live my life 😀

ew, gross. Sorry reading the flutter docs made my eyes bleed. I THINK the main difference is all states are proven by rust at compile time, and no unit testing is required because the compiler proves that there are no possible invalid states in your whole codebase.

not quite!

Big fan of typescript, when I work in a js framework, this is what I use!

You've got it!